multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 115 viability of using wind turbines for electricity generation in electric vehicles francisco rubio1 , carlos llopis-albert1 1centro de investigación en ingeniería mecánica (ciim), universitat politècnica de valència, camino de vera s/n, 46022 valencia, spain emails: frubio@mcm.upv.es, cllopisa@upvnet.upv.es received: 2019-02-08; accepted: 2019-04-25 abstract this paper presents a feasibility study of applying a fluid energy recovery system by means of wind turbines for charging batteries of electric vehicles. this is because the main disadvantage of electric vehicles with regard to conventional fuel automobiles is the scarce capacity of storing sufficient energy to run long distances. this can be carried out by recovering a percentage of the energy used to overcome the aerodynamic drag of the vehicle. this work analysis different case studies, with different driving modes, to quantify the theoretical energy recovered from the vehicle aerodynamics. results have shown the theoretical possibility to implement this technology in actual electric vehicles. keywords electric vehicles, wind turbines, energy recovery, aerodynamics, battery charging. http://orcid.org/0000-0003-3465-702x http://orcid.org/0000-0002-1349-2716 https://doi.org/10.4995/muse.2019.11743 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 116 1. introduction. a wind turbine can operate as an energy recovery system (ers) similar to the brakes, i.e., regenerative braking (valero et al., 2017). when a car changes its speed in any sense or direction, its amount of energy varies (bangi et al., 2017; ferdous et al., 2011). when it loses speed, that energy tends to dissipate. traditionally the dissipated energy has been wasted. that is, the kinetic energy of the vehicle is transformed into heat during braking. in recent years, due to greater awareness of society about environmental issues, pollution and climate change, there is a great interest in developing energy recovery systems. one of the best known is the regenerative braking systems, which is based on the kinetic energy recovery system (kers) during vehicle braking. this allows reductions in consumption (efficiency increases) of up to 45%. in this article we analyze the feasibility of using a wind turbine as an energy recovery system, quantifying the savings that can be made in its two possible uses: as an energy recovery system and as a system using the aerodynamic drag, i.e., the force acting opposite to the relative motion of the vehicle moving with respect to the surrounding air (wen-long yao and chiu., 2015; valero et al., 2019). the recovered energy can be used for electric vehicle charging, thus reducing costs (llopis-albert et al., 2015; 2018; 2019). this can play a major role since electric vehicles sales have increased significantly during last years (zheng et al., 2018). in addition, a procedure for shape optimization of the wind turbine should be performed to increase the energy recovered (llopis-albert et al., 2018a). there are many optimization procedures in the literature in different research areas (rubio et al., 2015; 2016; 2019; llopis-albert and pulido-velazquez, 2015; llopisalbert and capilla, 2010; llopis-albert et al., 2016). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 117 2. case studies. this work uses a wind turbine with a horizontal axis and 50 cm of diameter installed in the frontal part of a vehicle. this vehicle will be subjected to three different driving scenarios that will be characterized by speeds, accelerations and time of circulation. 2.1 first scenario: corresponds to a worldwide harmonized vehicle test procedure (wltp). in this cycle, the vehicle undergoes a 30-minute ride with certain characteristics of speeds and accelerations and routes (table 1). accelerations from 0 to 50 km/h must be made between 5 and 10 seconds and a distance of 27 km is travelled. it is intended to measure the power used to get the vehicle to move under the stipulated driving conditions. more specifically, the wltp cycle lasts 30 minutes and consists of 4 phases: phase 1: low speed (589 s-9.18 min); maximum velocity (vmax) = 56.5 km/h. phase 2: average speed (433 s.-7.22 min); v maximum velocity (vmax) = 76.6 km/h. phase 3: high speed (455 s-7.58 min); maximum velocity (vmax) = 97.4 km/h. phase 4: very high speed (323 s.-5.38 min); maximum velocity (vmax) = 131.3 km/h. different driving modes (rubio et al., 2019) are simulated covering city (urban), secondary road, autonomous or national road and freeway. in the freeway the maximum speed will be 131 km/h and the average protocol speed of 46.5 km/h. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 118 table 1: circulation characteristics for scenario 1 (wltp cycle). v: velocity, a: acceleration phase time (s) stop (s) distance (m) % stop v_max (km/h) a_min (m/s2) a_max (m/s2) low 589 156 3095 26.5 56.5 -1.47 1.47 medium 433 48 4756 11.1 76.6 -1.49 1.57 high 455 31 7158 6.8 97.4 -1.49 1.58 super high 323 7 8254 2.2 131.3 -1.21 1.03 total 1800 242 23262 2.2 second scenario: it corresponds to a purely urban driving (in city) with the following characteristics: cycle time: 20.25 min. time in circulation: 15.55 min stop time: 4.7 min distance travelled: 8.84 km maximum speed: 50 km/h during the journey there are ups, downs, accelerations, decelerations and stops. 2.3 third scenario: it corresponds to an interurban driving (highway) with the following characteristics: cycle time: 1.7 h time in circulation: 1.7 h stop time: 0 min distance travelled: 198.79 km multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 119 maximum speed: 120 km/h during the journey we have considered ups, downs, accelerations and decelerations. 3. driving modes and acting forces. table 2 shows the driving modes of the vehicle and the braking or driving force that must be provided for the vehicle to move under the conditions set by the corresponding driving mode. table 2: driving modes case 1: acceleration in plain 𝑭𝒎 = −𝑭𝒊 + 𝑭𝒓 + 𝑭𝐚+𝑭𝒘 case 2: deceleration in plain 𝑭𝒇𝒓𝒆𝒏 = −𝑭𝒊 + 𝑭𝒓 + 𝑭𝐚+𝑭𝒘 case 3: constant velocity 𝑭𝒎 = −𝑭𝒊 + 𝑭𝒓 + 𝑭𝐚 + 𝑭𝒘 fm fi fr fa ffren fi fr fa fm fr fa multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 120 table 2: driving modes (continued) case 4: acceleration upwards 𝑭𝒎 = −𝑭𝒊 + 𝑭𝒓 + 𝑭𝐚 + 𝑭𝐰 case 5: constant velocity upwards 𝑭𝒎 = −𝑭𝒊 + 𝑭𝒓 + 𝑭𝐚 + 𝑭𝐰 fm fa fr fi fw fm fa fr fw multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 121 table 2: driving modes (continued) case 6: deceleration upwards 𝑭𝒇𝒓𝒆𝒏 = −𝑭𝒊 + 𝑭𝒓 + 𝑭𝐚 + 𝑭𝐰 case 7: deceleration downwards 𝑭𝒇𝒓𝒆𝒏 = −𝑭𝒊 − 𝑭𝐰 + 𝑭𝒓 + 𝑭𝐚 case 8: constant velocity downwards 𝑭𝒇𝒓𝒆𝒏 = −𝑭𝒊 − 𝑭𝐰 + 𝑭𝒓 + 𝑭𝐚 fa fr fi fw ffren fa fr fi fw ffren fa fr fw ffren multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 122 table 2: driving modes (continued) case 9: acceleration downwards 𝑭𝒇𝒓𝒆𝒏 = −𝑭𝒊 − 𝑭𝐰 + 𝑭𝒓 + 𝑭𝐚 the forces considered in the conduction of the automobile are: 𝑭𝒎 = driving force f 𝑭𝒊 = inertia force 𝑭𝒓 = rolling force 𝑭𝒂 = dragging force 𝑭𝒘 = weight 𝑭𝒇𝒓𝒆𝒏 = braking force the power is calculated as follows: 𝑷 = 𝑭𝒎/𝒇𝒓𝒆𝒏 ∙ 𝒗. 4. results. analysis of the consumed power the most important vehicle characteristics and the set of parameters used in the calculation of the different forces that act on it are presented in table 3: ffren fa fr fi fw multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 123 table 3: parameters a 2,2 m 2 c 0,32 m 1000 kg θ 0 rad ρ 1,225 kg/m 3 g 9,81 m/s 2 where a is the front area of the vehicle; c: drag coefficient; m: mass of the vehicle; θ: angle of the ramp up or down; ρ: air density; g: acceleration of gravity; for the calculation of the rolling force, the coefficient of rolling resistance is 𝒇𝒓 = 𝟎. 𝟎𝟏 ∙ (𝟏 + 𝑽 𝟏𝟔𝟎 ), where the velocity (v) is given in km/h. the value of the air density has been considered ρ = 1.25 kg/m3 at atmospheric pressure and at 15 ºc and θ corresponds to the slope of the up and down ramps. the acceleration in each section is calculated using the equations of the uniformly accelerated rectilinear movement taking into account the initial, final speed and the elapsed time. the values obtained for the different scenarios allows to determine the motor power and braking power required to drive according to the circulation characteristics described for each scenario. in these scenarios, the different driving modes described in table 2 have been taken into account. the driving power affects the energy consumption of the vehicle to maintain the desired circulation characteristics. the braking power corresponds to the power dissipated in the form of heat to maintain the vehicle speed. it appears when the brakes of the motion regulation intervene. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 124 the dissipated power can be converted back into energy recovered by the use of regenerative brakes or wind turbines. in this analysis we propose the recovery made by using a wind turbine. table 4 summarizes the results about the recovered power. table 4: ideal percentage of power that can be recovered recovery system recovery syst. + drag force theoretical % theoretical % scenario 1 21.9 63.45 scenario 2 40.95 52.06 scenario 3 19.73 78.34 5. conclusions this paper is a first step to investigate the feasibility of implementing a technology for energy recovery using wind turbines in electric vehicles. this is carried out by considering the effects of the airflow through wind turbines and the vehicle aerodynamic drag during its motion. this allows to recover a percentage of the energy supplied by the batteries to the vehicle engine. dissipative forces such as the tyre rolling resistance force are responsible for not being able to recover all the energy supplied by those batteries. results have shown the theoretical viability to successfully develop this technology. as important fact, this study has shown that the use of wind turbines allows the possibility of recovering an important percentage of the energy provided by the batteries, although it strongly depends on the assumptions of each case study. however, further research is needed to verify the data with experimental tests. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 125 references -bangi, v.k.t., chaudhary, y.; guduru, r.k.; aung, k.t., reddy, g.n. (2017). preliminary investigation on generation of electricity using micro wind turbines placed on a car. international journal of renewable energy development, 6(1), pp. 75-81. doi: 10.14710/ijred.6.1.75-81. -ferdous, s.m, salehin, s, bin khaled, w. (2011). electric vehicle with charging facility in motion using wind energy. world renewable energy congress 2011 – sweden sustainable transport (st), 8-11 may 2011, linköping, sweden. doi: 10.3384/ecp110573629. -llopis-albert, c., rubio, f., valero, f., (2019). fuzzy-set qualitative comparative analysis applied to the design of a network flow of automated guided vehicles for improving business productivity. journal of business research, doi: 10.1016/j.jbusres.2018.12.076. -llopis-albert, c. rubio, f., valero, f. (2018). designing efficient material handling systems via automated guided vehicles (agvs). multidisciplinary journal for education, social and technological sciences, 5(2), 97-105.doi: 10.4995/muse.2018.10722. -llopis-albert, c., merigó, j.m., xu, y.j. (2016). a coupled stochastic inverse/sharp interface seawater intrusion approach for coastal aquifers under groundwater parameter uncertainty. journal of hydrology 540, 774-783. doi: 10.1016/j.jhydrol.2016.06.065. -llopis-albert, c. rubio, f., valero, f. (2018a). optimization approaches for robot trajectory planning. multidisciplinary journal for education 5(1), 1-16. doi: 10.4995/muse.2018.9867. -llopis-albert, c., rubio, f., valero, f. (2015). improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research 68, 1429–1431. doi: 10.1016/j.jbusres.2015.01.027. -llopis-albert, c., pulido-velazquez, d. (2015). using modflow code to approach transient hydraulic head with a sharp-interface solution. hydrological processes 29(8), 2052-2064. doi: 10.1002/hyp.10354. -llopis-albert, c., capilla, j.e. (2010). stochastic simulation of non-gaussian 3d conductivity fields in a fractured medium with multiple statistical populations: case study. journal of hydrologic engineering 15(7), 554-566. doi: 10.1061/(asce)he.1943-5584.0000214. -rubio, f., llopis-albert, c., valero, f., besa, a.j. (2019). a new approach to the kinematic modeling of a three-dimensional car-like robot with differential drive using computational mechanics. advances in mechanical engineering, doi: 10.1177/1687814019825907. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11743 social and technological sciences issn: 2341-2593 rubio and llopis-albert (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 115-126 | 126 -rubio, f., valero, f., llopis-albert, c. (2019a). a review of mobile robots: concepts, methods, theoretical framework, and applications. international journal of advanced robotic systems, 16(2). doi: 10.1177/1729881419839596. -rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2016). industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. robotics and autonomous systems 86, 106-112. doi: 10.1016/j.robot.2016.09.008. -rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2015). assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering, vol. 2015, article id 931048, 10 pages, 2015. https://doi.org/10.1155/2015/931048. -valero, f., rubio, f., llopis-albert, c. (2019). assessment of the effect of energy consumption on trajectory improvement for a car-like robot. robotica, 1-12. doi:10.1017/s0263574719000407. -valero, f., rubio, f., llopis-albert, c., cuadrado, j.i. (2017). influence of the friction coefficient on the trajectory performance for a car-like robot. mathematical problems in engineering, vol. 2017, article id 4562647, 9 pages. doi: 10.1155/2017/4562647. -wen-long yao, a. and chiu, c.-h (2015). development of a wind power system on trucks. universal journal of mechanical engineering, 3(5), pp. 151–163. doi: 10.13189/ujme.2015.030501. -zheng, x., lin, h., liu, z., li, d., llopis-albert, c., zeng, s (2018). manufacturing decisions and government subsidies for electric vehicles in china: a maximal social welfare perspective. sustainability, 10(3), 672. doi: 10.3390/su10030672. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 1 characterization and assessment of composite materials via inverse finite element modeling carlos llopis-albert1 , francisco rubio1 , francisco valero1 1centro de investigación en ingeniería mecánica (ciim), universitat politècnica de valència, camino de vera s/n, 46022 valencia, spain emails: cllopisa@upvnet.upv.es, frubio@mcm.upv.es, fvalero@mcm.upv.es received: 13 january 2019; accepted: 26 august 2019 abstract characterizing mechanical properties play a major role in several fields such as biomedical and manufacturing sectors. in this study, a stochastic inverse model is combined with a finite element (fe) approach to infer full-field mechanical properties from scarce experimental data. this is achieved by means of non-linear combinations of material property realizations, with a certain spatial structure, for constraining stochastic simulations to data within a non-multigaussian framework. this approach can be applied to the design of highly heterogenous materials, the uncertainty assessment of unknown mechanical properties or to provide accurate medical diagnosis of hard and soft tissues. the developed methodology has been successfully applied to a complex case study. keywords inverse modeling; finite element; mechanical properties; heterogeneity characterization; biomedical; uncertainty assessment. https://doi.org/10.4995/muse.2019.12374 http://orcid.org/0000-0002-1349-2716 http://orcid.org/0000-0003-3465-702x http://orcid.org/0000-0003-2295-4035 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 2 1. introduction. recently, there has been increased interest in highly heterogeneous materials and composite materials due to its wide engineering applications. nevertheless, heterogenous materials show a large spatial and temporal variability, which lead to significant uncertainties in the estimation of the full-field material properties (wu and zhu, 2017). for instance, composite material encompasses three components: discontinuous multiphases, the matrix as the continuous phase and the fine interface area. examples of such materials can be found in bioengineering such as musculoskeletal tissue and bone or implants in orthopedics, porous ceramics, and metal-composite joints in automotive and aerospace applications (ni et al., 2007). as a result, there are composite materials with mechanical properties changes within the vicinity with up to five orders of magnitude (wu and zhu, 2017). a comprehensive review on the uncertainty representation of material properties can be found in charmpis et al., 2007; sriramula and chryssanthopoulos 2009. there are also attempts in the literature for modelling the effects of the heterogeneity on the mechanical response (e.g., zottis et al., 2018; zhang et al., 2018). the material property characterization is hampered by the problem of obtaining reliable experimental data either by the financial cost or by technical impediments. experimental tests for determining material properties cover a wide range of techniques, including non-destructive methods with the ability to access internal variables such as strains (e.g., mortazavi et al., 2014). https://doi.org/10.4995/muse.2019.12374 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 3 examples of experimental techniques that can be used in conjunction with this methodology comprise extensometers, photoelasticity, x-ray techniques, thermography, digital image correlation (dic) and digital volume correlation (dvc) (li et al., 2014). this paper makes use of the experimental information obtained with these approaches within an inverse model framework to provide accurate predictions of the effects of heterogeneous material properties. the reliability in numerical models strongly depends on the properties of the underlying material, with justifies the use of inverse models to reduce the uncertainty of highly heterogeneous materials. the methodology has been successfully applied to a complex case study, while providing an uncertainty assessment of the results. 2. material and methods. inverse problems are often ill-posed although there are new computational schemes to properly overcome this drawback. inverse methods are intended to determine the input and the characteristics of a system from some of the output from the same system. the methodology encompasses a finite element (fe) approach embedded into a stochastic inverse framework for calculating effective material properties for heterogeneous materials. the stochastic correlation structure of the material properties relies on an indicator conditional simulation technique (gómez-hernández and srivastava, 1990). as a first step, this technique generates a set of material property realizations, named as seed fields, that honours the material property measurements within a non-multigaussian framework. then seed https://doi.org/10.4995/muse.2019.12374 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 4 parameter fields are conditional to measurements, and also to secondary data, for example, from expert judgement. the a priori stochastic structure of seed parameter fields is defined through the conditional probability distribution function (cpdf) and the indicator variograms, which allow to estimate variables at any unsampled location using indicator kriging algorithms (e.g., goovaerts, 1997). this enables to adopt any random function (rf) model and to reproduce the coalescence and connectivity among phases and existing crack patterns, which are of vital importance to provide reliable safety factors and fatigue life predictions. the second step entails a numerical approximation of the mechanical stress (𝜎) and displacement (u) field by means of the finite element method (fem). the ansys software is used for that purpose (www.ansys.com). in the third step, the method carries out an iterative optimization procedure based on successive non-linear combinations of two seed realizations and the previous optimized parameter fields (the elastic modulus (e) and poisson’s ratio (ν)). the procedure is based on an iterative minimization of a penalty function which expresses the discrepancy between the experimentally measured and the numerically computed response of the underlying physical system. in this sense, the optimization technique is based on data assimilation to identify stochastic structures of uncertain mechanical parameters. this way of proceeding has been widely used in the literature in many research fields (llopis-albert et al., 2015; 2018; 2018a; llopis-albert and pulido-velazquez, 2015; rubio et al., 2015; 2016; 2019). further information about the inverse https://doi.org/10.4995/muse.2019.12374 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 5 method used in this paper can be found in llopis-albert and capilla, 2009; 2010; 2010a; llopisalbert et al., 2014; 2016. 3. case study. the methodology is used for the simulation of uncertain material properties (elastic elastic modulus and poisson’s ratio) conditional to measurements of those parameters and also to stress and strain data. the case study deals with the bending of a composite beam. the beam has a length of 4 m, and a height and width of 0.4 m. it has been discretized using blocks of 0.1 m, thus leading to 640 blocks. several boundary conditions are applied. the boundary conditions applied is that the beam is fixed at one end in all degrees of freedom and the blocks belonging to the freeend are subjected to a bending moment as a result of a pressure of 1 mpa at the top face. a set of 76 data are used as conditioning data for each variable. they are uniformly distributed along the domain. 4. results and discussion. the gradual deformation process for constraining simulations to measurements leads to significant differences between unconditional and conditional fields, i.e., important changes in the seed parameter fields are induced. in this sense, results show a good agreement between the estimated effective young’s modulus and poisson’ ratio in the conditional simulations regarding their corresponding values in the reference field. https://doi.org/10.4995/muse.2019.12374 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 6 after the iterative optimization procedure, fig. 1 represents for a given conditional realization the material property fields of the elastic modulus and poisson’s ratio, together with its corresponding displacement and stress fields. it depicts how the heterogeneity in the material properties is also present in the displacement and stress fields. the conditional fields also show the presence of non-gaussian features. the perturbation field is able to partially change the stochastic structure of seed fields to come close to data, thus reducing the uncertainty in the results. to better analyse the results a performance measure is defined as the square root of a weighted mean of the square departures of computed values from the measured values after a certain iteration. the conditional field presents, after fifty iterations of the inverse model, a performance measurement of 1.34e-09 m for the displacement field (u), and 16.47 mpa for the von mises stress field (𝜎). this entails a reduction in the performance measurement, regarding the unconditional field, of around 63% for the (u) field and 44% for the (𝜎) field. https://doi.org/10.4995/muse.2019.12374 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 7 figure 1. material property fields for a given conditional realization: elastic modulus (a), poisson’s ratio (b); and its corresponding displacement (c) and stress fields (d). 5. conclusions. a stochastic inverse model combined with a finite element (fe) approach is presented to characterize heterogeneous mechanical properties within a non-multigaussian framework. the fe method allows solving the problem in hand and obtaining the mechanical stress and https://doi.org/10.4995/muse.2019.12374 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 8 displacement fields. the iterative optimization process allows constraining simulations to available data by minimizing an objective function that penalizes the difference between measured and computed data. the methodology allows characterizing the structural parameter fields of the effective modulus of elasticity and the poisson’s ratio and their corresponding mechanical response. finally, it has been successfully applied to a complex case study. references -charmpis, d. c., g. i. schueller, m. f. pellissetti (2007). the need for linking micromechanics of materials with stochastic finite elements: a challenge for materials science. computational materials science 41(1), 27–37. https://doi.org/10.1016/j.commatsci.2007.02.014 -gómez-hernández, j.j., r.m. srivastava, (1990). isim3d: an ansi-c three-dimensional multiple indicator conditional simulation program. computer geoscience 16(4), 395–440. https://doi.org/10.1016/0098-3004(90)90010-q -goovaerts, p. (1997). geostatistics for natural resources evaluation. oxford university press, ny, ny, 483 pp. -ni, y., m. y.m. chiang (2007). prediction of elastic properties of heterogeneous materials with complex microstructures. journal of the mechanics and physics of solids 55, 517–532. https://doi.org/10.1016/j.jmps.2006.09.001 -li, g., f. xu, g. sun, q. li (2014). identification of mechanical properties of the weld line by combining 3d digital image correlation with inverse modelling procedure. international journal of advanced manufacturing technology 74, 893–905. https://doi.org/10.1007/s00170-014-6034-x llopis-albert, c., capilla, j.e. (2009). stochastic inverse modeling of non multigaussian transmissivity fields conditional to flow, mass transport and secondary data. 2 demonstration on a synthetic aquifer. journal of hydrology, 371, 53–65. https://doi.org/10.1016/j.jhydrol.2009.03.014 llopis-albert, c., capilla, j.e. (2010). stochastic simulation of non-gaussian 3d conductivity fields in a fractured medium with multiple statistical populations: case study. journal of hydrologic engineering, 15, 554-566. https://doi.org/10.1061/(asce)he.1943-5584.0000214 https://doi.org/10.4995/muse.2019.12374 https://doi.org/10.1016/j.commatsci.2007.02.014 https://doi.org/10.1016/0098-3004(90)90010-q https://doi.org/10.1016/j.jmps.2006.09.001 https://doi.org/10.1007/s00170-014-6034-x https://doi.org/10.1016/j.jhydrol.2009.03.014 https://doi.org/10.1061/(asce)he.1943-5584.0000214 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 9 llopis-albert, c., capilla, j.e. (2010a). stochastic inverse modelling of hydraulic conductivity fields taking into account independent stochastic structures: a 3d case study. journal of hydrology 391, 277–288. https://doi.org/10.1016/j.jhydrol.2010.07.028 -llopis-albert, c., palacios-marqués, d., merigó, j.m. (2014). a coupled stochastic inversemanagement framework for dealing with nonpoint agriculture pollution under groundwater parameter uncertainty. journal of hydrology 511, 10–16. https://doi.org/10.1016/j.jhydrol.2014.01.021 -llopis-albert, c., rubio, f., valero, f. (2015). improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research 68, 1429–1431. https://doi.org/10.1016/j.jbusres.2015.01.027 -llopis-albert, c., pulido-velazquez, d. (2015). using modflow code to approach transient hydraulic head with a sharp-interface solution. hydrological processes 29(8), 2052-2064. https://doi.org/10.1002/hyp.10354 -llopis-albert, c., merigó, j.m., xu, y.j. (2016). a coupled stochastic inverse/sharp interface seawater intrusion approach for coastal aquifers under groundwater parameter uncertainty. journal of hydrology 540, 774-783. https://doi.org/10.1016/j.jhydrol.2016.06.065 -llopis-albert, c. rubio, f., valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences 5(1), 1-16. https://doi.org/10.4995/muse.2018.9867 -llopis-albert, c. rubio, f., valero, f. (2018a). designing efficient material handling systems via automated guided vehicles (agvs). multidisciplinary journal for education, social and technological sciences, 5(2), 97-105. https://doi.org/10.4995/muse.2018.10722 -mortazavi, f., e. ghossein, m. lévesque, i. villemure (2014). high resolution measurement of internal full-field displacements and strains using global spectral digital volume correlation. optics and lasers in engineering 55, 44–52. https://doi.org/10.1016/j.optlaseng.2013.10.007 -rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2015). assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering, vol. 2015, article id 931048, 10 pages, 2015. https://doi.org/10.1155/2015/931048 -rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2016). industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. robotics and autonomous systems 86, 106-112. https://doi.org/10.1016/j.robot.2016.09.008 https://doi.org/10.4995/muse.2019.12374 https://doi.org/10.1016/j.jhydrol.2010.07.028 https://doi.org/10.1016/j.jhydrol.2014.01.021 https://doi.org/10.1016/j.jbusres.2015.01.027 https://doi.org/10.1002/hyp.10354 https://doi.org/10.1016/j.jhydrol.2016.06.065 https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.4995/muse.2018.10722 https://doi.org/10.1016/j.optlaseng.2013.10.007 https://doi.org/10.1155/2015/931048 https://doi.org/10.1016/j.robot.2016.09.008 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.12374 social and technological sciences issn: 2341-2593 llopis-albert et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 1-10 | 10 -rubio, f., valero, f., llopis-albert, c. (2019). a review of mobile robots: concepts, methods, theoretical framework, and applications. international journal of advanced robotic systems, 16(2). https://doi.org/10.1177/1729881419839596 -sriramula, s., m. k. chryssanthopoulos (2009). quantification of uncertainty modelling in stochastic analysis of frp composites. composites part a: applied science and manufacturing 40(11), 1673–1684. https://doi.org/10.1016/j.compositesa.2009.08.020 -wu, x., y. zhu (2017). heterogeneous materials: a new class of materials with unprecedented mechanical properties. materials research letters 5:8, 527-532. https://doi.org/10.1080/21663831.2017.1343208 -zhang, z., c. zhan, k. shankar, e.v. morozov, h. kumar, t. ray (2017). sensitivity analysis of inverse algorithms for damage detection in composites. composite structures 176, 844-859. https://doi.org/10.1016/j.compstruct.2017.06.019 -zottis, j., theis, c.a., da silva, a (2018). evaluation of experimentally observed asymmetric distributions of hardness, strain and residual stress in cold drawn bars by fem-simulation. journal of materials research and technology. https://doi.org/10.1016/j.jmrt.2018.01.004 https://doi.org/10.4995/muse.2019.12374 https://doi.org/10.1177/1729881419839596 https://doi.org/10.1016/j.compositesa.2009.08.020 https://doi.org/10.1080/21663831.2017.1343208 https://doi.org/10.1016/j.compstruct.2017.06.019 https://doi.org/10.1016/j.jmrt.2018.01.004 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 69 the importance of knowing the starting level of knowledge r. barajas, p. saavedra, j. albéniz, i. carrillo* dpto. química industrial y polímeros, e.u.i.t. industrial, universidad politécnica de madrid, c/. ronda de valencia 3, 28012 madrid, e-mail: isabel.carrillo@upm.es received: 2013-12-15; accepted: 2014-01-28 abstract the aim of this work is to demonstrate the importance of the fact that from the beginning of the course students and teachers identify what is the initial knowledge level and the level that students must reach at the end of the course. this is important for the student because it will demonstrate the importance of the continuous assessment (bologne methodology) and in the other hand, for the teachers to know the initial level of their students. it can give influence to detect deficiencies and solve them to a well ending course. this helps a lot to students with an initial lack of knowledge of the subject. this study carries out specifically in the chemistry subject in the degree of ingeniería de diseño industrial y desarrollo de producto (industrial design engineering and product development) at escuela universitaria de ingeniería técnica industrial (e.u.i.t.i.) of the universidad politécnica de madrid (upm). keywords chemistry, teaching-learning, educational innovation. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 70 1. introduction “chemistry is all around you” as you can see in the video of the same title ( http://www.youtube.com/watch?v=y6zl7msxbag) about the contributions of chemistry at all areas of the society produced by the european petrochemical association (epca), the united nations educational, scientific and cultural organization (unesco) and international union of pure and applied chemistry (http://www.unesco.org/new/es/natural-sciences/science-technology/basicsciences/chemistry/international-year-of-chemistry/). the chemistry is, in spite of non being an essential subject in secondary education, absolutely important in all our surrounding living things. it has many applications in other scientific areas, such as medicine, materials technology, pharmaceutical industry, food industry, electronic industry, construction, environment,... it is necessary to point out that chemistry subject at high school curriculum in the engineering area, is not obligatory in 2nd course (orden esd/1729/2008). this fact joints with non-empathy from students to chemistry and causes many of them to enter the university with an almost non-knowledge of this subject. however in almost all degrees of upm, chemistry is a compulsory subject in all the industrial first year degrees area (http://www.upm.es/institucional/estudiantes/estudios_titulaciones/estudiosoficialesg rado). chemistry teachers of first-year engineering degrees of upm have created a platform called "punto de inicio" in the virtual campus (moodle), only for students enrolled on their first time in the subject (https://moodle.upm.es/puntodeinicio/niv/login.php) and a public one called open course ware (http://ocw.upm.es/apoyo-para-la-preparacion-de multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 71 los-estudios-de-ingenieria-y-arquitectura/quimica-preparacion-para-la-universidad) for the retaken students in order to solve the lack of chemistry knowledge in the previous years. the "punto de inicio" platform was created in the upm to help students in basic subjects like: mathematics, physics, chemistry, english and technical drawing, in order to inform the student what should be the minimum knowledge needed to follow that subjects. in this platform questions of different levels of difficulty and with multiple choice options are proposed. all of them are offered in moodle and are free for all students enrolled in the first course. students can access from july after their enrolment in the university. (figure 1). figure 1. image of the “punto de inicio” plataform. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 72 our school grade degrees have been introduced in 2010/2011 academic year and since the first moment the student has been informed about the existence of this platform, inviting him to use it. however, unfortunately, it has not got much participation. currently the e.u.i.t.i offers five degrees: electrical engineering, industrial electrical engineering and automation, industrial design engineering and product development, mechanical engineering and chemical engineering. in its five undergraduate degrees, first-year chemistry teachers have agreed to develop an initial diagnostic test of chemistry of the "punto de inicio" platform. this test was common to all students and performed in their first day of class in september. in this paper the results achieved by the degree in industrial design engineering and product development are presented. 2. methodology among the five degrees taught in euiti, we have chosen the students of the degree in industrial design engineering and product development. these students enter in our school with the highest cut-off mark in the pau (table 1), however, as they mostly did not take chemistry on secondary school they generally have got less knowledge on it (http://innovacioneducativa.upm.es/observatorio/ficha-estudio/informe-demanda-upm2011-12). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 73 table 1. scores required to enter the e.u.i.t.i’s degree programs. (course 2012/13) grade in score electrical engineering 8.378 industrial electronics and automation engineering 9.733 industrial design engineering and product development 10.664 mechanical engineering 10.356 chemical engineering 7.907 to develop this experience, the "punto de inicio" platform has been used as a starting point. when the student enters in "punto de inicio", he can carry out some questionnaires, both to review and to know their level of knowledge. the questions proposed in the virtual campus (moodle) tell the student what should be the minimum level of knowledge they must have in chemistry in order to take the degree subject (figure 2a). it is interesting to remark that the issues proposed in "punto de inicio" are classified into three levels depending on the grade of difficulty. each question has four possible answers to choose, obviously the response time is restricted once the quiz starts, as it is shown in figure 2b for the theme of atomic and molecular structure. the student can pre-select the level of the test he wants to do and he can also make as many quizzes as he wants as the questions are randomized and the possibility of doing it again is minimal. nevertheless, as the application in moodle is little demanded by students, teachers of first-year course in chemistry of the five degrees taught at euiti (upm) agreed to develop a diagnostic test including questions of the "punto de inicio” where wrongness multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 74 do not subtract points in the final marks. the idea of use these questions is that those students who had used the chemistry platform feel encouraged and motivated to see that their work was being rewarded and, on the other hand students who had not used the application will be feel invited to use it. figure 2. a) image of the test of the theme of atomic and molecular structure. b) model of questionnaire. therefore, since the first day of class, after giving them the rules of the course, explaining the schedule with instructional design and the planning of learning activities, the students made the diagnostic test of previous knowledge of 20 questions. this test was performed anonymously in the classroom. an active methodology was applied along the course, consisting of: cooperative actions (each two weeks group works tutored by the teacher) (barajas 2008), solving and delivery problems (individual problems which are solved in the classroom by the teacher or by a student), individual preparation of two simple themes closely related to multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 75 the initial summary, in these themes a number of very important questions are included to focus the student on the relevant items, fixing the day to correct them in the class. as chemistry is an experimental subject, students attend to laboratory every two weeks, trying to apply theory to practice. likewise class settled test and exams are performed, to establish their knowledge and to set goals along the course goals along the course. the final exam, common to all students in the five bachelor degrees, takes place in january. 3. results the number of students in industrial design engineering and product development, who performed the diagnostic test were 76 students. the test results were not very satisfactory, as the 85.5% of the students who made it do not pass the test. the highest average mark obtained by any student in the class was 8.21 (figure 3). the test results were reported in a few days letting them know the basic knowledge they should have to follow the course. therefore, in addition, in the first week of the course students were given a sheet with the minimal knowledge that they should know from their previous secondary school. students should prepare a short work of these contents in order to revise them and thus be aware of what they need to know to start and carry on with the subjects, for example: periodic table of the elements, formulation, stoichiometry, concentration units, etc. these contents would not be developed in the classroom but if they have doubts they could assist to tuitions. we have found that it has been very helpful to make students know their lacks in chemistry. chemistry is based on symbols, formulas and terminology, if anyone is not multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 76 aware of these things from its beginning, little or nothing can be done to understand and learn the subject. unfortunately, in our classrooms we have got a lot of students with a great unknown in chemistry. figure 3. number of correct answers in the diagnostic test the above mentioned activities were scheduled and the time spent on each of them was established based on the number of ects credits according to the subject academic guidance. in table 2, the credits dedicated to the different activities in chemistry are shown and also compared with other subjects in the same semester of degree in industrial design engineering and product development (http://www.euiti.upm.es/euitindustrial/estudiantes/estudiostitulaciones/ettitulosgr ado/ettitulosoficialesgrado/gradingdisind/a9c9d51cf6069210vgnvcm10000009c7 648arcrd). 0% 2% 4% 6% 8% 10% 12% 14% 16% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 % s tu d e n ts mark diagnostic test 2012-2013 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 77 table 2. number off credits dedicated in each basis subject of the first semester calculus linear algebra physics i chemistry technical drawing and computer aid design theory 1.2 1.2 1.2 1.2 0.6 problems 1.2 1.2 1.2 1.2 1.2 laboratory 0.6 0.6 0.6 0.6 1.2 personal study 2.8 2.8 2.8 2.8 2.8 exams 0.2 0.2 0.2 0.2 0.2 total credits 6 6 6 6 6 after the implementation of the plan of studies, teachers that tutors the different subjects have been requested to check the credit number of it. finally, it was established, that it was very similar to the initially proposed number of credits (table 2) [10]. in order to confirm the previous data of table 2, students have been pollen about the ects measurements. for this purpose, a survey was done, asking students the number of credits that had been used to study each subject making the different activities of it in the first semester (see table 3). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 78 table 3. number of ects credits of student work, in his opinion, for each subject of the first semester. ects calculus linear algebra physics i chemistry technical drawing and computer aid design total credits according to plan of study 6 6 6 6 6 30 according to students 5.9 5.8 7.0 6.8 6.3 31.8 comparing table 2 and 3 shows that ects applied to each subject and work for their development are well dimensioned. the discrepancy observed in the chemistry subject could be due to the low level of knowledge of the students. this fact implies that students should make a greater effort in chemistry than in other subjects to achieve the required level at the course effort to achieve the required level at the course. the total number of students enrolled in the subject was 106, table 4. it is interesting to remark that not all of them have followed the ongoing assessment, because as it is established in the course’s teaching guide (http://www.euiti.upm.es/euitindustrial/estudiantes/estudiostitulaciones/ettitulosgr ado/ettitulosoficialesgrado/gradingdisind/a9c9d51cf6069210vgnvcm10000009c7 648arcrd) students can chose exclusively for a final exam in january to pass the subject. however, 96.8% of the students who attended the final exam have followed the ongoing assessment. the data demonstrate low absenteeism rate obtained in the current multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 79 academic year (7.55 %) (table 5), it is interesting to observe that only 2.8% of those following the ongoing assessment give up. likewise, better results have been achieved over the previous year as it is reflected in the efficiency rate (42.45 %) and the success rate (45.92 %) (table 6). both rates are improved if we compare the results achieved by students enrolled in the course by first time, which are those that mostly follow ongoing assessment, reaching a success rate of 50% on first examination. most of the students belonging to this group are those who performed the diagnostic test at the beginning of the course, since students that fail chemistry the year before and repeat it incorporated to the class later. table 4. students of chemistry of the degree in industrial design engineering and product development enrolled not presented presented faillers passed continuos assesssment 94 3 91 47 44 give up ong. ass. 12 5 7 6 1 total 106 8 98 53 45 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 80 table 5. percentage ratio between the number of absent students and the number of students enrolled in the course nº enrolled students nº absent students absenteeism rate once 64 5 7.81 twice 31 1 3.23 three or more times 11 2 18.18 total 106 8 7.55 the fact of reaching the 50% in the rate success students that study this subject by the first time and remembering that only 14.5% of them pass the diagnostic test pleases and encourages us to continue working in this direction for the future academic years. table 6. percentage ratio between the number of passed students and the number of students presented on their first time and resitted. nº of attemps presented passed success rate once 60 30 50.00 twice 24 10 41.67 three times 9 2 22.22 four times 5 3 60.00 total 98 45 45.92 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 81 4. conclusions the results achieved during the development of this course are very encouraging. the purpose of this project is that a student at the beginning of the course be aware of his knowledge level and provide him appropriate tools to learn forcing him to study more deeply the concepts that he did not understand. in single words, he will make a better distribution on his study time and work. this also allowed teachers to have a better planning of student workload emphasizing the non-clear concepts. furthermore, the calculation of ects credits from the students fits well with the one proposed by the teachers, although students’ appreciation were slightly higher. despite the students’ low level of knowledge in chemistry shown in the diagnostic test, the workload of the student was well sized and did not suppose them an enormous extra work. obviously those students with lower level have had to study harder. acknowledgements this work has been supported by the universidad politécnica de madrid in the frame of the education innovation project nº ie12_13-56002. references albéniz, j. saavedra, p. barajas, r. carrillo, i. (2008) actas de incece’08 ii jornadas internacionales upm sobre innovación educativa y convergencia europea 2008, supuesto de consumo de profesores para propuesta de nuevo plan de estudios y estimación de los ects que deberá cumplir el profesorado a tiempo completo, madrid. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2193 social and technological sciences barajas et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 69-82 | 82 barajas, r. carrillo, i. albéniz, j. reinoso, c. saavedra, p. (2008) actas de 16 congreso universitario de innovación educativa en las enseñanzas técnicas (16º cuieet), ¿cómo inducir a los alumnos al estudio a través de una acción cooperativa?, cádiz. orden esd/1729/2008, de 11 de junio, por la que se regula la ordenación y se establece el currículo del bachillerato. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 97 designing efficient material handling systems via automated guided vehicles (agvs) carlos llopis-albert1, francisco rubio1, francisco valero1 1centro de investigación en ingeniería mecánica (ciim). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding autors: carlos llopis-albert: cllopisa@upvnet.upv.es francisco rubio: frubio@mcm.upv.es francisco valero: fvalero@mcm.upv.es received: 2018-04-08; accepted: 2018-08-15 abstract the designing of an efficient warehouse management system is a key factor to improve productivity and reduce costs. the use of automated guided vehicles (avgs) in material handling systems (mhs) and flexible manufacturing systems (fms) can help to that purpose. this paper is intended to provide insight regarding the technical and financial suitability of the implementation of a fleet of agvs. this is carried out by means of a fuzzy set/qualitative comparative analysis (fsqca) by measuring the level of satisfaction of managerial decision makers. keywords fuzzy sets, qualitative comparative analysis, autonomous guided vehicles, conflict resolution, decision-making, material handling systems, flexible manufacturing systems. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 98 1. introduction logistic processes are a key factor in the industrial performance (sarker and gurav, 2005), which can be improved with the use of automated guided vehicles (agvs). they are programmable and driverless vehicles used in industrial applications by means of a communication and navigation system to carry materials around a manufacturing facility (pillac et al., 2013). the benefits of its use span from more efficient warehousing systems and inventory control, lower labor costs, increased safety and production, to more flexible manufacturing systems (fms). the latter is because they can easily deal with changes in both products demand and labor force (fazlollahtabar et al., 2015). the design of an efficient agv network system should also take into account the dimensions of the aisles, stacking areas, workstations, and fixed structures on the facility layout. there are several ways of navigation and guidance technologies of agvs, which range from physical guide-paths using laser, magnetic tape, optical sensors, wire, to gyroscope based inertial guidance and wireless. the latter provides the advantage that can be easily modified. in addition, the agvs can connect different points of the facility using unidirectional, bidirectional, multi-lane and mixed systems. there are many traffic control algorithms to tackle the routing and scheduling problems, which cover static and dynamic approaches (biçer and seifert, 2017). in static approaches the route is known in advance, thus hampering the adaptation to changes in both traffic conditions and logistic system. in dynamic approaches the route uses real-time information (for instance, with regard traffic conditions and obstacles) to calculate the route, thus providing calable mhs and fms (gourgand et al., 1995). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 99 2. methodology this paper designs an efficient agv fleet using the fuzzy set/qualitative comparative analysis (fsqca) (ragin, 2008). the methodology considers many factors or conditions, covering management, financial and technical ones. the factors or antecedent conditions used in the fsqca are presented in table 1. the aim is to determine which configuration of conditions leads to the actors’ satisfaction, which cover shareholders, managers, labor unions and workers. the fuzzy set/qualitative comparative analysis, uses boolean algebra and fuzzy logic to develop principles of comparison in the qualitative studies of social and economic phenomena (ragin, 2008; berbegal-mirabent and llopis-albert, 2015; llopis-albert and palacios-marqués 2016; llopis-albert et al., 2017). an exhausted explanation of the methodology can be found in mendel and korjani 2012, 2013. the methodology is intended to find which combination of conditions (configuration) are minimally necessary and/or sufficient for obtaining a certain outcome (meyer et al., 1993). the methodology is applied to small and medium-sized enterprises (smes) of several industrial sectors in spain based on a wide range of information such as reports, surveys and expert judgment. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 100 table 1. factors or antecedent conditions used in the fsqca. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 101 the actors’ preference degree regarding the conditions is determined using a continuous fuzzy set, ranging from 0 (for low degree of acceptance) to 1 (for high degree of acceptance). these values are obtained based on the available information and after a calibration process. this allows to define the truth table, which is subsequently analyzed by means of the fsqca software (ragin, 2008). because there are 13 factors, so that the matrix dimensions are (213) rows and 13 columns, which lead to 8192 possible configurations. table 2 shows the necessary conditions for the actors’ satisfaction and also for the negation of those factors, which is marked with the tilde (~) sign. necessary conditions have a consistency score that exceeds the threshold of 0.9 (schneider et al., 2010). only 2 (c2 and c4) out of 13 conditions show a consistency above the threshold. however, there are other existing conditions in most of the configurations (e.g., annual savings). this shows that the outcome (stakeholders’ satisfaction) depends on a several conditions because of the high heterogeneity of stakeholders with conflicting interests. another step is to reduce the number of rows in the matrix, which is performed using the quine–mccluskey algorithm (quine, 1952). in this way the matrix is minimized by means of boolean algebra to obtain a set of combinations of causal conditions, where each of them is minimally sufficient to lead to the outcome. the minimization step is performed through the coverage and consistency values. table 3 depicts the 13 solutions that are found by the algorithm, in which black circles () indicate the presence of a condition, white circles (�) denote its absence, and blank cells represent ambiguous factors (ragin and fiss, 2008). the variety of these configurations suggests that they are sufficient but not necessary. hence, any configuration explains by itself the actors’ satisfaction. additionally, all configurations present acceptable consistency values (<0.80) and raw coverage values. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 102 table 2. analysis of necessary conditions and consistency and coverage values for all conditions table 3. sufficient configurations of factors for stakeholders’ satisfaction multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 103 4. conclusions nowadays, the use agvs are becomingly more common in the manufacturing industries for transport and warehousing purposes. this paper presents a powerful technique, based on the fsqca, to design efficient warehouse management system and improve productivity and reduce costs. the method has been successfully applied to a real case study of small and medium-sized enterprises (smes) of several industrial sectors in spain. it can help the decision-making process when designing and implementing a network flow of agvs systems. results show that actors are more interested in management and financial conditions than in technical ones. nevertheless, shareholders and managers pay more attention to the achievement of profits, while labor unions and workers are more prone to keep their jobs and concerned about the technical factors that can hinder the mhs and in safety issues. the unique necessary conditions are the improvement of productivity and performance and the attainment of fms, since they presumably favor all stakeholders. the methodology increases the business performance and leads to flexible manufacturing systems that can quickly reconfigurable due to production changes. moreover, it provides more agreement among stakeholders, thus reducing possible delays in the implementation of the agvs system because of work union strikes. references berbegal-mirabent, j., llopis-albert, c. (2015). applications of fuzzy logic for determining the driving forces in collaborative research contracts. j. bus. res., 69 (4), 1446–1451. doi: 10.1016/j.jbusres.2015.10.123. biçer, i., seifert, r.w., (2017). optimal dynamic order scheduling under capacity constraints given demand-forecast evolution. production and operations management 26(12), 2266–2286. doi: 10.1111/poms.12759. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 104 fazlollahtabar, h., saidi-mehrabad, m. (2015). autonomous guided vehicles: methods and models for optimal path planning. springer. doi: 10.1007/978-3-319-14747-5. isbn 978-3-319-14747-5. gourgand, m., sun, x.c., tchernev, n., 1995. choice of the guide path layout for an agv based material handling system. emerging technologies and factory automation, 1995. etfa '95, proceedings. 1995 inria/ieee symposium on. doi: 10.1109/etfa.1995.496688. llopis-albert, c., rubio, f., valero, f. (2015). improving productivity using a multiobjective optimization of robotic trajectory planning. journal of business research 68, 1429–1431. doi: 10.1016/j.jbusres.2015.01.027. llopis-albert, c., palacios-marqués, d. (2016). applied mathematical problems in engineering. multidisciplinary journal for education 3(2), 1-14. doi: 10.4995/muse.2016.6679. llopis-albert, c., merigó, j.m., xu, y., liao, h. (2017). application of fuzzy set/qualitative comparative analysis to public participation projects in support of the eu water framework directive. water environment research, 89. doi: 10.2175/werd1600372. mendel, j. m.; korjani, m. m. (2012). charles ragin’s fuzzy set qualitative comparative analysis (fsqca) used for linguistic summarizations. inf. sci., 202, 1–23. mendel, j. m., korjani, m. m. (2013). theoretical aspects of fuzzy set qualitative comparative analysis (fsqca). inf. sci., 237, 137–161. meyer, a. d., tsui, a.s. and hinings, c.r. (1993). configurational approaches to organizational analysis. academy of management journal, 36(6), 1175–1195. quine, w. v. (1952). the problem of simplifying truth functions. the american mathematical monthly, 59(8), 521–531. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10722 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 97-105 | 105 pillac, v., gendreau, m., guéret, c., medaglia, a.l. (2013). a review of dynamic vehicle routing problems. european journal of operational research 225, 1-11. doi: 10.1016/j.ejor.2012.08.015. ragin, c. c. (2008). redesigning social inquiry: fuzzy sets and beyond. chicago: university of chicago press. sarker, b.r., gurav, s.s. (2005). route planning for automated guided vehicles in a manufacturing facility. international journal of production research 43(21), 4659–4683. << /ascii85encodepages false /allowtransparency false /autopositionepsfiles true /autorotatepages /none /binding /left /calgrayprofile (dot gain 20%) /calrgbprofile (srgb iec61966-2.1) /calcmykprofile (u.s. web coated \050swop\051 v2) /srgbprofile (srgb iec61966-2.1) /cannotembedfontpolicy /error /compatibilitylevel 1.4 /compressobjects /tags /compresspages true /convertimagestoindexed true /passthroughjpegimages true /createjobticket false /defaultrenderingintent /default /detectblends true /detectcurves 0.0000 /colorconversionstrategy /cmyk /dothumbnails false /embedallfonts true /embedopentype false /parseiccprofilesincomments true /embedjoboptions true /dscreportinglevel 0 /emitdscwarnings false /endpage -1 /imagememory 1048576 /lockdistillerparams false /maxsubsetpct 100 /optimize true /opm 1 /parsedsccomments true /parsedsccommentsfordocinfo true /preservecopypage true /preservedicmykvalues true /preserveepsinfo true /preserveflatness true /preservehalftoneinfo false /preserveopicomments true /preserveoverprintsettings true /startpage 1 /subsetfonts true /transferfunctioninfo /apply /ucrandbginfo /preserve /useprologue false /colorsettingsfile () /alwaysembed [ true ] /neverembed [ true ] /antialiascolorimages false /cropcolorimages true /colorimageminresolution 300 /colorimageminresolutionpolicy /ok /downsamplecolorimages true /colorimagedownsampletype /bicubic /colorimageresolution 300 /colorimagedepth -1 /colorimagemindownsampledepth 1 /colorimagedownsamplethreshold 1.50000 /encodecolorimages true /colorimagefilter /dctencode /autofiltercolorimages true /colorimageautofilterstrategy /jpeg /coloracsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /colorimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000coloracsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000colorimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasgrayimages false /cropgrayimages true /grayimageminresolution 300 /grayimageminresolutionpolicy /ok /downsamplegrayimages true /grayimagedownsampletype /bicubic /grayimageresolution 300 /grayimagedepth -1 /grayimagemindownsampledepth 2 /grayimagedownsamplethreshold 1.50000 /encodegrayimages true /grayimagefilter /dctencode /autofiltergrayimages true /grayimageautofilterstrategy /jpeg /grayacsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /grayimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000grayacsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000grayimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasmonoimages false /cropmonoimages true /monoimageminresolution 1200 /monoimageminresolutionpolicy /ok /downsamplemonoimages true /monoimagedownsampletype /bicubic /monoimageresolution 1200 /monoimagedepth -1 /monoimagedownsamplethreshold 1.50000 /encodemonoimages true /monoimagefilter /ccittfaxencode /monoimagedict << /k -1 >> /allowpsxobjects false /checkcompliance [ /none ] /pdfx1acheck false /pdfx3check false /pdfxcompliantpdfonly false /pdfxnotrimboxerror true /pdfxtrimboxtomediaboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxsetbleedboxtomediabox true /pdfxbleedboxtotrimboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxoutputintentprofile () /pdfxoutputconditionidentifier () /pdfxoutputcondition () /pdfxregistryname () /pdfxtrapped /false /createjdffile false /description << /ara <feff06270633062a062e062f0645002006470630064700200627064406250639062f0627062f0627062a002006440625064606340627062100200648062b062706260642002000410064006f00620065002000500044004600200645062a064806270641064206290020064406440637062806270639062900200641064a00200627064406450637062706280639002006300627062a0020062f0631062c0627062a002006270644062c0648062f0629002006270644063906270644064a0629061b0020064a06450643064600200641062a062d00200648062b0627062606420020005000440046002006270644064506460634062306290020062806270633062a062e062f062706450020004100630072006f0062006100740020064800410064006f006200650020005200650061006400650072002006250635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e0635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e> /bgr <feff04180437043f043e043b043704320430043904420435002004420435043704380020043d0430044104420440043e0439043a0438002c00200437043000200434043000200441044a0437043404300432043004420435002000410064006f00620065002000500044004600200434043e043a0443043c0435043d04420438002c0020043c0430043a04410438043c0430043b043d043e0020043f044004380433043e04340435043d04380020043704300020043204380441043e043a043e043a0430044704350441044204320435043d0020043f04350447043004420020043704300020043f044004350434043f0435044704300442043d04300020043f043e04340433043e0442043e0432043a0430002e002000200421044a04370434043004340435043d043804420435002000500044004600200434043e043a0443043c0435043d044204380020043c043e0433043004420020043404300020044104350020043e0442043204300440044f0442002004410020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200441043b0435043404320430044904380020043204350440044104380438002e> /chs <feff4f7f75288fd94e9b8bbe5b9a521b5efa7684002000410064006f006200650020005000440046002065876863900275284e8e9ad88d2891cf76845370524d53705237300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /cht <feff4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef69069752865bc9ad854c18cea76845370524d5370523786557406300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /cze <feff005400610074006f0020006e006100730074006100760065006e00ed00200070006f0075017e0069006a007400650020006b0020007600790074007600e101590065006e00ed00200064006f006b0075006d0065006e0074016f002000410064006f006200650020005000440046002c0020006b00740065007200e90020007300650020006e0065006a006c00e90070006500200068006f006400ed002000700072006f0020006b00760061006c00690074006e00ed0020007400690073006b00200061002000700072006500700072006500730073002e002000200056007900740076006f01590065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f007400650076015900ed007400200076002000700072006f006700720061006d0065006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076011b006a016100ed00630068002e> /dan <feff004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000620065006400730074002000650067006e006500720020007300690067002000740069006c002000700072006500700072006500730073002d007500640073006b007200690076006e0069006e00670020006100660020006800f8006a0020006b00760061006c0069007400650074002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /deu <feff00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f006300680077006500720074006900670065002000500072006500700072006500730073002d0044007200750063006b0065002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /esp <feff005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f00730020005000440046002000640065002000410064006f0062006500200061006400650063007500610064006f00730020007000610072006100200069006d0070007200650073006900f3006e0020007000720065002d0065006400690074006f007200690061006c00200064006500200061006c00740061002000630061006c0069006400610064002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /eti <feff004b00610073007500740061006700650020006e0065006900640020007300e4007400740065006900640020006b00760061006c006900740065006500740073006500200074007200fc006b006900650065006c007300650020007000720069006e00740069006d0069007300650020006a0061006f006b007300200073006f00620069006c0069006b0065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740069006400650020006c006f006f006d006900730065006b0073002e00200020004c006f006f0064007500640020005000440046002d0064006f006b0075006d0065006e00740065002000730061006100740065002000610076006100640061002000700072006f006700720061006d006d006900640065006700610020004100630072006f0062006100740020006e0069006e0067002000410064006f00620065002000520065006100640065007200200035002e00300020006a00610020007500750065006d006100740065002000760065007200730069006f006f006e00690064006500670061002e000d000a> /fra <feff005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f0075007200200075006e00650020007100750061006c0069007400e90020006400270069006d007000720065007300730069006f006e00200070007200e9007000720065007300730065002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /gre <feff03a703c103b703c303b903bc03bf03c003bf03b903ae03c303c403b5002003b103c503c403ad03c2002003c403b903c2002003c103c503b803bc03af03c303b503b903c2002003b303b903b1002003bd03b1002003b403b703bc03b903bf03c503c103b303ae03c303b503c403b5002003ad03b303b303c103b103c603b1002000410064006f006200650020005000440046002003c003bf03c5002003b503af03bd03b103b9002003ba03b103c42019002003b503be03bf03c703ae03bd002003ba03b103c403ac03bb03bb03b703bb03b1002003b303b903b1002003c003c103bf002d03b503ba03c403c503c003c903c403b903ba03ad03c2002003b503c103b303b103c303af03b503c2002003c503c803b703bb03ae03c2002003c003bf03b903cc03c403b703c403b103c2002e0020002003a403b10020005000440046002003ad03b303b303c103b103c603b1002003c003bf03c5002003ad03c703b503c403b5002003b403b703bc03b903bf03c503c103b303ae03c303b503b9002003bc03c003bf03c103bf03cd03bd002003bd03b1002003b103bd03bf03b903c703c403bf03cd03bd002003bc03b5002003c403bf0020004100630072006f006200610074002c002003c403bf002000410064006f00620065002000520065006100640065007200200035002e0030002003ba03b103b9002003bc03b503c403b103b303b503bd03ad03c303c403b503c103b503c2002003b503ba03b403cc03c303b503b903c2002e> /heb <feff05d405e905ea05de05e905d5002005d105d405d205d305e805d505ea002005d005dc05d4002005db05d305d9002005dc05d905e605d505e8002005de05e105de05db05d9002000410064006f006200650020005000440046002005d405de05d505ea05d005de05d905dd002005dc05d405d305e405e105ea002005e705d305dd002d05d305e405d505e1002005d005d905db05d505ea05d905ea002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e05d005de05d905dd002005dc002d005000440046002f0058002d0033002c002005e205d905d905e005d5002005d105de05d305e805d905da002005dc05de05e905ea05de05e9002005e905dc0020004100630072006f006200610074002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e> /hrv (za stvaranje adobe pdf dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. stvoreni pdf dokumenti mogu se otvoriti acrobat i adobe reader 5.0 i kasnijim verzijama.) /hun <feff004b0069007600e1006c00f30020006d0069006e0151007300e9006701710020006e0079006f006d00640061006900200065006c0151006b00e90073007a00ed007401510020006e0079006f006d00740061007400e100730068006f007a0020006c006500670069006e006b00e1006200620020006d0065006700660065006c0065006c0151002000410064006f00620065002000500044004600200064006f006b0075006d0065006e00740075006d006f006b0061007400200065007a0065006b006b0065006c0020006100200062006500e1006c006c00ed007400e10073006f006b006b0061006c0020006b00e90073007a00ed0074006800650074002e0020002000410020006c00e90074007200650068006f007a006f00740074002000500044004600200064006f006b0075006d0065006e00740075006d006f006b00200061007a0020004100630072006f006200610074002000e9007300200061007a002000410064006f00620065002000520065006100640065007200200035002e0030002c0020007600610067007900200061007a002000610074007400f3006c0020006b00e9007301510062006200690020007600650072007a006900f3006b006b0061006c0020006e00790069007400680061007400f3006b0020006d00650067002e> /ita <feff005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f00620065002000500044004600200070006900f900200061006400610074007400690020006100200075006e00610020007000720065007300740061006d0070006100200064006900200061006c007400610020007100750061006c0069007400e0002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /jpn <feff9ad854c18cea306a30d730ea30d730ec30b951fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e305930023053306e8a2d5b9a306b306f30d530a930f330c8306e57cb30818fbc307f304c5fc59808306730593002> /kor <feffc7740020c124c815c7440020c0acc6a9d558c5ec0020ace0d488c9c80020c2dcd5d80020c778c1c4c5d00020ac00c7a50020c801d569d55c002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /lth <feff004e006100750064006f006b0069007400650020016100690075006f007300200070006100720061006d006500740072007500730020006e006f0072011700640061006d00690020006b0075007200740069002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b00750072006900650020006c0061006200690061007500730069006100690020007000720069007400610069006b007900740069002000610075006b01610074006f00730020006b006f006b007900620117007300200070006100720065006e006700740069006e00690061006d00200073007000610075007300640069006e0069006d00750069002e0020002000530075006b0075007200740069002000500044004600200064006f006b0075006d0065006e007400610069002000670061006c006900200062016b007400690020006100740069006400610072006f006d00690020004100630072006f006200610074002000690072002000410064006f00620065002000520065006100640065007200200035002e0030002000610072002000760117006c00650073006e0117006d00690073002000760065007200730069006a006f006d00690073002e> /lvi <feff0049007a006d0061006e0074006f006a00690065007400200161006f00730020006900650073007400610074012b006a0075006d00750073002c0020006c0061006900200076006500690064006f00740075002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006100730020006900720020012b00700061016100690020007000690065006d01130072006f00740069002000610075006700730074006100730020006b00760061006c0069007401010074006500730020007000690072006d007300690065007300700069006501610061006e006100730020006400720075006b00610069002e00200049007a0076006500690064006f006a006900650074002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006f002000760061007200200061007400760113007200740020006100720020004100630072006f00620061007400200075006e002000410064006f00620065002000520065006100640065007200200035002e0030002c0020006b0101002000610072012b00200074006f0020006a00610075006e0101006b0101006d002000760065007200730069006a0101006d002e> /nld (gebruik deze instellingen om adobe pdf-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. de gemaakte pdf-documenten kunnen worden geopend met acrobat en adobe reader 5.0 en hoger.) /nor <feff004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d00200065007200200062006500730074002000650067006e0065007400200066006f00720020006600f80072007400720079006b006b0073007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /pol <feff0055007300740061007700690065006e0069006100200064006f002000740077006f0072007a0065006e0069006100200064006f006b0075006d0065006e007400f300770020005000440046002000700072007a0065007a006e00610063007a006f006e00790063006800200064006f002000770079006400720075006b00f30077002000770020007700790073006f006b00690065006a0020006a0061006b006f015b00630069002e002000200044006f006b0075006d0065006e0074007900200050004400460020006d006f017c006e00610020006f007400770069006500720061010700200077002000700072006f006700720061006d006900650020004100630072006f00620061007400200069002000410064006f00620065002000520065006100640065007200200035002e0030002000690020006e006f00770073007a0079006d002e> /ptb <feff005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020006d00610069007300200061006400650071007500610064006f00730020007000610072006100200070007200e9002d0069006d0070007200650073007300f50065007300200064006500200061006c007400610020007100750061006c00690064006100640065002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /rum <feff005500740069006c0069007a00610163006900200061006300650073007400650020007300650074010300720069002000700065006e007400720075002000610020006300720065006100200064006f00630075006d0065006e00740065002000410064006f006200650020005000440046002000610064006500630076006100740065002000700065006e0074007200750020007400690070010300720069007200650061002000700072006500700072006500730073002000640065002000630061006c006900740061007400650020007300750070006500720069006f006100720103002e002000200044006f00630075006d0065006e00740065006c00650020005000440046002000630072006500610074006500200070006f00740020006600690020006400650073006300680069007300650020006300750020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e00300020015f00690020007600650072007300690075006e0069006c006500200075006c0074006500720069006f006100720065002e> /rus <feff04180441043f043e043b044c04370443043904420435002004340430043d043d044b04350020043d0430044104420440043e0439043a043800200434043b044f00200441043e043704340430043d0438044f00200434043e043a0443043c0435043d0442043e0432002000410064006f006200650020005000440046002c0020043c0430043a04410438043c0430043b044c043d043e0020043f043e04340445043e0434044f04490438044500200434043b044f00200432044b0441043e043a043e043a0430044704350441044204320435043d043d043e0433043e00200434043e043f0435044704300442043d043e0433043e00200432044b0432043e04340430002e002000200421043e043704340430043d043d044b04350020005000440046002d0434043e043a0443043c0435043d0442044b0020043c043e0436043d043e0020043e0442043a0440044b043204300442044c002004410020043f043e043c043e0449044c044e0020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200431043e043b043504350020043f043e04370434043d043804450020043204350440044104380439002e> /sky <feff0054006900650074006f0020006e006100730074006100760065006e0069006100200070006f0075017e0069007400650020006e00610020007600790074007600e100720061006e0069006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b0074006f007200e90020007300610020006e0061006a006c0065007001610069006500200068006f0064006900610020006e00610020006b00760061006c00690074006e00fa00200074006c0061010d00200061002000700072006500700072006500730073002e00200056007900740076006f00720065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f00740076006f00720069016500200076002000700072006f006700720061006d006f006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076016100ed00630068002e> /slv <feff005400650020006e006100730074006100760069007400760065002000750070006f0072006100620069007400650020007a00610020007500730074007600610072006a0061006e006a006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b006900200073006f0020006e0061006a007000720069006d00650072006e0065006a016100690020007a00610020006b0061006b006f0076006f00730074006e006f0020007400690073006b0061006e006a00650020007300200070007200690070007200610076006f0020006e00610020007400690073006b002e00200020005500730074007600610072006a0065006e006500200064006f006b0075006d0065006e0074006500200050004400460020006a00650020006d006f0067006f010d00650020006f0064007000720065007400690020007a0020004100630072006f00620061007400200069006e002000410064006f00620065002000520065006100640065007200200035002e003000200069006e0020006e006f00760065006a01610069006d002e> /suo <feff004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f00740020006c00e400680069006e006e00e4002000760061006100740069007600610061006e0020007000610069006e006100740075006b00730065006e002000760061006c006d0069007300740065006c00750074007900f6006800f6006e00200073006f00700069007600690061002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a0061002e0020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /sve <feff0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400200073006f006d002000e400720020006c00e4006d0070006c0069006700610020006600f60072002000700072006500700072006500730073002d007500740073006b00720069006600740020006d006500640020006800f600670020006b00760061006c0069007400650074002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /tur <feff005900fc006b00730065006b0020006b0061006c006900740065006c0069002000f6006e002000790061007a006401310072006d00610020006200610073006b013100730131006e006100200065006e0020006900790069002000750079006100620069006c006500630065006b002000410064006f006200650020005000440046002000620065006c00670065006c0065007200690020006f006c0075015f007400750072006d0061006b0020006900e70069006e00200062007500200061007900610072006c0061007201310020006b0075006c006c0061006e0131006e002e00200020004f006c0075015f0074007500720075006c0061006e0020005000440046002000620065006c00670065006c0065007200690020004100630072006f006200610074002000760065002000410064006f00620065002000520065006100640065007200200035002e003000200076006500200073006f006e0072006100730131006e00640061006b00690020007300fc007200fc006d006c00650072006c00650020006100e70131006c006100620069006c00690072002e> /ukr <feff04120438043a043e0440043804410442043e043204430439044204350020044604560020043f043004400430043c043504420440043800200434043b044f0020044104420432043e04400435043d043d044f00200434043e043a0443043c0435043d044204560432002000410064006f006200650020005000440046002c0020044f043a04560020043d04300439043a04400430044904350020043f045604340445043e0434044f0442044c00200434043b044f0020043204380441043e043a043e044f043a04560441043d043e0433043e0020043f0435044004350434043404400443043a043e0432043e0433043e0020043404400443043a0443002e00200020042104420432043e04400435043d045600200434043e043a0443043c0435043d0442043800200050004400460020043c043e0436043d04300020043204560434043a0440043804420438002004430020004100630072006f006200610074002004420430002000410064006f00620065002000520065006100640065007200200035002e0030002004300431043e0020043f04560437043d04560448043e04570020043204350440044104560457002e> /enu (use these settings to create adobe pdf documents best suited for high-quality prepress printing. created pdf documents can be opened with acrobat and adobe reader 5.0 and later.) >> /namespace [ (adobe) (common) (1.0) ] /othernamespaces [ << /asreaderspreads false /cropimagestoframes true /errorcontrol /warnandcontinue /flattenerignorespreadoverrides false /includeguidesgrids false /includenonprinting false /includeslug false /namespace [ (adobe) (indesign) (4.0) ] /omitplacedbitmaps false /omitplacedeps false /omitplacedpdf false /simulateoverprint /legacy >> << /addbleedmarks false /addcolorbars false /addcropmarks false /addpageinfo false /addregmarks false /convertcolors /converttocmyk /destinationprofilename () /destinationprofileselector /documentcmyk /downsample16bitimages true /flattenerpreset << /presetselector /mediumresolution >> /formelements false /generatestructure false /includebookmarks false /includehyperlinks false /includeinteractive false /includelayers false /includeprofiles false /multimediahandling /useobjectsettings /namespace [ (adobe) (creativesuite) (2.0) ] /pdfxoutputintentprofileselector /documentcmyk /preserveediting true /untaggedcmykhandling /leaveuntagged /untaggedrgbhandling /usedocumentprofile /usedocumentbleed false >> ] >> setdistillerparams << /hwresolution [2400 2400] /pagesize [612.000 792.000] >> setpagedevice multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 121 use of new methodologies for students assessment in large groups in engineering education b. tormos, h. climent *, p. olmeda, f. arnau cmt-motores térmicos, universitat politècnica de valència, camino de vera s/n, 46022, valencia, españa, tel. 963 877 650, fax 963 877 659, hcliment@mot.upv.es* received: 2013-12-15; accepted: 2014-01-28 abstract in this paper, a student evaluation methodology which applies the concept of continuous assessment proposed by bologna is presented for new degrees in higher education. an important part of the student's final grade is based on the performance of several individual works throughout the semester. the paper shows the correction system used which is based on using a spreadsheet with macros and a template in which the student provides the solution of each task. the employ of this correction system together with the available e-learning platform allows the teachers to perform automatic tasks evaluations compatible with courses with large number of students. the paper also raises the different solutions adopted to avoid plagiarism and to try that the final grade reflects, as closely as possible, the knowledge acquired by the students. keywords assessment methodology, large groups, technical subjects, continuous assessment. mailto:hcliment@mot.upv.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 122 1. introduction during the development of new degrees in higher education in spain, new subjects have to be adapted to bolonia’s philosophy and have to include continuous assessment systems (shipman et al. 2003; morales et al. 2012; christoforou and yigit 2008; dixon et al. 1987). these systems are easily applied in subjects of latest academic years, where the number of students in the subject is reduced and the required effort in the evaluation process is reasonable (heywood 2000; shorter and young 2011). however, in the case of subjects in first or second courses, a large number of students (115 approx.) is usually found in spanish universities. the adoption of a continuous assessment in this scenario is highly time consuming for the teachers, who have to reduce their dedication to research and management activities. the present paper describes an experience in the use of new methods for the evaluation of different tasks developed by students in a subject with a large number of course assistants. 2. background the methodology presented in this paper is applied in the subject “thermal engineering”, which starts in the second semester of the secod course in the mechanical engineering degree in the escuela técnica superior de ingeniería del diseño (etsid) from the universitat politècnica de valència (upv). 2.1. subject description it is important to remark some characteristics of the subject since the developed methodology may not be effective for subjects that do not fulfill some requirements. the number of students is approximately 230, split in two groups of similar size. the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 123 subject, with 4.5 ects credits, combines theoretical lectures, problems solving in classroom and laboratory activities. the students’ assessment system consists of several evaluation acts which are detailed in table 1 and fulfill the requirements given by the university guidelines. the written exam is a limited-time activity that takes place at the end of the course. it consists of numerical problems and the students do not have access to books or additional material during the performance of the exam. there is another opportunity two weeks later, with the same characteristics, for students that did not pass the ordinary exam. there are three tests for the evaluation of each part of the subject. these tests are fulfilled using the on-line e-learning platform developed by the upv, which will be described in the next section. the tests are multiple-choice tests with only one correct answer and can be reached in the e-learning system during three consecutive days. once the students open the test, they have limited time (usually half an hour) and there is no possibility to save and continue the test in other moment. half of the points in the qualification mark can be achieved with individual tasks, where the students present numerical exercises similar to those solved in class. these tasks are evenly distributed all along the semester and each one has its own deadline in order to promote a continuous learning process in the students. each task involves a duration of 3 or 4 hours and should be performed individually, although the students may have access to additional material if they want. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 124 table 1. students assessment system in the subject “thermal engineering” evaluation method evaluation act weight [%] written exam: 30% ordinary extraordinary 30% (30%) tests: 20% test 1 5% test 2 10% test 3 5% individual tasks: 50% task 1 15% task 2 10% task 3 10% task 4 10% task 5 5% by observing the weights of the different assessment methods in table 1 it is observed that only the 30% of the qualification mark is achieved under teacher controlled conditions. the other 70% relies on the students’ code of honor. in order to avoid plagiarism and provide qualification marks that correlate with the students’ acquired competences, an additional written activity is included in the evaluation system; it is called the “task checking test”. this additional test is carried out in a similar way as the written exam. however, the contents of this test are very different. specifically, this test is designed in such a way that it is quick to perform and to correct. it consists of a collection of identical paragraphs to the students individual tasks. the aim is that the results of this test provide very high values for students who have actually carried out their tasks with interest and rigor, and mediocre or very low values for the rest of the students. the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 125 result of this test, on a scale between 0 and 1, directly multiply the grade obtained in the individual tasks, so that it is impossible to pass the subject by being evaluated only with methods were the authoring cannot be perfectly verified. 2.2. e-learning platform in the upv poliformat is the e-learning platform that gives support the teaching activities in all the subjects of the upv, which is a member of the educational project sakai. using this platform, students and teachers relationship breaks the traditional barriers. poliformat is the upv adaptation from the open source sakai platform, used worldwide by more than 100 universities. poliformat has a lot of features but, concerning the purpose of this work, only two of them are important to describe: tasks and grades sections. the tasks tool allows the creation, distribution and correction of the students’ activities. students’ activities are private and can only be seen by the student and the teacher. on the other side, grades tool allows the calculation, saving and distribution of the students marks. information related to students’ marks in the tasks tool is transferred directly into the grades section, where the students can follow the scores as they get their tasks and tests corrected. 3. methodology for students’ tasks assessment as described before, each one of the five tasks consists of similar numerical exercises to those solved in class throughout the teaching units that make up the subject. a classic correction (such as that carried out when correcting a written test) of the five tasks is not reasonable to perform due to the large number of students involved. thus, the tasks are evaluated taking into account only the final numerical results when solving each multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 126 problem. it is advisable to divide the exercises that make up a task on a large number of independent sections so that the student can easily detect where in the resolution process has failed. a task is created when a lesson is finished and the task typically remains open for about three weeks. in this way, the students can organize themselves and manage the other subjects of the course. the creation of the task is carried out in poliformat and includes two files: a text file containing the instructions and the exercises and a spreadsheet file that serves as a template for the presentation of the results. 3.1. on the instructions of the task and deadlines the text file is twofold: to clearly state the task instructions (with indications on how to deliver it), and on the other hand, to present the set of exercises to solve. in relation to the latter, and in order to try to reduce the cases of plagiarism, it is remarked that the input data of the exercises are modified with each student (they depend on the personal passport number). there are two important dates concerning the delivery of the task. on the one hand, a task delivered after the “deadline date” will not be corrected. this deadline is established at the time of creating the task in poliformat platform, which will prevent a student to upload the task. in addition, the “recovery date” is defined as a date prior to the deadline (around a week before). all the delivered tasks prior to the “recovery date” will be corrected and the results of its evaluation transmitted to students. students have now the opportunity to amend the results where they have made a mistake and reupload the corrected task before the deadline. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 127 3.2. on the spreadsheet template the spreadsheet helps the students to enter their numerical results in the appropriate cells. in this way, the correction process can be carried out as automatically as possible. to do this, it is advisable to give this spreadsheet the following: (1) it must be locked in order to avoid inserting and deleting rows and columns, except in cells where the end result of each exercise section is expected, (2) it is advisable to have a cell showing the input data calculated from the passport number (and should not show anything if the student has not entered the value of the passport number in the corresponding cell), and (3) it is advisable to include errors handling in the cells fulfilled by the students (typical examples are: using the wrong decimal symbol or confusion between the capital “o” and the zero digit). once the students have fulfilled the template with their numerical results, they upload the spreadsheet file to poliformat platform and wait for the correction, either on the “recovery date”, either at the “deadline date”. it should be noted that a student can upload a file to the application, even if it is not completely solved. however, since the server is able to store several files, it is important to say to the students in the delivery instructions that there must be only one file before the correction date. otherwise, the system is unable to know which is the last file that was uploaded, as that information is unfortunately lost. the use of a reasonable filename to the uploaded file to the server is also a good practice. 3.3. on the correction process once the “recovery date” or “deadline date” have passed, the correction process starts. poliformat greatly eases the management of the tasks delivered by the students. the process is described in the following paragraphs. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 128 firstly, the teacher enters the tasks section in poliformat and downloads all the files submitted by students. poliformat allows a massive download in a compressed zip file with all the information. this action, along with the extraction of the compressed file in a folder on the local hard drive, usually takes about five minutes. the extracted information is structured as follows: (1) a main folder named the same way as the task, (2) a microsoft excel© file, called “grades.xls” which contains a list of all students, and (3) a folder for each student called after his name, first name and passport number in brackets. within each student folder there is a subfolder called “task attachments” which contains the spreadsheet template file with the numeric results. then, every student file is corrected in an automatic procedure using a spreadsheet with internal macros programmed in visual basic©. the basic functions that are programmed for this automatic correction are able to: (1) create a file for each student in a subfolder within each student folder called “task comments”, (2) compare the student's solution and indicate in the file if the exercise’s section is correct or not, (3) manage potential standard errors (most of them are related to problems in the units and their detection can be carried out automatically), (4) calculate the final mark based on the errors in the different sections, and (5) insert the mark in the corresponding row in the global file “grades.xls”. this process usually takes about ten minutes. finally, once the correction process is finished, the teacher must make a compressed zip file containing the main task folder and upload the resulting file to the task section in poliformat where the students’ marks are published. the time spent on this process is around five minutes. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 129 4. results and discussion despite the large available period for tasks delivery, students generally tend to deliver the tasks in the last moment. in figure 1 the number of deliveries for two different tasks are shown as a function of time. they are split depending on “recovery date” or “deadline date”. task 1, shown above, was open for 30 days while task 2, plotted below, was open for 19 days. it can be clearly detected when the “recovery date” was scheduled in both tasks (day 12 in task 1 and day 14 in task 2). it can be observed a second peak when the task deadline occurs. figure 1. daily deliveries in two different tasks. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 130 figure 2 shows the histograms of the marks obtained in the previously described tasks 1 and 2. we present both the mark achieved before “recovery date” and before “deadline date”. a significant improvement in the students marks is detected thanks to the feedback provided in the first correction process. in the first task, it is observed that only 21 students from 152 deliveries before the “recovery date” achieve the maximum qualification. in the second task, the results are even lower; only 4 students (from 114) reach the maximum mark. the rest of the students have the possibility of making another delivery including the appropriate corrections before the “deadline date”. figure 2. marks histograms for tasks 1 and 2 depending on recovery and deadline dates. figure 3 shows a histogram of the mark improvement between tasks presented before “recovery date” and “deadline date”. high values are found in the 2 and 3 points. it is also remarkable the high percentage of students that do not improve their marks (observed in the values given in 0 points), either because they are not able to solve the task properly, either because they settle for their first mark. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 131 figure 3. marks improvement between recovery and deadline dates. as already mentioned, activities outside the classroom control can give rise to plagiarism. this is an undesired effect that should be minimized for educational and impartiality reasons. the “task checking test” aims to correct this possible mismatch between acquired knowledge and attained mark. the left plot in figure 4 shows the results of the “task checking test” and the average mark of the individual tasks. a very low correlation exists between the two variables. the right plot presents the variation in the obtained mark between the “task checking test” and the tasks average. positive values mean that the student has obtained a better mark in the “task checking test” than in the tasks average while negative values means the opposite. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 132 figure 4. correlation between the “task checking test” and the average marks of the individual tasks (left) and histogram of both marks difference (right). interesting results can be obtained from the plot on the right. half of the students present a marks difference between -1 and 1 point. 10% of the students have a better mark in the “task checking test” than in the average of the individual tasks. 20% of the students have -2 or -3 points difference and the other 20% have larger differences between both marks. 5. conclusions we have developed an automatic correction process for numerical exercises, whose input data vary with each student, based on the use of a programming environment within a spreadsheet file. the application of this system is advisable in subjects with large numbers of students and where there are several tasks to correct. the time spent by the teacher when correcting a task with 200 students is around 40 minutes (including two corrections: recovery and deadline dates). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 133 concerning the application of this methodology to a technical subject in second course following conclusions are obtained: tasks deliveries occur on the same day as the deadline date. for this reason it is very convenient to include the “recovery date” with teacher’s feedback, which promotes the continuous learning process of the subject. around 75% of the students make use of the correction before the “recovery date”. a very high percentage of these students make a second delivery before the “deadline date”. the mark usually increases with the second delivery. although the values depend largely on the performed task, it is possible to state that the mark rises up to 2 or 3 points (in a scale from 0 to 10) in the revised student delivery. lastly, since there is a low correlation between the final mark and the average marks of the individual tasks, it is advisable to include the “task checking test”, which balances the weight of the marks obtained in the individual tasks. it is a quick exam (not only for the student but also in the correction process) that tries to capture plagiarism cases in the evaluation acts in non-controlled situations. in our experience, up to 20% of the students fall into this plagiarism group due to the high discrepancies between the mark in the “task checking test” and the average mark in the individual tasks. moreover, if this checking test would not exist, around 20% of the students would have 2 or 3 points more in the average mark of the tasks than what they deserve. references a.p. christoforou and a.s. yigit (2008). improving teaching and learning in engineering education through a continuous assessment process, european journal of engineering education, 33(1), 105. doi: 10.1080/03043790701746405. r. dixon, g. rawlings, e.a. jones (1987). experiences with continuous assessment, assessment and evaluation in higher education, 12(1), 24. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2198 social and technological sciences tormos et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 121-134 | 134 j. heywood, assessment in higher education, jessica kingsley publishers ltd, londres, (2000). d. morales, g. centeno, c. vallellano, f. j. doblas, a. j. martínez-donaire, a. estévez, f. j. garcía-lomas, sistema de autoevaluación continua en la asignatura tecnología de fabricación, xx congreso universitario de innovación educativa en las enseñanzas técnicas, universidad de las palmas de gran canaria, (2012). d. shipman, s.l. aloi, e.a. jones (2003). addressing key challenges of higher education assessment, journal of general education, 52(4), 335 n.a. shorter, c.y. young (2011). comparing assessment methods as predictors of student learning in an undergraduate mathematics course, international journal of mathematical education in science and technology, 42(8), 1061. anexo 1 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 153 valencian network of educational innovation in optics a. pons-martí 1 , j. c. barreiro 1 , m. fernández-alonso 2 , ll. martínez-león 2 , m. h. giménez 3 , and j. a. monsoriu 3 1 grupo de innovación docente en óptica. dep. óptica, universitat de valència, 46100 burjassot, valencia, españa. amparo.pons-marti@uv.es 2 equipo de enseñanza y divulgación de la física, departament de física/init, universitat jaume i, campus de riu sec, 12071, castelló de la plana, españa. 3 departamento de física aplicada, escuela técnica superior de ingeniería del diseño, universitat politècnica de valència, 46022 valencia, españa. received: 2013-12-15; accepted: 2014-01-28 abstract the valencian network of educational innovation in optics consists of three groups from the universitat de valència, universitat jaume i, and universitat politècnica de valència. the participants in the network present an extensive background on performing actions to improve teaching practice. they have been involved for years in scientific outreach activities and in a number of educational innovation projects, which have developed innovative teaching materials. with the aim of sharing their experience and enhance their performance, the three groups have decided to join forces to become a network of educational innovation whose main projects are described in this communication. keywords educational innovation, collaboration, teaching materials, popular science. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 154 1. motivation and general guidelines of the network's activities the valencian network of educational innovation in optics (red valenciana de innovación educativa en óptica) was established this academic year 2012-13, thanks to a grant for the development of educational innovation from the vicerrectorado de cultura e igualdad at the universitat de valència. it consists of three groups (primarily made up of professors, although they also include research personnel and labratory technicians) from three of the public universities in the valencian region: the universitat de valència (uveg), the universitat jaume i of castellón (uji), and the universitat politècnica de valència (upv). its members have a broad experience in developing programs to improve teaching and, for many years, they have been participating in various science outreach activities and in educational innovation projects, for which they have devised innovative teaching materials. up to now, the groups under the valencian network of educational innovation in optics have developed three important lines of work, which are briefly described below: designing digital simulations and virtual laboratories; designing new experiments for both laboratory sessions and classroom demonstrations; and carrying out scientific dissemination activities. designing digital simulations and virtual laboratories. virtual laboratories are tools designed to provide students with the means to develop their own understanding of the processes they are learning. with this goal in mind, the simulation must give users the possibility of controlling all the variables involved, so that they can analyse their influence on the final outcome. furthermore, the required visual representations (images, graphs, animation) must be used so that the concepts and underlying relationships are properly assimilated. the groups from the universitat de valència and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 155 the universitat politècnica de valència have developed several applications within this line, using matlab and easy java. design of laboratory sessions and classroom demonstrations. generally speaking, the aim of laboratory sessions is to experiment with physical phenomena, taking measurements, analysing experimental data, and critically evaluating results. because of this, students who perform satisfactorily the laboratory classes have a considerable advantage for achieving the general course goals, both in terms of procedural objectives, and in terms of attitude and knowledge. in order to accomplish these goals, the methodology applied in the laboratory sessions must be meticulously devised, so that the maximum theory-practice transference is achieved. keeping in mind all of these points, the three network’s groups have taken an interest in designing and setting up new experiments and classroom demonstrations that approach the study of physical phenomena in innovative ways. science outreach activities. in order to disseminate the study of physics and improve the relationship between the university and society at large, it is always desirable to have direct and continuous contact with schools located in the area to fulfil the social responsibility of public institutions of higher education. we must emphasise that, in this role, it is necessary to supervise and collaborate with the transition from high school to college and, in our field, to also encourage scientific vocations. this is why the three groups of the network have paid special attention to their relationship with secondary schools, promoting different activities addressed to teachers and students, as well as the general public. in order take advantage of their expertise in the above areas, the three groups from the universitat de valència, the universitat jaume i of castellón and the universitat multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 156 politècnica de valència, have joined forces to create a network of educational innovation. the specific goals proposed by the valencian network of educational innovation in optics are, first of all, the exchange of teaching experiences and, secondly, the development of specific activities in the members' previous lines of action. to acomplish the first objective, the group leaders from each university have met several times throughout the academic year to begin outlining their joint actions. as for the second goal, it was decided to work together on designing simulations and applets. the following sections provide a summary of the main activities carried out by each one of the network’s groups. the development of all the activities of the three groups was funded by various several for innovative education projects from their respective universities. 2. the universitat de valència group (uveg) the group for innovative teaching in optics (indoptic) at the university of valencia is made up of professors with extensive experience in the field of optics and a common interest in the development of programs for innovative education. coordinated by dr. amparo pons martí, this group was created several years ago, and is recognized as a consolidated innovation group at its university. in fact, it was this group that suggested the creation of the valencian network of educational innovation in optics, in which dr. amparo pons martí serves as a general coordinator. so far, this group has focused on designing and setting up the virtual optics laboratory and on proposing new laboratory experiences and classroom demonstrations, although most of its members also participate in different scientific dissemination activities. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 157 the virtual optics laboratory (pons 2009) is conceived as an educational tool to support the theoretical and practical teaching of optics, which includes experiments in instrumental and physical optics. special care has been taken in the design of the virtual environment so that the approach used for the practices is very realistic. the contents are presented sequentially using diagrams, videos, and simulations of the expected results. questions about the devices are interspersed, for students to check their knowledge about the physical phenomena involved. in developing these virtual experiments, measuring components and self-editing tables of values have also been included to give the students the possibility of facing some of the experimental problems they will find in a real laboratory. the introduction of this kind of items related to the measurement process is an innovative characteristic of this virtual laboratory. as a complementary feature for the virtual experiments, an applet was designed last year for studying the linear polarization of light. the applet allows malus’s law to be verified by using up to three linear polarizers (see fig.1). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 158 figure 1. some examples of the virtual optics laboratory during these years, various laboratory experiments have also been proposed that could be used either as qualitative classroom demonstrations or as laboratory practices. for example, fig. 2 shows some pictures of two of the proposals carried out: a mechanical model to study the behaviour of a micro-structured optical fibre (stevenson 2011), and the determination of the average size of the red blood cells in a sample of human blood, based on the measurement of the diffraction pattern generated when the sample is illuminated with a laser beam. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 159 in the context of the network, and in collaboration with the group at the universitat politècnica de valència, work has also been done for developing another kind of virtual laboratory to study the diffraction properties of fractal elements, and for proposing new laboratory experiments, which are described below in section 4. in all cases, an effort has been made to incorporate innovative components to make them attractive to students. as for scientific dissemination, most group members regularly participate in all kinds of activities aimed at teachers and secondary-school students. among these activities, we would like to highlight those related to the physics classroom experimenta of the faculty of physics, laboratory specially designed for high school students in which training courses for teachers are also taught (http://experimenta.blogs.uv.es/]). finally, we would like to mention, for its high difussion and social impact, the experimenta science fair-competition (which held its eighth edition last april) in which high school students present to the general public their own experimental projects (ferrer-roca 2013). figure 2. two examples of laboratory experiments multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 160 also, in collaboration with the group at the universitat politècnica de valència, they participate in the summer science campus "un viaje fantástico con los pies en la física" -“a fantastic walk through physics” (http://www.campuscientificos.es/). 3. the group of the universitat jaume i de castellón at the universittat jaume i, in castellón de la plana, the team of professors in the field of optics has been working for years in the areas of innovative education and scientific dissemination, and has strengthened ties with secondary schools. currently, it is established as an educational innovation group, recognized by an internal grant at the university in castellón under the title team for the teaching and outreach of physics, coordinated by dr. mercedes fernández alonso. the group members teach in the technological degree programs at the school of technology and experimental sciences, which include degrees in industrial engineering, technical architecture and design and development of video-games, as well as chemistry. in the area of educational innovation, the group has carried out different projects pertaining to the development of new university degrees and the process of european harmonization. from the beginning, they have participated in pilot experiences, even before the establishment of the new degrees (martínez-león 2006, martínez-león 2008). after the new degrees were put in place, the continued work on educational improvement has centred its efforts on: encouraging student autonomous learning; preparing new educational resources or selecting existing ones; devising and implementing an efficient continuous assessment system; providing support for teaching physics subjects in english and for the educational coordination of the study of physics multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 161 in the first years of the new degree programs (martínez-león 2010, fernández-alonso 2011). we should stress that, in the last few years, this educational team has been responsible for the subject physics ii, which is part of the degrees in industrial technology, mechanical engineering, electrical engineering, chemical engineering, and agricultural and environmental engineering. this subject is taught in a coordinated way in six different groups made up of students from all these degree programs. among their dissemination activities, the group organises the local events for the physics olympiad, as well as talks and demonstrations that take place in secondary schools or during secondary-school visits to the university campus (such as those included in the practica la uji program). they participate in the university’s open day, and in the science week at the different extension centres of university throughout the region of castellón. the group has also participated in training activities for teachers, such as the courses organized by cefire in castellón, and is teaching classes in the master in teaching secondary school, vocational education and training and foreign languages. different aspects of physics are addressed in these outreach activities, although in some demonstrations emphasis is placed on the subject closest to this group’s research field: optics. in fact, they have a series of experiments on light diffraction and interference, polarization, and optical transmission of information, which have been used in science dissemination sessions for all kinds of audiences: undergraduate and high school students, teachers in training, and the general public [martínez-león 2013). some of these demonstrations are shown, by way of summary, in the pictures in fig. 3. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 162 figure 3. some dissemination activities done by the group at uji for many of these activities the group has relied on the collaboration of their colleagues at the universitat de valència and the universitat politècnica de valència, now integrated in the valencian network of educational innovation in optics. together, they have begun to devise new educational resources that take advantage of information technologies to promote the students' autonomous learning. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 163 4. the universitat politècnica de valència group the third team of the valencian network of educational innovation in optics is constituted by professors from the departments of physics and applied mathematics at the universitat politècnica de valència. this multidisciplinary team, coordinated by dr. juan a. monsoriu, has extensive experience in mathematical modelisation through virtual laboratories, used as an aid for the teaching-learning process. for several years, they have been developing digital simulations (using matlab and easy java) of physical processes, such as the movement of blocks and wheels, oscillations, waves (giménez 2009), etc. within the specific context of the network, they have developed a virtual laboratory for the mathematical modelisation of the diffraction properties of fractal gratings (giménez 2011) (see fig.4) which, in turn, is complemented with real experiments (monsoriu 2011). they are currently working on the application of these diffraction gratings to other aperiodic sequences (fibonacci, thue-morse, etc.). figure 4. virtual laboratory difract. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 164 in addition to the virtual laboratories, in recent years the group has also worked on the development of new, highly innovative laboratory experiences. with the aim of enhancing the theory-practice transference, they have made a point of introducing innovative elements in their laboratory experiments which will arouse the students' interest in carrying out the practice, such as can be, for example, the video analysis of physical phenomena (monsoriu 2005) or taking measurements with the accelerometer of a smartphone (castro-palacio 2013). in collaboration with the group at the universitat de valència, they have proposed some of these innovative experiments, such as determining the spatial resolution of a lcd or ccd through diffraction (barreiro 2012, barreiro 2014). figure 5 shows some images of these laboratory experiences. currently, the valencian network of innovative education is studying the possibility of using the built-in lux meter of a smartphone in different physics experiments. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 165 figure 5. a few innovative proposals for laboratory experiments multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 166 acknowledgements the authors of each of the groups of educational innovation, as well as indoptic (gcid35/2009) at the universitat de valència, moma at the universitat politècnica de valència, and the team for the teaching and dissemination of physics at the universitat jaume i, would like to thank the institutional and economic support received from their respective universities for the development of their innovation projects. also, the three groups would like to thank the vicerrectorado de cultura e igualdad at the universitat de valència for its support in creating the valencian network of educational innovation. references barreiro, j., monsoriu, j.a., andrés, p., barreiro, j.c. and pons, a. (2012). difracción y estructura espacial de componentes electrónicos de uso cotidiano: experiencias motivadoras para estudiantes de ingeniería, proceedings xx cuieet, las palmas de gran canaria. barreiro, j., pons-martí, a. barreiro, j.c., castro-palacio, j.c. and monsoriu, j.a. (2014). diffraction by electronic components of everyday use. american journal of physics (accepted). castro-palacio, j.c., giménez, f., giménez, m.h. and monsoriu, j.a. (2013). physics experiments using the mobile acceleration sensor, inted2013 proceedings, valencia. cuenca, r. (2011). la enseñanza de la física en los nuevos grados o cómo orientar el aprendizaje de la física para la generación facebook. proceedings iii jornada nacional sobre estudios universitarios: "el presente de los nuevos títulos", castellón. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 167 ferrer-roca, ch., andrés, m.v. and pons-martí, a. (2013). el aula de física experimenta de la universidad de valencia. revista española de física, 27(1), 68-71. giménez, m.h., riera, j., vidaurre a. and monsoriu, j.a. (2009). virtual laboratory for the study of one-dimensional waves. edulearn09 proceedings, barcelona. giménez, m.h., monsoriu, j.a., giménez, f., pons, a., barreiro, j.c. and furlan, w.d. (2011). difract: un nuevo laboratorio virtual para la modelización matemática de las propiedades de difracción de redes fractales. modelling in science education and learning, 4, 223-229. martínez-león, ll., tajahuerce-romera, e., climent-jordà, v., lancis-sáez, j., fernández-alonso, m. and mínguez-vega, g. (2006). estrategias de mejora educativa del laboratorio de física en el marco del proceso de armonización europea. proceedings iv congrés internacional de docència universitària i innovació, barcelona. martínez-león, ll., fernández-alonso, m., mínguez-vega, g., lancis-sáez, j., tajahuerce-romera, e. and climent-jordà, v. (2008). metodologías armonizadas en grupos tradicionales: la reforma universitaria, entre la realidad y la utopía. proceedings i jornada nacional sobre estudios universitarios: “de los proyectos de convergencia a la realidad de los nuevos títulos”, castellón. martínez-león, ll., darraud, c., cosset, f., bessaudou, a., brevier, j., fernándezalonso, m., mínguez-vega, g., tajahuerce, e., lancis, j. and climent, v. (2010). strategies for improving science education on both sides of pyrenees. proceedings of inted2010 conference, valencia. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2200 social and technological sciences pons-martí et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 153-168 | 168 martínez-león, ll., fernández-alonso, m., lancis, j., tajahuerce, e., mínguez-vega, g., alós-valls, h., durán, v., climent, v., ferriol, n.s., pérez-vizcaíno, j., clemente, p. and mendoza-yero, o. (2013). focusing on people’s curiosity: science popularization through experiments and demonstrations on optics for a variety of audiences, proceedings of inted2013 conference, valencia. monsoriu, j.a., riera, j., giménez, m.h. and vidaurre, a. (2005). measuring coupled oscillations using an automated video analysis technique based on image recognition. european journal of physics, 26, 11491155. doi:10.1088/0143-0807/26/6/023. monsoriu, j.a., furlan, w.d., pons, a., barreiro j.c. and giménez, m.h. (2011). undergraduate experiment with fractal diffraction gratings. european journal of physics, 32, 687-694. pons, a. , andrés, p., barreiro, j.c., furlan, w.d., martínez-corral, m., monsoriu, j.a., navarro, h., sánchez, e., saavedra, g. and silvestre, e. (2009). laboratorio virtual de óptica: diseño y puesta en marcha. proceedings xvii cuieet, valencia. stevenson, a., silvestre, e., castelló-lurbe, d., beltrán-mejía, f. and monsoriu, j.a. (2011). modelo mecánico de una fibra de cristal fotónico. proceedings xxxiii reunión bienal de la real sociedad española de física 21º encuentro ibérico para la enseñanza de la física, santander. multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 20 the effectiveness of using augmented reality in teaching geography curriculum on the achievement and attitudes of omani 10th grade students khalfan al shuailia, ali al musawib, raja muznahb aminstry of education, p.o.box 3, muscat pc 100, oman bsultan qaboos university, p.o.box 32, al-khodh pc 123, oman * correspondence: asmusawi@squ.edu.om; phone: +968-9931-3289 received: 17 january 2020; accepted: 02 september 2020; published: october 2020 abstract this study aimed to investigate the effectiveness of using a mobile augmented reality (ar) application in the geography unit for tenth grade students on their achievements and attitude. the unit is part of the omani’s social studies curriculum and entitled, “environmental problems and hazards”. to meet this purpose, a quasi-experimental study design was conducted. the instruments designed, validated and used to collect data were pre-post achievement tests and attitudinal scale. the participants were 64 male students, divided into two experimental and control groups. the field application period took four weeks in which the control group studied the unit using the normal teaching method, while the mobile ar application and the hp reveal groups were supported with a website that supports their features. findings show that there was a statistically significant difference between the pre and post-field applications in terms of students’ achievement and attitudes. nevertheless, there was no impact observed when the control and experimental groups’ achievement were compared. the researcher suggested conducting further investigations to examine the value of integrating ar in the educational system considering various factors and variables. keywords: augmented reality; students’ achievement; attitude, social studies curriculum; geography; hp reveal. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 21 1. introduction the role of technology is important in our lives, particularly in the field of education. the integration of information and communication technologies (ict) in teaching and learning shows positive effects and has become a new trend in the field of education (georgieva, 2006). in addition, earlier research studies showed that supporting education with technologies to deliver instructions helps teachers to carry out innovative teaching approaches (alkhattabi, 2017) and helps learners to improve their curiosity and understanding. in addition, it help them to “increase their learning capacities and personal development” (gómez-ejerique & lópez-cantos, 2019, p.1). using technology is not exclusive to a specific area, as students can use different ict tools, applications, and services at any time and place. therefore, integrating ict in the educational process increases students’ motivation toward learning and makes them responsible for their learning (safar, al-jafar, & al-yousefi, 2017). since individual students vary on their characteristics and interests, using ict in teaching and learning has to be managed according to their needs and levels. moreover, researches showed that, students at the elementary level prefer to use concrete rather than abstract thinking; therefore, utilizing technology tools that help to provide content with real experience can influence their critical thinking skills and pace of learning (al-asheeri, 2017). new technology such as virtual and/or augmented reality can simulate real-life situations and enrich the educational environment with interactive content. romero forteza & carrió pastor (2014) find that virtual learning environments must be adaptable for implementation with various teaching approaches. augmented reality (ar) technologies are specifically characterized by their ability to depict reality by transferring abstract elements to dynamic ones. primarily, this technology brings direct and in-direct real-world environments through digital devices, with which students can interact and manipulate the presented content (chang, wu, & hsu, 2013). many students in schools have difficulties in understanding or imagining the studied concepts, phenomena, and/or processes spatially. the traditional way to solve this problem is by providing two-dimensional (2d) images or video clips that can show how the processes or phenomena occur (shelton & hedley, 2002), but ar technology can now easily show the depth of these phenomena (chang, wu, & hsu, 2013). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 22 2. significance of the study the importance of this study lies on measuring the effectiveness of a technological trend in educational technology, which is ar technology. based on the best of our knowledge, this study is considered first work in the field of using ar in geography teaching in oman according to researcher findings. the results of this study may help the curriculum designers/planners to apply new strategies through the use of such emerging technologies. furthermore, it may increase the omani teachers’ awareness of integrating new technologies such as ar in their teaching methods. it is also envisaged that the study may open doors to integrate new learning styles to enrich geography lessons with technology-enriched environments. through this field experience, the researchers believe that the findings could offer solutions to the difficulties faced by geography (and social studies) teachers in terms of site visits, natural phenomena observations or lack of finance. 3. statement of the problem currently, removing boundaries of time and location is an important factor that supports lifelong learning. with the advancement of new mobile technologies, teachers had become able to integrate ar as an innovative teaching method (al-azawi and shakkah, 2018). in addition, using ar to augment abstract topics on the printed books, by using multimedia and three-dimensional objects can enhance students’ learning experience. as a result, students can move through the curriculum topics with a deep understanding of what they are learning. with the advent of modern technologies, geography teachers are now able to bring the real world to their students and eliminate the traditional barriers of learning. introducing interactive technologies, such as ar, can make the learning process more exciting and elongate the information retention (alhumaidan, lo, & selby, 2018). oman is one of the rare countries that have rich geographical diversity (al maashri, al-asadi, tageldin, al-lawati, & al shidhani, 2015). therefore, the ministry of education in oman has integrated a geography curriculum that explains oman’s diversity from different perspectives. furthermore, findings by (al-maamari, al-nofli, & al-gharibi, 2014) showed that both teachers and students were interested in the variation of the topics presented in the geography units. in addition, the topics that attract students’ interest most were weather/climate, natural disasters (earthquakes, volcanoes, and cyclones) and environmental issues such as deforestation, soil erosion, and the greenhouse effect. however, teachers face problems to accurately represent spatial phenomena or https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 23 topics (such as desertification) by using the traditional methods of 2d multimedia (shelton & hedley, 2002). omani geography teachers are no exception. sometimes they want their students to discover a specific location, but it is remote from their school’s position and unreachable in a short time. recent research calls for the implementation of new technologies, particularly ar, in the omani educational system to overcome these and other logistical obstacles that may impede learners from experiencing the depth of the natural phenomena (basha, abbas, yusufi, & rajbunisa, 2019; al musawi, ambusaidi, al-balushi, al-sinani, & al-balushi, 2017; almusawi, resheidi, jadeedi, alsaadi, & riyami, 2016; al maashri, et al, 2015). from this perspective, this research aims to study the effectiveness of using augmented reality in teaching the geography unit of social studies curriculum on the achievements and attitudes of 10th-grade students. 4. methodology 4.1. research design this research emphasizes the use of quasi-experimental design and focuses on a specific variable namely, the attitudes. therefore, to achieve the objectives of the study and to get reliable data on the impact of ar on students’ learning, this study used a quasi-experimental research design to measure two main variables, which are achievement and attitudes. the study sought to answer two questions, the first question was answered using an achievement test and the data collected using this instrument helped to identify the systematic change in students’ learning and understanding. the second question utilized the attitudinal scale to analyze students’ tendency to adopt this technology in their future learning and life. all the data by these two instruments were collected and analyzed quantitatively. 4.2. population and sample the target population is 10th-grade students in al shiekh nasser bin rashid school in almawaleh region in the muscat governate in oman. the targeted sample of this research is 64 students. according to meta-analysis investigation, most of the previous studies in the field of ar conducted the investigation among samples not exceeding 100 (akçayır & akçayır, 2017; garzón & acevedo, 2019). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 24 there were seven sections of 10 grade in the school, and two sections were selected based on the research need. random selection of the sample and mixing students was not allowed due to school and the ministry of education (moe) policies. the first section was contained 31 students and they will be under the control group and the other section includes 33 students and they will be under an experimental group; table 1 illustrates sample distribution. table 1 sample distribution among the groups group students number treatment measurement tool experimental 33 learning with ar achievement test and attitudinal scale control 31 traditional class achievement test 4.3. procedures with a limited time of implementation, the study was established in one of the government schools in muscat. implementing ar in students’ teaching and learning was unique, which caused full acceptance from the school administration and the cooperative teachers. the experiment did not require a change in the classroom setting, but it was based on using mobile devices. the following points summarize the procedures followed by the researcher until finishing the research. the aim of this step is to show for beginner researchers a minimum required step for making research within the school context. • reading the horizon reports of schools and higher education to get more insight into the new promises technologies in the field of education. • after deciding which technology to use, reviewing previous works conducted in that field was necessary to start from the researchers to stop and learn from their recommendations. • selecting the study population and sample; school, students, and content. • since the study was conducted in a government school, the formal procedure was required, which included a no objection letter released from squ to moe and the selected school. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 25 • looking for logistic support, in terms of providing smart devices to conduct the experiment. different request letters were sent to squ and moe for that purpose. • selecting the appropriated ar application based on what resources were available, the used model was sections model. • developing evaluation tools (achievement test and attitudinal scale) and revising them from different experts from squ and moe. • preparing the learning materials that include different types of multimedia using the internet and producing them. • preparing the ar application from creating accounts, integrating multimedia, and creating auras. • selecting students randomly, and conducting a pre-test for both groups and a pre attitudinal scale for the experimental group. • training teachers on using the new instructional materials, and deciding the suitable learning strategies that fit with the experiment and the moe learning goals. • start the experiment over four weeks in the second semester of the academic year 2018/2019. • assessing students after they finished the experiment through the post-test for two groups and the attitudinal scale for the experimental group. • analyze the collected data by adding the data in spss and according to the results, a summary of the result with the experts from squ and moe. 5. findings the findings showed that there was an increase in students’ learning (achievement) before and after the process of treatment was established. furthermore, students’ motivation and interaction increase as observed by the instructor. however, there were no significant differences between the control and the experimental group after using ar in terms of their learning, despite the attitude of the experimental group increasing. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 26 6. recommendations according to the experiment and its findings in term of using ar in teaching geography unit, there are several recommendations provided by the researcher for the purpose of increasing reader insight about the findings and providing suggestions for future research in that field of using ar in education in general, and in the omani context in specific. • since the findings in terms of testing students’ achievement after using ar did not match with what came in most of the previous literature, further research is required to be conducted in this area controlling some factors such as: a. ensuring that each student has his own mobile device in the process of implementation. b. assigning different teachers for teaching both the control and experimental group. c. applying the experiment in parallel with both groups. d. the prepared achievement test marks should be written in students’ records to ensure students' realism in selecting answers and their credibility. e. developing a method that ensures students’ access to the materials used in their homes to enable them to review what they learned in class. f. ensure that there is enough time to teach all the content, or the ar may become a distracting tool. g. focusing on innovative learning strategies and new classroom settings in which students are able to interact easily. • there is a need to investigate the effect of using ar in teaching geography content, but with using different assessment tools to traditional achievement tests to show the effect. this suggestion rises because students who used ar focused on interaction with content and understanding it through different aspects in which the normal test might not be measuring the effect of. • this research investigated the effect of mobile-based ar on teaching geography. oman is one of the countries that is geographically diverse. another type of ar, which is location-based, could be implemented in students’ teaching in the form of field trips to specific sites. therefore, there is a need to investigate the effect of location-based ar in teaching geography on different https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 27 factors such as students’ achievement, motivation, attitude, and interaction and comparing the findings with mobile based methods. • since this research investigated 64 male students from one school in four weeks, there is a need for a wider investigation that includes both genders from different governances to generalize the findings for the decision makers to take the necessary measures. 6. conclusion most of the technologies used are accessible, but they are not self-explanatory. teachers should find creative strategies to combine the implemented technology and their instructional strategies in a way that learning occurs in an easy and fast manner. in addition, virtual technology depends on designing the interactive content, therefore giving much attention to the instructional design process, as well as the application’s user interface design playing a key role in making changes. moreover, technical problems that appear during the use of ar applications should be anticipated to let students concentrate on their learning without any distractions. references akçayır, m., & akçayır, g. (2017). advantages and challenges associated with augmented reality for education: a systematic review of the literature. educational research review, 20, 1-11. https://doi.org/10.1016/j.edurev.2016.11.002. al-asheeri, h. a. y. (2017). effectiveness of using augmented reality strategy in enhancing learning english for cycle one students in the elementary stage in the kingdom of bahrain. us-china foreign language, 15(7), 409-419. https://doi.org/10.17265/1539-8080/2017.07.001. al-azawi, r. (2018, april). embedding augmented and virtual reality in educational learning method: present and future. paper presented at the 9th international conference on information and communication systemsicics, irbid, jordan. retrieved from https://ieeexplore.ieee.org/document/8355470. al-maamari, s., al-nofli, m., & al-gharibi, z. (n.d.). the state of social studies in basic education schools in oman. asian social science, 10(7), 213–220. https://doi.org/10.5539/ass.v10n7p213 al maashri, a., al-asadi, s., tageldin, m., al-lawati, s., & al shidhani, a. (2015, february). augmented reality for tourism in oman using free open source software. paper presented at the 2nd free and open source software conference foss-2015, muscat, oman. retrieved from https://fossc.om/images/papers/ahmed_almaashri_fossc_oman_2015.pdf https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1016/j.edurev.2016.11.002 https://ieeexplore.ieee.org/document/8355470 https://fossc.om/images/papers/ahmed_almaashri_fossc_oman_2015.pdf multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 28 al musawi, a., ambusaidi, a., al-balushi, s., al-sinani, m., & al-balushi, k. (2017). effectiveness of learning with 3d-lab on omani basic education students' achievement, attitudes and scientific thinking. journal of education and training studies, 5(11), 177-188. https://doi.org 10.11114/jets.v5i11.2743. alhumaidan, h., lo, k. p. y., & selby, a. (2018). co-designing with children a collaborative augmented reality book based on a primary school textbook. international journal of child-computer interaction, 15, 24–36. https://doi.org/10.1016/j.ijcci.2017.11.005 alkhattabi, m. (2017). augmented reality as e-learning tool in primary schools’ education: barriers to teachers’ adoption. international journal of emerging technologies in learning, 12(2), 91–100. https://doi.org/10.3991/ijet. v12i02.6158 al musawi, a., resheidi, a., jadeedi, m., alsaadi, a., riyami, h.a. (2016). the impact of an augmented reality system in teaching machine dynamic course for engineering students, turkish online journal of educational technology, special issue, july 2016; 562-564. http://www.tojet.net/special/2016_7_1.pdf. basha, s., abbas, m., yusufi, g., & rajbunisa. (2019). augmented reality based education for the improvement for sustainable learning ability in oman educational system. international journal of advanced trends in computer science and engineering, 8(12), 110 – 115. https://doi.org/10.30534/ijatcse/2019/1881.22019 chang, h.-y., wu, h.-k., & hsu, y.-s. (2013). integrating a mobile augmented reality activity to contextualize student learning of a socioscientific issue. british journal of educational technology, 44(3), 95–99. https://doi.org/10.1111/j.1467-8535.2012.01379.x garzón, j., & acevedo, j. (2019). meta-analysis of the impact of augmented reality on students’ learning gains. educational research review, 27, 244-260. https://doi.org/10.1016/j.edurev.2019.04.001 georgieva, e. (2006, june). a comparison analysis of mobile learning systems. paper presented at at international conference on computer systems and technologies compsystech’ 2006, veliko tarnovo, bulgaria, retrieved from https://pdfs.semanticscholar.org/2e3c/61c97bf06dcb86c8e94125407c79bb0b2a9f.pdf. gómez-ejerique, c. & lópez-cantos, f. (2019). application of innovative teaching-learning methodologies in the classroom. coaching, flipped-classroom and gamification. a case study of success. multidisciplinary journal for education, social and technological sciences, 6(1), 46-70. https://doi.org/10.4995/muse.2019.9959. romero forteza, f. & carrió pastor, m. (2014). virtual language learning environments: the standardization of evaluation. multidisciplinary journal for education, social and technological sciences, 1(1), 135152, https://doi.org/10.4995/muse.2014.2199. safar, a. h., al-jafar, a. a., & al-yousefi, z. h. (2017). the effectiveness of using augmented reality apps in teaching the english alphabet to kindergarten children: a case study in the state of kuwait. eurasia journal of mathematics, science & technology education, 13(2), 417-440. https://doi.org/ 10.12973/eurasia.2017.00624a. shelton, b. e., & hedley, n. r. (2002, september). using augmented reality for. teaching earth-sun relationships to undergraduate geography. students. paper presented at the first ieee international https://creativecommons.org/licenses/by-nc-nd/4.0/ http://www.tojet.net/special/2016_7_1.pdf https://doi.org/10.30534/ijatcse/2019/1881.22019 https://doi.org/10.1016/j.edurev.2019.04.001 https://pdfs.semanticscholar.org/2e3c/61c97bf06dcb86c8e94125407c79bb0b2a9f.pdf https://doi.org/10.4995/muse.2019.9959 https://doi.org/10.4995/muse.2014.2199 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13014 social and technological sciences issn: 2341-2593 al shuaili et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 20-29 | 29 augmented reality toolkit workshop, darmstadt, germany. retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.116.3323&rep=rep1&type=pdf https://creativecommons.org/licenses/by-nc-nd/4.0/ http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.116.3323&rep=rep1&type=pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 101 review of professionals accreditation systems and their relation to higher education studies in the branch of engineering juan r. lama ruiz, alejandro m. martín gómez, mª jesús ávila gutiérrez, francisco aguayo gonzález e.p.s universidad de sevilla, virgen de áfrica nº 7, 41011 sevilla, teléfono 954552827, fax 954552826, jrlama@us.es received: 2013-12-15; accepted: 2014-01-28 abstract this paper aims to analyze and compare the accreditation systems of people applying to higher education professionals independently of the area of accreditation to which they belong. to do this, we will analyze five certification systems: project management institute (pmi), registration and certification center of people (cerper), general council of industrial engineering (cogiti), association of naval architects and ocean (coin) and national qualifications authority (incual) based on eleven indicators derived from the study of different sources of information. this comparative allows us to analyze and evidence the similarities and differences of existing accreditation systems of people. keywords certification systems, competencies, professional qualifications. mailto:jrlama@us.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 102 1. introduction in the development of higher education european system, appears a new scenario that seems to blur the relationship between academic qualifications and professions, especially in the field of engineering, which will be enhanced as a result of the next passing of professional services act. actually the role of the engineer in society is changing, it must be prepared to deal not only technical issues, but to solve problems and formulate an interdisciplinary mode integrated solution alternatives. the goal to strive for greater integration in terms of application of skills in the academic and professional field has led to a profound modification in the concept of competence, understood as a combination of attributes applied knowledge, abilities, skills and responsibilities describing the level proficiency in performing a function. from the educational standpoint competition focuses on the teaching-learning process, while in the professional field concept focuses on the effectiveness and efficiency of work. although until now it was possible to identify the qualified professional based on their qualifications, it is expected that in a near future we need new mechanisms to achieve this, such as the accreditation of professionals. the process of accreditation seeks to declare publicly that a product, process or service conforms to the requirements. the areas of activity of an engineer can be classified according to the type of company where you work or service function to develop. depending on the type of company include industrial business or service (management: technical, commercial, production, ...; projects: products, facilities, ...; organization: quality, stocks, production, ...), government or public company (technical assessments, project management, ...), and freelance (projects, technical reports, ratings, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 103 ...). depending on the work to develop: leadership and management, process control, product marketing, r & d, technical (technical office), consulting etc. the main objective of this paper is to analyze and compare systems of accreditation of people that to be applied to professionals in higher education and who have a national or international recognition independently of the area to which they belong. so we will describe different accreditation systems to establish indicators that allow a comparison of the systems to establish the similarities and differences that may exist between the systems analyzed. 2. conceptual basis 2.1. accreditation systems of people the current situation in the workplace and mobility of people who exist in the european union has made that to be needed the recognition of professional qualifications. through the directive 2005/36/ce of the european parliament and of the council of 7 september 2005, has established the procedure for the recognition of professional qualifications. the directive distinguishes five levels of professional qualifications: certificate of competence: general training of primary or secondary education, or professional experience of three years. certificate: training at secondary education technical or professional. accrediting title: formation of post-secondary level. accrediting title of higher or university education: at least three years. accrediting title of higher or university education: minimum of four years. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 104 with regard to higher education the rd 1027/2011, of 15 july, provides a map of national qualifications (levels, learning outcomes and descriptors), dividing it in four levels: technician, grade, master and doctor. the qualifications, in formal education, is defined in article 2.a) of rd as "any degree, diploma or certificate from an educational institution accredited that have acquired a set of learning outcomes, having successfully passed a training program in a legally recognized institution in the field of higher education. " however, the law 5/2002 on qualifications and vocational training, defines the vocational qualifications in article 7.3.a) as "the set of competences with significance in employment that can be acquired through training or other modular training as well as through work experience". giving special importance to work experience for the recognition of professional qualifications. as a combination of the facts appear different institutions that acquire an interest in the certification of professional competences. being the certificate of people the process by which an independent third party ensures that a person meets the requirements contained in a certification scheme. including criteria to ensure skills, competencies, experience and training (tallo 2009). the iso 17024 standard for organizations engaged to certification of persons provides in paragraph 4.3 the importance of impartiality due to the impact of the certificates to be issued by accredited organizations. the enac establishes that "the organization of certification must be structured to provide confidence in its competence, impartiality and integrity to stakeholders" (cga-enac 2009), as a prerequisite for accreditation. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 105 2.2. taxonomy of engineering competences it has professional competence who has the knowledge, skills and attitudes necessary to practice a profession, can solve professional problems independently and flexibly and is trained to assist in their professional environment and work organization. the main requirements of skills demanded to the workers by business are synthesized into three groups: technical, human and analytical. as shown in table 1, the tasks that are grouped in each of three identified groups are diverse: table 1. professional requirements in european companies type of knowledge functional task or skills technical management, production, organization and systems design, quality control, scheduling, etc. human creativity, organization and coordination of tasks, decision making, teamwork, leadership, communication, etc. analytical development of diagnostics, information and analysis of problems, forecast future scenarios, design business strategies, etc. in the table 2 (torres and abud 2004), is showed generic skills of the engineer in order to further establish the possible indicators for its evaluation. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 106 table 2. integrated view of generic competences of mechanical engineer knowledge, skills, attitudes and values "the engineer has skill / ability / willingness / attitude ..." investigate, generate and manage information and data analyze, propose and solve real engineering problems designing systems to meet needs complementary skills communicate effectively interact and work in teams encourage self-development and continuous improvement commitment to ethics and professional, legal, social and environmental responsibility assess the social, artistic and cultural diversity 3. methodology the summary of the methodological process has been carried out to define and compare the different systems of accreditation of people analyzed are shown in table 3. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 107 table 3. summary of the methodology factor initial considerations object of study systems certification: pmi, aec-cerper, cogiti, coin, fp sources information used primary sources of information such as books, magazines, reports and websites. secondary sources of information such as libraries and databases. data collection is employed document analysis analysis and interpretation of data study of indicators that are clear, concise, and show relevant information and are in line with the objectives. for definition of indicators have followed the next steps of review of information sources, selecting the relevant information, identifying and developing indicators and legend definition for each indicator. 4. accreditation system analysis in this section we will investigate the different certification systems together with their indicators and comparative. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 108 4.1. project management in project management certification exists mainly four systems of certification at international level: ipma, pmi, p2m y prince2. these systems possess organization of knowledge, standards and certification processes themselves. given the similarities between these standards (cardoza 2011), is propose to analyze the pmi system because it is very interesting in the field of engineering. pmi (project management institute) is a non-profit association for the profession of project management. founded in 1969 in pennsylvania (usa), currently has more than half a million members and certified professionals, and has credentials in 185 countries. the pmi is accredited by the american national institute of standards (ansi). pmi objetive seeks to promote the profession of project management through standards and certificates recognized worldwide accreditation ansi accredited to iso 17024 and iso 9001:2008 accreditation levels three levels of certification in project management: program management professional (pgmp). project management professional (pmp). certified associate in project management (capm). scope of international international multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 109 recognition certification process pgmp certification process: application for presentation, review of completed application, payment application process, audit process, review before the exam of multiple-choice eligibility, evaluation of several reviewers panel. pmp: request for production, review of completed application, payment application process, audit process, exam of multiple-choice eligibility. capm: request for production, review of completed application, payment application process, audit process, exam of multiple-choice eligibility. evaluation audit assessment of the application and review cost 400 – 600 € 4.2. quality systems and environment the registration and certification center of people (cerper) belonging to the aec (spanish association for quality) plays the mission of the personnel certification schemes based on the european organization for quality (eoq). aec-cerper objetive find certified professionals in quality and environment. accreditation accredited by enac, according to the une en iso 17024 certification of persons, the national accreditation body (enac), providing 12 certificates in the fields of environmental multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 110 quality and the prl, food security and innovation. accreditation levels these certifications are divided into three levels (cerper 2011): coach manager and auditor. scope of international recognition national certification process the certification process consists of three phases: initial certification documentation phase evaluation phase evaluation all certification processes include audit process the request and both written and oral examination. cost 250-400 € multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 111 4.3. professional accreditation systems for engineers from the general council of industrial engineering (cogiti) has developed and implemented a procedure for accreditation of continuing professional development (cpd) for engineers. cogiti objetive the dpc patent and certify professional competence, consisting of training and professional experience throughout the professional life of an engineer. (dpc 2011) accreditation -- accreditation levels accreditation is structured in four levels: junior senior advance expertise the first three of which are divided into two sublayers (a and b), in order to achieve a greater distinction. scope of international recognition national certification dcp accreditation process comprises the following steps: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 112 process submission of documentation and initiation. rating and qualification records. dossier evaluation by the accreditation cogiti table. certificate delivery. accreditation audits. evaluation this certification process is to audit and verification of the curriculum presented. cost 25-150 € 4.4. professional accreditation systems for naval engineers the association of naval architects and ocean (coin) has a future plan for their professional certifications based, assessing the need for such certification be accredited by enac, since they allow eu recognition. coin objetive its mission is to ensure the quality of the work done in the field of naval engineering-and by extension in the maritime sector through adequate monitoring of vocational training of people working in this field. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 113 accreditation accredited by enac according to iso 17024 accreditation levels this plan establishes certification areas in which the professional can certify their skills, whether in one or more of these areas (lomo 2011). the system defines two levels of qualification: enabling professional generic: certified professional in one of three areas (design, production and repair). partial professional enabling: certifying professional competence in one of the established areas. these sub-areas are: basic design, prl in nautical industries, stability, hydrodynamics, naval electronics, ports, shipping, etc. scope of international recognition international certification process the certification process comprises: filing of the application to the president of the commission for access. evaluation by the entry fee of the professional activity of the candidate. issue report by the commission. opinion of the committee. if it is favorable, related multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 114 ratings, but information of training activities by the coin. evaluation audit application and written evaluation. cost 7.500-20.000 € 4.5. professional qualifications in the field of professional training, the national qualifications authority (incual) has been commissioned to develop the national catalogue of professional qualifications (rodríguez 2012), in view of the law 5/2002 of 19 june. fp objetive trainings aimed at insertion, update and labor reinsertion, whose main objective is to increase and adapt the knowledge and skills of current and future workers throughout life. accreditation incual (national qualifications) accreditation levels the national catalogue is divided into 26 professional and 5 levels of qualification families, according to the degree of knowledge, initiative and autonomy required to perform this work activity responsibility. each skill is divided into units of competence. as the unit of competency "minimum aggregate of professional skills recognition susceptible to partial accreditation" (rodriguez 2012). scope of national multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 115 international recognition certification process the certification process of people has left the institutes, centers and agencies of the corresponding autonomous communities and / or the office of state employment service (seps). evaluation audit application and written evaluation. cost 300-500 € / year 4.6. indicators after the review of the different certification systems of persons analyzed, are presented the 11 indicators to be used for comparison of the 5 certification systems studied (cardoza 2011). in table 4 are presented the indicators selected: table 4. indicators nº indicators legend description 1 accreditation system differentiates according to systems validation processes that have passed: 1: do not certified by any standards 2: accredited iso 17024 3: certificate iso 9001 and iso 17024 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 116 2 accreditation body identifies the agency that accredits the certification process 3 certification levels: number difference systems according to levels of certification: 1: set 2 levels of certification 2: sets 3 levels of certification 3: set 4 levels of certification 4: sets 5 levels of certification 4 levels of certification: accessibility unlike the systems according give access from one level to another certification: 1: it is not necessary to have a lower level for admission to a higher 2: you need to possess a certain level below to access higher levels 5 scope / recognition difference according recognition systems in the geographical area: 1: national 2: international 6 model requirements: experience establish levels of experience are requested to enter the certification process: 1: no experience required for any level of certification 2: minimum years of experience for some levels and lower levels are required is not mandatory 7 model requirements: training or education required sets the requirement for initial training or education to enter the certification process: 1: no initial training or education is required 2: training or education is required 8 certification process: grading system difference evaluation systems that are made: 1: audit request 2: audit of the application and written evaluation. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 117 9 as experience sets the unit of measure of experience that is required to access certain levels of certification: 1: no requests experience 2: experience is requested in years 3: experience is requested in number of projects 10 validity difference systems according to time expended validity of certificates: 1: certificates 1 and 3 years 2: certificates 3 and 5 years 3: certificates only 5 years 11 coste y mantenimiento difference systems according to the initial cost and maintenance of certificates expended: 1: less than 150 € 2: between 150-1000 € 3: superior to 1000 € 4.7. comparative analysis of certification systems each of the objects systems study was analyzed according to the indicators described above, obtaining the following comparative table, table 5. tabla 5. comparison of certification systems of persons. indicadores pmi aeccerper cogiti coin incual accreditation system 3 3 -2 1 (law) accreditation body ansi enac -enac incual multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 118 certification levels: number 2 2 3 1 4 levels of certification: accessibility 1 1 1 1 2 scope / recognition 2 1 1 2 2 model requirements: experience 2 2 2 2 1 model requirements: training or education required 2 2 2 2 2 certification process: grading system 2 2 1 2 2 as experience 2 2 2 3 1 validity 2 2 1 3 - cost and maintanance 2 2 1 3 3 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 119 5. conclusions analyzed and compared the five proposed systems, it is possible to identify a number existing commonalities between most of them despite being of certification of persons of different application areas and in different geographical areas. first, organizations and institutions take a special interest in taking as a framework for system certification iso 17024 certification of persons. (une-en iso/iec 17024 2012) as well as being accredited by a recognized certification company certifying compliance with this standard within their system because they recognize the value as it relates to transparency and reliability of the system for both the institution and the people who certify. secondly, the evaluation systems of most institutions consist of audit record, written and sometimes include an interview. so as to ensure that the person tested has the powers he seeks are recognized. another important factor common to pmi, cerper, coin and incual is to establish and clearly define the units of competency, so that certification is oriented to a specific field of knowledge (there different levels within each). in many cases the institutions develop standards that contain the knowledge to certify competence, such as the pmbok guide (pmbok 2013) in the pmi and can thus ensure homogeneity within the same group that owns a certification. finally, there is an evident need for external reference models for the development of new qualifications in the ehea, which accreditation systems could play a decisive role. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2197 social and technological sciences lama et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 101-120 | 120 references cardoza, a. (2011). comparación de cuatro sistemas de certificación del ámbito de la dirección de proyectos. xv congreso internacional de ingeniería de proyectos, 1(1), 411-428. cga-enac (2009). criterios generales de acreditación. competencia técnica de las entidades de certificación de personas. entidad nacional de acreditación. dcp (2011). normativa de acreditación de la formación continua y la experiencia profesional (dpc) de ingenieros técnicos industriales e ingenieros de grado de la rama industrial en españa, cogiti. lomo, l. (2011). coin plan de futuro de las instituciones. madrid, colegio oficial de ingenieros navales. pmbok (2013). guide to the project management body of knowledge, 5ª ed, pennsylvania, project management body of knowledge. cerper (2011). aec proceso de certificación de personas. asociación española para la calidad. rodríguez, r. (2012). incual catálogo nacional de cualificaciones profesionales. instituto nacional de las cualificaciones. tallo, d. (2009). la certificación de las personas en la empresa, un mecanismo para garantizar los criterios de competencias. 2(1), 22-24. torres, f. and abud, i. (2004). análisis mediante categorías universales de las competencias exigidas al ingeniero industrial por los organismos internacionales de acreditación. xii congreso universitario de innovación educativa en las enseñanzas técnicas, 1(1), 1048-1059. une-en iso/iec 17024 (2012). evaluación de conformidad, requisitos generales para los organismos que realizan certificación de personas. multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 1 teaching technical english: corrective feedback based on students’ and teachers’ beliefs oksana chugaia , olena ogienkob department of english for specific purposes. national technical university of ukraine “igor sikorsky kyiv polytechnic institute”, 37, peremohy ave., kyiv, 03056, ukraine. department of pedagogy. a. s. makarenko sumy state pedagogical university, 87, romenska st., sumy, 40002, ukraine. * correspondence: ochugai@meta.ua received: 13 july 2020; accepted: 10 march 2021; published: april 2021 abstract the present study investigates the beliefs of students and teachers about corrective feedback (cf) regarding necessity, timing and the most effective types in technical english setting. this study was conducted at the national technical university of ukraine “igor sikorsky kyiv polytechnic institute” with students majoring in information technology, mathematics or physics ranged between 18 and 22 years of age. methods included a questionnaire, interviews and classroom observations. the feedback from 55 students and 33 teachers was collected during practical classes in february – march at the beginning of the second semester of 2019-2020 academic year. this research has confirmed the necessity of cf for students and teachers. it also demonstrated the differences regarding timing: teachers preferred delayed to immediate cf, while students equally appreciated both. the research also indicated that both teachers and students preferred indirect types of cf; teachers had stronger attitudes though. the research demonstrated the most effective types of cf for students: meta-linguistic clue and repetition, while for teachers it was elicitation. completing the survey, both teachers and students chose at least two different types of cf. the necessity of effective cf, which requires learning more about students’ and teachers’ beliefs, benefits and drawbacks of each cf type, is emphasized. the results of the present research may be used by practitioners, who would like to use cf effectively in teaching technical english. future studies may explore cf in terms of students’ and teachers’ beliefs and behavior in different learning environments. keywords: corrective feedback (cf); recast; explicit correction; meta-linguistic clue to cite this article: chugai, o., ogienko, o. (2021). teaching technical english: corrective feedback based on students’ and teachers’ beliefs. multidisciplinary journal for education, social and technological sciences, 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0002-2118-8255 https://orcid.org/0000-0002-3089-6288 https://doi.org/10.4995/muse.2021.14016 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 2 1. introduction interactive nature of learning and teaching foreign languages suggests a two-way flow of information. while communicating, students experiment with a new language inevitably making errors, which makes teachers searching for the best strategy to provide corrective feedback (cf) aiming at activating students’ potential (scrivener, 2005, p. 298). being treated as indicators of student progress, errors provide valuable information teachers use to meet the needs of learners and adjust the english course accordingly. kartchava (2016) claims cf is necessary to provide in a language classroom setting, which implies distinguishing between types of errors and corresponding types of cf (p. 36). errors may be treated in different ways, therefore, various terms were developed by experts in assessment and practitioners. the term “error correction”, which used to be common in traditional pedagogy, means one way of responding to learner errors known as explicit correction. at the same time, the term “corrective feedback” covers various responses, which are classified in many different ways (li, 2018). considering active participation of students is impossible without involving them into the learning process by means of interaction and collaboration (lytovchenko et al., 2018, p. 74). learner-centeredness is beneficial for adult students in terms of real interaction, meeting students’ needs, making them more responsible (lytovchenko et al., 2020, p. 54). meeting the needs of learners is possible by trying the eclectic approach and choosing the methods that work in a particular environment (chugai et al, 2017, p. 76). feedback in general is connected with motivation and accuracy: positive feedback indicates that the learner’s response is correct, while negative feedback means that the answer is wrong. corrective feedback is considered to be negative; it aims at indicating an error and improving the language a learner used (ellis, 2009). cf may be simple or complex, online or offline; direct like recast, explicit correction, meta-linguistic clue, and indirect like repetition, clarification, elicitation and miming (li, 2013, p. 197). time constrains make cf even more complicated as it includes noticing an error and taking the decision about ignoring or correcting it. if a teacher decides that an error should be corrected, next steps are related to the agent (who is to correct an error), time (when it is better to correct it), priority (which error to correct) and strategies (how to correct it) (ellis, 2009). https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 3 multiple papers which focus on theoretical and practical aspects of cf (ellis, 2010; li, 2010), learners’ beliefs about cf (basturkmen, 2012; dornyei & ryan, 2015; horwitz, 2015; zhang & rahimi, 2014;), harmful effects of mismatch between students’ and teachers’ beliefs about cf (russel, 2009; horwitz, 1990), prove the importance of further investigations in this area. although some studies have considered comparison between students’ and teachers’ beliefs about cf (russel, 2009; horwitz, 1990; mori, 2002), none has focused on cf related to teaching technical english. the aim of the paper is to investigate the issues related to providing corrective feedback (cf) in technical english setting by means of comparing students’ and teachers’ perspectives. considering the design of this study, three research questions were formulated: 1. is cf necessary to provide? 2. what is the best timing for cf? 3. which type of cf strategy is more effective? 2. materials and methods 2.1. context and participants this study was conducted at the national technical university of ukraine “igor sikorsky kyiv polytechnic institute”. university students majoring in information technology, mathematics or physics ranged between 18 and 22 years of age lived in a foreign language environment in which they shared the same first language (ukrainian). english was the main language of instruction, but in some classes ukrainian was still used. there were few foreign students who knew at least two languages at the level of independent users. participants were not familiar with theoretical part related to cf strategies before. esp teachers were aware of cf concept and regularly practiced some of cf strategies during classes. corrective feedback instruction. the cf strategies were introduced to the students through teacher’s presentation and description in the handouts. seven types of cf were defined as follows: 1. recasts or reformulating the sentence keeping the same meaning; 2. repetition which means saying the part of the sentence which contains an error again; https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 4 3. clarification or making a request to indicate that something is wrong; 4. elicitation which means prompting a learner to identify and correct an error; 5. explicit correction which is about revealing the error and presenting the right variant; 6. meta-linguistic clue is providing extra information related to the error 7. miming involves facial expression, gestures and other kinds of body language with attempts to make a learner correct the error (li, 2018). in order to check students’ comprehension and provide another opportunity for them to learn more about cf, an activity on matching cf strategies with the examples, which were typed on separate cards, was suggested. for example, “elicitation” was to be matched with “his roles in films…. ?”, “explicit correction” with “not “made”, it is wrong, “have made” should be used instead”, etc. students worked in pairs and completed the matching by moving the cards. checking the results provided another opportunity for discussing the differences between some cf strategies, which were new for students. esp teachers conducted the same matching activity as students, but the aim was to familiarize teachers with the particular terminology used in the questionnaire considering the fact that there are different ways of classifying cf strategies. 2.2. data collection and analysis overall design. methods included a questionnaire, interviews and classroom observations with students and teachers of the national technical university of ukraine “igor sikorsky kyiv polytechnic institute”. the feedback from 55 students and 33 teachers was collected in february – march at the beginning of the second semester of 2019-2020 academic year. procedure. the questionnaire was administered to the students and teachers in the middle of the academic year. they were told that it was not a test, they did not have to identify themselves and the results would be treated with confidentiality. the participants were asked to think about their own experience and express their own opinion when responding to the questionnaire. the participants were told that they would be informed about the results of the study shared with an international community of esp teachers and learners. https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 5 questionnaire. the questionnaire written in english consisted of main questions, which were in the focus of this study: about the best timing for cf and the most effective type of cf strategy. there were three options for the best timing: during the activity, after the activity and at the end of the class. for the second question, the participants could choose some out of seven types of cf strategies by putting ticks in corresponding boxes. interviews. in order to compensate for the weaknesses of the questionnaire, the interviews in written form were conducted for the participants to express their opinions and add comments to explain their choices and attitudes towards cf strategies. observation. classroom observations were conducted in order to check how students reacted to different cf strategies used by teachers or their peers, which cf strategies were most common in observed esp classrooms. 3. results and discussion in spite of the fact that respondents had to tick choices in order to complete the questionnaire, some of them left their comments, ranked the choices by numbering, etc. technically, it was possible as pen and paper version was used. all the comments were taken into consideration as they provided valuable information related to personal views and opinions. research question 1. is cf necessary to provide? all students and teachers who completed the survey were sure that errors had to be noticed and corrected. those students who did not get effective feedback on their errors during the lesson, referred to supplementary literature like grammar books, online dictionaries, forums in order to find the explanations. students often approached a teacher and asked for clarification, which would help them to avoid an error in the future. research question 2. what is the best timing for cf? the results of the survey research show that the number of students who prefer being corrected during the activity (49%) is practically the same as the number of those who would like to be corrected after the activity (50%) (chugai, 2020). some students added comments that the best timing for cf depended on the subject: during the activity in mathematics, but after the activity in english (figure 1). https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 6 figure 1. students’ beliefs about the best timing for cf according to the survey, teachers preferred cf after the activity (70%) to cf during the activity (33%). some teachers chose “after the activity” but added comments that the best timing for cf depended on the activity and its aim. choosing doing cf “during the activity”, some teachers added that it is “on-the-spot comprehension of the mistake”. one response combined timing and different types of cf: recasts, repetition, clarification and elicitation were chosen the best to do during the activity, elicitation, explicit correction and meta-linguistic clue to do after the activity, recasts and explicit correction at the end of the class (figure 2). figure 2. teachers’ beliefs about the best timing for cf finally, more teachers (6%) than students (1%) thought it was better to correct errors at the end of the class (table 1). cf before the activity 49% cf after the activity 50% cf at the end of the class 1% students' beliefs cf during the activity 30% cf after the activity 64% cf at the end of the class 6% teachers' beliefs https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 7 table 1. students’ and teachers’ beliefs about best time to correct the error time for cf students teachers during the activity 49% 33% after the activity 50% 70% at the end of the class 1% 6% dividing cf into immediate (during the activity) and delayed (after the activity, at the end of the class), we conclude that overall students equally appreciate both types of cf, which differs from teachers who prefer delayed cf (76%) to immediate (33%). these results contrast to those obtained by li et.al (2016) which demonstrated an advantage of immediate cf. such result could be explained by intention of students to use the opportunity to apply immediately the knowledge, which was learned in class. research question 3. which type of cf strategy is more effective? considering direct (recast, explicit correction and meta-linguistic clue) and indirect cf types (repetition, clarification, elicitation and miming), the results of the research show that both students and teachers demonstrated preferences towards indirect cf. however, teachers had stronger attitudes (table 2). table 2. students’ and teachers’ beliefs about direct and indirect cf types of cf students teachers direct cf 43% 36% indirect cf 57% 63% the survey indicated that each respondent chose at least two ways of cf, one respondent chose all 7 types of cf. the results showed that meta-linguistic clue (19%) and repetition (18%) were chosen most frequently by students, then came explicit correction (16%) and miming (16%) less frequently chosen clarification (11%) and elicitation (11%), and recasts (8%) were the least frequently chosen (figure 3). https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 8 figure 3. students’ beliefs about the best timing for cf the most frequently chosen kind of cf by teachers was elicitation (21%), leaving all the others behind (figure 4). figure 4. teachers’ beliefs about the best timing for cf some teachers indicated their individual preferences by numbering the choices, for example: elicitation as the first choice, meta-linguistic clue as the second, and explicit correction as the third (table 3). 8 18 11 11 16 19 16 0 2 4 6 8 10 12 14 16 18 20 1 2 3 4 5 6 7 percentage of choices 1 recasts ; 2 repetition; 3 clarification; 4 elicitation; 5 explicit correction; 6 meta-linguistic clue; 7 miming which kind of cf is better? students' beliefs 14 15 14 21 10 12 14 0 5 10 15 20 25 1 2 3 4 5 6 7 percentage of choices 1 recasts ; 2 repetition; 3 clarification; 4 elicitation; 5 explicit correction; 6 meta-linguistic clue; 7 miming which kind of cf is better? teachers' beliefs https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 9 table 3. students and teachers beliefs about the best kind of cf types of cf students teachers 1 recasts 8% 14% 2 repetition 18% 15% 3 clarification 11% 14% 4 elicitation 11% 21% 5 explicit correction 16% 10% 6 meta-linguistic clue 19% 12% 7 miming 16% 14% these findings are in keeping with the studies conducted by schulz (2001), who claimed that learners and teachers had different beliefs about providing cf: the majority of students expected feedback on their spoken errors while prevailing number of teachers did not meet those expectations. however, the results of this study differ from those reported by yoshida (2008), brown (2009), kartchava (2016) who claimed that learners expected teachers to use prompts and recasts. kartchava (2016) writes about the respondents’ experiences as passive recipients being unaware of other education systems, which determine the preferable cf type (p. 34). thus, recast may be chosen because respondents were exposed to it before or because they would like to try it in the future, while prompts appeal to students because they call for students’ autonomy, interactive nature (kartchava, 2016, p. 32-33). according to the approach called “prompt-then-provide”, cf should be applied only when necessary and be used individually (lantolf, 2000). therefore, teacher correction should be preceded by self-correction (ellis, 2010). moreover, indirect cf like repetition, clarification, elicitation and miming, should prevail due to the fact that direct cf like recast, explicit correction and meta-linguistic clue maybe harmful for learner autonomy (li, 2013, p. 197). such incentives as participants’ experience in general, their previous beliefs, their own learning experience when they encountered a particular cf may influence the results of the research (schulz, 2001; loewen et al., 2009). the results need to be interpreted with caution taking into the account some limitations like new terms for some participants, which they could not fully https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 10 understand. despite these drawbacks, the results of this research represent real tendencies in students’ and teachers’ beliefs. 4. conclusions this research has confirmed the necessity of cf for students and teachers. at the same time, it demonstrated the differences in timing preferences: students expressed the same preferences about being corrected before and during activities, while teachers preferred doing cf after activities. in addition, more teachers than students thought it was possible to do cf at the end of the lesson. overall, students equally appreciated both types of cf, which differs from teachers who prefer delayed to immediate cf. the research also indicated that both teachers and students preferred indirect types of cf; teachers had stronger attitudes though. the research also demonstrated that meta-linguistic clue and repetition were the most frequently chosen by students, while for teachers it was elicitation. completing the survey, both teachers and students chose at least two different types of cf. in conclusion, effective cf suggests learning more about students’ and teachers’ beliefs, benefits and drawbacks of each cf type. the results of the present research may be used by practitioners, who would like to use cf effectively in teaching technical english. future studies may explore cf in terms of students’ and teachers’ beliefs and behavior in different learning environments. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references basturkmen h. (2012). review of research into the correspondence between language teachers’ stated beliefs and practices. system, 40(2), 282–295. http://dx.doi.org/10.1016/j.system.2012.05.001 brown, a. (2009). students` and teachers` perceptions of effective foreign language teaching: a comparison of ideals. the modern language journal, 93(1), 46-60. http://dx.doi.org/10.1111/j.1540-4781.2009.00827.x https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://dx.doi.org/10.1016/j.system.2012.05.001 http://dx.doi.org/10.1111/j.1540-4781.2009.00827.x multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 11 chugai, o. (2020). providing effective corrective feedback in esp class. proceedings of the ii annual conference on current foreign languages teaching issues in higher education, 90–91. https://kamts2.kpi.ua/sites/default/files/img/i%d1%80%d0%bd%d0%b8%d0%ba%20 14.05.2020%281%29.pdf chugai, o., terenko, o., & ogienko, o. (2017). methods that work: best practices of adult educators in the usa. advanced education, 8, 72-77. https://doi.org/10.20535/24108286.109216 dornyei, z., ryan, s. (2015). the psychology of the language learner revisited. routledge. https://doi.org/10.4324/9781315779553 ellis, r. (2009). corrective feedback and teacher development. l2 journal, 1(1), 3–18. https://repositories.cdlib.org/uccllt/l2/vol1/iss1/art2/ ellis, r. (2010). cognitive, social, and psychological dimensions of corrective feedback. in r. batstone (ed.), sociocognitive perspectives on language use and language learning (pp. 151–165). oxford: oxford university press. horwitz, e.k. (1990). attending to the affective domain in the foreign language classroom. in s.magnan (ed.), shifting the instructional focus to the learners (pp. 15–33). middlebury, vt: northeast conference on the teaching of foreign languages. horwitz, e.k. (2015). beliefs about language learning and the experience of second language learning: asking useful questions about language learning. in h.m. mcgarrell and d.wood (eds.), contact-refereed proceedings of tesl ontario research symposium, 41(2), (pp. 20–30). kartchava, e. (2016). learners’ beliefs about corrective feedback in the language classroom: perspectives from two international contexts. tesl canada journal 33(2), 19–45. https://doi.org/10.18806/tesl.v33i2.1235 lantolf, j. (2000). sociocultural theory and second language learning. oxford university press. li, s. (2013). oral corrective feedback. elt journal, 68(2), 196–198. https://doi.org/10.1093/elt/cct076 li, s. (2018). corrective feedback in l2 speech production. the tesol encyclopedia of english language teaching, (1st ed.). https://doi.org/10.1002/9781118784235.eelt0247 li, s., zhu, y., & ellis, r. (2016). the effects of the timing of corrective feedback on the acquisition of a new linguistic structure. modern language journal, 100, 276–295. https://doi.org/10.1111/modl.12315 loewen, s., li, s., fei, f., thompson, a., nakatsukasa, k., ahn, s., & chen, x. (2009). second language learners’ beliefs about grammar instruction and error correction. the modern language journal, 93, 91-104. https://doi.org/10.1111/j.1540-4781.2009.00830.x lytovchenko, i., lavrysh, y., lukianenko, v., & o. ogienko, o. (2020). how to teach grammar to adult esp learners at technical university more communicatively: task-based https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.20535/2410-8286.109216 https://doi.org/10.20535/2410-8286.109216 https://doi.org/10.4324/9781315779553 https://repositories.cdlib.org/uccllt/l2/vol1/iss1/art2/ https://doi.org/10.1093/elt/cct076 https://doi.org/10.1002/9781118784235.eelt0247 https://doi.org/10.1111/modl.12315 https://doi.org/10.1111/j.1540-4781.2009.00830.x multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 chugai and ogienko (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 1-12. https://doi.org/10.4995/muse.2021.14016 12 approach. multidisciplinary journal for education, social and technological sciences, 7(1), 54-71. https://doi.org/10.4995/muse.2020.12419. lytovchenko, i., ogienko, o., sbruieva, a., & sotska, h. (2018). teaching english for specific purposes to adult learners at university: methods that work. advanced education, 10, 69– 75. https://doi.org/10.20535/2410-8286.149741 mori, r. (2002). teachers’ beliefs and corrective feedback. jalt journal, 24(1), 48–69. https://doi.org/10.37546/jaltjj24.1-3 russel, v. (2009). corrective feedback, over a decade of research since lyster & ranta (1997): where do we stand today? electronic journal of foreign language teaching, 6(1), 21–31. https://e-flt.nus.edu.sg/archive/v6n12009.html schulz, r. a. (2001). cultural differences in student and teacher perceptions concerning the role of grammar instruction and corrective feedback: usa – columbia. the modern language journal, 85(2), 244-258. https://doi.org/10.1111/0026-7902.00107 scrivener, j. (2005). learning teaching, a guide for guidebook for english language teachers. 2nd ed. macmillan education. yoshida, r. (2008). teachers’ choice and learners’ preference for corrective-feedback types. language awareness, 17(1), 78-93. https://doi.org/10.2167/la429.0 zhang, l. j., rahimi, m. (2014). efl learners’ anxiety level and their beliefs about corrective feedback in oral communication classes. system, 42, 429–439. https://doi.org/10.1016/j.system.2014.01.012 https://doi.org/10.4995/muse.2021.14016 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4995/muse.2020.12419 https://doi.org/10.20535/2410-8286.149741 https://doi.org/10.37546/jaltjj24.1-3 https://doi.org/10.1111/0026-7902.00107 https://doi.org/10.2167/la429.0 https://doi.org/10.1016/j.system.2014.01.012 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 keeping teachers up-to-date is essential to reach and maintain a high quality university education l. aguilar-santelises1, 3, m.t. corona-ortega1, m. cruz-millán1, a.g. rojas-fernández1, m. aguilar-santelises2, a. garcía-del valle1 1national autonomous university and 2national polytechnique institute, mexico. 3 corresponding author: email: lsante02@yahoo.com.mx; faculty for higher education (fes) zaragoza, unam. batalla del 5 de mayo s/n col. ejército de oriente, iztapalapa, 09230, méxico city. phone: +52 5556 23 07 93 received: 2013-12-31; accepted: 2014-04-14 abstract a multidisciplinary group of teachers and scientists from the faculty for higher education (fes) zaragoza from the national autonomous university of mexico (unam) is dedicated to the continuous improvement of biochemistry teaching within the programme of chemistry, pharmacy and biology (qfb) at our university. to provide our teachers with modern teaching abilities is our top priority. therefore, we offer various activities to promote their continuous scientific and didactic updating. we also encourage self-awareness on the strong need for continuous adaptation and utilization of information and communication technologies (ict) that have become very popular among young generations. teaching is not an isolated discipline. on the contrary, teaching is intimately related with social and economic circumstances. teachers and students must be aware of their social environment, continuously improving their capability to approach national problems with an accurate utilization of national resources. research and development have to be linked to the best teaching techniques that stimulate teachers and scientists to grow as educators and experimenters. our scientific meetings and courses on scientific topics and teaching techniques have contributed not only to promote individual teaching and scientific capabilities but also to update the study programme from the qfb curricula at our university. keywords biochemistry, ict, teaching updating aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 1 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 1. introduction a strong knowledge of biochemistry provides the right foundations for a good education of students following the chemistry, pharmacy and biology (qfb) curricula at any university. such kind of instruction requires in turn, professional teachers with a high level of theoretical knowledge and practical experience in science and education. to fulfil their mission and vision, modern universities require good educators, acting with passion and reason, with ability to act as role models, leaders and colleagues in order to develop proper and rewarding students-faculty interactions. those who wish to become teachers at primary or intermediate level must follow special education in specialized schools before they can actually work as teachers. however, no such demand for a special preparation exists to become a teacher at university level. instead, university teachers must have a degree in the specialty that they are going to teach and prove that they possess the knowledge and the skills to teach it by being approved in an examination made by representatives from the academy, school directives and the union. professionals following this path in mexico acquire the right to teach after succeeding on such examination and their efficiency as teachers is guaranteed by their experience in the field and the time they spend acting as educators. however, modern education demands professionals critically aware of their own capacities, able to recognize valuable information, to learn new didactic strategies and evolve, improving their own abilities to continue a successful interaction with their students. it is not enough to obtain a degree and work for a number of years teaching what they knew when they obtained a degree. modern societies change continuously, and education must also change, adapting to present needs and resources. according to unesco, there is an unprecedented demand for a great diversification in higher education, as well as an increased awareness of its vital importance for sociocultural and economic development (unesco 1998 and 2008). in order to satisfy aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 2 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 such demand, universities must employ well-prepared educators and life-long learners with a deep knowledge of the subject that they teach, and the skills to teach it. therefore, we have been devoted during several years to provide the means to elevate and maintain a high level of scientific knowledge and didactic competence from qfb teachers at our university. various activities have been scheduled to achieve these goals, including a number of courses, discussion meetings and support activities for students and teachers from the faculty for higher education (fes) zaragoza from the national autonomous university of mexico (unam). 1.1 updating knowledge an elevated quality of education is required to produce the critically thinking, creative, flexible and entrepreneurial young professionals who will shape our future. the quality of teaching and learning is vital for countries aiming to face modern challenges with success. moreover, education should focus on curricula that deliver relevant, up-to-date knowledge and skills allowing the new professionals to successfully compete in the labour market (high level group on the modernization of higher education 2013). teachers cannot accomplish their function if they are satisfied with the knowledge acquired at the time they graduated. on the contrary, they must constantly renovate and improve their knowledge. nowadays, knowledge is everywhere, easy to get, and teachers must frequently evaluate the new information. teachers must also be aware that the purpose of teacher-student interactions at university level is not only to set the basis for exchange of information but also to provide a suitable start for students that must become professionals with problem solving capability. teachers must show their students how to perform critical thinking and to analyse great amounts of information before they adopt or reject some of that information. the experience of learning from well-prepared university teachers is crucial to make students to consider their school time as a motivating and rewarding period. aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 3 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 which ones are the most important capabilities of university teachers? firstly, they must have high levels of expertise and subject knowledge. secondly, they must possess effective teaching strategies to transmit relevant information. thirdly, they must appear not only as authority figures but also as educators. science, technology and pedagogy become all together essential components of the processes of teaching and learning. teaching at university level demands a great capacity to adapt and face contemporary challenges. therefore, continuous updating of knowledge is the first step to reach a position where teachers may act as a positive influence for their students (romeo et al. 2012). unfortunately, the quality of universities is often overlooked and undervalued (high level group on the modernization of higher education 2013). universities must struggle to adopt efficient mechanisms for reaching and maintaining the high level of education that our society demands. teachers must be subject of a never-ending professional development, and they should always have an active role in their own process of learning and teaching (bushati et al. 2013). 1.2 updating communication skills the competence of university teachers goes along their willingness to communicate their knowledge. transmission of information often also conveys thoughts and emotions to the receivers. in order to effectively communicate, we have to take advantage of every didactic tool available to transmit our knowledge and promote development of our students. understanding how is that our students understand the knowledge that we provide is a key factor to achieve our goals (felder and brent 2005). sharing information with the general purpose of education and specific purposes oriented to particular topics is easier when we have developed a personality that gives the impression of being both knowledgeable and approachable. to achieve this kind of personality, university teachers must be well informed and experienced but also have to aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 4 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 develop effective communication skills (high level group on the modernization of higher education 2013, ihmeideh et al. 2010). to listen and to be able to interact with students is as important as having good theoretical and practical knowledge. university teachers must constantly gather, analyse and sort relevant information. then, they should be able to present it as easy to assimilate information. to build up skills for effective communication is an important task and university teachers appreciate all the help they can get in form of courses, lectures and meetings with their fellow teachers in and out from their own university. to increase the potential to communicate among university teachers is as important as having well prepared, knowledgeable and experienced teachers. such kind of personnel constitutes the basis to sustain a high level of education at any university. 1.3 forming professionals academic and scientific professionals require an excellent basic formation, continuous update and development. forming professionals must be understood as a limitless, wide and complete activity for educators. teachers and universities are responsible to warrant that the person obtaining a university degree has the corresponding knowledge, experience and merits to function as a professional. then, the graduated professional is responsible for keeping himself able to perform at the expected level and to increase continuously his/her knowledge and aptitudes. forming professionals with the ability to take right decisions is probably the most important goal from universities working to get the best from the individuals in benefit of the society (ferreira et al. 2014). in mexico, the national association of universities and high education institutions (anuies) has been determined to find out the most efficacious ways to constantly improve the formation of professionals. among other strategies, anuies is promoting a better use of information and communication technologies (ict) to reinforce the acquisition and transmission of knowledge at university level. although must youngsters aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 5 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 are nowadays acquainted with digital cameras and phones, not enough students (and teachers) have a good knowledge and utilization of ict. excessive use of electronic means often deteriorates students’ skills for learning, when they just copy and paste instead of read and analyse. taking a photograph instead of trying to understand the topic is another detrimental activity. many students even lose their abilities to read and write as well as to make basic calculations, since they prefer to copy, to take a photograph or to use a computer instead of using their own brain. the extensive use of videogames and electronic devices often undermines mental abilities instead of improving them for learning “real life” issues. well used, radio, tv, internet and all kind of electronic devices are fantastic means for learning and stimulating the brain function but they are too often unduly used, replacing cognitive and physical functions and leading to mental disabilities (drain et al. 2012). in consequence, and despite of extensive nets of communication in the country, electronic means are still not fully exploited with educational purposes. 1.4 teaching and searching collaboration between educators and scientists is critical to maintain a high level of education at any university. more often than seldom, teachers at our university are also involved in research activities of various kinds, from basic to translational science. teaching and searching are complementary activities that benefit from each other, summarizing knowledge, solving problems and discovering many more questions that need to be solved. interest and curiosity from teachers and learners grow as result of being permanent or temporarily associated with adequate research environments. countries are nowadays as developed and independent as they have invested in scientific research and high-level education. mexico has understood that and it is trying to catch up through research and development in education. universities are demanding higher levels of knowledge and promoting collaboration between multidisciplinary groups of teachers and scientists, in and out the academic institutions. at fes zaragoza, we have an aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 6 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 institutional program giving high priority to good organization and proper use of resources. accordingly, to develop collaboration between teachers and scientists is an important goal of this project. linking research and teaching stimulates teachers and researchers with new study fields that help to maintain both teaching and searching as critical, demanding and motivating activities. a combined teaching and research interest also provides feedback for educators and scientists. giving the students the possibility to participate early in scientific projects helps them to reinforce the importance of their professional formation and to direct their interests to particular fields of knowledge. educators who are simultaneously devoted to pedagogic and scientific activities also need institutional support and guidance. teaching and searching give structure to the institution. universities having both activities in the same group of individuals, elevate the quality of the institution, renovate teaching and help their academic personnel to keep the highest possible level. this is a phenomenon that has marked the universities in modern times, making a great difference between those institutions that evolve positively and those, which do not do it. 2. objectives and methodology • to increase awareness on the need for continuous updating of knowledge and skills from biochemistry teachers at the faculty for higher education (fes) zaragoza from the national autonomous university of mexico (unam). • to examine and evaluate the program of cell and tissue biochemistry at the fes zaragoza in order to identify areas requiring modifications and modernization. • to integrate multidisciplinary groups actively working for exchange of information with renowned scientists and teachers experienced on biochemistry and education. aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 7 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 • to design and offer courses, workshops and discussion meetings with focus on modern molecular biology and technology. • to obtain additional didactic resources and make them available to our community. joined efforts of multidisciplinary groups are required to approach the vast number of technological and scientific advances that exist today. academic groups with various specialties must work together, gaining information through combined efforts to maintain a high scientific and educational level. biochemistry requires continuous experimentation to confirm, rectify and evolve before a concept can be considered established. new methodologies are important to maintain a high level of experimentation. newly formed professionals need to master established technologies as much as new ones. cell biology has made an enormous progress during the last century but there are even more immense amounts of knowledge missing in cell and molecular biology, pharmacology, medicine, phytochemistry, nanotechnology, genomics, proteomics and others. we started with workshops to analyse the content of the program of cell and tissue biochemistry at fes zaragoza. then, we detected topics and laboratory sessions that were missing or old-fashioned. our laboratory had sessions teaching basic techniques but missed modern techniques to study dna and proteins. feasible modifications were selected in order to renovate our practical sessions, with diminished production of chemical and biological waste (table 1). discussion meetings with scientists from our own and other universities were essential to develop new techniques and sessions to be included in the modified program. aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 8 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 table 1. cell and tissue biochemistry laboratory laboratory session former techniques additional techniques instruments 1. analytical scale 2. clinical centrifuge 3. spectrophotometer 4. potentiometer 1. micropipettes 2. biophotometer 3. gel electrophoresis of proteins, dna and rna 4. elisa 5. nucleic acids extraction. 6. nucleic acids purification and quantification 6. cdna and pcr proteins 1. standard curves lowry biuret 1. microscale techniques 2. analysis of plasma proteins gamma globulin precipitation protein quantification immunoelectrophoresis immunoprecipitation carbohydrates 1. demonstrative extraction of rat liver 2. identification of polysaccharides with lugol 3. hydrolysis and identification of monosaccharides with benedict 1. extraction of rat liver 2. identification of polysaccharides and monosaccharides lipids 1. quantification of phosphorus by spectrophotometry 1. microscale techniques 2. phosphorus content in phospholipids 3. thin layer chromatography dna 1. dna extraction from wheat germ 1. dna extraction, purification and quantitation from human leukocytes 2. electrophoresis of normal dna 3. electrophoresis of fragmented dna mitosis 1. observation of acetocarmin stained nuclei 1. technique explanation 2. description of mitotic phases 3. microscopic and digital observation of stained nuclei from mitotic onion cells enzymes 1. observation of enzyme activity starch degradation by salivary amylase effect of temperature and heavy metals on salivary amylase activity 1. determination of enzyme activity optimal invertase activity 2. enzyme properties invertase activity under various ph and temperature conditions invertase activity in relation to substrate and enzyme concentration. plotting enzyme activity determination of velocity curves km constant results and discussion aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 9 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 3. results and discussion we have organized courses in experimental biochemistry, with particular attention to the detection, analysis and quantification of proteins, lipids and carbohydrates, enzyme pharmacokinetics, elisa, western blot, cell culture, cytometry and apoptosis. we also introduced practical sessions of molecular biology where students at pre-graduated level learn to study nucleic acids with pcr and various other techniques (table 1). ict have also been studied and adopted by our teachers and students in the middle stage of their studies, learning to elaborate and use digital materials, as well as exploiting all kinds of possibilities of using electronic materials, digital graphs, posters, presentations and other resources such as electronic files and platforms (table 2). table 2. continuous education activities courses & workshops discussion meetings continuous support 1. biochemistry cells, eukaryotes and prokaryotes cell structure organelles and function molecules enzyme kinetics cell biochemistry proteins carbohydrates lipids antigen-antibody reaction (ag-ab) optical microscopy uv microscopy cytometry samples with different cell types 2. experimental biochemistry spectrophotometers standard curves protein quantification lipid quantification enzyme kinetics biomolecules structure and function methods to study biomolecules ag-ab clinical applications clinical drug development liposomes microorganisms, environment and industry samples and reagents assistance for developing good laboratory practices (glp) and use of equipment 3. microscale techniques proteins, lipids, carbohydrates and nucleic acids. basics of microscales technique applications equipment, reagents assistance for developing glp aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 10 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 table 2. continuous education activities courses & workshops discussion meetings continuous support 4. elisa methods, practical work antibodies, antigens, enzymes antibodies and antigens detection clinical and research applications results and discussion elisa methods sensitivity, reproducibility, research and clinical applications cell and molecules separation agglutination and precipitation immunoprecipitation immunofluorescence and cytometry equipment and reagents assistance for developing glp 5. western blot sample, cell sources and purpose denaturation, solubilisation, protein determination basic set-up. sds-page, transfer and detection enzymes and chemoluminiscence antibodies, antigens and the ag-ab reaction wb clinical and research applications power blot and other alternatives equipment and reagents assistance for developing glp 6. human dna human genetics forensic genetics genes and sex molecular biology techniques pcr applications apoptosis assistance for better utilisation of molecular biology techniques and study of human dna 7. information and communication technologies (ict) digital tools demonstration microsoft office package information systematization chemsketch for creation and handling of 3d chemical structures prezi, calameo, blogs electronic presentations graphs, drawings, texts and posters chemical structures presentations, data bases, analysis and publication of results 8. tic. advanced course digital tools training electronic files and mail posters (creation and editing) digital material (power point and prezi) digital platforms (moodle) learning management system (lms) platform moodle on line courses blogs, discussion sites electronic publications reusable digital materials assistance access to platforms and blogs assistance for preparation of digital material discussion meetings have been organized with the purpose of bringing updated information into our program. scientists and teachers have come to discuss their work in biochemistry, molecular biology, pharmacology, genetics and immunology. as a consequence, a number of students have got the opportunity to work during several weeks between semesters in research laboratories around the city. this experience is aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 11 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 called “youngsters towards investigations” and has allowed many students to get in contact with several groups of investigators in various hospitals and institutions, participating with small but rewarding tasks which have produced the most positive impact in our future professionals. limitations that we encountered during our work were due to lack of time (from teachers) and resources (from the institution). however, this was compensated by enthusiastic collaboration and interest shown by all the participants (institution, teachers and students). we are indebted to all scientists, educators and administrative personnel that made our work possible and contributed to positive results. our contributions to elevate the standard of education at the fes zaragoza can be summarized as follows: • increased awareness of the necessity for constant improvement of knowledge and didactic capacity. • integration of a multidisciplinary group that provides multiple forums for exchange of information with distinguished scientists and educators. • increased availability of ict and technical support to use it. • teachers have been updated in various topics of biochemistry, cell biology and molecular biology. • teachers have shown personal commitment to assiduous learning and education. more than 80 % of teachers working within the program have participated, forwarded their interests and learned diverse techniques that have widened their professional capacity as scientist and educators. • teaching and searching are closer than before and most teachers participate somehow in both activities. aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 12 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 • our activities promoting up-to-date knowledge and communication based on tight collaboration between science and education are contributing to improve the level of education of our students in an extent that will be very much appreciated afterwards, when they act as professionals, problem solvers, educators and scientists. 4. acknowledgements this project was supported by unam (dgapa papime pe206913). the authors gratefully acknowledge dr. arnoldo aquino galvez and ms maria lourdes vega navarrete for their kind collaboration. we also thank the students araceli carrillo, sergio albarran, fabian serrano and margarita aguilar for their enthusiastic participation. 5. references bushati s, ninka i, bushi f. (2013). assimilation techniques and technological methods, implements in the teaching and learning process. mediterranean j soc. sci. 4: 763–767. drain ts, grier le, sun w. (2012). is the growing use of electronic devices of benefit for academic performance? issues inform systems 13: 225-231. felder rm, brent r. (2005). understanding student differences. j eng. edu. 94: 57-72. ferreira n, haddad m, faria a. (2014). educational technology and educational management in the higher education: new ways of forming professionals. open j soc. sci. 2: 7-11. high level group on the modernisation of higher education. (2013) report to the european commission on improving the quality of teaching and learning in europe’s higher education institutions. available at modernisation of higher education european commission europa accessed on july 8, 2014. aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 13 http://www.google.se/url?sa=t&rct=j&q=&esrc=s&source=web&cd=27&ved=0cfoqfjagobq&url=http%3a%2f%2fec.europa.eu%2feducation%2flibrary%2freports%2fmodernisation_en.pdf&ei=yvwmu5arfsaiyah0-ydocg&usg=afqjcngomunq_f-ifyqci_5usl9m893gyg http://www.google.se/url?sa=t&rct=j&q=&esrc=s&source=web&cd=27&ved=0cfoqfjagobq&url=http%3a%2f%2fec.europa.eu%2feducation%2flibrary%2freports%2fmodernisation_en.pdf&ei=yvwmu5arfsaiyah0-ydocg&usg=afqjcngomunq_f-ifyqci_5usl9m893gyg multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2204 social and technological sciences eissn: 2341-2593 ihmeideh fm, al-omari aa, al-dababneh ka. (2010). attitudes towards communication skills among students-teachers in jordanian public universities. australian j teacher edu. 35: 1-12. romeo g, lloyd m, downes t. (2012). teaching teachers for the future: building the ict in education capacity of the next generation of teachers in australia. australasian j edu. technol. 28: 949-964. unesco (2008). ict competency standards for teachers. available at http://unesdoc.unesco.org/images/0015/001562/156207e.pdf accessed on july 8, 2014. unesco (1998). world declaration on higher education for the twenty-first century: vision and action. available at http://www.unesco.org/education/educprog/wche/declaration_eng.htm accessed on july 8, 2014. aguilar-santelises et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 1-14 | 14 http://unesdoc.unesco.org/images/0015/001562/156207e.pdf http://www.unesco.org/education/educprog/wche/declaration_eng.htm multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 30 development of lower-limb rehabilitation exercises using 3-prs parallel robot and dynamic movement primitives rafael j. escarabajal1 , fares j. abu-dakka 2 , josé l. pulloquinga1 , vicente mata3 , marina vallés1 , ángel valera1 1instituto de automática e informática industrial, universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain 2department of electrical engineering and automation (eea), aalto university, espoo, finland 3centro de investigación de ingeniería mecánica (ciim). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding author: rafael j. escarabajal, e-mail address: raessan2@upv.es received: 03 june 2020; accepted: 04 september 2020; published: october 2020 abstract the design of rehabilitation exercises applied to sprained ankles requires extreme caution, regarding the trajectories and the speed of the movements that will affect the patient. this paper presents a technique that allows a 3-prs parallel robot to control such exercises, consisting of dorsi/plantar flexion and inversion/eversion ankle movements. the work includes a position control scheme for the parallel robot in order to follow a reference trajectory for each limb with the possibility of stopping the exercise in mid-execution without control loss. this stop may be motivated by the forces that the robot applies to the patient, acting like an alarm mechanism. the procedure introduced here is based on dynamic movement primitives (dmps). keywords: parallel robot, rehabilitation robot, dynamic movement primitives, position control https://creativecommons.org/licenses/by-nc-nd/4.0/ mailto:raessan2@upv.es https://orcid.org/0000-0003-1061-6039 https://orcid.org/0000-0003-1061-6039 https://orcid.org/0000-0003-1061-6039 https://orcid.org/0000-0003-1061-6039 https://orcid.org/0000-0003-1061-6039 https://orcid.org/0000-0003-1061-6039 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 31 1. introduction parallel robots are made up of closed kinematic chains that connect a fixed platform with a mobile one, which usually includes the end effector. comparing to a serial robot, a parallel robot (pr) has better precision and dynamic performance, providing many applications such as rehabilitation, manufacturing, etc. (patel & george, 2012). in this paper, we focus on medical rehabilitation applications, which are intended to aid injured patients. many applications of robotics for rehabilitation can be found in (xie, 2016), and lower limb rehabilitation devices are discussed in (díaz et al., 2011). this subfield includes gait trainers (hesse & uhlenbrock, 2000; reinkensmeyer et al., 2006) and ankle rehabilitation (dai et al., 2004; liu et al., 2006; sui et al., 2009). in the context of sprained ankle rehabilitation, the aim of the robot is to imitate and adjust the exercises done by the patient. control strategies for this kind of applications can be found in (saglia et al., 2013; tsoi et al., 2009). many problems related to singular configurations can be found during the design of a pr (gosselin & angeles, 1990), so techniques like robot limb reconfiguration are arising in order to widen the range of movements required to run the exercise (yoon et al., 2006). in this paper, a 3 dof pr is proposed to carry out the desired movements for the ankle; the first degree of freedom allows dorsiflexion/plantarflexion movements, the second permits inversion/eversion (brockett & chapman, 2016), and the third is translational and allows the physical ascent and descent of the limb according to the height of the patient. in addition to the suitable kinematic and dynamic model of the robot, which is described in the section 2, a position control system based on dynamic movement primitives (dmps) is provided (ijspeert et al., 2013; schaal, 2006). this approach converts the time variable into a manipulable variable called phase, and its application cover several intelligent robotics applications, such us humanoids robots (ijspeert et al., 2002) and biped locomotion (nakanishi et al., 2004). dmps encode the time-dependent reference trajectories to a phase-dependent and nonlinear system which is the actually followed. the phase variable can be manipulated to slow down and adapt the exercise or simply stop it, providing more flexibility, and so there is no need to accurately design a trajectory for every situation. conversely, having a finite set of trajectories and adapting them according to some parameters tuned offline or online is viable, thus generating a training and https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 32 learning environment (nemec & ude, 2012). this can be customized to each patient with the help of medical personnel. the dmp system is an approximation that can be optimized using locally weighted regression (atkeson et al., 1997) or gaussian process regression (fanger et al., 2016). specifically, dmps are used in this paper with two purposes: 1) to encode the predefined references in the cartesian space by the medical personnel, and 2) to stop the execution of an unfinished exercise keeping the control of the robot in the final position, and being able to resume the exercise from the last point if needed. this application emulates an alarm mechanism. the paper is structured as follows: section 2 presents the kinematic and dynamic model of the robot, section 3 describes the dmp system regarding the trajectory generation and phase stopping, section 4 includes the control scheme with the proposed approach, section 5 presents an application of the system on a lower-limb rehabilitation exercise, and section 6 expounds the conclusions. 2. kinematic and dynamic model of the 3 dof parellel robot the rehabilitation exercises are performed in a 3 dof pr (see figure 1a). it was developed at universitat politècnica de valència (vallés et al., 2012), and consists of three kinematic chains (legs) which enable two angular rotations (roll and pitch) and a linear motion (heave). each limb has a ball screw actuator as a prismatic joint (p) and an intermediate coupling bar to connect to the mobile platform. the bar is connected to the actuator with a revolute joint (r), and to the mobile platform with a spherical joint (s). the limbs are arranged in equilateral triangular configuration (figure 1b). figure 1. (a) the 3-prs parallel robot; (b) kinematic diagram, joints and generalized coordinates. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 33 the active (i.e., actuated) generalized coordinates 𝑞𝑞1, 𝑞𝑞6 and 𝑞𝑞8 correspond to the prismatic joints; 𝑞𝑞2, 𝑞𝑞7 and 𝑞𝑞9 are the passive generalized coordinates associated with the revolute joints (r), and the coordinates 𝑞𝑞3, 𝑞𝑞4 and 𝑞𝑞5 represent the spherical joint (s). the inverse kinematic model to get the actuated coordinates is obtained by applying denavit-hartenberg parameters based on paul’s notation (paul, 1981) and geometric constraints. the forward kinematic model uses the newtonraphson method to get q2, q7 and q9, for a given position and orientation of the mobile platform: � 𝑞𝑞2 𝑞𝑞7 𝑞𝑞9 � 𝑖𝑖+1 = � 𝑞𝑞2 𝑞𝑞7 𝑞𝑞9 � 𝑖𝑖 − 𝐽𝐽𝑖𝑖 −1 � 𝑓𝑓1(𝑞𝑞2, 𝑞𝑞7) 𝑓𝑓2(𝑞𝑞2, 𝑞𝑞9) 𝑓𝑓3(𝑞𝑞7, 𝑞𝑞9) � 𝑖𝑖 (1) in the equation (1), 𝐽𝐽𝑖𝑖 is the jacobian matrix of 𝑓𝑓𝑖𝑖 with respect to the variables [𝑞𝑞2 𝑞𝑞7 𝑞𝑞9], and 𝑓𝑓𝑖𝑖 considers the geometrical constraints among the variables (vallés et al., 2012). regarding the pr dynamics, when independent generalized coordinates are used to model the pr, the equation of motion can be expressed as follows, 𝑀𝑀(�⃗�𝑞)�⃗̈�𝑞 + 𝐶𝐶��⃗�𝑞, �⃗̇�𝑞��⃗̇�𝑞 + �⃗�𝐺(�⃗�𝑞) = 𝜏𝜏 (2) where 𝑀𝑀 is the system mass matrix, 𝐶𝐶 is the matrix which includes the centrifugal and coriolis terms, �⃗�𝐺 is a vector that contains the gravitational terms and 𝜏𝜏 is the vector of generalized forces. an exhaustive dynamic model identification of the robot can be found in (díaz-rodríguez et al., 2010). 3. dynamic movement primitives this section describes the two utilities covered by the dmp system, as mentioned earlier: firstly, the reference trajectory generation in the phase-domain from the time-domain, which is provided by the physiotherapist, and secondly, the possibility of stopping the exercise in mid-execution. 3.1. trajectory adaptation using dmp the trajectories designed by the medical personnel for the exercises are periodic, so a rhythmic dmp variant suits well for this problem, which encodes the trajectories by means of the following nonlinear equations (a. j. ijspeert et al., 2013): https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 34 𝜏𝜏�̇�𝑧 = 𝛼𝛼𝑧𝑧(𝛽𝛽𝑧𝑧(𝑔𝑔 − 𝑦𝑦) − 𝑧𝑧) + 𝑓𝑓(𝜙𝜙, 𝑟𝑟) (3) 𝜏𝜏�̇�𝑦 = 𝑧𝑧 (4) 𝜏𝜏�̇�𝜙 = 1 (5) the basic idea behind these equations consists of building a stable dynamical system which is modulated by means of nonlinear subsystems 𝑓𝑓(𝜙𝜙, 𝑟𝑟) to accomplish the desired behavior. on this purpose, instead of the time dimension, the system reacts based on the phase variable 𝜙𝜙, which is updated according to a periodic linear system (5) with 𝜙𝜙 ∈ [0, 2𝜋𝜋], 𝜙𝜙0 = 0. this is also called canonical system. 𝑦𝑦 is the periodic trajectory defined by the physiotherapist, 𝑔𝑔 is its mean, and 𝑧𝑧 is an auxiliary variable to create a second order system in state-space model. the parameters 𝛼𝛼𝑧𝑧, 𝛽𝛽𝑧𝑧, 𝜏𝜏, 𝑟𝑟 define the dynamics of the system. by setting 𝛼𝛼𝑧𝑧 = 4𝛽𝛽𝑧𝑧 and 𝛼𝛼𝑧𝑧, 𝛽𝛽𝑧𝑧 > 0 the stability is ensured, 𝜏𝜏 is the time constant which must be set to the period divided by 2𝜋𝜋, and 𝑟𝑟 is set to 1. the functions 𝑓𝑓(𝜙𝜙, 𝑟𝑟) are linear combinations of 𝑁𝑁 von mises basis functions. they drive the system according to the phase 𝜙𝜙 and can be learned to follow the desired trajectory: 𝑓𝑓(𝜙𝜙, 𝑟𝑟) = ∑ 𝑤𝑤𝑖𝑖 𝑁𝑁 𝑖𝑖=1 𝜓𝜓𝑖𝑖(𝜙𝜙) ∑ 𝜓𝜓𝑖𝑖(𝜙𝜙)𝑁𝑁𝑖𝑖=1 𝑟𝑟 (6) 𝜓𝜓𝑖𝑖(𝜙𝜙) = exp (ℎ𝑖𝑖(cos(𝜙𝜙 − 𝑐𝑐𝑖𝑖) − 1)) (7) in these equations, 𝜓𝜓𝑖𝑖(𝜙𝜙) are the basis functions whose values are modulated by 𝑤𝑤𝑖𝑖. the centers of the basis functions 𝑐𝑐𝑖𝑖 must be chosen properly to make good approximations, and ℎ𝑖𝑖 is related to the width of the shape. the values of 𝑤𝑤𝑖𝑖 are estimated from the trajectories using regression (atkeson et al., 1997; fanger et al., 2016). an important decision is to establish a common canonical system for the three degrees of freedom (so they are synchronized by the same phase). however, each limb is encoded as a separate dmp to generate its own reference trajectory. the larger the value of 𝑁𝑁, the greater will be the precision of the resulting dmp. figures 2 shows the fitting of the basis functions and the original trajectories based on this parameter 𝑁𝑁, and figure 3 shows the error in terms of their difference. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 35 figure 2. original and generated reference trajectories in function of n. figure 3. error between both trajectories in function of n. 3.2. dmp phase stopping the specialist can stop momentarily the robot’s movement. this is useful if the physiotherapist detects an anomaly during the exercise (for example, because the patient is not performing it correctly), or the patient experiences pain, fatigue, nervousness, etc. figure 4 shows the evolution of the phase. the blue line corresponds to the natural evolution if no obstacle is found. in red, the phase https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 36 is stopped twice: in 𝑡𝑡 ∈ [6. .10] s and 𝑡𝑡 ∈ [43. .50] s. this phase stopping will be reflected in the robot trajectory tracking described in section 4. figure 4. phase normal evolution (blue) and voluntarily stopped (red). 4. parallel robot task space controller robot control can be addressed by using two types of strategies: joint space control and task space control. the first one assumes direct action upon the motors of each joint. in this case, position, velocity and acceleration references are usually provided as time-dependent variables and the objective of the controller is to follow asymptotically the desired trajectory regardless of possible disturbances or non-modelled dynamics. on the other hand, in task space control the desired position, velocity and acceleration are provided with respect to the end effector. in this work, a pd with gravity compensation in task space has been developed. the control action 𝜏𝜏𝑐𝑐 is given by the expression: 𝜏𝜏𝑐𝑐 = 𝐽𝐽𝑇𝑇(𝑞𝑞) ∙ 𝐾𝐾𝑝𝑝 ∙ (𝑥𝑥𝑑𝑑 − 𝑥𝑥) − 𝐽𝐽𝑇𝑇(𝑞𝑞) ∙ 𝐾𝐾𝑑𝑑 ∙ 𝐽𝐽(𝑞𝑞) ∙ �̇�𝑞 + 𝐺𝐺(𝑞𝑞) (8) https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 37 where 𝐾𝐾𝑝𝑝 and 𝐾𝐾𝑑𝑑 are symmetric positive definite matrices, 𝐽𝐽 is the jacobian matrix of the robot manipulator, 𝑥𝑥 and 𝑥𝑥𝑑𝑑 are the actual and reference end effector positions, and 𝑞𝑞 is the vector of joint coordinates. if 𝐽𝐽 is full-rank for all joint configurations 𝑞𝑞, then 𝑥𝑥 = 𝑥𝑥𝑑𝑑 �̇�𝑥𝑑𝑑 = 0 (9) is a globally asymptotically stable equilibrium point for the closed-loop system (2) and (8). figure 5 shows the control scheme implemented in this paper. the reference generation for the mobile platform of the robot is generated using the dmp technique presented in section 3. the dmp block has an input (phase stopping) which allows to freeze the phase 𝜙𝜙. while it is in low level, the signal evolves normally outputting the estimated reference 𝑥𝑥𝑑𝑑 as the canonical system evolves, and when it turns high, the canonical system stops and the dmp keeps emitting the last value of 𝑥𝑥𝑑𝑑 until the phase stopping turns back low, after which the systems resumes the operation. the value of 𝑥𝑥𝑑𝑑 contains the heave and orientation references since a task space controller is employed thereafter. figure 5. control scheme based on task space controller and dmp. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 38 figure 6 shows the reference and actual position of the mobile platform’s pitch angle, and figure 7 plots the error, which demonstrates the precision of the controller. the values of the dmp parameters are 𝛼𝛼𝑧𝑧 = 48 and 𝛽𝛽𝑧𝑧 = 12. the task space controller has proportional gains 200000 for heave and 150000 for pitch and roll, and the derivative term is 1000. figure 6. reference and actual pitch angle position using dmp-based task space controller. figure 7. measured pitch angle error using dmp-based task space controller. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 39 5. lower limb rehabilitation exercises figure 8 shows four of the possible movements of the ankle: plantar/dorsiflexion, inversion and eversion. the most common injuries are ankle sprains, representing 38% of locomotor system injuries. sudden movements in the eversion direction cause these sprains by stretching and tearing the ligaments (safran et al., 1999). figure 8. ankle movements: plantar/dorsiflexion, and inversion/eversion. in many cases rehabilitation is not performed, so sprains are not treated. this fact leads to a likely chronic injury between 80% and 90% of times and causes ligament instability in absence of rehabilitation, which keeps increasing over time as new injuries occur. in order to avoid this, proper rehabilitation exercises are indispensable. there are different exercises that can be performed to treat or strengthen injured ankles, and the parallel robot presented in section 2 is intended to help in this context. the exercises can be either passive or active. passive exercises are those in which the patient does not make any voluntary movement and so the robot dictates the full trajectory based on its program (abu-dakka et al., 2015), while active exercises imply certain voluntary movements performed by the patient, thus increasing the interaction between human and robot. figure 9 shows the rehabilitation robot. as mentioned in section 2, it has three kinematic limbs that control the mobile platform (roll/pitch orientations and heave translation). the human lower limb is tied to an orthopedic boot that is attached to the mobile platform by means of an ati delta https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 40 si-330-30 force/torque sensor. thanks to this, the specialist can set the required movements to make in order to reproduce a trajectory or rehabilitation exercise. figure 9. rehabilitation robot and exercise preparation. once these movements are stored in the control unit of the robot, they are encoded using the procedure shown in section 3.1. the dmp trajectory adaptation is used as reference (𝑥𝑥𝑑𝑑) for the task space controller. the rehabilitation robot incorporates different safety devices. firstly, the forces and torques applied to the ankle are monitored during the exercise. moreover, an emergency stop button allows to stop the robot motion and remove the power if an emergency occurs. finally, there is an additional button that stops the phase variable when necessary, until new command is sent to continue the exercise. figure 10 shows the results of a plantar/dorsiflexion passive ankle rehabilitation exercise. the robot is programmed to follow a specific position reference defined by a specialist, which consists of a sinusoidal-based signal for the roll angle. the blue signal is the original reference (without any phase stopping), and the black one is the response of the robot for that situation. conversely, the cyan and red signals represent, respectively, the same variables when performing a phase stopping in 𝑡𝑡 ∈ [6. .10] s and 𝑡𝑡 ∈ [43. .50] s. in such intervals, the reference does not change. both experiments perform a very accurate tracking. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 41 figure 10. rehabilitation exercise with and without phase stopping. 6. conclusions in this paper, a learning algorithm based on dynamic movement primitives has been proposed for trajectory adaptation of a parallel robot in ankle rehabilitation tasks. this algorithm adjusts the exercises by making the system subject to a manipulable phase variable that intervenes in the reference signal generation, which is integrated in the control scheme. the position control has been designed in task space, which allows to handle cartesian coordinates directly by means of the jacobian matrix. this is possible due to the direct and inverse kinematic models that had been solved in previous work. therefore, the adaptation mechanism lies in the fact that the robot can be provided with external feedback by altering the normal course of the dmp phase variable. this feedback is received by using a device, such as a button, which causes the phase variable to stop. when it occurs, the robot keeps controlling the position, although the reference does not move forward. this is a convenient way to prevent undesired movements from happening on the patient. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 42 under this premise, passive exercises have been performed for one of the degrees of freedom. the experiment shows that the reference generation was successful, and the algorithm could accurately follow this reference while being able to stop the exercise. in future developments, improvements to this algorithm can be designed for more complex ankle rehabilitation exercises (for example, by using more degrees of freedom) and an automatic adaptation of the phase signal can be performed, increasing the flexibility of the system. furthermore, the applied forces can be sent to the controller thanks to the force sensor, improving the control performance. finally, as the patient performs the same exercise several times, such exercise can be intelligently modified in order to adapt to the hopefully improved condition of their ankle. acknowledgements this work has been partially funded by feder-cicyt project with reference dpi2017-84201r financed by ministerio de economía, industria e innovación (spain). references abu-dakka, f. j., valera, a., escalera, j. a., vallés, m., mata, v., & abderrahim, m. (2015). trajectory adaptation and learning for ankle rehabilitation using a 3-prs parallel robot. lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 9245, 483–494. https://doi.org/10.1007/978-3-319-22876-1_41 atkeson, c. g., moore, a. w., & schaal, s. (1997). locally weighted learning. artificial intelligence review, 11(1–5), 11–73. https://doi.org/10.1007/978-94-017-2053-3_2 brockett, c. l., & chapman, g. j. (2016). biomechanics of the ankle. orthopaedics and trauma, 30(3), 232– 238. https://doi.org/10.1016/j.mporth.2016.04.015 dai, j. s., zhao, t., & nester, c. (2004). sprained ankle physiotherapy based mechanism synthesis and stiffness analysis of a robotic rehabilitation device. autonomous robots, 16(2), 207–218. https://doi.org/10.1023/b:auro.0000016866.80026.d7 díaz-rodríguez, m., mata, v., valera, á., & page, á. (2010). a methodology for dynamic parameters identification of 3-dof parallel robots in terms of relevant parameters. mechanism and machine theory, 45(9), 1337–1356. https://doi.org/10.1016/j.mechmachtheory.2010.04.007 díaz, i., gil, j. j., & sánchez, e. (2011). lower-limb robotic rehabilitation: literature review and challenges. journal of robotics, 2011(i), 1–11. https://doi.org/10.1155/2011/759764 fanger, y., umlauft, j., & hirche, s. (2016). gaussian processes for dynamic movement primitives with application in knowledge-based cooperation. ieee international conference on intelligent robots and systems, 2016-novem, 3913–3919. https://doi.org/10.1109/iros.2016.7759576 gosselin, c., & angeles, j. (1990). singularity analysis of closed-loop kinematic chains. ieee transactions on https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1007/978-3-319-22876-1_41 https://doi.org/10.1007/978-94-017-2053-3_2 https://doi.org/10.1016/j.mporth.2016.04.015 https://doi.org/10.1023/b:auro.0000016866.80026.d7 https://doi.org/10.1016/j.mechmachtheory.2010.04.007 https://doi.org/10.1155/2011/759764 https://doi.org/10.1109/iros.2016.7759576 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 43 robotics and automation, 6(3), 281–290. https://doi.org/10.1109/70.56660 hesse, s., & uhlenbrock, d. (2000). a mechanized gait trainer for restoration of gait. journal of rehabilitation research and development, 37(6), 701–708. ijspeert, a. j., nakanishi, j., hoffmann, h., pastor, p., & schaal, s. (2013). dynamical movement primitives: learning attractor models formotor behaviors. neural computation, 25(2), 328–373. https://doi.org/10.1162/neco_a_00393 ijspeert, a. j., nakanishi, j., & schaal, s. (2002). movement imitation with nonlinear dynamical systems in humanoid robots. proceedings ieee international conference on robotics and automation, 2, 1398– 1403. https://doi.org/10.1109/robot.2002.1014739 liu, g., gao, j., yue, h., zhang, x., & lu, g. (2006). design and kinematics simulation of parallel robots for ankle rehabilitation. 2006 ieee international conference on mechatronics and automation, icma 2006, 2006, 1109–1113. https://doi.org/10.1109/icma.2006.257780 nakanishi, j., morimoto, j., endo, g., cheng, g., schaal, s., & kawato, m. (2004). learning from demonstration and adaptation of biped locomotion. robotics and autonomous systems, 47(2–3), 79–91. https://doi.org/10.1016/j.robot.2004.03.003 nemec, b., & ude, a. (2012). action sequencing using dynamic movement primitives. robotica, 30(5), 837– 846. https://doi.org/10.1017/s0263574711001056 patel, y. d., & george, p. m. (2012). parallel manipulators applications—a survey. modern mechanical engineering, 02(03), 57–64. https://doi.org/10.4236/mme.2012.23008 paul, r. p. (1981). robot manipulators: mathematics, programming, and control : the computer control of robot manipulators (p. 279). reinkensmeyer, d. j., aoyagi, d., emken, j. l., galvez, j. a., ichinose, w., kerdanyan, g., maneekobkunwong, s., minakata, k., nessler, j. a., weber, r., roy, r. r., de leon, r., bobrow, j. e., harkema, s. j., & reggie edgerton, v. (2006). tools for understanding and optimizing robotic gait training. journal of rehabilitation research and development, 43(5), 657–670. https://doi.org/10.1682/jrrd.2005.04.0073 safran, m. r., benedetti, r. s., bartolozzi, a. r., & mandelbaum, b. r. (1999). lateral ankle sprains: a comprehensive review part 1: etiology, pathoanatomy, histopathogenesis, and diagnosis. in medicine and science in sports and exercise (vol. 31, issue 7 suppl., pp. s429–s437). https://doi.org/10.1097/00005768-199907001-00004 saglia, j. a., tsagarakis, n. g., dai, j. s., & caldwell, d. g. (2013). control strategies for patient-assisted training using the ankle rehabilitation robot (arbot). ieee/asme transactions on mechatronics, 18(6), 1799– 1808. https://doi.org/10.1109/tmech.2012.2214228 schaal, s. (2006). dynamic movement primitives -a framework for motor control in humans and humanoid robotics. in adaptive motion of animals and machines (pp. 261–280). https://doi.org/10.1007/4-43131381-8_23 sui, p., yao, l., lin, z., yan, h., & dai, j. s. (2009). analysis and synthesis of ankle motion and rehabilitation robots. 2009 ieee international conference on robotics and biomimetics, robio 2009, 3, 2533–2538. https://doi.org/10.1109/robio.2009.5420487 tsoi, y. h., xie, s. q., & graham, a. e. (2009). design, modeling and control of an ankle rehabilitation robot. studies in computational intelligence, 177, 377–399. https://doi.org/10.1007/978-3-540-89933-4_18 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1109/70.56660 https://doi.org/10.1162/neco_a_00393 https://doi.org/10.1109/robot.2002.1014739 https://doi.org/10.1109/icma.2006.257780 https://doi.org/10.1016/j.robot.2004.03.003 https://doi.org/10.1017/s0263574711001056 https://doi.org/10.4236/mme.2012.23008 https://doi.org/10.1682/jrrd.2005.04.0073 https://doi.org/10.1097/00005768-199907001-00004 https://doi.org/10.1007/4-431-31381-8_23 https://doi.org/10.1007/4-431-31381-8_23 https://doi.org/10.1109/robio.2009.5420487 https://doi.org/10.1007/978-3-540-89933-4_18 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13907 social and technological sciences issn: 2341-2593 escarabajal et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 30-44 | 44 vallés, m., díaz-rodrguez, m., valera, á., mata, v., & page, á. (2012). mechatronic development and dynamic control of a 3-dof parallel manipulator. mechanics based design of structures and machines, 40(4), 434– 452. https://doi.org/10.1080/15397734.2012.687292 xie, s. (2016). advanced robotics for medical rehabilitation: current state of the art and recent advances. in springer tracts in advanced robotics (issue 108). yoon, j., ryu, j., & lim, k. b. (2006). reconfigurable ankle rehabilitation robot for various exercises. journal of robotic systems, 22(suppl.), 15–33. https://doi.org/10.1002/rob.20150 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1080/15397734.2012.687292 https://doi.org/10.1002/rob.20150 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 59 the sustainable development goals (sdgs) applied to mechanical engineering carlos llopis-albert1 , francisco rubio1 , shouzhen zeng2 , juan grima-olmedo3, carlos grima-olmedo4 1 instituto universitario de ingeniería mecánica y biomecánica (i2mb). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding author: carlos llopis-albert, cllopisa@upvnet.upv.es 2 ningbo university, ningbo 315211, china, zengshouzhen@nbu.edu.cn 3 instituto geológico y minero de españa (igme-csic), cirilo amorós, 42. 46004 valencia, spain, j.grima@igme.es 4 escuela técnica superior de ingenieros de minas y energía. departamento de ingeniería geológica y minera. universidad politécnica de madrid; c/ ríos rosas 23, 28003 madrid, spain, carlos.grima@upm.es received: 13 february 2022; accepted: 29 march 2022; published: april 2022 abstract in 2015, the united nations approved the 2030 agenda on sustainable development, an opportunity for countries and their societies to face the challenge of successfully addressing the transition to a more sustainable future for people and the planet, leaving no one behind. the agenda defines a total of 17 sustainable development goals (sdgs) of universal application to promote economic growth, a commitment to social needs and the protection of the environment. for the successful achievement the ambitious and demanding sdgs, individuals, businesses, governments, and higher education institutions should work collaboratively. this paper aims to contribute to the implementation of the sdgs in the mechanical engineering subjects, and to define appropriate indicators and validation processes to assess their level of fulfillment as established in the 2030 agenda. as a result, better decisions can be made regarding the efforts that universities including students, academics and researchers and professional services staff must invest to incorporate the sdgs into their initiatives, structures, and policies. keywords: sustainable development goals; mechanical engineering; indicators to cite this article: llopis-albert, c., rubio, f., zeng, s., grima-olmedo, j, grima-olmedo, c. (2022). the sustainable development goals (sdgs) applied to mechanical engineering. multidisciplinary journal for education, social and technological sciences, 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://orcid.org/0000-0002-1349-2716 http://orcid.org/0000-0003-3465-702x https://orcid.org/0000-0002-3604-0843 https://doi.org/10.4995/muse.2022.17269 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 60 1. introduction the 2030 agenda for sustainable development, approved in 2015 by the united nations, represents the international benchmark for excellence in this matter and is being adopted by governments, organizations, and companies in many countries. the commitments that all these agents put in place will determine the success of this strategy, as well as their collaboration to achieve sustainable development that leaves no one behind. the agenda comprise an extensive action plan based on three axes: people, the planet and prosperity, which should guide development strategies worldwide for the next few years. the 2030 agenda proposes 17 sustainable development goals (sdgs) with 169 targets that cover the economic, social, and environmental spheres. the sdgs are interconnected and interdependent, referring to the great challenges facing the world today. it establishes a roadmap towards social, economic, and environmental sustainability, and underlines the need to carry out a significant effort of alignment and coordination on the part of all the agents involved. the agents include governments, civil society, private sector, universities, labor unions, non-governmental organizations (ngos), etc. the 17 sustainable development goals (sdgs) are subsequently presented (united nations, 2015):  sdg 1: no poverty. end poverty in all its forms everywhere.  sdg 2: zero hunger (no hunger). end hunger, achieve food security and improved nutrition, and promote sustainable agriculture.  sdg 3: good health and well-being. ensure healthy lives and promote well-being for all at all ages.  sdg 4: quality education. ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.  sdg 5: gender equality. achieve gender equality and empower all women and girls. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 61  sdg 6: clean water and sanitation. ensure availability and sustainable management of water and sanitation for all.  sdg 7: affordable and clean energy. ensure access to affordable, reliable, sustainable, and modern energy for all.  sdg 8: decent work and economic growth. promote sustained, inclusive, and sustainable economic growth, full and productive employment, and decent work for all.  sdg 9: industry, innovation, and infrastructure. build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation.  sdg 10: reduced inequality. reduce income inequality within and among countries.  sdg 11: sustainable cities and communities. make cities and human settlements inclusive, safe, resilient, and sustainable.  sdg 12: responsible consumption and production. ensure sustainable consumption and production patterns.  sdg 13: climate action. take urgent action to combat climate change and its impacts by regulating emissions and promoting developments in renewable energy.  sdg 14: life below water. conserve and sustainably use the oceans, seas, and marine resources for sustainable development.  sdg 15: life on land. protect, restore, and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss.  sdg 16: peace, justice, and strong institutions. promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable, and inclusive institutions at all levels.  sdg 17: partnership for the goals. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 62 strengthen the means of implementation and revitalize the global partnership for sustainable development. figure 1. sdgs proposed by united nations (2015). universities as agent of change have an important role in the fulfillment of this agenda given their responsibility in the field of education, research, development, innovation, and transfer of knowledge to society (pérez-sánchez et al., 2020). this is because higher education broadens people's options to develop valuable lives. it constitutes a powerful tool for the achievement of sustainable human development. a good part of the existing inequalities in terms of income or ability to influence relevant political processes is directly related to the possibility of accessing advanced educational levels. likewise, countries with a higher quality of life have invested considerable resources and efforts in promoting higher education. that is why the objective of development policies and strategies must be to make effective the human right for all people to access higher education. therefore, the contribution to the united nations mandate is still a challenge and requires universities to rethink their activity in a pro-active and operational way towards their specific goals. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 63 this raises a wide range of monitoring indicators for the achievement assessment of the different goals. this paper is intended to contribute to the theoretical reflection, which must be accompanied by practical proposals that propose concrete and pertinent measurement instruments on the degree of achievement of the sdgs in universities, so that, in the light of the data, decisions can be made regarding the efforts that universities must invest to incorporate the 2030 agenda into their initiatives, structures and policies. it is necessary to be aware what is being done within the universities, assessing its degree of relevance and effectiveness in order to be able to consider what remains to be done and what are the most appropriate instruments to carry it out. 2. indicators to measure the degree of compliance of the sdgs at upv. the upv is intended to analyze the role that higher education should play in the new paradigm of sustainable human development and in its contribution to the 2030 agenda. the upv has defined a set of indicators that measures how universities are addressing the 2030 agenda and their degree of compliance. the indicators are based on relevant, transparent, accessible, and understandable data, not only for experts but for the entire university community, with the goal of normalizing the 2030 agenda (upv, 2019; 2020). the set of indicators are flexible, adaptable, and with different levels of depth with the aim of measuring the university commitment to achieve the sdgs. this operational deployment from the most general to the most concrete, also forces universities reconsider the scope and contents of the sdgs, the goals, and the indicators in their context. as a result, a methodological process was carried out for the construction and measurement of an extensive battery of specific and differentiated indicators for each of the 17 sdgs and structured into different degrees of measurement. they range from the most general and fundamental aspects (degree 1), to the most concrete and specific (degree 4).  degree 1: it covers the most general aspects of the corresponding sdg. these are quick response indicators (yes / no / in process) and the result of the measurements is presented in percentage terms, with only three possibilities: o% / 50% / 100%. this level assesses the existence of minimum compliance by the university institution in https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 64 relation to a series of variables referred to 8 categories: organizational structure, preparation / publication of reports, academic / training offer, r+d + i projects, normative framework, relations with the local environment, participation in public policies and existence of alliances. in short, it is a very broad level, covering multiple categories, but with little depth or not very sensitive to intensity.  degree 2: it provides specific information through numerical indicators. it is also a general level but one that provides a greater degree of specificity. it should be noted that levels 1 and 2 have been conceived to be able to be compared between the different universities, so the results are always reflected in comparable measurement units, such as percentages, indices, etc., never in absolute terms.  degree 3: it is specific for each university and unlike the previous ones, it does not revolve around comparability, but rather its usefulness lies in the internal use of the information by each entity. for this, specific compliance goals are set for each indicator, which will be defined by each university individually, taking into account their priorities, needs and capacities. this level recovers most of the indicators proposed in level 2, which, along with others considered relevant, will appear associated with specific quantified goals, established by each university.  degree 4: it is a specific and concerning each area, unit, service, or department, establishing indicators adapted to the functions and responsibilities of the organizational structure considered. the objective is to obtain useful tools for control and decision-making by university authorities, placing emphasis on the sdgs that have a higher level of connection with their specific objectives and purposes. 3. sdgs implementation in mechanical engineering subjects universities as a socially committed institutions, must contribute to the achievement of the sggs from their different facets, academic, research, transfer, and innovation, involving the entire university community. all subjects taught in the university should carry out a reflection exercise on how they can contribute to the improvement of society within the framework of the sdgs. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 65 in this sense, this paper deepens in the implementation of the sdgs for the mechanical engineering subjects taught at upv, and in defining appropriate indicators and validation processes to assess their level of fulfillment as established in the 2030 agenda. a wide range of subjects are taught by the department of mechanical and materials engineering (dmme) at upv and many research areas are tackled by the different research institutes dealing with the mechanical engineering field. they cover subjects such as:  mechanism and machine design (including kinematics and dynamics).  mechanical design: fatigue and fracture.  failure analysis of mechanical components.  strength of materials and solid mechanics.  materials engineering, composites.  manufacturing engineering, technology, and processes.  maintenance engineering technology, industrial facilities.  mechanical vibrations.  structural integrity.  mechatronics and robotics.  biomechanics.  vehicle dynamics.  engineering design and product design.  engineering, services, and work risk prevention  instrumentation and measurement.  acoustics, noise, and vibration control technology.  computational mechanics.  computer-aided design (cad).  computer-aided manufacturing (cam).  computer-aided engineering (cae).  finite element method. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 66 due to the closely relation between the sdgs and the different courses offered by the department of mechanical and materials engineering (dmme), most of them can be easily implemented in those courses. moreover, some of the sdgs are already worked in the different courses, although further improvements would be advisable. in fact, this is the main goal of the present paper, i.e., to provide tools, concepts, and ideas to further implement corrective actions to incorporate the sdgs in the mechanical engineering courses. for instance, in the current stage of the study program the following sustainable goals are partially worked: sdg 6 (clean water and sanitation), 7 (affordable and clean energy), 9 (industry, innovation, and infrastructure), sdg 11 (sustainable cities and communities), sdg 12 (responsible consumption and production), and sdg 13 (climate action). it is worth mentioning the extensive experience of authors of the present paper with many of these subjects. note that to effectively transmit the sdgs to students, it is important that teachers keep up to date with educational and technological advances in the subjects. in this sense, they have published many articles in recent years closely related with the aforementioned subjects. for instance, llopis-albert et al. (2018); rubio et al., (2021); valero et al. (2017, 2019); hu et al. (2021); valera et al. (2021); zeng et al. (2022). as stated in the previous section, general rules and indicators are defined by the upv to deal with the sdgs. based on that information, we proposed several measures and indicators that can be applied to the syllabus of the mechanical engineering subjects to further implement the sdgs. this is carried out by defining three levels: a first level (level 1) of immediate application, a second level that would be applied in the short term (level 2) and a third level corresponding to the medium and long term, which would imply proposals that would modify the title verification report as certified by the national agency for quality assessment and accreditation (aneca) of spain. despite the time needed for its implementation, the third level can lead to more integrated and articulated changes throughout the study plan of the title.  level 1: introduce in the mechanical engineering courses content related to the 2030 agenda in a transversal way, so that all students know its content. it could be implemented by proposing a compulsory training activity for first-year students. the activity would explain the objectives pursued by the 2030 agenda, the 17 sdgs and the 169 targets. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 67  level 1: students must incorporate in all laboratory practices and academic works an explicit reflection that justifies the extent to which their works are related to certain sdgs. this can be extended to the final degree projects and master's degree final projects offered by the department of mechanical and materials engineering (dmme).  level 1: students must take lessons related to sdgs in each subject. it is proposed to hold meetings with the teachers responsible of each subject, together with experts in the 2030 agenda, and to address changes in the syllabus of the subject so that the sdgs are present in the learning results of the students. this level corresponds to improvement proposals that do not involve a modification of the certificate verification report of the university degree.  level 2: defining new transversal competences (upv, 2020a; llopis-albert and rubio, 2021) related to the sdgs.  level 3: incorporation of the sdgs in the degrees will be encouraged. elective and specific training courses on each of the sdgs should be offered, either compulsory or optional courses. this action would modify the title verification report as certified by aneca.  level 3: promote the mechanical engineering field among high school students to increase the enrollments and bridge the still existing gap between men and women. in addition to studying how to introduce the sdgs into the mechanical engineering courses, we also must consider how to verify that students have achieved these learning outcomes. as a first step, for a validation of the effectiveness of the actions, all students must pass different activities of evaluation (including exams, laboratory practices, academic works, etc.) to certify that they have acquired the knowledge taught in each subject. this can be extended this can be extended to the final degree projects and master's degree final projects. therefore, the contents related to the sdgs that are introduced in each subject are relatively easy to assess. however, it will be necessary to correctly design the evaluation acts so that this statement is valid. to implement this action an evaluation rubric should be defined to analyze learner development to incorporate the sdgs in their curriculum. this rubric should be designed defining 4 levels (a: ineffective; b: developing; c: effective; d: highly effective), which will be available to both students and teachers. the adaptation of the rubric to 4 levels has been carried out to maintain the same qualification system that is carried out at the upv to evaluate the transversal competences present in all its degrees (upv, 2020a). https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 68 as previously mentioned, a way to assess the level of fulfillment as established in the 2030 agenda is to use some indicators. some of the indicators presented for the whole university in upv (2019, 2020) are easily transferable to the mechanical engineering subjects. it should be noted that mechanical engineering subjects mainly deal with sdgs 6, 7, 9, 11, 12 and 13. subsequently, for the sake of conciseness only a few examples of indicators at degrees of measurement 1 and 2 are presented in table 1 for those sdgs. table 1. examples of indicators for assessing the level of fulfillment as established in the 2030 agenda. the department of mechanical and materials engineering (dmme) and the different research institutes dealing with the mechanical engineering field at upv mainly deal with sdgs 6, 7, 9, 11, 12 and 13. sdg 6 degree 1 -do the dmme or the research institutes participate at the local, national, regional or international level in the debate and / or elaboration of institutional policies on the subject of sustainable water management and the promotion of sufficient and adequate water and sanitation systems? sdg 7 degree 1 -do the different subjects offered by dmme contemplate academic / training in sustainable energy, efficient energy management and renewable energy? -are there r+d+i and transfer projects / programs at the research institutes focused on the production of sustainable energy, its sustainable management, and the promotion of renewable energies? sdg 8 degree 2 -temporality rate for hired personnel. -employability rate of graduates in the mechanical engineering degrees and masters. -percentage of graduates who have carried out external internships in companies sdg 9 degree 1 -do the different subjects offered by dmme contemplate academic / training in the development of sustainable infrastructures, innovation, and promotion of industrial sectors? -are there r+d+i and transfer projects / programs at the research institutes aimed at promoting sustainable infrastructures and supporting industry and the productive sector? degree 2 -percentage of personnel totally or partially dedicated to innovation and technology transfer -percentage of revenues from r&d with external funds over the total budget of the department or research institute. sdg 11 degree 1 -do the dmme or the research institutes include objectives related to the integral sustainability, for example related to mobility, use of ecological transport, management of green areas, promotion of cultural and natural heritage, etc.? sdg 13 degree 1 -do the dmme or the research institutes promote initiatives, such as actions, campaigns, projects, or programs related to the fight against climate change and its effects (environmental, social, politicians, etc.)? https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 69 4. conclusions the 2030 agenda constitutes a roadmap towards sustainable development with the aim to end poverty, protect the planet, and ensure all people enjoy peace and prosperity. this work constitutes a first effort to implement, measure and validate the degree of compliance with the sdgs in the mechanical engineering subjects taught at upv. several actions have been defined at three different levels to incorporate the sdgs into the students’ curriculum. they differ from the time needed to be implemented, i.e., immediate, short, medium, and long-term. additionally, a set of indicators have been developed to assess the significance and the degree of fulfillment of the actions undertaken. they cover all the actors involved, students, teachers, and university. eventually, we propose to define a rubric similar to those existing for the transversal competences to facilitate the evaluation. author contributions: conceptualization, c.l.a; methodology, c.l.a and f.r.; validation, c.l.a and f.r.; formal analysis, c.l.a and f.r.; investigation, c.l.a and f.r.; resources, c.l.a and f.r.; data curation, c.l.a and f.r.; writing original draft preparation, c.l.a and f.r.; writing review and editing, c.l.a and f.r.; visualization, c.l.a and f.r. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references hu, y., zeng, s., llopis-albert, c., ullah, k., yang, y. (2021). social network group decision-making method based on q-rung orthopair fuzzy set and its application in the evaluation of online teaching quality. axioms 10(3), 168. https://doi.org/10.3390/axioms10030168 llopis-albert, c., rubio, f., valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences, 5(1), 1-16. https://doi.org/10.4995/muse.2018.9867 llopis-albert, c., rubio, f. (2021). methodology to evaluate transversal competences in the master's degree in industrial engineering based on a system of rubrics and indicators. multidisciplinary journal for education, social and technological sciences 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 pérez-sánchez, m., díaz-madroñero, m., mula, j., sanchis, r. (2020). the sustainable development goals (sdgs) applied to higher education. a project-based learning proposal integrated with the sdgs in bachelor degrees at the campus alcoy (upv). edulearn20 proceedings, 12th international conference on education and new learning technologies online conference, iated, 6-7 july, 2020. https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.3390/axioms10030168 https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.4995/muse.2021.15244 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17269 social and technological sciences e-issn: 2341-2593 llopis-albert et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 70 isbn: 978-84-09-17979-4, pages 3995-4005. https://doi.org/10.21125/edulearn.2020 rubio, f., llopis-albert, c., valero, f. (2021). multi-objective optimization of costs and energy efficiency associated with autonomous industrial processes for sustainable growth. technological forecasting and social change 173, 121115. https://doi.org/10.1016/j.techfore.2021.121115 united nations (2015). resolution adopted by the general assembly on 25 september 2015. transforming our world: the 2030 agenda for sustainable development. (retrieved on december 21th 2021 from https://sdgs.un.org/). upv. (2019). the sustainable development goals (sdgs). keys to a constantly changing university. universitat politècnica de valència (upv). isbn 978-84-121798-7-3. upv (2020a). proyecto institucional competencias transversales. universitat politècnica de valència (upv). valencia. spain. https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf upv. (2020). the sustainable development goals (sdgs) in spanish universities: a proposal from the upv to measure their degree of compliance. keys to a constantly changing university. universitat politècnica de valència. valera á., valero f., vallés m., besa a., mata v., llopis-albert c. (2021). navigation of autonomous light vehicles using an optimal trajectory planning algorithm. sustainability. 2021; 13(3):1233. https://doi.org/10.3390/su13031233 valero, f., rubio, f., llopis-albert, c., cuadrado, j.i. (2017). influence of the friction coefficient on the trajectory performance for a car-like robot. mathematical problems in engineering, 9 pages. article id 4562647. https://doi.org/10.1155/2017/4562647 valero, f., rubio, f., besa, a.j. (2019). efficient trajectory of a car-like mobile robot. industrial robot: the international journal of robotics research and application, 46(2), 211–222. https://doi.org/10.1108/ir10-2018-0214 zeng, s., zhang, na, zhang, c., su, w., llopis-albert, c. (2022) .social network multiple-criteria decisionmaking approach for evaluating unmanned ground delivery vehicles under the pythagorean fuzzy environment. technological forecasting and social change 175, 121414. https://doi.org/10.1016/j.techfore.2021.121414 https://doi.org/10.4995/muse.2022.17269 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.techfore.2021.121115 https://sdgs.un.org/ https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf https://doi.org/10.1155/2017/4562647 https://doi.org/10.1108/ir-10-2018-0214 https://doi.org/10.1108/ir-10-2018-0214 https://doi.org/10.1016/j.techfore.2021.121414 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 45 students’ perception to integrate education 4.0 in science program. siti hajar halilia , shukri sulaimanb a university of malaya, jalan professor ungku aziz, 50603, kuala lumpur, malaysia universiti sains malaysia, 11800, penang, malaysia * ajai912@gmail.com received: 11 december 2020; accepted: 10 march 2021; published: april 2021 abstract this study aims to explore students’ perception of integrating education 4.0 in the science program. the technology acceptance model was used to determine students’ understanding of integrating education 4.0 focusing on the science program. this research was conducted using a qualitative approach. the interview was used to collect the data. five respondents among science students were interviewed. they were undergraduate students pursue study in a science education program. the finding of the interview result showed that students had positive responses in integrating education 4.0 for technological sustainable development. the results of the study showed that students had a positive understanding of the three aspects namely, readiness in integrating education 4.0 for the science program, the impact of technology facilities, and enhancing digital skills for employability. we recommend further research to evaluate the preparation or readiness of students to integrate training using technology 4.0. based on the results, further research is proposed to take into account new education skills among science students in line with the industrial revolution 4.0. keywords: industrial revolution 4.0; science program; technological advancement; digital skills; students to cite this article (example): halili, s.h., sulaiman, s. (2021). students’ perception to integrate education 4.0 in science program. multidisciplinary journal for education, social and technological sciences, 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0001-5712-9364 https://orcid.org/0000-0002-4869-2267 https://doi.org/10.4995/muse.2021.14768 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 46 1. introduction in higher education institutions there has been a rise in understanding of sustainability and an increase in science (othman & othman, 2012; ahmad, 2012). the next generation of people who will help the sustainability movement needs to be developed in higher education institutions as skills demand grows (swaim et.al, 2014). john and cedric (2004) pointed to the relevance of higher education as new fields have become increasingly essential to continued growth, and the organization needs to remain relevant to learning through that period. science and technological education demands continue to grow. in the scope of higher education institutions, there are different needs of each university as to how sustainability is important to them. in the training and learning programs, higher education institutions have implemented environmental growth concepts and practices. it has been important to further establish ideas about what sustainability implies and the value of educational advancement to implement sustainable development. in higher education, sustainable growth is necessary because students are now ready to reach the job market, develop expertise, and translate knowledge with what they have learned. jennie et al. (2008) claimed that higher education can serve as a change-maker in multiple communities and environments. anthony (2003) also realized that higher education could act as a sustainable model by combining all elements of campus life. the first industrial revolution began in the 1800s with the existence of mechanical advances such as the use of steam engines, cotton spinning, and trains. the second industrial revolution, which took place in the 1900s, initiated large production through installations and electrification. in the 1970s, the third technological revolution saw the presence of major computer skeletons, the presence of personal computers, and the internet. whereas nanotechnology, brainwaves, 3d printing as well as mobile and computer networks, became the subject of the fourth revolution. industrial revolution 4.0 brings the vast usage of intelligent networked networks to innovate innovative goods, processes, and facilities into account. since human lives will be transformed in the fourth revolutionary revolution, many countries including malaysia, want to become an industrial society (dzulkifli, 2017). in compliance with the malaysian education higher education strategy 2015-2025 and the global industrial revolution 4.0, the malaysian minister for higher education introduces a higher education framework. this framework is focused upon information and awareness align with the national vision 2050 (tn50). the malaysian government has initiated many programs and built the https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 47 necessary infrastructure to achieve tn50. one of the programs is to translate the training curriculum, in line with the revolutionary trends in the industrial revolution, as a way for organizations to remain successful (malaysia ministry of higher education malaysia, 2018). malaysia is changing to a knowledge-driven culture in 2050 with a shift toward industrialization. 1. 1. industrial revolution 4.0 in science program one of the most challenging issues in malaysia is the industrial revolution of 4.0. industrial revolution 4.0 has evolved to change the employees' work and skills. new jobs and the introduction of new products and services are being developed in the industrial revolution 4 (thomas & gerold, 2016). malaysia is bound towards a scientist-led economy by 2010 (ng, 2014). malaysia has been primarily concerned about the value of science education. from kindergarten to the higher education institution, the sciences subject is taught. science, technology, engineering and mathematics (stem) is one of the initiatives listed in the malaysian roadmap that will help educationists integrate technology into the educational and learning process (rose, 2016). in other words, stem allows students to incorporate information and to think structurally and in a systematic way through subjects. the sciences education curriculum must be thoroughly trained and aware of the latest industrial technology in compliance with the mandate of the malaysian minister for higher education to promote higher education in industrial revolution 4.0 in 2018. the use of technology in learning needs to be emphasized in line with the technical change in education, as stated by the malaysian qualifications agency in 2018. it is limited to date and still considered at the early stage of adoption to accept the industrial revolution in malaysia's context. since the malaysian education system requires studies examining industrial revolution 4.0, limited information was gathered. science students need to be equipped and ready for the emergence of the industrial revolution 4.0. they will compete in the global markets for jobs and must be knowledgeable of emerging technology industries. there will be jobs replaced that can be automated. the rapid revolutions during the industrial revolution 4.0. katharina et.al., (2015), also agreed to equip students and train them to fulfil industry demands 4.0. hence the education system needs to be revised. in this respect, universities must ensure that their academic curriculum or services for science students are compatible with industrial revolution 4.0. malaysian universities must integrate different techniques to enhance the progress of learners in their studies in the integration of science education with the industrial revolution 4.0. https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 48 technologies such as big data, automated robotics, simulation; uniform systems integration; cybersecurity; cloud computing; additive manufacturing; and growing reality are among the nine foundations of technological advancement in industrial revolution 4.0. the use of modern technology will help improve the teaching and learning experience and generate a learner's interest in the learning materials by incorporating them into teaching and learning science programs. casasus et.al., (2018) stated that the usage of multimedia is an effective learning material to support the teaching and learning process. since the newest technical development 4.0 in education and training has evolved, blaschke (2012) claims the need to bring modern pedagogical approaches into consideration. the role of technologies is to promote digital competencies with the growth of new digital technologies (halili, 2019). the digital competencies are essential in industry 4.0 which are relevant for students (ulrike, 2018). digital competencies include related digital skills, ranging from basic literacy skills to the total population in digital skills related to emerging digital technology, and products and services innovations. educators must however ensure that students are educated and aware of these emerging technologies. therefore, it is important to ensure that science students are prepared to fulfil the needs of their potential employers with the knowledge, expertise, and understanding of industrial revolution 4.0. this article is therefore intended to respond to the following research question: what is the students’ perception in integrating education 4.0 in science program? 2. methodology the technology acceptance model (tam) by davis (1989) is used in this study as an instrument to identify factors influencing science students’ perception in integrating education 4.0 for technological sustainable development. the rationale of using the tam model in this study is that this model is widely used in various studies such as the world wide web, mobile banking, multimedia, and healthcare. the tam model is one of the most well-known models in the field of technology adoption and has great potential for testing and predicting attitudes toward information technology use. (moore, 2012; park, 2009). tam model stipulates that factors are influencing the users’ acceptance toward the use of particular technology. in this study, the tam model is focused on two main constructs such as perceived usefulness (pu) and perceived ease of use (peou). these https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 49 two constructs are key determinants of the tam model that is the level at which a user believes that using a particular technology will improve individual performance and the degree to which a person believes that the use of technology would enhance his/her job performance (pai & huang, 2011). the research focused on the science program due to its challenging process with the highest difficulty compared with other subjects (kubilay & ozden, 2012). the selected population for this study consisted of science students in one of the top universities in malaysia. they were undergraduate students pursue study in a science education program. the researcher selected the sample using purposive sampling. since the choice of sample is based on purposive sampling; the sample characteristics have already been defined. six students were invited and selected as the subjects of research. they were chosen because they were the senior in the university and have learned course in the technology-based subject since the first year. they were named u1, u2, u3, u4, u5, and u6. two males and four females participated. qualitative analysis is performed in natural systems and uses words instead of numbers. for a small sample size of six, the qualitative method such as the interview would be a more appropriate way to collect data (tracy, 2013). patton (2002) also agreed that six to ten people are suitable to be involved in the qualitative method and usually involved in a small focus group interview. further, this interview aims to improve participants’ confidence in explaining their ideas. the researcher chooses to implement the semistructured interview because its casual style allows more flexibility and freedom in communication. this is because according to kajornboon (2004), semi-structured is a freestyling interview compared to a structured interview. the interview questions were modified from previous research done by suguneswary (2016). the questions were focused on two main constructs in the tam model such as perceived usefulness (pu) and perceived ease of use (peou). the duration of an interview session was carried out in ten to fifteen minutes after the class session in the fourteen weeks. the researcher referred to miles and huberman (1994) on the analyzing procedure in analyzing the data from the interview session. in this study, after conducting the semi-structured interview with a group of respondents, the researcher immediately processes, and records raw data collected during the interview session. the researcher gathers data that is significant to the research and simplifies it to be understood by the reader. all answers and feedback received from the interview session are categorized based on the interview questions. the content was characterized accordingly to answer https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 50 the interview questions. after identifying the contents into a theme, the researcher arranges the data and reports it descriptively according to the themes. 3. findings the finding of the interview result showed that students had positive responses in integrating education 4.0 for technological sustainable development. based on the students’ agreement in the interview process, the majority of the respondents hold positive views regarding the use of technology in the science program for sustainable development. in this study, the researcher classifies the raw data related to students’ perception into specific themes. the themes are the overall discussions which are based on the interview findings. it has become easier for the researcher to put it into themes for analysis. three interlocking themes emerged during the interview process in terms of students’ learning: (1) readiness in integrating education 4.0 for the science program, (2) impact of technology facilities, and (3) enhancing digital skills for employability. 3.1 theme 1: readiness in integrating education 4.0 for science program in this section, when the researcher began asking students about their perceptions in integrating education 4.0, all of them responded that they were ready to embrace education 4.0 in the science program. the majority of the students mentioned that they were motivated and gained more knowledge using technology 4.0 compared to traditional approaches. u1 and u3 declared, “i like the way this subject was taught. i can get more knowledge in learning.” (u1 & u3). u2 stated, “yes, i am actively involved in-class activity and i am more motivated to learn.” (u2). u4 commented, “yes, i am ready to embrace education 4.0 in learning science. i am more motivated to learn science” (u4). u6 said, “i can use the teachers' learning resources. as a student, i need to equip myself with the new current of the educational system for my knowledge” (u6). https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 51 based on the interview analysis, the majority of the science students agreed that they were able to increase students’ interest, gain more knowledgege and motivation in learning science. the finding is coherent with a study by ali (2016) that revealed students had a positive attitude and ready to use technology in learning. this is aligned with the research done by contreras and hilles (2015) which stated that students are ready to embrace the latest technology in the future. however, u5 mentioned that he was not ready to embrace education 4.0 in learning science. they stated that he had no information and exposure on how the integration of industrial revolution 4.0 embraces education. thus, the researcher believes that more awareness needs to be delivering to students since the study on education 4.0 is limited to date. 4.2 theme 2: impact of technology facilities respondents were asked to describe facilities to embrace education 4.0 for technological sustainable development. almost all respondents mentioned that they were not able to use technologies 4.0. in other words, most respondents mentioned that lack of technology facilities will give an impact to embrace education 4.0 for technological sustainable development. u1 noted that "the facilities should be considered as a positive factor for planning education 40." (u1). student (u2) agreed and stated: “lack and poor learning facilities can foster negative impacts and hinder the growth of sustainable development” (u2). student (u4) and student (u6) noted that inadequate facilities may give an impact on the development of education 4.0 in the science program. student (u6) mentioned: “the ability to serve more students with advanced technology facilities is needed to sustain in this era.” (u6). almost all students agreed that lack of technology facilities can contribute to negative impacts. since students were ready to use technology in their learning, it is a challenge for the https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 52 institutions to provide sufficient infrastructures to students (anuar et.al., 2016; afolabi, 2015). penaloza and vargas (2017) also agreed that there are many challenges to integrating technology in the teaching and learning process. the rresearcher believes that although improving the technology facilities is costly, however, it can give a positive impact on the institutions to sustain as a reward for investments. 4.3 theme 3: enhancing digital skills for employability although the students agreed that they were ready to embrace education 4.0 for technological sustainable development, however, they still need guidance from teachers to enhance their digital skills to increase employability. student (u1) mentioned: “the teacher can make a difference as guidance to me. the teacher should expose us more to the use of technology 4.0 in class. quality of teacher learning experience is important to enhance our digital skills for our future employment.” (u1). student (u2) agreed and stated: “i need more hands-on activity to enhance my digital skills in learning science. it is an important requirement for future jobs.” (u2). student (u4) mentioned: “i need to constantly up-skill my knowledge in digital usage so that i can meet future skills demand.” (u4) student (u5) also mentioned: “i believe that as a student, i need to accelerate skills acquisition in the age of intelligent technologies i need guidance from the lecturer to enhance my digital skills so that i can be marketable in future.” (u5) student (u6) reported: “i understand the demand for skills in future, thus i need guidance from lecturer to guide me.” (u6). the finding of this study showed that digital skills are becoming globally requirements and it is important for students for their employment. previous research related to industrial focus more on employability skills to fulfil the needs of industry and graduate (thi et.al., 2018: track, 2017; https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 53 beaumont et. al., 2016; dacre et. al., 2014; bridgstock, 2009). to embrace education 4.0 for technological sustainable development, students need to equip with high-skilled job. thus, there is an increased demand for digital skills and acquiring these skills is important in this era. 4. conclusion and recommendations sustainable organizations consider the economic, social, and environmental effects of their activities to ensure they work in a well-balanced manner towards the achievement of an equitable society. with current technologies, information is accessible almost everywhere and learning has become very dynamic. this means the science curriculum needs to be designed in such a way that it creates more room for students to fulfil their learning needs. the technological advancements of the fourth revolution were mentioned by klaus (2017) in different fields, economies, industries, and almost all facets of daily life. this article has a strong effect on the malaysia industrial 4.0 trends, which will help institutions establish industrial revolution 4.0 education by malaysia education blueprint 2013-2025. this article has many direct consequences. to remain relevant, higher education institutions need to change the framework of academic programs. it will help students confront and recognize the demands of the industrial revolution 40 (centre for teaching excellence and academic quality, 2017). in this research, it was found that students were ready to accept education 4.0 for technical sustainable development; but they need to be equipping with technology facilities and will need feedback from teachers to improve their digital skills to increase employability. policymakers sometimes neglect the effect of facilities, hence the results of this study indicated that greater focus should be provided on the impacts of technology facilities as it will provide long-term cost-benefit aspects to keep maintain in this period. in the malaysian science academic program, the introduction of industrial revolution 4.0 has already been incorporated and has been taken into account in early implementation phases. in this respect, it is important for the universities, to fulfil global market demands after their graduation, to ensure their curriculum and programs for science students are in line with industrial revolution 4.0. this study only investigated the understanding of science students in the integration of education 4.0 using the tam model. we recommend further research to evaluate the preparation or https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 54 readiness of students to integrate training using technology 4.0. further analysis is also proposed in line with the industrial revolution of 4.0 and takes into account the latest educational skills amongst science students. it is a clear way of recognizing the education skills needed to bring about a 4.0 transition in business. education skills, such as teamwork, communication, critical thinking, innovation, and collaboration are important in 21 st -century education (lorna, 2016; deanna, 2016; miller, 2015). therefore, it is important to learn the training skills needed for the sense of the industrial revolution 4.0 to fulfil the needs of the employer and enable science students in the malaysian education blueprint 2013-2025 to become energetic, imaginative, and innovative. acknowledgements: the authors would like to acknowledge the ministry of higher education malaysia for the financial support through the fundamental research grant scheme (frgs) fp024-2018a (frgs,/1/2018/ssi09/um/02/3). author contributions: “writing original draft preparation, siti hajar halili; writing review and editing, siti hajar halil; methodology, shukri sulaiman.; validation, shukri sulaiman and formal analysis, shukri sulaiman. funding: “this research was funded by the ministry of higher education malaysia for the financial support through the fundamental research grant scheme (frgs) fp024-2018a (frgs,/1/2018/ssi09/um/02/3)”. conflicts of interest: “the authors declare no conflict of interest.” references afolabi, a.a. (2015). availability of online learning tools and the readiness of teachers and students towards it in adenkule ajasin university, akungba-akoko, ondo state, nigeria. procedia – social and behavioral sciences, 176, 610-615. https://doi.org/10.1016/j.sbspro.2015.01.517 ahmad, j. (2012). can a university act as a corporate social responsibility (csr) driver? an analysis. social responsibility journal, 8(1), 77-86. https://doi.org/10.1108/17471111211196584 ali, w. (2016). nursing students’ readiness for e-learning experience. gynecology & obstetrics. 6. https://doi.org.10.4172/2161-0932.1000388. anthony, d.c. (2003). the critical role of higher education in creating a sustainable future. planning for higher education, 15-22. anuar, r., wan zakaria, w.z., md noor, h. & othman, n. f. (2016). tpack in vae: a study on students’ readiness to use e-learning in the teaching and learning of visual art education. in c.y. fook et.al (eds.), proceedings of the 7th international conference on university learning and teaching (incult 2014) (pp.811-822). singapore: springer science + business media singapore. https://doi.org/10.1007/978-981287-664-5_64 https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.sbspro.2015.01.517 https://doi.org/10.1108/17471111211196584 https://doi.org.10.4172/2161-0932.1000388 https://doi.org/10.1007/978-981-287-664-5_64 https://doi.org/10.1007/978-981-287-664-5_64 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 55 beaumont, e., gedye, s., & richardson, s. (2016). ‘am i employable?’: understanding students’ employability confidence and their perceived barriers to gaining employment. journal of hospitality, leisure, sport & tourism education, 19, 1–9. https://doi.org/10.1016/j.jhlste.2016.06.001 blaschke, l. m. (2012). heutagogy and lifelong learning: a review of heutagogical practice and selfdetermined learning. international review of research in open and distance learning, 13(1), 56-71. https://doi.org/10.19173/irrodl.v13i1.1076 bridgstock, r. (2009). the graduate attributes we’ve overlooked: enhancing graduate employability through career management skills. higher education research & development, 28(1), 31–44. https://doi.org/10.1080/07294360802444347 casasus, e.t., ivars, e.a., & lopez, r.m.i. (2018). present and future of the e-learning in economics schools and faculties. multidisciplinary journal for education, social and technological sciences, 5(1), 44-64. https://doi.org/10.4995/muse.2018.9777. centre for teaching excellence & academic quality. (2017). criteria for academic program 4.0. http://www.ums.edu.my/ppsav2/index.php/ms/muat-turun/kriteria-untuk-program-akademik-4-0 contreras, j.o. & hilles, s.m.s. (2015). assesment in e-learning environment readiness of teaching staff, administrators and students of faculty of nursing-benghazi university. international journal of the computer, the internet and management, 23(1), 53-58. dacre pool, l., qualter, p., & j. sewell, p. (2014). exploring the factor structure of the careeredge employability development profile. education & training, 56(4), 303–313. https://doi.org/10.1108/et-012013-0009 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology, mis quarterly, 13(3), 319–340. https://doi.org/10.2307/249008 deanna, z. (2016). connecting the 4cs of 21st century education (with a 5th c!). https://www.linkedin.com/pulse/connecting-4cs-21st-century-education-5th-c-deanna-zaucha dzulkifli abdul razak. (2017). 4th industry revolution: can i technology. https://www.majalahsains.com/revolusi-industri-ke-4-mampukah-menginsankan-teknologi/ halili, s.h. (2019). technological advancements in education 4.0. the online journal of distance education and e-learning, 7(10), 63-69. jennie, c. s., marian, e.h., mikael, r., amanda, c.g. & roland, s. (2008). higher education as a change agent for sustainability in different cultures and contexts. international journal of sustainability in higher education, 9(3), 317-338. https://doi.org/10.1108/14676370810885916 john, b., & cedric, c. (2004). the sustainability curriculum: the challenge for higher education. london. kajornboon, a. b. (2004). creating useful knowledge: a case study of policy development in e-learning at chulalongkorn university language institute. (dissertation). australia. klaus, s. (2017). the fourth industrial revolution. london. https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.jhlste.2016.06.001 https://doi.org/10.19173/irrodl.v13i1.1076 https://doi.org/10.1080/07294360802444347 https://doi.org/10.4995/muse.2018.9777 https://doi.org/10.1108/et-01-2013-0009 https://doi.org/10.1108/et-01-2013-0009 https://doi.org/10.2307/249008 https://doi.org/10.1108/14676370810885916 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 56 kubilay, k., & ozden, t. (2012). challenges for science education. procedia social and behavioral sciences, 51(2012), 763 – 771. https://doi.org/10.1016/j.sbspro.2012.08.237 lorna, k. (2016). pedagogy and technology: integrating 5c’s of 21st century education. http://fishtreeblog.tumblr.com/post/112037114985/pedagogy-and-technology-integrating-the-5-cs-of malaysia ministry of higher education malaysia. (2018). framing malaysian higher education 4.0: futureproof talents. putrajaya. malaysian qualifications agency. (2018). code of practice for open and distance learning, the standards division, malaysian qualifications agency, 1-34. miller, b. (2015). the 6 c’s squared version of learning skills for the twenty-first century. http://flipped4science.blogspot.my/p/the-6-cs-of-education-for-future-during.html moore, r. k. (2012). a bayesian explanation of the ‘uncanny valley’ effect and related psychological phenomena. nature scientific reports, 2,10-23. https://doi.org/10.1038/srep00864 ng, s.b. (2014). malaysian school science education: challenges and the way forward. http://research.utar.edu.my/sose2014/1.challenges%20and%20issues%20in%20science%20education.pdf othman, r., & othman, r. (2014). higher education institutions and social performance: evidence from public and private universities. international journal of business and society, 15(1), 1-18. pai, f., & huang, k. (2011). applying the technology acceptance model to the introduction of healthcare information systems. technological forecasting and social change, 78(4), 650-660. https://doi.org/10.1016/j.techfore.2010.11.007 park, n. (2009). user acceptance of e-learning in higher education: an application of technology acceptance model. paper presented at the annual meeting of the international communication association, new york. patton, m. q. (2002). qualitative research and evaluation methods (3rd ed.). london. penaloza, f., j.l., & vargas, p.c. (2017). big-data and the challenges for statistical inference and economics teaching and learning. multidisciplinary journal for education, social and technological sciences, 4(1), 64-87. https://doi.org/10.4995/muse.2017.6350 rose, a. (2016). stem pedagogical approach for primary science teachers’ through early engineering training program. http://eprints.um.edu.my/16793/1/0001.pdf suguneswary, r. (2014). teachers’ acceptance of using information communication and technology in teaching tamil language in a primary school, master thesis, university malaya. swaim, j. a., maloni, m. j., napshin, s. a., & henley, a. b. (2014). influences on student intention and behaviour toward environmental sustainability. journal of business ethics, 124(3), 465-484. https://doi.org/10.1007/s10551-013-1883-z thi, t. h. t., ronald, s. l., & kylie, s. (2018). the importance of developing soft skill sets for the employability of business graduates in vietnam: a field study on selected business employers. journal of education and culture studies, 2(1), 32-45. https://doi.org/10.22158/jecs.v2n1p32 https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.sbspro.2012.08.237 https://doi.org/10.1038/srep00864 https://doi.org/10.1016/j.techfore.2010.11.007 https://doi.org/10.4995/muse.2017.6350 http://eprints.um.edu.my/16793/1/0001.pdf https://doi.org/10.1007/s10551-013-1883-z https://doi.org/10.22158/jecs.v2n1p32 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 halili and sulaiman (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 57 thomas, w., & gerold, w, l. (2016). academic education 4.0: university of applied science upper austria (austria) international conference on education and new developments 2016 conference: conference: end 2016 international conference on education and new developments, at ljubljana, 155-159. track, d. (2017). preparing sports graduates for employment: satisfying employers expectations. higher education, skills and work-based learning, 7(4), 354-368. https://doi.org/10.1108/heswbl-02-20170017 tracy, s.j. (2013). qualitative research method: collecting evidence, crafting analysis, communicating impact. united kingdom. ulrike, s.f. (2018). the need for digital and soft skills in the romanian business service industry. management & marketing. challenges for the knowledge society. 13(1), 831-847. https://doi.org/10.2478/mmcks-20180008 vasja, r., maja, m., & alojz, k. (2016). a complex view of industry 4.0. https://doi.org/10.1177/2158244016653987 https://doi.org/10.4995/muse.2021.14768 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1108/heswbl-02-2017-0017 https://doi.org/10.1108/heswbl-02-2017-0017 https://doi.org/10.2478/mmcks-2018-0008 https://doi.org/10.2478/mmcks-2018-0008 https://doi.org/10.1177/2158244016653987 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 30 methodology to evaluate transversal competences in the master's degree in industrial engineering based on a system of rubrics and indicators carlos llopis-albert1 , francisco rubio1 1 centro de investigación de ingeniería mecánica (ciim). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding author: carlos llopis-albert, e-mail address: cllopisa@upvnet.upv.es received: 12 november 2020; accepted: 28 march 2021; published: april 2021 abstract this paper presents a methodology to evaluate transversal competences in the context of the subject “design and application of industrial equipment” in the master's degree in industrial engineering at universitat politècnica de valència (spain). the competency-based education implies several activities, such as a project-based learning that must be eventually defended in public by students in groups. evidence of learning is collected based on a well-defined system of rubrics and indicators, which are known in advance by students. we have observed that the use of such techniques improves the students learning on the contents of the subject, allows to acquire the transversal competences related to the analysis and problem solving, and enhances the ability to understand concepts intuitively. moreover, results clearly show a positive influence on the use of such tools for improving the professional and ethical commitment to the issues raised. keywords: transversal competences; project-based learning; innovation; creativity and entrepreneurship; knowledge of contemporary problems to cite this article: llopis-albert, c., rubio, f. (2021). methodology to evaluate transversal competences in the master's degree in industrial engineering based on a system of rubrics and indicators, 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://orcid.org/0000-0002-1349-2716 http://orcid.org/0000-0003-3465-702x https://doi.org/10.4995/muse.2021.15244 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 31 1. introduction this paper presents a methodology to evaluate transversal competences in the context of the subject “design and application of industrial equipment” in the master's degree in industrial engineering at universitat politècnica de valència (upv) in spain. this master qualifies for the industrial engineering profession. the degree of industrial engineer has been accredited by the american abet (accreditation board for engineering and technology) agency, which recognizes it as equivalent to the master's degree studies taught in the united states. it also has the eur-ace international seal of excellence, which certifies that these studies meet the professional requirements of engineering (table 1). table 1. indicators in the master's degree in industrial engineering at upv. academic year 2015/2016 2016/2017 2017/2018 2018/2019 degree performance rate 91,71 % 89,76 % 88,05 % 88,62 % degree drop-out rate 1,84 % 1,41 % 2,09 % graduate efficiency rate 98,5 % 96,94 % 94,1 % 94,16 % graduation rate 73,36 % 73,72 % 67,25 % supply-demand rate 161 % 172,33 % 194,67 % 199,67 % registration rate 99,67 % 100,67 % 102,33 % 106,67 % phd rate for academic staff 85,49 % 82,22 % 83,16 % 81,52 % full time academic staff rate 77,25 % 72,22 % 69,7 % 67,66 % number of students enrolled 505 701 827 864 number of students graduated 89 154 236 235 this degree provides the students with a deep professional and scientific background and with a great variety of knowledge about industrial technologies like mechanics, electricity, electronics, automation, materials, industrial building, projects, environment, energy and industrial organization and management. all of this implies a multidisciplinary education that makes graduates able to develop their professional work in industries, companies, professional practice of industrial engineering and public organizations. this teaching innovation project is in line with the upv objective to accredit the transversal competences (tc) to graduate students in any of the official degrees taught at the university (upv, 2020). the tc are intended to ensure that students acquire the necessary skills to be able to have an https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 32 adequate labor insertion. this training must be seen from a broad perspective, linked to the integral training cycle of students, which includes both undergraduate and postgraduate degrees. in this regard, this research evaluates for the subject “design and application of industrial equipment” the transversal competences ct-04 and ct-10, which are explain in detail in the next section. 2. evaluation of transversal skills 2.1. transversal competences at upv this research aims to develop a teaching innovation project to evaluate the transversal competences (tc) as defined by the upv. tc are intended to synthesize a competency profile for its graduate students, ensuring a reference framework of all degrees. these cover a set of cognitive skills and metacognitive and instrumental and attitudinal knowledge of great value for the knowledge society. they are related to a set of attitudes and values (know-how to be) and procedures (knowhow to do / act) and can be translated from one specific professional field to another. furthermore, they are key and transferable in relation to a wide variety of personal, social, academic, and work contexts throughout of the life. thirteen transversal competences are defined at upv (upv, 2020):  tc-01. comprehension and integration. demonstrate understanding and integration of knowledge both from own specialization and in other broader contexts.  tc-02. application and practical thinking. apply theoretical knowledge and establish the process to follow the achievement of certain objectives, carry out experiments and analyze and interpret data to draw conclusions.  tc-03. analysis and problem solving. analyze and solve problems effectively, identifying and defining the significant elements that constitute them.  tc-04. innovation, creativity, and entrepreneurship. innovate to respond satisfactorily and in an original way to personal, organizational, and social needs and demands with an entrepreneurial attitude.  tc-05. design and project. design, direct and evaluate an idea effectively until it is concretized in a project. https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 33  tc-06. teamwork and leadership. work and lead teams effectively for the achievement of common objectives, contributing to the personal and professional development of themselves.  tc-07. ethical, environmental, and professional responsibility. act with ethical responsibility, environmental and professional to oneself and others.  tc-08. effective communication. communicate effectively, both orally and written, adequately using the necessary resources, and adapting to the characteristics of the situation and the audience.  tc-09. critical thinking. develop critical thinking by taking an interest in foundations on which ideas, actions, and judgments, both own and others, are based.  tc-10. knowledge of contemporary problems. identify and interpret problems contemporaries in their field of expertise, as well as in other fields of knowledge, paying special attention to aspects related to sustainability.  tc-11. permanent learning. use learning strategically, autonomously, and flexible, throughout life, depending on the objective pursued.  tc-12. planning and time management. adequately plan the time available and program the activities necessary to achieve the objectives in academic, professional, and personal terms.  tc-13. specific instruments. use techniques, skills, and the necessary updated tools for the practice of the profession. the development of each competence is staggered on a continuum that can be called curve of learning, in which partial results that provide information to students, teachers and tutors. it must consider the reference level of development that will be required at the end of the training process to accredit students’ achievement. 2.2. transversal competences ct-04, tc-05 and ct-10 at upv in the subject “design and application of industrial equipment” taught in the master's degree in industrial engineering at upv the transversal competences tc-04, tc-05 and tc-10 are assessed. https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 34 2.2.1. transversal competences ct-04 the transversal competence ct-04 is named “innovation, creativity and entrepreneurship” and refers to the students’ capability to innovate and respond satisfactorily and in an original way to personal, organizational, and social needs and demands with an entrepreneurial attitude. innovation is understood as the ability to respond satisfactory to personal, organizational, and social needs, modifying processes and / or results to generate new value. the development of this competence requires both the thinking of another way to bring different perspectives (creativity) and committing certain resources on their own initiative, to explore an opportunity, assuming the risk that it implies (entrepreneurship). the procedure to detect advances in the development of the competence entails three levels of domain, which are based on what learning outcomes would be desirable to reach at the end of master's course by the student. for each of these three learning outcomes, we will define a series of indicators, which are the concretions of those, of helping to focus attention on the issues that have been to analyze to detect progress (upv, 2020). level 1:  learning outcome: questioning reality, identifying needs of improvement and ideas that can generate value.  indicators: o question reality. o come up with ideas. o express ideas formally. o identify results. level 2:  learning outcome: come up with original ideas and approaches that add value through creativity strategies and techniques.  indicators: o identify opportunities and / or aspects for improvement. o come up with ideas and original approaches. https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 35 o employ creative strategies and / or techniques to capture formal ideas and solutions. o control results. level 3:  learning outcome: propose an action plan, including a global analysis of the innovation value.  indicators: o integrate knowledge from other disciplines. o adopt creative approaches to contents and development of classes. o propose an action plan. o analyze the value of innovation. 2.2.2. transversal competences ct-05 the transversal competence ct-05 is named “design, direct, develop and evaluate an idea effectively until concretize it in a service or product”. the development of this competence favors the student learn by doing and integrating knowledge and skills from different disciplinary areas, developing high-level intellectual skills, promoting learning and autonomous work, teamwork, and self-assessment. three levels of mastery are specified in which it is intended that students reach certain learning outcomes. for each of them, a series of indicators are defined to detect progress (upv, 2020). level 1:  learning outcome: design a working project at the draft level (without get to its execution).  indicators: o reasonably justify the need for the project. o establish clear objectives of the project. o propose actions to achieve the objectives (and assign responsible in the case of group work). https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 36 o assign the necessary deadlines to complete the planned actions (and assign responsible in the case of group work). o develop actions to achieve the objectives. o identify possible risks inherent to the project. level 2:  learning outcome: planning projects in collaboration with others in situations poorly structured, anticipate incidents and risks (planning without execution).  indicators: o identify the causes that have led to the current situation of a problem. o identify the social, economic, cultural, and diverse consequences of the problem in the context of their professional field. o distinguish the parts that make up a problem and relate them to each other, identifying the main actors in their different dimensions (economic, social, ethical, technological…). o reason (critically) the solutions already proposed for a problem. o propose new solutions to the problem that has arisen based on one's own experience and available information. level 3:  learning outcome: assess and become aware of contemporary problems that affect their professional field and related fields.  indicators: o propose solutions to certain important contemporary problems in their professional field and related fields. o evaluate the proposed solutions to the most important contemporary problems in your professional field and related fields. o prioritize the best solution to the problem raised based on own experience and available information. o reframe the problem in terms of a new scenario. https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 37 o evaluate the consequences and implications of the proposed solutions to the problem in terms of a new scenario. 2.2.3. transversal competences ct-10 the transversal competence ct-10 entails to “identify and interpret contemporary problems in your field of expertise, as well as in other fields of knowledge”. this competence refers to the need for students to understand contemporary political, social, legal, and environmental issues and values, as well as the mechanisms of expansion and dissemination of knowledge. it is about developing the ability to "stay up-to-date" with current events in their field of knowledge and in society in general. to work on this competence, formative scenarios should be raised in which students discuss these types of issues in depth, being able to summarize the most relevant aspects and defend a position on it. in the same way, it is very important that they learn to evaluate complex situations using different approaches, such as: economic aspects, quality of life, environmental repercussions, local and national policies, etc. again, several levels of student's command and indicators are defined: level 1:  learning outcome: recognize contemporary issues that affect your professional field.  indicators: o identify what contents of the subject are related to a contemporary problem. o identify contemporary problems related to your professional field. o describe the problem and its main characteristics with an appropriate vocabulary in different contexts, orally or in writing. o identify valid sources of information related to the assigned problem (reliable, independent, recognized sources…). o identify solutions to the problem based on the information available level 2:  learning outcome: planning projects in collaboration with others in situations https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 38 poorly structured, anticipate incidents and risks (planning without execution). indicators: o support the context and the need for the project with evidence and data. o formulate the objectives of the project with coherence regarding the needs detected in context. o plan actions effectively (achieve objectives). o plan actions efficiently (use resources optimally). o establish monitoring mechanisms for planning. o identify possible risks inherent to the project. o review expected results. level 3:  learning outcome: designing projects in unstructured contexts by part of the professor on global scopes, contemplating the execution of the same.  indicators: o assess the needs in a real context of intervention. o specify the operational objectives of the project and establish deadlines. o plan actions and foresee contingencies. o properly manage available resources. o track the implementation of the project. o properly manage the risks of the project. 3. implementing the tcs in the framework of “design and application of industrial equipment” subject. the subject “design and application of industrial equipment” is framed in the field of industrial constructions and mechanical engineering, which is taught in the first year of the master's degree in industrial engineering at upv. it basically deals with the design of machine components (materials https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 39 in the design of machines, transmission elements, elastic elements, cams and unions, etc.), and about indoor transport systems and maintenance. students must have the skills, knowledge, and abilities to carry out the verification and control of facilities, processes, and products, as well as knowing the methods and techniques of transport and industrial maintenance. this trains students to work in different types of companies in the industrial field. therefore, in addition to learning to design and calculate elements and components of machines that are common to the different types of industrial equipment (brakes, clutches, springs, screw connections, power screws, bearings, etc.), the students will be able to apply the methods and techniques of indoor transport and material handling systems in any type of industrial facility. for instance, problems of movement of light or heavy loads arise, either in a discrete way (level change systems, lifting devices, overhead cranes, handling trolleys, etc.) or continuous (conveyor systems by belts, rollers, wheels, splints, chains, paternosters, aerial, etc.). the subject comprises 11 evaluation acts, none of which individually exceed 40% of the final grade. the evaluation will be carried out through two objective tests with multiple choices, two openresponse written answers, mark of three academic works, a project-based learning carried out in groups or 2-3 students and orally defended in classroom and a case study (table 1). table 1. evaluation system. description number of acts weight (%) objective tests (multiple choice) 2 35 open-response written answers 2 25 academic work 3 7,5 case study 1 7,5 project 1 25 the activities carried out for the acquisition of the competences entails carrying out an academic work in groups of 2-3 students, solving different problems, the use of several techniques for generating ideas, and a swot analysis. this analysis is a strategic planning technique that helps students to identify strengths, weaknesses, opportunities, and threats related to subject. in this sense, students must carry out in groups of 2-3 components a series of classroom and laboratory practices, with a weight of 17.5% in the final grade. in the practices they must analyze the indoor transport and maintenance of products in industrial facilities of any type of activity. each group must carry out the https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 40 analysis of the problem raised, trying to find technically viable solutions, as well as proposing creative and innovative solutions that improve the systems currently on the market, with respect to the characteristics of the equipment and facilities proposed in practice. in one of the practices students will apply a technique for generating ideas, innovation and creativity of new concepts and technical solutions, applied to proposals for the design of equipment and industrial facilities. for example, the scamper technique could be applied (eberle, 1996). this technique is an activity-based thinking process that consists of the application of a checklist used to create new ideas about improvements or new products / services while avoiding leaving out concepts that may be interesting. it is an acronym for the words / terms that define each of its letters and that identify the following ideas:  substitute comes up with another topic that is equivalent to the present topics.  combine adds information to the original topic.  adjust identifies ways to construct the topic in a more flexible and adjusted material.  modify, magnify, minify creatively changes the topic or makes a feature/idea bigger or smaller.  put to other uses identifies the possible scenarios and situations where this topic can be used.  eliminate removes ideas or elements from the topic that are not valuable  reverse, rearrange evolves a new concept from the original concept. therefore, this technique as a teaching strategy helps the students to analyze the knowledge in its creative form and helps the teacher to make teaching creative and interesting. with regard the academic work, it will consist of carrying out an analysis of facilities for indoor transport of goods and materials in a real company. the students must justify the proposals leading to improvements regarding the current situation of the industrial plant. for instance, the topics of the academic work include:  docks for loading and unloading goods, with their maneuvering areas.  storage and transportation facilities that use load level compensation equipment.  facilities for the production, storage, or dispatch of materials where handling equipment supported on the ground is used. https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 41  freight transport facilities that use motorized load lifting equipment.  automated freight transport facilities between production areas.  facilities with automated cargo palletizing systems.  robotization of industrial facilities. after the completion of the academic work, students should be able to answer the following questions:  how is the system / facility currently implemented?  what could be done to improve its performance, reliability, or security?  what element or mechanical system of the equipment or installation can students design and calculate? the analysis of the industrial activities of the company, and the proposal to improve the selected facilities, will allow the development of the ct-10 competence. the teachers who evaluate the evidence of compliance of ct-10, will apply to justify its achievement, the rubric shown in the following shown in table 2. table 2. rubrics for evaluating ct-10 indicators descriptors 0. not reached 1. adequate 2. good / excellent proposes solutions to the problems treated according to the subject of the work does not propose viable solutions appropriate to the problem posed proposes few solutions or little elaborate solutions it proposes solutions with an optimal or satisfactory level of detail. demonstrates good application of the contents of the subject evaluate the proposed solutions does not evaluate the feasibility of the proposed solutions evaluate solutions superficially evaluate the solutions with an optimal or satisfactory level of detail, as discussed in the subject relate any of the objectives of the sustainable development goals or challenges 4.0 as a contemporary problem it does not relate the proposal to any of the sdgs or challenges 4.0 little does the proposal relate to the sdgs or challenges 4.0 relate the proposal with an optimal or satisfactory level of detail with the sdgs or challenges 4.0 https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 42 the use of the rubric will offer a grade between 0 and 4, with the following scale:  grade between 0.00 and 0.99: not reached (d)  grade between 1.00 and 1.99 in development (c)  grade between 2.00 and 2.99 adequate (b)  grade between 3.00 and 4.00 excellent (a) among the different teachers of this subject are the authors of this paper, professors llopis-albert and rubio, which have an extensive experience in this field. to be able to transmit knowledge effectively, especially at the master's level, it is important that teachers keep up to date with technological advances in the subject. for instance, they have published articles closely related with the subject: llopis-albert et al., 2015; 2018; 2019; 2019a; 2020; 2020a; rubio et al., 2015; 2016; 2019; 2019a; 2020; and valero et al., 2017; 2019; 2019a. 4. results and discussion there are 300 students enrolled in the subject of subject “design and application of industrial equipment” in the master's degree in industrial engineering at upv for the 2019-2020 academic year. the results obtained have been very satisfactory given the great acceptance by the students, which have actively participated in all activities. regarding the activities related to the competences, all students but seven have passed the evaluation activities based on the aforementioned indicators and rubrics. furthermore, most of students reached a high grade in the competences, i.e., an a or b mark. this shows that the evaluated competences were widely addressed in classroom. although the academic works were less guided activities and student should work autonomously, they did them correctly. further improvement for the next academic year includes to better fit in the calendar the different installments of the academic works, considering the workload of the students, and the elaboration of improved indicators and rubrics for their evaluation. the main indicators of success of the activities carried out are the excellent pass rate and the high correlation that exists among the evaluation of the academic works, the open response written answers, the objective tests, the case studies, and the final grade of the subject. https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 43 4. conclusions this paper presents a teaching innovation project based on applying a system of rubrics and indicators to evaluate several transversal competences as defined at upv. the proposed evaluation activities have been proven to be a valid tool for the achievement of such competences, and to improve both the students’ performance in a mechanical engineering subject, and their level of satisfaction. in fact, the student’s satisfaction survey regarding the subject, evaluation activities and teachers are very positive. results have shown an excellent pass rate and a high correlation among the evaluation of academic works, the open response written answers, the objective tests, the case studies, and the final grade for the subject. as future enhancements for next academic years it is expected to better fit in the calendar the different installments of the academic works and the elaboration of improved indicators and rubrics for the evaluation. author contributions: conceptualization, c.l.a; methodology, c.l.a and f.r.; validation, c.l.a and f.r.; formal analysis, c.l.a and f.r.; investigation, c.l.a and f.r.; resources, c.l.a and f.r.; data curation, c.l.a and f.r.; writing original draft preparation, c.l.a and f.r.; writing review and editing, c.l.a and f.r.; visualization, c.l.a and f.r. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references eberle, b. (1996). scamper: games for imagination development. prufrock press inc. isbn 978-1-88266424-5. llopis-albert, c., rubio, f., valero, f. (2015). improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research, 68 (7), 1429-1431. https://doi.org/10.1016/j.jbusres.2015.01.027 llopis-albert, c., rubio, f., valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences, 5(1), 1-16. https://doi.org/10.4995/muse.2018.9867 llopis-albert, c., rubio, f., valero, f. (2019). fuzzy-set qualitative comparative analysis applied to the design of a network flow of automated guided vehicles for improving business productivity. journal of business research, 101, 737-742. https://doi.org/10.1016/j.jbusres.2018.12.076 llopis-albert, c., rubio, f., valero, f., liao, h., zeng, s. (2019a). stochastic inverse finite element modeling https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.jbusres.2015.01.027 https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.1016/j.jbusres.2018.12.076 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 44 for characterization of heterogeneous material properties. materials research express, 6(11), 115806. https://doi.org/10.1088/2053-1591/ab4c72 llopis-albert, c., valero, f., mata, v., pulloquinga, j.l., zamora-ortiz, p., escarabajal, r.j. (2020). optimal reconfiguration of a parallel robot for forward singularities avoidance in rehabilitation therapies. a comparison via different optimization methods. sustainability, 12(14), 5803. https://doi.org/10.3390/su12145803 llopis-albert, c., valero, f., mata, v., zamora-ortiz, p., escarabajal, r.j., pulloquinga, j.l. (2020a). optimal reconfiguration of a limited parallel robot for forward singularities avoidance. multidisciplinary journal for education, social and technological sciences, 7(1), 113-127. https://doi.org/10.4995/muse.2020.13352 rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2015). assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering, 10 pages. article id 931048. https://doi.org/10.1155/2015/931048 rubio, f., llopis-albert, c., valero, f., & suñer, j. l. (2016). industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. robotics and autonomous systems, 86, 106–112. https://doi.org/10.1016/j.robot.2016.09.008 rubio, f., llopis-albert, c. (2019). viability of using wind turbines for electricity generation in electric vehicles. multidisciplinary journal for education, social and technological sciences, 6(1), 115-126. https://doi.org/10.4995/muse.2019.11743 rubio, f., valero, f., & llopis-albert, c. (2019a). a review of mobile robots: concepts, methods, theoretical framework, and applications. international journal of advanced robotic systems, 16(2), 172988141983959. https://doi.org/10.1177/1729881419839596 rubio, f., llopis-albert, c., valero, f., besa, a.j. (2020). sustainability and optimization in the automotive sector for adaptation to government vehicle pollutant emission regulations. journal of business research 112, 561-566. https://doi.org/10.1016/j.jbusres.2019.10.050 upv, 2020. proyecto institucional competencias transversales. universitat politècnica de valència (upv). valencia. spain. https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf valero, f., rubio, f., llopis-albert, c., cuadrado, j.i. (2017). influence of the friction coefficient on the trajectory performance for a car-like robot. mathematical problems in engineering, 9 pages. article id 4562647. https://doi.org/10.1155/2017/4562647 valero, f., rubio, f., llopis-albert, c. (2019). assessment of the effect of energy consumption on trajectory improvement for a car-like robot. robotica, 37(11), 1998-2009. https://doi.org/10.1017/s0263574719000407 valero, f., rubio, f., besa, a.j. (2019a). efficient trajectory of a car-like mobile robot. industrial robot: the international journal of robotics research and application, 46(2), 211–222. https://doi.org/10.1108/ir10-2018-0214 https://doi.org/10.4995/muse.2021.15244 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1088/2053-1591/ab4c72 https://doi.org/10.3390/su12145803 https://doi.org/10.4995/muse.2020.13352 https://doi.org/10.1155/2015/931048 https://doi.org/10.1016/j.robot.2016.09.008 https://doi.org/10.4995/muse.2019.11743 https://doi.org/10.1177/1729881419839596 https://doi.org/10.1016/j.jbusres.2019.10.050 https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf https://doi.org/10.1155/2017/4562647 https://doi.org/10.1017/s0263574719000407 https://doi.org/10.1108/ir-10-2018-0214 https://doi.org/10.1108/ir-10-2018-0214 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 71 replicating a study about children’s drawings concerning radiation t. plotz*, f. hollenthoner (orcid id: 0000-0002-7265-8149) aecc physik. university of vienna, porzellangasse 4/2/2, 1090 vienna, austria. * corresponding author: email: thomas.plotz@univie.ac.at; phone: + 43 1427760333 received: 2018-08-15; accepted: 2019-02-21 abstract radiation surrounds us in various forms and plays a huge role in our everyday life. however, little is known about student and children’s conceptions of this topic. this study is part continuation part replication of the studies carried out by neumann and hopf (2013). the method employed in both studies was identical. 459 students drew pictures associated with the concept “radiation” under observation. the resulting motives were subsequently categorized and compared. in this study the children barely associate the concept of “radiation” with the fukushima nuclear disaster. moreover, a number of differences could be realized when compared to the reference study. for instance, significantly more students drew cell phones and computer monitors in the current study. additionally, a greater number of drawings related to radioactivity could be observed. overall, the findings of this work indicate that not only are students exposed to the media at a much younger age, but also more frequently. this leads to the conclusion that more and more children build their own understanding of a particular subject, which could potentially result in misconceptions. keywords radiation, children drawings, replication study, fukushima, radioactivity https://doi.org/10.4995/muse.2019.10390 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 72 1. introduction students’ conceptions about radiation are a field of little interest in empirical science education research. however, this topic influences our everyday life on a broad basis. from mobile phone radiation to x-rays in medicine we are surrounded by radiation every day. neumann and hopf (2013) investigated the drawings of over 500 children to answer the question, what motifs are drawn and do these motifs change over time? this study replicates the study by neumann and hopf and expands the question. after five years the topic of radiation and fukushima has disappeared from the media. therefore it is interesting to find out if the increase of motifs related to radioactivity documented by neumann and hopf is visible today. makel and plucker (2014) stated the importance of replication studies in education research. they point out the fact, that only 0,13% of all published studies are replication studies. replication studies have the potential to “identify, diagnose, and minimize many of the methodological biases (makel and plucker p. 305)“. there are many ways to design a replication study (for a good overview, see schmidt 2009). in our replication study we use the same design and method to analyze the drawings. we also tried to investigate children from the same schools to enable a good comparison. additional children from a rural area were part of this study to compare them to children from the city. 2. previous findings many studies in the mid-nineties (h. eijkelhof and millar 1988; h.m.c. eijkelhof 1996; h. m. c. eijkelhof et al. 1990; lijnse et al. 1990; millar et al. 1990; millar 1994; millar and gill 1996) revealed frequent misconceptions (false conceptions) on nuclear radiation. they documented problems with the concept of contamination and irradiation or the concept of activation: an object emits radiation after being exposed to radiation. additionally, students linked the effects of nuclear radiation to other environmental issues like the greenhouse effect or the ozone layer. a lot of these misconceptions can be found multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 73 today. sesen and ince (2010) reported in their study that those misconceptions are widely spread on the internet. this source for students to learn can stabilize their misconceptions. other studies focusing on invisible radiation as distinct from nuclear radiation came up with very concerning results. rego and peralta (2006) found that students were unable to distinguish between non-ionizing and ionizing radiation. most of the students could not tell the difference between various types of radiation. libarkin et al. (2011) focused on infrared (ir) and ultraviolet (uv) radiation. the majority of students (age 10 to 16) believed that the sun is the only origin for uv-radiation. they also described uv as “light,” “bright light,” “strong rays,” “very violet,” “a color like red, blue, purple light,” or “harmful rays.” concerning ir-radiation a lack of knowledge was revealed; scarcely anybody had heard of this kind of radiation. plotz (2017) summarized conceptions of nuclear radiation and various types of electromagnetic radiation. he provides a recent overview of the excising literature. a lot of the mentioned studies dealt with misconceptions and associations linked with radiation. overall we see a nonpoint picture of the students’ conceptions. there is a clear gap in the knowledge of students’ conceptions in the field of radiation. due to the design of this study, the most important findings are those of neumann and hopf (2013). they showed first and foremost, that children are likely to draw the sun, when they are asked to draw about radiation. about 70 percent of the students drew a picture of the sun or sunlight. overall, neumann and hopf showed, that the motifs are often connected to visible light and to sources of radiation like mobile phones and monitors. there was also a shift in motifs from younger (sun, visible light) to older children (artificial light sources). in addition to the first study 2009 neumann and hopf (2011, 2012), also investigated the change in motifs in 2011 after the nuclear accident in fukushima. in figure 1, this change in percentage of motifs is visible. neumann and hopf documented a significant change in the motifs connected to radioactivity. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 74 figure 1. results according to neumann and hopf (2013) they explained the change with the enormous amount of coverage of the accident in fukushima in the media. this connection has been documented in the drawings and in interviews with the students after they drew the first set of pictures. they also saw a significant increase of the motifs related to visible light sources and a significant decrease in the pictures of mobile phones. both changes were not explained in the study. as mentioned above, there is a limited set of known conceptions concerning radiation. 3. research method the method of drawing associations is used to identify students’ perception of the concept “radiation”. students are asked to draw pictures associated with the subject under consideration. in general, this method originated in the psychological field in the middle of the 19th century. there it was used to get some impression of the psychological state of children, to get a “window into his/her thoughts and feelings”. later on it was believed, that drawings of children could tell us something about their intelligence. this multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 75 assumption did not prove right and was dismissed. however, the idea that children’s drawings could provide information about their understanding of a particular concept proved fruitful and was further examined. in natural sciences, this technique was already used by chambers (1983) between 1966 and 1977 in his famous “draw-a-scientist test”. in this test children had to draw a picture of what they believed a scientist looked like. white and gunstone (1992) also used this methodology to investigate processes of understanding. children were asked to draw a teacher while he is teaching, at the beginning of the school year. most of the drawings showed conventional teaching sequences, where a teacher stands in front of the class and children sit in rows facing him. at the end of the year those school students were asked to repeat the exercise. the resulting drawings looked quite different, as this year the children were taught in a very open way and the pictures reflected that. they showed a lot of mixed class structures, where for example a lot of teamwork sessions without teachers could be seen. dikmenli (2010) used this method to discover perceptions concerning cell division. rennie and jarvis (1995) investigated what children understood under the term “technology”. the following picture (figure 2) makes clear, that the results of rennie and jarvis (1995) are comparable to the drawings of this study. in both studies the children had to draw pictures of abstract concepts or terms. figure 2. a year 6 boy’s drawing (rennie and jarvis 1995, p.246) multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 76 in general, this method has a lot of advantages, as well as a number of disadvantages. its biggest advantage is that it turns out to be ideal to examine concepts of progresses inherited by students. dove et al. (1999) used it in their study of children between the age of nine and eleven, who were asked to draw a water cycle. it turned out, that most of the proponents did not know where a river starts, however they did know where it ends. most of the time, rivers were drawn flowing from left to right. all this information probably would not have been observed with, for instance, interviews. another advantage of this method is that it is very open. questionnaires, especially multiple-choice tests have a lot of limiting factors, which limit the forms of expression of the proponents. on the other hand, questionnaires are far easier to interpret than drawings. this leads to one of the biggest problems of this method. it is very difficult to interpret drawings. the statement that drawings enable us to look into the “heads” of children is an assumption, not a fact. it must also be considered, that drawing is not an easy exercise for children. they need to have certain cognitive skills and it could happen that they are faced with “organizational and procedural problems”. for example, (brown et al. 1987) asked children to draw pictures of what they associated with nuclear power plants. no human beings could be observed around the power plants in most of the drawings. the reason for that was not that they did not imagine any people in that environment. it is very difficult for school students to draw humans and so the children avoided that task. in addition to that, children between the age of nine and ten start to criticize their drawings. it is frustrating to them, when a drawn object does not look like the real one. last but not least, the common education systems lack the facilitation of drawing skills. from the age of six to ten school children still draw a lot but then at each step up the education ladder this skill is replaced by writing and reading. in summary, this method gets more and more impractical as students get older. nevertheless sometimes emotions of students are revealed through a drawing. there are examples where the picture shows happy children in the sun. other one, obviously multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 77 associated power plants with negative feelings because the look at the face of the person is sab. in general, the method is useful with younger children. it is real fun and therefore motivates students to participate. thomas and silk (1990) wrote, „children make drawings largely for the satisfaction they get from the activity”. nearly every tested class welcomed the diversion from writing and reading. it seemed that the task had even a relaxing effect for most of the children. in conclusion, the method is highly underestimated in its values to examine conceptions of children. however, it must be considered, that its usefulness depends on the subject under consideration and the age of students. currently a study is being carried out, testing this technique with older children (age 16 to 18) on the subject of radiation. we implemented the procedure described by neumann and hopf to collect drawings from the students. the students received a blank sheet of paper and the word “strahlung” (the german word for radiation, as used in the term “elektromagnetische strahlung”) was written on the board in the classroom. we or the teacher asked the students to draw whatever comes to their minds, whenever they read or hear the word. during the period of drawing, the teacher did not answer any subject-specific questions and the students were encouraged to draw any motif. after 10 to 15 minutes the drawings were collected. to analyse the drawings, we categorised the motifs using the five main categories from neumann and hopf.  the sun  artificial source of light (lamps, flashlight, …)  motifs related to radioactivity (nuclear power plants, radioactive warning sign…)  mobile phones  monitors (all types of different screens like, tv, computer or laptop) in addition to these categories, we found new motifs, which we grouped into several side categories. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 78 in this study, we collected drawings from 459 students (age 9 to age 12) from seven different schools in austria. three schools are located in vienna and four schools are in the countryside of upper-austria. the drawings were made and collected in december 2015. as shown in figure 4 the process of data collection was done four years after the second and six years after the first study. to be able to compare our data to the previous study of neumann and hopf we decided to collect drawings in two schools from their study (both in vienna). table 1. qverview of the collected sample. 2015 town countryside total 4th grade 54 39 93 5th grade 68 109 177 6th grade 45 144 189 total 167 292 459 to conclude the setting of the replication study was as similar to the original study as possible to be able to compare the numbers. however, there was a major change in the data collection process. we did not differentiate the gender of the students. this decision was based on the fact that the results did not vary significantly for most categories in the original study. we focused more on the overall numbers and the variation between the different age and location of our students. we addressed the following research questions. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 79 figure 3. timeline of the different studies 3.1 research questions due to the fact that this study is a replication study, the most important research question is if the results are the same as in the study of neumann and hopf (2013). therefore the first two research questions match the questions from the previous study.  what do younger students (age 9 to 12) associate with the term “radiation”?  do these associations change with the age of the students? the fatal accident in fukushima occurred in march 2011. since then the attention in the media has vanished. neumann and hopf (2013) assume in their discussion that media (newspaper, tv) and the discussions in school caused the increase of motifs related to radioactivity. hence, we formulate the hypothesis that the number of associations with radioactivity should decrease to the level of 2009. in addition to the questions above, we investigated two research questions.  do students from the countryside associate other motifs than students from the city?  are there other motifs in our drawings beside the main categories? multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 80 4. results first we compared the overall numbers with the results of the study from neumann and hopf (2013). in figure 4, the results are shown for the five main categories. the percentage of drawings picturing the sun (χ2=1.145, df=1, p<0.01) and radioactivity (χ2=1.317, df=1, p<0.01) has not changed in a significant way, compared to 2011. what stands out is the vast increase of pictures of mobile phones (χ2=38.1, df=1, p<0.01) and monitors (χ2=30.99, df=1, p<0.01). both doubled the percentage from 2011 to 2015. we assume, that the different accessibility to mobile phones and computers are the main reasons for this increase. our hypothesis, that the increase in radioactive motifs from 2009 to 2011 should vanish, can be rejected, due to the slight but not significant increase in the percentage (to about 35%). we can also see a significant increase of pictures containing artificial light sources (χ2=12,11, df=1, p<0.01). figure 4. comparison of the five main categories multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 81 to investigate the second question further, we ordered the appearance of different motifs to the different grades. we can observe a very similar trend to neumann and hopf (see also figure 5). (a) (b) figure 5. percentage of motifs from neumann hopf 2013 (a) and from actual study (b) there is a decrease in motifs showing the sun and artificial light sources. for both motifs we see a significant drop-off between grade four and five. for the sun we have χ2=5,68, df=1, p<0.05 and for artificial light source the calculated χ2 is χ2=7,34, df=1, p<0.05. looking to the other categories, a increase in the frequency of the appearance of mobile phone, radioactivity and monitors in various forms can be seen. in the categories mobile phone and radioactivity there is a clear significant jump from grade four and grade five. however, the difference between grade five and grade six is not significant for the first four categories. there is an interesting new finding in the comparison of the different grades. in our analyses we found a decent amount of pictures depicting laser in various forms. these motifs are new and were not documented before. they appeared on 11,3% of the drawings in all grades. however, there was a difference in the grades. the younger children drew the motif more often (27%) than the older one (5% in grade 6). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 82 in addition to the appearance of laser-themed drawings we identified two interesting subcategories. neumann and hopf found, that the vast majority (81,4%) of the drawings related to nuclear radiation had a negative connotation, especially after the accident in fukushima. they even pointed out that only three drawings showed positive aspects of radioactivity. although our results showed a similar amount of drawings related to radioactivity, we also found in 13,7% of our drawings positive aspects related to radioactivity or radiation in general. there are different examples for this in the drawings like a smiling worker in a power plant or a smiling person in a “radiation chamber”. the second subcategory we called the etymology category. to fit into this category the motifs should contain a connection to the word radiation or radiate. one example for those motifs was a picture of a smiling face. the german language normally uses the term “to radiate with joy” instead of “to beam with joy”. so therefore there is a connection for the children to the word radiation. the same argument can explain the appearance of the number line in some drawings. the german translation would be “zahlenstrahl” (“number ray”). we found those motifs in about 4,1% of our drawings. the comparison of the students from the schools in vienna to their colleagues in the countryside can be seen in figure 6. there is no significant difference between those two groups in the two main categories sun and artificial light source. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 83 figure 6. comparison of the main categories town/countryside in the main categories mobile phone (χ2=27,21, df=1, p<0.01), radioactivity (χ2=26,34, df=1, p<0.01) and monitors/screens (χ2=17,88, df=1, p<0.01) there is a significant difference. in all of those categories the students from the countryside drew the motifs more often. the subcategory laser shows that the difference is the other way round (χ2=11,5, df=1, p<0.01). that the proposed methodology is on the way of being introduced right now, as the subject is taught this year for the first time (academic year 2012-2013), and during the spring semester. consequently, the results that will be presented and discussed will be, up to now, incomplete. finally some observations, which seem to show some disadvantages or difficulties that may appear during the evaluation process, together with some proposals to overcome them, will be mentioned. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 84 5. discussion and conclusion looking into the results there is not much of a connection to the nuclear disaster of fukushima. the slight increase in the radioactivity category occurred because of the pictures of applications like cancer treatment that were linked to nuclear radiation. overall, the results correspond with the previous study and the overall trend in the appearance of the different motifs, although there are some interesting exceptions, like the picture of lasers from above. at the time of the data collection the new star wars film (the awakening of the force) premiered and laser swords were part of the everyday life of the students. therefore it is reasonable to link the appearance of this topic relate to radiation in the media to the frequency of the pictures of this topic in our research. this link is also a possible explanation for the increase of motifs for radioactivity in the original study. neumann and hopf hinted this link in their discussion as they wrote: „the analysis of the interviews reinforced our hypothesis that the reason for this change in the students’ associations could be found in the tragic events of fukushima.“ (neumann and hopf 2013) however, we are not able to explain the result that in 2015 the frequency of radioactive motifs is as high as in 2011. the media coverage has rapidly decreased in the years after 2011 and so there should be a decrease. the results did not show much of a connection to the nuclear disaster of fukushima. the slight increase in the radioactivity category maybe occurred, because of pictures of applications like cancer treatment that were linked to nuclear radiation. we think that this point should be investigated further in the future. the initial hypotheses that the frequency of motifs related to radioactivity should decrease could not be confirmed. the increase in the initial study can therefore not be linked easily to the media coverage. in our analyses we found pictures with smiling faces or the number ray and we decided to put these motifs into a new category. we called it the etymology category. neumann and hopf also discussed the impact of the language to their results. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 85 “we assume that these results are strongly influenced by the german language since the german word strahlung (radiation) is commonly used in everyday speech, especially in connection with the word ‚sun’“. (neumann and hopf 2013) in a similar fashion knowledge of the german language is necessary, to understand the connection between a smile and radiation. the phrase “a shining smile” means, the face is radiating in german. and the “zahlenstrahl” contains the word “strahl” which can be translated with ‘ray’. keeping those explanations in mind it seems obvious, that further research is necessary. therefore we plan to conduct a study in different european countries (denmark, italy, france,… ) to validate the hypotheses that there are certain motifs depending on the language and others that are independent thereof (for example the motif of the sun). due to the very simple instruction to the students the collected data should be comparable and we hope to get a better insight into the conceptions of children concerning radiation. new are the results from the comparison between the drawings of children living in a town to those living in the countryside. it is interesting, that there is a significant difference in three of the five main categories. in addition to these three there is an inverse difference in the laser-motif. there is no good explanation for this result. overall we think, that this study helped to solidify some results from the original study. it also showed the importance of replication studies. therefore we strongly recommend, that more replication studies should be conducted. we also see a possibility and a necessity to investigate this topic further. 6. reference brown, j. m., henderson, j., & armstrong, m. p. (1987). children's perceptions of nuclear power stations as revealed through their drawings. journal of environmental psychology, 7(3), 189-199, doi: http://dx.doi.org/10.1016/s0272-4944(87)80029-4. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 86 chambers, d. w. (1983). stereotypic images of the scientist: the draw‐a‐scientist test. science education, 67(2), 255-265. dikmenli, m. (2010). misconceptions of cell division held by student teachers in biology: a drawing analysis. scientific research and essays, 5(2), 235-247. dove, j. e., everett, l. a., & preece, p. f. w. (1999). exploring a hydrological concept through children's drawings. international journal of science education, 21(5), 485-497, doi: http://dx.doi.org/10.1080/095006999290534. eijkelhof, h., & millar, r. (1988). reading about chernobyl: the public understanding of radiation and radioactivity. school science review, 70(251), 35-41. eijkelhof, h. m. c. (1996). radiation risk and science education. radiation protection dosimetry, 68(3-4), 273-278. eijkelhof, h. m. c., klaassen, c. w. j. m., lijnse, p. l., & scholte, r. l. j. (1990). perceived incidence and importance of lay-ideas on ionizing radiation: results of a delphi-study among radiation-experts. science education, 74(2), 183–195, doi: http://dx.doi.org/10.1002/sce.3730740205. libarkin, j. c., asghar, a., crockett, c., & sadler, p. (2011). invisible misconceptions: student understanding of ultraviolet and infrared radiation. astronomy education review, 10(1), 10105, doi: http://dx.doi.org/10.3847/aer2011022. lijnse, p. l., eijkelhof, h. m. c., klaassen, c. w. j. m., & scholte, r. l. j. (1990). pupils’ and mass-media ideas about radioactivity. international journal of science education, 12(1), 67–78, doi: http://dx.doi.org/10.1080/0950069900120106. makel, m. c., & plucker, j. a. (2014). facts are more important than novelty. educational researcher, 43(6), 304-316, doi: http://dx.doi.org/10.3102/0013189x14545513. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 87 millar, r. (1994). school students' understanding of key ideas about radioactivity and ionizing radiation. public understanding of science, 3(1), 53–70, doi: http://dx.doi.org/10.1088/0963-6625/3/1/004. millar, r., & gill, j. s. (1996). school students’ understanding of processes involving radioactive substances and ionizing radiation. physics education, 31(1), 27–33. millar, r., klaassen, k., & eijkelhof, h. (1990). teaching about radioactivity and ionising radiation: an alternative approach. physics education, 25(6), 338. neumann, s., & hopf, m. (2011). was verbinden schülerinnen und schüler mit dem begriff "strahlung". zeitschrift für didaktik der naturwissenschaften, 17, 157–176. neumann, s., & hopf, m. (2012). students’ conceptions about "radiation": results from an explorative interview study of 9th grade students. journal of science education and technology, 21(6), 826–834, doi: http://dx.doi.org/10.1007/s10956-012-9369-9. neumann, s., & hopf, m. (2013). children’s drawings about "radiation" before and after fukushima. research in science education, 43(4), 1535–1549, doi: http://dx.doi.org/10.1007/s11165-012-9320-3. plotz, t. (2017). students’ conceptions of radiation and what to do about them. physics education, 52(1), 014004, doi: http://dx.doi.org/10.1088/1361-6552/52/1/014004. rego, f., & peralta, l. (2006). portuguese students’ knowledge of radiation physics. physics education, 41(3), 259–262. rennie, l. j., & jarvis, t. (1995). children's choice of drawings to communicate their ideas about technology. research in science education, 25(3), 239-252. schmidt, s. (2009). shall we really do it again? the powerful concept of replication is neglected in the social sciences. review of general psychology, 13(2), 90-100, doi:10.1037/a0015108. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10390 social and technological sciences issn: 2341-2593 plotz and hollenthoner (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 71-88 | 88 thomas, g. v., & silk, a. m. (1990). an introduction to the psychology of children's drawings: new york university press. white, r., & gunstone, r. (1992). probing understanding. london, new york: routledge falmer. multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 1 technical university students’ feedback on studying english online under the covid-19 o. chugaia , l. svyrydovaa a department of technical english. national technical university of ukraine, 37, peremohy ave, kyiv, 03056, ukraine. *correspondence: ochugai@meta.ua received: 29august 2021; accepted: 02 february 2022; published: april 2022 abstract the paper reveals the results of the survey, carried out by non-linguistic students of the national technical university of ukraine “igor sikorsky kyiv polytechnic institute”. special attention is devoted to comparative analysis of feedback from respondents studying english online at the institute of physics and technology (ipt) and instrumentation engineering faculty (ief) under covid-19. the actuality of the study is determined by the necessity of learning about technical students’ experiences studying english online during the 2020–2021 academic year for educators to design effective online courses and strategies and adjust teaching practices accordingly. to achieve the aim, the following tasks were completed: technical, methodological and psychological aspects of studying english online during the 2020–2021 academic year under covid-19, to use descriptive statistics for processing the obtained data. the results of the survey are analyzed, described and presented in the paper; recommendations are given. overall, the respondents positively assessed their experience of studying english online under the covid-19. keywords: technical university; students’ feedback; covid-19; online learning; studying english to cite this article: gold, n.e., purves, r., himonidesa, e. (2021). playing, constructionism, and music in early-stage software engineering education. multidisciplinary journal for education, social and technological sciences, 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 https://doi.org/10.4995/muse.2022.16142 https://orcid.org/0000-0002-2118-8255 https://orcid.org/0000-0002-9762-4092 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 2 1. introduction the covid-19 pandemic brought drastic changes to practically any aspect of people’s lives all over the world. strict regulations had to be followed in order to stop the disease from spreading rapidly. the necessity to provide social distancing became a threat to the quality of education. face-to-face classes were replaced by remote ones conducted online, changing educational environments around the world. the closure of educational institutions was painful not only for educational authorities, teachers, and students, but for many more people, as it affected whole families. a rapid shift from brick to click, seminar to webinar raised a lot of questions to be answered (ubaid-ullah, 2021). the research on the issue of teaching online during the pandemic is devoted mostly to technical, psychological, and methodological aspects (adarkwah, 2021; albalinero et al., 2020; erdmann et al., 2021; gokuladas & baby sam, 2020; kimmel et al., 2020). technical issues became of utmost importance for many students and teachers, sharpening the problems that existed before the pandemic (mathur & singh, 2020).svyrydova providing the only way to continue teaching and learning, the very access to online education, technical issues disclosed the disparity between high-income and low-income countries. no devices, no internet access at home revealed the digital divide, which meant that millions of students were disconnected and their studying was disrupted (unesco, 2020). even when students had internet access and devices, they lacked the necessary skills which ensured their active participation in online lessons. methodological issues related to what, how, and when it was better to teach in order to reach the goals of the language course, were also important, as online is a totally different environment compared to conventional. therefore, adjusting the regular practices according to the quarantine restrictions and trying to maintain a continuous educational process, teachers had to use new methods related to planning and designing online courses, preparing students for learning english online, conducting a transparent and fair assessment, interacting effectively (kimmel et al., 2020, p. 4). teaching online was not what teachers of english usually did before the pandemic. however, it is considered a new reality impossible to deny and according to which teaching methods are chosen and exploited differently (yucesoy-ozkan et al., 2020, p. 51). psychological issues related to the difficulties to adapt to new reality, to communication online, which teachers and students had to do confined to their homes. the very thought of https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 3 impossibility to go anywhere, to have choices about the ways to study and spend their free time was difficult to accept for both, teachers and students. social distancing in public places, looking at black squares instead of human faces during video conferences, practically unlimited screen time certainly affect the mental health of students (erdmann et al., 2021, p. 23). the necessity of reflecting on students’ experience and their learning outcomes became the precondition for research in the area of teaching online (camilleri, 2021, p. 4). the issues of compatibility and readiness of students to study online, factors that motivate and hinder remote learning, strategies for the effective implementation of online learning at higher education institutions are of vital importance (khalil awan, 2021). researchers focused on the evaluation of online courses (lukianenko & vadaska, 2020), psychological and operational aspects of online education (gokuladas & baby sam, 2020), obstacles to effective learning online (aslam, 2021; chang, 2020; dogar et al., 2020; kimmel et al., 2020), supervision and evaluation of online learning experience (adarkwah, 2021, p. 1, 2). indeed, there are some important issues that are still uncovered by research. therefore, the actuality of the study related to feedback of technical students was determined by the necessity of learning what students of a technical university thought about their experience of studying english online for educators to develop more effective strategies, adjust teaching practices to students’ needs, and plan the language course accordingly. the article aims to conduct a comparative analysis of feedback from ukrainian technical university students studying english online at the institute of physics and technology (ipt) and the instrumentation engineering faculty (ief) under the covid-19. to achieve this aim, the following tasks are to be completed: to establish technical, psychological, and methodological aspects of studying english online during the 2020–2021 academic year under covid-19, to exploit descriptive statistics for processing the obtained data, and give recommendations for teachers of english. in the study, we propose two hypotheses: hypothesis a: technical university students had mostly positive experiences learning english online during the 2020–2021 academic year. hypothesis b: technical university students had mostly negative experiences learning english online during the 2020–2021 academic year. https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 4 2. materials and methods research design. a mixed-method research design was used in the study: quantitative and qualitative data were obtained from the surveys completed by students of the institute of physics and technology (ipt) and the instrumentation engineering faculty (ief) of the national technical university of ukraine “igor sikorsky kyiv polytechnic institute”. the students were instructed by the teachers who worked at the department of english for engineering. qualitative data was received through feedback via zoom sessions and in written form. participants. this study was conducted at the national technical university of ukraine “igor sikorsky kyiv polytechnic institute”. the online mode of studying was introduced in march 2020 in ukraine; therefore, technical university students had no regular face-to-face classes in the 20202021 academic year. the survey was completed by institute of physics and technology (ipt) students (n = 25) and instrumentation engineering faculty (ief) students (n = 43) in the first, second, and third years of study in june 2021. about 700 students’ responses in total were analyzed. the participants had different language proficiency levels, from intermediate to advanced. this study is ethical because all respondents volunteered to take part in the survey and gave their consent. instruments and procedure. the survey used a likert scale (prompt 1: “never” / “practically never” / “from time to time” / “often” / “always”; prompts 2-8: “strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”; prompt 9: “excellent” / “good” / “quite good” / “could be worse”; prompt 10 provided multiple responses) (see appendix a). the researchers exploited descriptive statistics; they calculated the median of each item, i.e., what most respondents believed, and the interquartile range of each item, i.e., how strongly respondents agreed with each other. the median (mdn) was used to measure central tendency, the inter-quartile range (iqr) showed the spread of the responses. the survey covered technical, psychological, and methodological aspects of studying english online during the 2020–2021 academic year under covid-19. in addition, there was an opportunity for the respondents to leave additional comments at the end of the survey in google form. 3. results and discussion the findings of the study showed that most ipt and ief students indicated agreement with prompt no. 1 (see appendix a), that they practically never experienced technical problems (mdn = https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 5 2, iqr =1). it could be partially explained by the fact that considering the negative experience at the beginning of the quarantine in march 2021, there was an official requirement for students to have the necessary equipment and internet access to ensure studying online at the beginning of the 2020-2021 academic year. these results are consistent with the previous studies at a spanish university, which showed that most students did not have technical problems (alba-linero et al., 2020, p. 85). however, some spanish students complained of poor internet connectivity (albalinero et al., 2020, p. 86). other difficulties included the high cost of internet data bundles, technical problems with online platforms, electricity supply, lack of it skills, and limited ict tools (adarkwah, 2021, p. 1). responding to prompt no. 2 about studying from home, ipt students strongly agreed (mdn = 1, iqr = 1), while ief students agreed (mdn = 1, iqr = 1.5) with that statement (see appendix a). some students mentioned in their comments that they managed to study while working; some stayed at the dormitories in kyiv, not in their hometowns. according to the responses, studying online was comfortable (prompt no. 3), as most ipt and ief students agreed with that statement (mdn = 2, iqr =1). they provided some reasons for such a choice: saving traveling time, cutting expenses on food and accommodation, focusing on the subjects that were important to them. the responses showed that students had enough free time to find a job, new hobbies, and take extra courses online (prompt no. 4, see appendix a), as most ipt and ief students agreed with that statement (mdn = 2, iqr =1). students explained in their comments that they discovered some free online courses during the pandemic, and they decided to use that opportunity to complete one or even more courses on different subjects. besides focusing on their specialty, students communicated in english and practiced using it in real-life situations. it is important to know that all the students experienced studying online in spring 2020 (march, april, may). some attended additional classes online in the summer, got part-time jobs, and, in spite of all the restrictions of the quarantine, managed to study and work. it was predictable that, because of the pandemic, there would not be face-to-face classes in the 2020-2021 academic year, and students had already accepted that adjusting to the new reality. technical university students were familiar with cooperative learning before the quarantine as teachers used various interaction patterns; learning online, they got used to the breakout rooms of a videoconferencing platform called zoom. most ipt and ief students agreed with prompt no. 5 (see appendix a) about enjoying work in pairs or groups (mdn = 2, iqr = 1). the experience https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 6 related to interaction patterns was closely related to interactive activities students were engaged in: before the pandemic, ipt and ief students also played games, completed quizzes, and did roleplays at the english lessons working individually, in pairs, small groups, or whole class. that is why some researchers indicated that familiar activities made communication online easier: gamification and collaboration fostered online learning acceptance by teachers and students (adarkwah, 2021, p. 1, 2); games, quizzes, and role-plays created a relaxing atmosphere at the english lesson (chugai, 2020; stavytska et al., 2021). according to another research, the introduction of gamification tools facilitated meaningful learning, having a positive impact on the professional motivation of the teacher and the academic performance of students as well (gómezejerique, & lópez-cantos, 2019, p. 67). overall, the results of our research confirmed that most students of both faculties enjoyed games and quizzes like quizlet, kahoot, wordwall, in their english class. it could be explained by the fact that engaging in interactive communication lowered the affective filter that was caused by negative emotions, and, consequently, enhanced students’ ability to use english freely (larsen-freeman & marti anderson, 2011, p. 294). however, a comparison of the results showed though that ipt students were more enthusiastic about gamification responding to prompt no. 6 (see appendix a), indicating “strongly agree” (mdn = 1, iqr = 1), while ief students chose “agree” and their opinions were somewhat polarized (mdn = 2, iqr = 2) which could be explained by individual differences. responding to prompt no. 7 about assessment (see appendix a), most ipt and ief students agreed that tests were easier to complete online than face-to-face (mdn = 2, iqr =1). students took quick classtime tests aimed at checking the material of the previous lessons or home assignments. unfortunately, taking tests online was attractive for some students because of the opportunity to cheat and get more points (chugai, 2021, p. 111). these results differ from those provided by another study when students of a spanish university did not indicate any changes resulting in improvement or worsening related to learning online (erdmann et.al., 2021, p. 33). in fact, some universities practiced open-book examinations (obe) when students could use any supplementary materials in order to complete a test and learn something new (ubaid-ullah, 2021, p. 1). however, it was not a usual way of conducting tests or examinations at technical universities in ukraine. still, teachers set time limits, created a friendly atmosphere, conducted error correction after revealing the answers and provided feedback, aiming at motivating students to be honest and not be afraid of making mistakes. detailed feedback on students’ performance was of vital https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 7 importance for motivating and encouraging them to take an active position, to be responsible for their progress (nikolaeva, korol, 2021). at the same time, some students were aware of the possibility to cheat when taking tests online. using alternative assessment methods such as presentations, peer-review, self-assessment, or rubric-based projects could encourage students to act in accordance with academic integrity principles. according to the feedback for prompt no. 8 (see appendix a), most ipt and ief students thought the english lessons were beneficial for them (mdn = 2, iqr =1). assessing their overall experience learning english online in response to prompt no. 9, most ipt and ief students thought it was good, but at the same time, ipt students were more decisive about that (mdn = 2, iqr = 0) than ief students (mdn = 2, iqr = 1). in general, the responses of technical university students to the prompts of the survey were quite similar (mdn = 2, iqr = 1), just the ones about working in pairs or groups were slightly different (mdn = 2, iqr =1.25). the results of the survey showed that while studying english online, non-linguistic students practically never experienced technical problems. furthermore, most studied from home, which was comfortable for them. the students had free time to find new hobbies, get a job, or do extra courses online. most respondents enjoyed working in pairs or groups, as well as interactive activities like games, quizzes, and role-plays. most students claimed that online tests were easier than face-to-face. the respondents considered english lessons online beneficial for them and assessed their overall experience as good, which is in keeping with the results of another survey according to which students of a spanish university were content with new digitalized learning environments within their institutions (erdmann et.al., 2021, p. 34). the results presented here do not support the findings of adarkwah (2021), who claimed that in spite of considering the adoption of online learning as the best way to continue education during the pandemic, students concluded that the online instruction was ineffective. it is worth mentioning that the results obtained in our research were proved by regular tests and the final module tests as well. responding to prompt no. 10 (see appendix a) about the ways of looking for help with english assignments, respondents could choose several options (see table 1). https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 8 table 1. responses about ways of looking for help with english assignments (prompt 10) options students ief, % students ipt, % difference, % ask a teacher 9 23 -14 ask a peer 15 17 -2 google it 44 23 +21 use the context 9 23 -14 all above 23 14 +9 ief students chose “google it” first of all (44% out of all the choices they had), while ipt students most frequently chose three options: “ask a teacher”, “google it”, and “use the context” (23% each). the average result for technical university students was the choice “google it”, which was predictable. in fact, even in face-to-face classes before the pandemic, some students could not resist the temptation to “google it” before asking a peer, teacher or using the context. to conclude, in this research the responses of the two groups of students to most prompts of the survey were quite similar. however, the responses of students from a spanish university differed depending on the study branches, which could be explained by familiarity with the use of technology and the level of social interaction in these faculties. in addition, some teachers overloaded students by assigning projects and essays for homework causing a negative attitude towards studying online (erdmann et.al., 2021, p. 32). a few students in our research responded to open-ended prompts and provided additional comments about english lessons. this activity was not obligatory, and students were reluctant to spend more time completing the survey. ipt students wrote that they especially enjoyed team games, sessions with native speakers, and solving puzzles. however, students felt a lack of face-toface communication. students of a spanish university also wrote about mental and psychological risks triggered by social isolation (erdmann et.al., 2021, p. 33). in addition, it was unusual for some ipt students to speak english only, and it took some time to get used to it (chugai, 2021, p. 112). speaking their native language in a foreign language class is a contentious issue. if a native language is often used, students do not consider english appropriate for real-life situations and find it strange when a teacher speaks english before, during, and after the class. in contrast, when all the participants in communication speak english, it is considered authentic, i.e., the language used in a real context (larsen-freeman & marti anderson, 2011, p. 294). ief students also provided positive https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 9 comments and described their lessons as informative, engaging, and useful. they mentioned games and tests as the most interesting activities, described the personality of a teacher, and were appreciative of such a positive experience. the students also used emoticons to indicate their positive feedback. to conclude, hypothesis a that technical university students had mostly positive experiences learning english online during the 2020–2021 academic year was proved to be true. our recommendation concerning technical issues is to make students aware of the requirements to have the necessary equipment to ensure studying online; methodological and psychological issues could be resolved by implementing cooperative learning, various interaction patterns possible by means of breakout rooms, games, quizzes, online tests, alternative assessment. 4. limitations to the study though the study provided valuable information on technical university students’ feedback on learning english online during covid-19, there were limitations to the study. the study was carried out during the second semester of the 2020–2021 academic year. however, more data is needed to draw conclusions about the effectiveness of online learning and its long-term consequences. additionally, the number of ipt and ief students was not equal. the next steps when conducting another research project will exclude these limitations. 5. conclusions a comparative analysis of the feedback of ukrainian technical university students studying english online at the institute of physics and technology (ipt) and instrumentation engineering faculty (ief) showed that while the responses to the prompts of the survey were quite similar, the responses about interaction patterns were slightly different. the results of the survey showed that while studying english online, technical university students practically never experienced technical problems studying from home. for most, studying online was comfortable: they had free time to find new hobbies, get a job, or do extra courses online. most respondents enjoyed working in pairs or groups, as well as interactive activities like games, quizzes, role-plays, and online tests. the respondents considered english lessons online beneficial for them and assessed their overall experience positively. the recommendations for teachers of english are related to implementing cooperative learning, various interaction patterns, games, alternative assessment. https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 10 author contributions: authors contributed equally. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references: adarkwah, m. a. (2021). “another year lost?”: a novel approach to the online learning in ghana. academia letters. article 365. https://doi.org/10.20935/al365. alba-linero, c., moral-sanchez, s. n., & gutierrez-castillo, p. (2020). impact of covid-19 on education in a spanish university: what should we change? in i. sahin & m. shelley (eds.), educational practices during the covid-19 viral outbreak: international perspectives (pp.81-106). istes organization. https://eric.ed.gov/?id=ed608253 aslam, s., saleem, a., akram, h., parveen, k., hali, a. u. (2021). the challenges of teaching and learning in the covid-19 pandemic: the readiness of pakistan. academia letters, article 2678. https://doi.org/10.20935/al2678 camilleri, m. a. (2021). shifting from traditional and blended learning approaches to a fully virtual and remote course delivery: implications from covid-19. academia letters, article 481. https://doi.org/10.20935/al481. chang, h. s. (2020). online learning in pandemic times. revista romaneasca pentru educatie multidimensionala. 12(2sup1), 111-117. https://doi.org/10.18662/rrem/12.2sup1/296 chugai, o. (2020). games and competitions to transform an english for specific purposes class into student-centered. pedagogy. sofia, 92 (3), 442-449. https://pedagogy.azbuki.bg/en/pedagogics/pedagogyarticle/sadarzhanie-na-sp-pedagogika2020-g/sp-pedagogika-knizhka-3-2020-godina-xcii/ chugai, o. (2021). technical students’ perspectives: studying english during the pandemic. results of modern scientific research and development. proceedings of the 5th international scientific and practical conference, spain, 108-115. https://sciconf.com.ua/vmezhdunarodnaya-nauchno-prakticheskaya-konferentsiya-results-ofmodern-scientificresearch-and-development-25-27-iyulya-2021-goda-madrid-ispaniyaarhiv/ dogar, a. a., shah, i., ali, s. w., & ijaz, a. (2020). constraints to online teaching in institutes of higher education during pandemic covid-19: a case study of cui, abbottabad https://doi.org/10.4995/muse.2022.16142 https://doi.org/10.20935/al365 https://eric.ed.gov/?id=ed608253 https://doi.org/10.18662/rrem/12.2sup1/296 https://pedagogy.azbuki.bg/en/pedagogics/pedagogyarticle/sadarzhanie-na-sp-pedagogika-2020-g/sp-pedagogika-knizhka-3-2020-godina-xcii/ https://pedagogy.azbuki.bg/en/pedagogics/pedagogyarticle/sadarzhanie-na-sp-pedagogika-2020-g/sp-pedagogika-knizhka-3-2020-godina-xcii/ https://sciconf.com.ua/v-mezhdunarodnaya-nauchno-prakticheskaya-konferentsiya-results-ofmodern-scientific-research-and-development-25-27-iyulya-2021-goda-madrid-ispaniyaarhiv/ https://sciconf.com.ua/v-mezhdunarodnaya-nauchno-prakticheskaya-konferentsiya-results-ofmodern-scientific-research-and-development-25-27-iyulya-2021-goda-madrid-ispaniyaarhiv/ https://sciconf.com.ua/v-mezhdunarodnaya-nauchno-prakticheskaya-konferentsiya-results-ofmodern-scientific-research-and-development-25-27-iyulya-2021-goda-madrid-ispaniyaarhiv/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 11 pakistan. revista romaneasca pentru educatie multidimensionala. 12(2sup1), 12-24. https://doi.org/10.18662/rrem/12.2sup1/285 erdmann, a., estrada,a., & de miguel, m. (2021). digital transformation of universities: the influence of covid-19 and student’s perception. multidisciplinary journal for education, social and technological sciences, 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 gómez-ejerique, c., & lópez-cantos, f. (2019). application of innovative teaching-learning methodologies in the classroom. coaching, flipped-classroom and gamification. a case study of success. multidisciplinary journal for education, social and technological sciences, 6(1), 46-70. https://doi.org/10.4995/muse.2019.9959 gokuladas, v. k., & baby sam, s. k. (2020). psychological and operational challenges in educational field amidst pandemic. revista romaneasca pentru educatie multidimensionala. 12(2sup1), 143-15. https://doi.org/10.18662/rrem/12.2sup1/300 khalil awan, r., afshan, g., bano memon, a. (2021). adoption of e-learning at higher education institutions: a systematic literature review. multidisciplinary journal for education, social and technological sciences, 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 kimmel, h. s., carpinelli, j. d., spak, g. t., & rockland, r. h. (2020). a methodology for retaining student learning during the pandemic. in i. sahin & m. shelley (eds.), educational practices during the covid-19 viral outbreak: international perspectives (pp.1-18). istes organization. https://eric.ed.gov/?id=ed608253 larsen-freeman, d. & marti anderson. (2011). techniques and principles in language teaching (3rd ed.). oxford university press. https://acasearch.files.wordpress.com/2015/03/techniques-in-language-teaching.pdf lukianenko, v., & vadaska, s. (2020). evaluating the efficiency of online english course for first-year engineering students. revista romaneasca pentru educatie multidimensionala [online]. 12(2sup1), 62-69. https://doi.org/10.18662/rrem/12.2sup1/290 mathur, s., & singh, a. (2020). the perception of teachers on unlocking technology by redesigning education system during and after covid-19 pandemic lockdown. in i. sahin & m. shelley (eds.), educational practices during the covid-19 viral outbreak: international perspectives (181-198). istes organization. https://eric.ed.gov/?id=ed608253 https://doi.org/10.4995/muse.2022.16142 https://doi.org/10.18662/rrem/12.2sup1/285 https://doi.org/10.4995/muse.2021.16007 https://doi.org/10.4995/muse.2019.9959 https://doi.org/10.18662/rrem/12.2sup1/300 https://doi.org/10.4995/muse.2021.15813 https://doi.org/10.18662/rrem/12.2sup1/290 https://eric.ed.gov/?id=ed608253 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 12 nikolaeva, s., korol, t. (2021). formative assessment in the translation classroom: closing a feedback loop. international journal of evaluation and research in education (ijere). vol. 10, no. 2, june 2021, pp. 738~746 issn: 2252-8822. https://doi.org/10.11591/ijere.v10i2.21274 stavytska, i. v., kutsenok, n. m., korbut, o. h., & svyrydova, l. h. (2021). features of application of interactive games in conditions of distance learning. pedagogical sciences reality and perspectives. 2(79), 126–129. https://doi.org/10.31392/npunc.series5.2021.79.2.27 ubaid-ullah, d. s. (2021). my experiences with obe. academia letters. article 929. https://doi.org/10.20935/al929. unesco. (2020). starting digital divides in distance learning. https://en.unesco.org/news/startlingdigital-divides-distance-learning-emerge yucesoy-ozkan, s., kaya, f., gulboy, e., altun, d. e., & oncul, n. (2020). general and special education practices during the covid-19 viral outbreak in turkey. in i. sahin & m. shelley (eds.), educational practices during the covid-19 viral outbreak: international perspectives (19-62). istes organization. https://eric.ed.gov/?id=ed608253 appendix appendix a the survey for students “studying english online under the covid-19” prompt 1. i experienced technical problems (“never” / “practically never” / “from time to time” / “often” / “always”); prompt 2. i studied from home (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); prompt 3. studying online was comfortable for me (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); prompt 4. i had free time to find new hobbies / job /do extra courses online (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); https://doi.org/10.4995/muse.2022.16142 https://doi.org/10.31392/npu-nc.series5.2021.79.2.27 https://doi.org/10.31392/npu-nc.series5.2021.79.2.27 https://doi.org/10.20935/al929 multidisciplinary journal for education, https://doi.org/10.4995/muse.2022.16142 social and technological sciences issn: 2341-2593 chugaia and svyrydovaa (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 1-13. https://doi.org/10.4995/muse.2022.16142 13 prompt 5. i enjoyed working in groups/pairs (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); prompt 6. i enjoyed games / quizzes / role-plays at the lessons (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); prompt 7. tests are easier to complete online than f2f (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); prompt 8. my english lessons were beneficial for me (“strongly agree” / “agree” / “neutral” / “disagree” / “strongly disagree”); prompt 9. i assess my overall experience learning english online as (“excellent” / “good” / “quite good” / “could be worse”); prompt 10. when i needed help with my english assignments (“ask a teacher” / “ask a peer” / “google it” / “use the context” / “all above”); provide additional comments on your experience studying english online (open). (self-developed) https://doi.org/10.4995/muse.2022.16142 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 123 best practices in syllabus design and course planning applied to mechanical engineering subjects francisco rubio1 , carlos llopis-albert1 , shouzhen zeng2 1 instituto universitario de ingeniería mecánica y biomecánica (i2mb). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding author: frubio@mcm.upv.es; cllopisa@upvnet.upv.es 2 ningbo university, ningbo 315211, china, zengshouzhen@nbu.edu.cn received: 03 april 2022; accepted: 10 september 2022; published: october2022 abstract the syllabus of a subject, that is part of the curriculum of a bachelor’s or master's degree, must provide the student with information about all the fundamental aspects of the subject. it is a piece of written document or multimedia file encompassing all topics and concepts that will be covered in a certain subject. the objective of the syllabus is to put the subject and the information related to it in context by means of clear, organized, concise and summarized style. it should not be limited only to the subject matter. instead, it is advisable to provide basic course information such as the number of credits; course content; transversal competences, skills and attitudes that are relevant for access to work and further learning; faculty staff; assessment and evaluation elements; calendar; venues, and facilities location; lesson plans and bibliography. moreover, information about the activities to be carried out and whether they are done individually or in groups. another important point is the evaluation of students and how to assess their achievements in terms of the level of acquisition of knowledge and skills planned in the subject. it helps students to meet the desired subject objectives and to motivate them. in short, it will lay the foundations so that at least contents, methods and techniques of the discipline that supports the subject can be taught and so that students can acquire the knowledge and competences committed. keywords: syllabus, transversal competences, effective teaching-learning process, effective teaching, learning targets, student evaluation elements, alumni motivation to cite this article: rubio, f., llopis-albert, c., zeng, s. (2022). best practices and syllabus design and course planning applied to mechanical engineering subjects. multidisciplinary journal for education, social and technological sciences, 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://orcid.org/0000-0003-3465-702x http://orcid.org/0000-0002-1349-2716 https://orcid.org/0000-0002-3604-0843 https://doi.org/10.4995/muse.2022.18230 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 124 1. introduction the beginning of a new term is a critical period in the student's learning process. students enrolled in a bachelor's or master's degree must face a situation of uncertainty related fundamentally to the content, the development of each subject, the teaching methods, and the way they are assessed and evaluated (nunan, 1988). there is also some degree of uncertainty regarding the knowledge that students have about the teaching staff, the resources that will be deployed, the places where the teaching will take place (classrooms, laboratories, etc.) and the most important dates related to them (dates of laboratory work, exam dates, handing assignments dates, etc.). this situation of uncertainty is usually addressed in the first-class session. the subject syllabus is intended not only to reduce the student’s uncertainty and anxiety, but also to act actively to enhance and encourage the acquisition of knowledge and skills of students in relation to the subject to be presented (millis, 2009; jones et al., 2015). the subject syllabus is very important since it is the first interaction between the student and the subject. it is equivalent to having a first impression of the subject. it fulfils at least two basic objectives: on the one hand, it serves to make known the most important aspects of the subject (contents and assessments acts), putting it in context and orienting the student; and on the other hand, it serves as a reference to recall and retrieve information related to the subject that the teacher has made available to the student at any time between the beginning and the end of the course (robb, 2012). the content of the subject syllabus should not be limited to reflecting the teaching guide but to contain complementary and necessary information for a correct understanding of the subject in the context of the degree or master's degree in which it is taught. furthermore, it should boost the alumni motivation and need to take the subject. when a student begins a term must deal with different subjects from which they have very little information. students face a situation in which they do not know the contents that await them throughout the four-month period, nor the workload that they will have to carry out in order to pass the different subjects. they usually wait until the first day of class of each subject to obtain the information they need and to check to what extent their expectations will be fulfilled in terms of https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 125 content and to gauge whether they have the necessary resources and skills to acquire an adequate level of knowledge and competencies. the name of the subject is usually indicative of what its content will be, but obviously the name is not enough to know the scope and importance of the subject, what the specific contents will be, the faculty in charge of teaching, etc. much of the information related to a given subject is made available to students on the online technological platforms. in the case of universitat politècnica de valència (upv) in spain, the tool that host all this information and supports teaching is called poliformat. it is based on sakai, which is a free, community source, educational software platform designed to support teaching, research, and collaboration. sakai is developed by a community of academic institutions, commercial organizations, and individuals. it is distributed under the educational community license (a type of open-source license). however, very few people take the time and trouble to consult this tool in order to access the available information. this circumstance may have a very simple explanation: the student expects the professor to shed light on the subject in question, at least on the first day of class. it is in this context that the subject syllabus takes on its full meaning. it will dispel all the questions that the student may have about the subject, and that will help to improve the students' performance, since they will not have the worries, uneasiness and anxiety linked to the uncertainty of what may be unknown (slattery and carlson, 2005). in addition, with the reasonable teaching differences inherent of each subject, they usually present many disparities in terms of content, learning methods used, evaluation elements for the assessment, evaluation dates, weights of each part in obtaining the final grade for the subject, etc. it is difficult for the student to retain in memory all the details of all the subjects that make up the fourmonth period (tiana et al., 2011). in this sense, the subject syllabus fulfils its second major task, to be a reference document or multimedia file throughout the term for the student in relation to the content and continent of each subject. in this sense, they can consult through the online platform anywhere at any time all kinds of information related to each subject, particularly information on https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 126 dates, group formation, methods of delivery of laboratory works, percentages that each element of evaluation has on the final grade of the subject, evaluation system, etc. it should also be noted that many subject syllabi are too brief and simplified. on the contrary, there are others that are very dense, with too much information and even with contents that are not appropriate for a subject syllabus (tokatli and kesli, 2009). whether it meets the established objectives or not will depend on the degree of elaboration of the presentation topic and whether the contents are presented in a clear and organized way. 2. best practices in syllabus design and course planning. the subject syllabus can be approached from different perspectives. for example, one could distinguish between first-year subjects from the rest. for the first year, everything is unknown to the student, including the university and its facilities, so the type of information required is different. regardless of the perspective with which the subject syllabus is prepared, it could be said that it will be good when the student finds it unnecessary to search for any information related to the subject by using sources other than those provided for in the subject itself. at that point it could be concluded that an excellent subject syllabus has been prepared in relation to the relevant subject. therefore, from the perspective of a student who is already in college or university, this topic should contain at least the following sections: a) baseline course information and teaching staff. b) fit and function of the subject in the degree program. c) calendar, venues, and facilities location. d) competencies. e) didactic units and teaching methods. f) evaluation systems and assessment acts. g) tutorships. h) student-teacher communication. i) bibliography. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 127 each of these sections is explained in detail below: a) baseline course information and teaching staff: it should include course name and number, term and year, credit hours, location and timetable of class meetings, and teaching staff. a virtual teaching contingency plan for online or blended/hybrid teaching in an exceptional situation caused by exceptional problem such as covid-19 should also be provided. in general, the quality and training of the university academic body and particularly of the faculty is fundamental for the academic, social, and cultural development of the students. descending to the level at which the subjects are taught, the importance of the professor is undeniable. on most occasions, he or she becomes the soul that gives a subject a life of its own. the role of the teacher is not limited to transferring the competencies, the contents of the subjects and to enabling and directing the evaluation process of that subject. they become a mirror in which students look at themselves. they constitute the main reference that students have in their formative process. these tend to emulate or repudiate their scale of values, their vision of reality, etc. there are many aspects to highlight about the teaching staff:  relevance in the ethical and moral formation of students.  well-being of the teaching staff in its triple aspect: manager, teacher, and researcher.  level of academic and skills training university teachers face many problems derived from a demanding work context produced by educational reforms related to the working of the university. they must remain attentive to the pedagogical level they deploy, enabling mechanisms to develop the democratization of educational scope, encouraging participation, and pluralism. they must undergo a continuous process of updating on issues such as new learning methodologies, content development and subject planning, as well as evaluation processes. in short, teachers must be trained and committed to the performance of their work and have a positive attitude to accept and understand the differences of the students they will be in charge of. in addition, they are also subject to student criticism and a survey-based assessment process. it is vital to introduce teacher training programs to professionalize the work, promote new competencies related to inclusion and attention to diversity in university teaching institutions. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 128 in all, the subject syllabus must contain at least the name of the coordinator or person in charge of the subject, although usually the entire teaching staff is included. b) the fit and function of the subject in the degree: it is essential to justify the existence of the subject in the degree, the relationship it has with the other in the course, as well as the applications and professional opportunities to which the acquired knowledge may lead. the subject goals, objectives and learning outcomes must be provided. it serves to give meaning to the subject, offering a context in which it is developed. the degree (or in its case the master's degree) must have a coherent corpus of studies that is made explicit through its syllabus. this is usually drawn up by specialists and experts and it must reflect the areas of knowledge that must be part of it in order to meet the training objectives. the different subjects come to fulfil the areas of knowledge and make up the curriculum of a given degree. this is the origin of the relationship between certain subjects in the curriculum. in order to adequately cover the areas of knowledge, the descriptors of the subject areas and sub-areas are used. they are indicative of the contents sought. the suitability of the subject within the syllabus of which it forms part is justified by these descriptors. the student must participate in the thematic areas of the degree and focus the subject as an important part to cover this thematic area. defining the thematic area of the subject and the set of competences that will be worked on will serve as motivation for the students to follow the subject. the subject syllabus should include a brief description of the subject and the minimum recommended knowledge, in order to review the knowledge acquired in other subjects. c) calendar and venues and facilities location: teaching sessions calendar, assignment due dates and assessment dates must be provided. also, how holiday periods and professional or university events affect class meetings. in addition, especially in universities, which sometimes have huge campuses with facilities distributed in different locations within the campus, it is very important to clarify the rooms, classrooms, laboratories, and other venues in which the teaching of the subject will take place. even more during the first days of the term, when students are lost and are not able to find the venues and facilities where a certain activity is going to take place, such as a master class, a laboratory work session, or a field activity. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 129 it is necessary to have good information about sites and places. the use of maps, the assignment of names to buildings, streets, classrooms, laboratories, etc. will help the students in the search for these facilities. it is a help that should be included in the subject syllabus, especially when there is a very heterogeneous student body that is not used to these places. this is often the case with erasmus-type students, with those who apply for a transfer to another university, etc. in a nutshell, this information makes life easier for all members of the university community. d) competencies: they are defined as the set of abilities, skills, knowledge, attitudes, and abilities that are acquired during participation in the teaching of a subject. on the other hand, the enumeration of the competencies that are expected to be developed in a subject are necessary in the degree accreditation processes. therefore, it is necessary not only to define them but also to implement the processes for their evaluation and to see to what degree these competencies have been achieved by the students. it is necessary to distinguish (according to upv, 2020) between:  specific competencies: those that are carried out in the development of the subject itself. each subject must indicate which are the basic competences, capacities and personal skills that must be acquired throughout the development of the subject.  transversal competences: these are skills related to personal development, which do not depend on a specific thematic or disciplinary field but appear in all domains of professional and academic performance. it is a very complex know-how, so it is necessary to specify it in more specific learning outcomes. within the scope of the subject syllabus, it is convenient to explain to the students which the competences (specific and transversal) are related to the subject. it is also necessary to make explicit the way in which they are assessed by defining several degrees of achievement that will help students to determine the level of skill acquired in such competences at the end of the teaching of the subject. e) didactic units and teaching methods: a didactic unit is a piece of information that provides the backbone of the contents of the subject. they imply a method to organize and plan the teaching and learning process. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 130 the didactic units must contain at least the following information:  name of the didactic unit or topic.  quantity: indicate how many didactic units or topics exist in the syllabus.  content: a brief index of each didactic unit. the set of contents of the didactic units makes up the teaching and academic corpus of the subject. it must include traditional materials and e-resources that students have at their disposal, such as: coursebooks and other learner materials; available information and communications technologies, such as educational platforms, recorded lectures, multimedia, online and computer-based materials; activities, tasks and exercises designed for use in and out of the classroom; and recommended bibliography. teaching-learning method is "a coherent set of logically coordinated techniques and actions to direct student learning towards certain learning outcomes" (upv, 2020). for an effective teaching development, the following headings should be designed:  theory sessions: these are sessions where the master class is developed.  seminar: group work technique, which is clearly different from the master class, students become familiar with the means of research and reflection.  classroom work  laboratory work  computer work  evaluation activities f) evaluation systems and assessment acts: the evaluation systems and assessment acts of each subject represent a very important point, which together with the faculty responsible for it must be specified in the teaching guide of the subject before the beginning of the subject. according to university regulations, the evaluation system cannot be modified during the course. a distinction must be made between assessment acts and evaluation systems. the evaluation system of each subject is an instrument for monitoring and assessing the results obtained by the students in the teaching-learning process. the evaluation system will include the means of evaluation used, the number of assessment acts, the weight of these in the final grade and the recovery procedure. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 131 the faculty in charge of teaching a subject will evaluate the students enrolled in that subject in accordance with the provisions of the regulations in force (considering the objectives of each degree and the resources available). an act of evaluation is understood as that activity that allows the orderly collection of information on the student's teaching-learning process, through one or more means of evaluation, in order to determine the degree of assimilation of the learning results foreseen in the subject. the means of evaluation that can be established in the teaching guide are the following:  open answer written test: timed test carried out under the control of the teacher, where the student must respond in their own words and in writing, to one or more questions related to the didactic units of the subject. occasionally, they may consult the notes, documentation, support material and/or access the internet.  objective test: written exam structured with several questions or items, in which the student either chooses the answer he/she considers correct or complements it with precise elements such as a word or short phrase.  oral exam: technique related to oral expression and the student's active participation in learning in relation to: mastery of contents, communicative skills, attitudes, reflective processes, etc. they can often be a complement to written tests, academic papers, etc.  academic work: evaluation technique that includes the work done by the students, from short and simple work to extensive and complex work typical of the last courses: tfg, tfm, doctoral thesis...  project: means of evaluation that allows the assessment of both the projects developed by students, as well as the skills, competencies and knowledge acquired with their development.  problems: means to evaluate the knowledge and skills used by the student when facing and solving a problem designed by the teacher. teaching-learning method whose starting point is a problem designed by the teacher, so that the student, not having all the information, must reflect and identify their needs. to solve it correctly, the student must search, understand, and integrate the basic concepts of the subject.  cases: evaluation technique especially indicated to evaluate the different cases solved by the students. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 132  portfolio: a structured set of documents, prepared by the student, which gathers evidence that demonstrates their knowledge and competencies (learning level) in a subject or course, in relation to the criteria-guidelines stipulated by the teacher.  journal: it is a personal report, a narrative about an assignment, in which students detail the actions undertaken in their learning, including their concerns, feelings, observations, questions, hypotheses, explanations, etc.  observation: strategy based on the systematic collection of data in the learning context itself, on students' performance, skills, abilities, and attitudes. it can be carried out through different instruments such as anecdotal records, checklists, and rating scales.  minute tests: these are open questions -one or twothat are asked minutes before the end of a class and they allow evaluating what the students have understood from what they have seen in class, in a specific reading, article, activity, etc. it is possible for the students themselves to generate closed-answer questions. upv may include any other method it deems appropriate. it should also include the attendance policy and expected classroom behaviour (e.g., cell phone and laptops use). g) tutorship: information on tutorship is an element to be taken into account in the subject syllabus. tutorship is an act that the teacher carries out on the students, and it consists of a particular help to the student to facilitate the teaching-learning process and therefore helps their academic progress. it can have a face-to-face or virtual format using an educational platform. therefore, contact information, and teacher's office location and mentoring hours must be provided. h) student-teacher communication: these are the channels through which the student and the teacher interact. apart from the direct contact that may exist during the development of a teaching session in its different formats, it is necessary to establish additional and complementary channels of communication. one of these methods can be through e-mail, announcements in the technological platform of communication with the student (poliformat), etc. i) bibliography: it is the list of books or other types of writings (articles, book chapters, etc.) and information sources that are used as support or consultation material for the understanding, development and research related to the content of the subject. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 133 a bibliographic reference must contain at least the following elements: author's name, full title, and year of publication. in general, the bibliographic reference should be concrete and concise, including only the necessary information; at the same time, it should be formal, exhaustive, and reliable, i.e., it should not omit or obviate important data. 3. improving and updating the subject syllabus when teachers prepare the subject syllabus and includes all the information referred to in the previous sections, they may have the feeling that the topic is finished, that it is complete and that there is no room for further improvements and contributions. however, it lacks the most important test, which is the acceptance and recognition as such by the students. that is why it is important to have a continuous improvement process that takes into account the suggestions done by the students (of the different courses or promotions that will receive it). there is always some nuance to be added or some clarification to be made. for this reason, the subject syllabus improves with the external contributions made mainly by the students. in some cases, things as simple as introducing icons, using a graphic or an image, or even some animation or video substantially improve the content and understanding of the subject syllabus. in any case, it is important do not disregard any suggestions received from external sources and study them in case they could be useful in improving the subject syllabus. it is also necessary to update the subject syllabus because there may be sections that change from one term to another. for example, the headings related to the schedule of classes and laboratory work sessions. also, the proposed dates of evaluation events, etc. therefore, in addition to a continuous process of improvement, a continuous updating of the subject is necessary. https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 134 4. discussion and conclusions the outcome of the subject syllabus can be measured by the students' satisfaction with the information provided by it. it is open here to all teachers and researchers to make their contributions to improve the content of the subject syllabus. thus, it is necessary for teachers to be involved in this issue. it is worth mentioning the extensive experience of the authors of the present paper with many subjects in the field of mechanical engineering. note that to effectively transmit knowledge to students it is important that teachers keep up to date with educational and technological advances concerning these subjects. in this sense, they have published many articles in recent years closely related with the aforementioned subjects. for instance, llopis-albert et al. (2015-2022); rubio et al., (2015-2021); valero et al. (2017, 2019); valera et al. (2021); zeng et al. (2022). the authors have successfully applied the present methodology and implement the best practices in syllabus design and course planning in the mechanical engineering subjects that they teach. the success of the developed methodology is demonstrated by the fact of the highly alumni satisfaction as shown by the mandatory surveys that the university implement in all the subjects as a measure of the teaching quality. furthermore, student satisfaction has increased over the years as the syllabus has been improved, while the best practices discussed above have been applied and suggestions from students and other faculty have been implemented. these surveys are in line with the current system of the spanish university system, that guarantees the training and competence of the teaching staff. this system rests with universities and, consequently, they must develop procedures for the assessment of their performance, as well as for their training and encouragement, guaranteeing their qualification and teaching competence. to favour this evaluation of teaching, the national agency for quality assessment and accreditation (aneca) launched in 2007, in close coordination with the regional evaluation agencies, the support program for the evaluation of the teaching activity of university professors (docentia) with the aim of supporting universities in the design of its own mechanisms to manage the quality of the teaching activity of university teaching staff and favour its development and recognition. the docentia program takes as a reference the recommendations for quality assurance in higher education institutions, contained in the document criteria and guidelines for quality assurance in https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 135 the european higher education area (2015), which was approved by the conference of ministers signatories of the bologna declaration in may 2015. likewise, in the design of the program, the standards established by internationally recognized organizations in the field of personnel evaluation have been taken into account, such as the personnel evaluation standards, prepared by the eu joint committee of standards for educational evaluation. at this moment, more than 90% of the universities participate in this program, throughout its different phases. in conclusion, reaching a lively classroom atmosphere and fruitful learning process can be enhanced by designing an efficient syllabus and course planning, for which an active involvement of the teaching staff and students is desirable. this leads to a higher degree of student satisfaction, their motivation to approach the subject and achieve the learning objectives is enhanced, and therefore, to an improvement in their academic performance is attained. author contributions: all authors have contributed equally funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references jones, s.k., noyd, r.k., sagendorf, k.s. (2015). building a pathway to student learning: a how-to guide to course design. stylus publishing, llc. llopis-albert, c., rubio, f., valero, f. (2015). improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research, 68 (7), 1429-1431. https://doi.org/10.1016/j.jbusres.2015.01.027 llopis-albert, c., rubio, f., valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences, 5(1), 1-16. https://doi.org/10.4995/muse.2018.9867 llopis-albert, c., rubio, f., valero, f. (2019). fuzzy-set qualitative comparative analysis applied to the design of a network flow of automated guided vehicles for improving business productivity. journal of business research, 101, 737-742. https://doi.org/10.1016/j.jbusres.2018.12.076 llopis-albert, c., rubio, f. (2021). methodology to evaluate transversal competences in the master's degree in industrial engineering based on a system of rubrics and indicators. multidisciplinary journal for education, social and technological sciences 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.1016/j.jbusres.2018.12.076 https://doi.org/10.4995/muse.2021.15244 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 136 llopis-albert, c., palacios-marqués, d., simón-moya, v. (2021). fuzzy set qualitative comparative analysis (fsqca) applied to the adaptation of the automobile industry to meet the emission standards of climate change policies via the deployment of electric vehicles (evs). technological forecasting and social change, 169, 120843. https://doi.org/10.1016/j.techfore.2021.120843 llopis-albert, c., rubio, f., zeng, s., grima-olmedo, j., grima-olmedo, c. (2022). the sustainable development goals (sdgs) applied to mechanical engineering. multidisciplinary journal for education, social and technological sciences 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 millis, b.j. (2009). the syllabus toolbox: a handbook for constructing a learningcentered syllabus, university of texas, san antonio. nunan, d. (1988). syllabus design. oxford: oxford university press. isbn: 0194371395, 177 pages. robb, m.s. (2012). the learner-centered syllabus. the journal of continuing education in nursing, 43(11), 489-490. https://doi.org/10.3928/00220124-20121024-26 rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2015). assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering, volume 2015, article id 931048. https://doi.org/10.1155/2015/931048 rubio, f., llopis-albert, c. (2019). viability of using wind turbines for electricity generation in electric vehicles. multidisciplinary journal for education, social and technological sciences, 6(1), 115-126. https://doi.org/10.4995/muse.2019.11743 rubio, f., llopis-albert, c., valero, f. (2021). multi-objective optimization of costs and energy efficiency associated with autonomous industrial processes for sustainable growth. technological forecasting and social change, 173, 121115. https://doi.org/10.1016/j.techfore.2021.121115 slattery, j.m., carlson, j.f. (2005). preparing an effective syllabus: current best practices. college teaching 53(4), 159-164. https://doi.org/10.3200/ctch.53.4.159-164 tiana, a., moya, j., luengo, f. (2011). implementing key competences in basic education: reflections on curriculum design and development in spain. european journal of education 46(3), 307-322. https://doi.org/10.1111/j.1465-3435.2011.01482.x tokatli, a.m., kesli, y. (2009). syllabus:how much does it contribute to the effective communication with the students? procedia social and behavioral sciences 1, 1491–1494. https://doi.org/10.1016/j.sbspro.2009.01.263 upv (2020). proyecto institucional competencias transversales. universitat politècnica de valència (upv). valencia. spain. https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf valera, á., valero f., vallés m., besa a., mata v., llopis-albert c. (2021). navigation of autonomous light vehicles using an optimal trajectory planning algorithm. sustainability, 2021; 13(3):1233. https://doi.org/10.3390/su13031233 valero, f., rubio, f., llopis-albert, c., cuadrado, j.i. (2017). influence of the friction coefficient on the https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.techfore.2021.120843 https://doi.org/10.3928/00220124-20121024-26 https://doi.org/10.1155/2015/931048 https://doi.org/10.1016/j.techfore.2021.121115 https://doi.org/10.3200/ctch.53.4.159-164 https://doi.org/10.1111/j.1465-3435.2011.01482.x https://doi.org/10.1016/j.sbspro.2009.01.263 https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf multidisciplinary journal for education https://doi.org/10.4995/muse.2022.18230 social and technological sciences e-issn: 2341-2593 rubio et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 123-137. https://doi.org/10.4995/muse.2022.18230 137 trajectory performance for a car-like robot. mathematical problems in engineering, 9 pages. article id 4562647. https://doi.org/10.1155/2017/4562647 valero, f., rubio, f., besa, a.j., llopis-albert, c. (2019). efficient trajectory of a car-like mobile robot. industrial robot: the international journal of robotics research and application, 46(2), 211–222. https://doi.org/10.1108/ir-10-2018-0214 valero, f., rubio, f., llopis-albert, c. (2019). assessment of the effect of energy consumption on trajectory improvement for a car-like robot. robotica, 37(11), 1998-2009. https://doi.org/10.1017/s0263574719000407 zeng, s., zhang, na, zhang, c., su, w., llopis-albert, c. (2022) .social network multiple-criteria decisionmaking approach for evaluating unmanned ground delivery vehicles under the pythagorean fuzzy environment. technological forecasting and social change, 175, 121414. https://doi.org/10.1016/j.techfore.2021.121414 https://doi.org/10.4995/muse.2022.18230 https://doi.org/10.4995/muse.2022.18230 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1155/2017/4562647 https://doi.org/10.1108/ir-10-2018-0214 https://doi.org/10.1017/s0263574719000407 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 an electronic record system in nursing education: evaluation and utilization. gonzález-chordá vm1*, mena-tudela d1, salas-medina p1, cevera gasch a1, lapeñamoñux y1, folch-ayora a1, orts-cortés mi1, maciá-soler ml1,2. (1) nursing department. university jaume i. avda sos baynat s/n, 12071. castellón de la plana. castellón. spain. (2) nursing department. university of alicante. carretera de san vicente del raspeig, s/n, 03690, san vicente del raspeig. alicante. spain. * corresponding author: e-mail: vchorda@uji.es phone: (+34) 964.387.744. received: 2013-12-15; accepted: 2014-08-04 abstract the main objective of the present work is to analyze the results of the utilization and evaluation of the loreto record system (lrs), providing improvement areas in the teaching-learning process and technology, in second year nursing students. a descriptive, prospective, cross sectional study using inferential statics has been carried out on all electronic records reported by 55 nursing students during clinical internships (april 1º-june 26º, 2013). electronic record average rated 7.22 points (s=0.6; cv=0.083), with differences based on the clinical practice units (p<0,05). three items assessed did not exceed the quality threshold set at 0.7 (p<0.05). record rate exceeds the quality threshold set at 80% for the overall sample, with differences based on the practice units. only two clinical practice units rated above the minimum threshold (p <0.05). record of care provision every 3 days did not reach the estimated quality threshold (p <0.05). there is a dichotomy between qualitative and quantitative results of lrs. improvement areas in theoretical education have been identified. the lrs seems an appropriate learning and assessment tool, although the development of a new app version and the application of principles of gamification should be explored. keywords nursing; nursing education; software; student records; m-learning. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 70 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 1. introduction. information and communication technology (ict) has become an effective tool in the daily activity of the healthcare professions and the improvement of the teaching-learning processes in higher education. in the case of nursing, practising professionals work in a changing context within the care environment. changes in science, technology and healthcare systems as a whole have important implications for nurses' expectations and the impact on users in relation to the provision of safe, quality care. the quality of patient medical record documentation has an important structural aspect both from an ethical and safety assurance perspectives for the provision of healthcare services. in spain, the minimum clinical data set in the national healthcare system (2010), including the nursing care reports, has been approved pursuant to the national health system cohesion and quality act (ley de cohesión y calidad del sistema nacional de salud) (2003) and the royal decree 1093/2010. on the other hand, the projects conducted by the national health system quality agency (agencia de calidad del sistema nacional de salud) for the creation of the national health system electronic health records (2010, 2012) foster the development of computer applications that allow the electronic record of nursing care (gonzález sánchez et al, 2004; sánchez ros et al, 2006; navarro armedo et al, 2007; garrido bartolomé et al, 2007; parra-estrada et al, 2010; medina-valverde et al, 2012; sánchez garcía, 2013). moreover, the use of ict in nursing education is being progressively introduced into the teaching-learning process, facilitating its adoption by future professionals, as dearnley, haigh and fairhall (2008) stated. it is also an emerging line of research (bogossian, kellet y mason 2009; wu et al, 2011; clay, 2011) as there is a commitment among universities and healthcare centers for the integration of information technology into their gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 71 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 processes. however, there have not been many experiences with the use of electronic records as skill learning and assessment tool (gordon et al, 2007; sayadi y rokhaforz, 2012; saba y feeg, 2005 boyle et al, 2008), considering this an important aspect in the training of future graduates in nursing, as woodill and udell (2011) and gardner and jones (2012) stated. a process of integration of electronic records into the student learning process in all the subjects in which skills are acquired through a methodology that integrates theoretical knowledge and practical and clinical skills (maciá soler et al, 2013) has been initiated in the nursing degree at the university jaume i (uji) (castellón, spain) during the 2012/2013 academic year. the implementation process begun in the second year, in which the skills that are to be achieved in specialized care are related to basic care. the students' length of stay in this environment is of 3 months. each student is assigned an electronic device (tablet with android system) that includes an electronic activity record system (loreto record system) with a methodology developed by the professors of the subjects on the basis of the skills involved and adapted to the learning needs. the loreto record system (lrs) is based on the international classification of diseases, such as the taxonomy of diseases and procedures. the lrs reflects the nursing method, with a structure based on the assessment, care prescription and re-evaluation of each patient according to their functional abilities (barthel index). this system also incorporates support tools for clinical decision making related to nursing sensitive outcomes (falls, pressure ulcers, malnutrition, etc.), and offers students an overview of process-based management applied to the nursing practice, while respecting individuality of care. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 72 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 the objective of this research is to know the use nursing students make of the lrs during supervised clinical practice and analyse the results of the evaluation, establishing qualitative and quantitative criteria in order to provide possible areas for improvement in the teaching and development of the application. 2. methodology. this is an analytical, prospective, observational study in which inferential statics has been used. the study population consists of all electronic records carried out by 55 nursing students during supervised clinical practice in the subject "nursing care in healthcare processes" in the second year of the nursing degree at the university jaume i (uji), which includes the following subjects: primary care nursing, nursing care in osteoarticular processes, nursing care in digestive, endocrine and renal processes and nursing care in cardiovascular and respiratory processes. the supervised clinical practice is conducted in 16 acute inpatient units of 5 public and private hospitals in the province of castellón (spain) with which a cooperation agreement exists. 2.1 inclusion and exclusion criteria the electronic records carried out in the trauma, internal medicine and surgery units of the public hospitals in the province of castellón (spain) were included in this study, as these are the ones that best suit the skills to be acquired by students. those considered data entry errors or with an age <18 were excluded. 2.2 variables studies 2.2.1 utilization of records • records estimated (re) (formula 1) gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 73 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 formula 1 : records estimated (er) re = (64 actual days of clinical practice/average stay in unit) * no. student beds • ri/re index: records issued/records estimated (formula 2) formula 2 : records issued/records estimated (ri /re index) ri/re index = number of records issued/number of records estimated 2.2.2 completion of electronic records. • percentage of completed records (% c): 21 variables included in the lrs, which meet the learning outcomes of the subject "nursing care in healthcare processes", were selected on faculty consensus. table 1 : variables selected from lrs. lrs variables socio-demographic variables age sex hospital unit clinical variables icd-9cm diagnosis type of process type of admission readmission complexity assessment at admission functional ability fall risk pu risk nutritional risk body mass index (bmi) social evaluation cognitive status pain primary caregiver care prescription and provision barthel index score every three days implementation of the standardized care plan every three days care report at discharge reason for discharge continuity of care functional ability • record rate (rr) of each electronic record (formula 3) gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 74 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 formula 3 : record rate; rr = (number of variables recorded/21 variables)*100 2.2.3 assessment of the quality of records the assessment of the quality of electronic records is carried out by means of a tool designed ad hoc on faculty consensus. this includes 10 items that are evaluated between 0 and 1 points and account for 25% of the final practice score. the variables considered are: • final score for each student (0-10 points). • identification of the reason and type of admission (0-1 points). • record of complete health history (0-1 points). • accurate record of health history (0-1puntos). • performance of physical examination (0-1 points). • utilization of assessment tools (0-1 points). • patient's evolution per shift (0-1 points). • record of planned activities (0-1 points). • accurate record of discharge report (0-1 points). • identification of care diagnosis (0-1 points). • use of a professional and technical language (0-1 points). 2.3 statistical analysis. 2.3.1 utilization of lrs. an estimate of the number of records to be carried out by the students in each practice unit included in the study and the records issued/records estimated index (ri/re index) were calculated, considering a ri/re index ≥ 0.7 as quality threshold. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 75 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 2.3.2 completion of records. the record rate (rr) and computerized record of each of the selected 21variables was calculated considering an average rr ≥ 80% as quality threshold. by means of the tstudent test, the achievement of the quality threshold (80%) was confirmed for the sample as a whole and for each of the units by comparing the unit rr average to the anova or the kruskal-wallis test, being the homoscedasticity previously analysed by means of the levenne test. in order to confirm whether each variable reaches a record proportion of 80%, the z-test for proportions was used, and a study of the possible dependence of the record proportion of the variables based on the clinical practice units was conducted by means of the chisquare test. fisher's exact test was used when the number of observations per group was n <5. 2.3.3 assessment of the quality of the records. a descriptive analysis (average, standard deviation, coefficient of variation and percentiles) of the scores obtained by students and of each of the items composing the assessment tool was performed. the kruskal-wallis test determined whether there were differences in scores based on the clinical practice units. a statistical analysis of the data was performed by means of the r commander application of the r 3.0.2 software. a 5% bilateral significance level was assumed in hypothesis tests. 2.4 data collection. the collection of information was carried out concurrently during the supervised clinical practice, from 1st april to 26th june 2013, in the clinical practice units that met the inclusion criteria. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 76 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 2.5 ethical considerations. there is an agreement among the professors of the subjects to use the results of the applied teaching methods for the elaboration of this research. moreover, the anonymity of patient records is guaranteed under the current legislation. 3. results. 3.1 utilization of lrs. a total of 334 computerized records, made in 7 clinical practice units that met the inclusion criteria, were included in the study. the ri/ re index for the whole sample was 0.55, which did not meet the quality threshold established. a ri/re index of 0.7 (table 2) was reached only in two units. 5 registration errors (data entry errors) (1.49%) and one age error (age <18), were excluded from the 334 electronic records, so the final sample consisted of 328 electronic records (table 2). table 2 : ri/re index. exclusion criteria. hospital/units electronic records re1 ri2 ri/re index 3 exclusion criteria. included error <18 no % no % no % hosp 1 surgery 60 29 0.48 0 0 1 2.9 28 96.5 hosp 2 surgery 70 47 0.67 0 0 0 0 47 100 internal medicine 210 96 0.45 2 2.08 0 0 94 97.9 traumatology 70 49 0.7 0 0 0 0 49 100 hosp 3 surgery 64 49 0.76 0 0 0 0 49 100 internal medicine 64 35 0.41 0 0 0 0 35 100 traumatology 64 29 0.54 3 10.3 0 0 26 89.6 total 602 334 0.55 5 1.4 1 0.3 328 98.2 gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 77 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 3.2 completion of records. the average record rate (rr) for all electronic records included in the study is 89.82%, exceeding the quality threshold set at 80% (p <0.05). except for the hospital 3 trauma unit, all units have an rr greater than 80% (p <0.05). there are also statistically significant differences in rr based on the units in which students undertake clinical practice (p <0.05) (table 3). table 3 : record rate (rr). units no 1 s t-student * levene anova hosp 1 surgery 28 93.27 3.72 <0,001 0.0016 <0,001 hosp 2 surgery 47 88.09 8.24 <0,001 hosp 2 internal med 94 88.19 7.14 <0,001 hosp 2 trauma 49 93.46 3.74 <0,001 hosp 3 surgery 49 90.27 7.01 <0,001 hosp 3 internal med 35 91.89 2.94 <0,001 hosp 3 trauma 26 81.48 14.12 0.2952 total 328 89.82 7.68 <0,001 the study conducted on the completion proportion of the 21 selected lrs variables shows that all socio-demographic and clinical variables are recorded by students in 100% of the sample, except for the complexity variable (92.99%), although it exceeds the 80% expected (p <0.05) (table 4). (*) student t-test results for a sample. p-value in hypothesis testing (p <0.05): h0: mu = 79.; h1: mu ≥ 80 (1) records estimated: re= (64 actual days of clinical practice/average stay in unit) * no. student beds (2) records issued, (3) ri/re index = records issued/records estimated gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 78 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 all variables included in the patient's admission assessment exceed the threshold established, except for the bmi (62.5%) and the evaluation of the pain level (41.16%), being the record of these variables based on the units in which students undertake clinical practice (p <0.05). the continuity of care is the only variable recorded at discharge that does not reach the quality threshold (14.94%) depending on the units of supervised clinical practice (p <0.05). table 5 shows how students' use of barthel index for assessing functional ability and prescribing care remains above the established threshold whenever the stay is shorter than 18 days. a connection between the increase in the stay and the decrease in the record proportion can be noted in this variable. a similar situation is observed in the record of the implementation of the standardized care plan based on the functional ability, except that in this variable only the record of the care plan at admission exceeds the 80% expected. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 79 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 table 4: completion proportion of the selected lrs variables. lrs variables %c1 no n inference z-test* ic 95% x2 so ci ode m og ra ph ic va ri ab le s age 100 328 328 sex 100 328 328 hospital 100 328 328 unit 100 328 328 c lin ic al v ar ia bl es diagnosis 100 328 328 type of process 100 328 328 type of admission 100 328 328 readmission 100 328 328 complexity 92.99 305 328 <0,01 0-9% a ss es sm en t a t a dm is si on functional ability 100 328 328 fall risk 96.95 318 328 <0,01 0-5% pu risk 96.95 318 328 <0,01 0-5% nutritional risk 96.95 318 328 <0,01 0-5% bmi 62.5 205 328 1 0-41% <0,01 social evaluation 91.77 301 328 <0,01 0-11% cognitive status 96.95 318 328 <0,01 0-5% pain 41.16 135 328 1 0-63% <0,01 primary caregiver 96.95 318 328 <0,01 0-5% c ar e re po rt a t di sc ha rg e reason for discharge 100 328 328 continuity of care 14.94 49 328 1 0-85% <0,01 functional ability 100 328 328 on the other hand, it cannot be said that there is a significant dependency between the record of the barthel index or the implementation of the care plan every three days and the units in which students undertake clinical practice, noting as a single exception the record of the implementation of the care plan based on the functional ability on the third day of stay (p <0.05). (1) completion proportion (*) z-test results for a sample. p-value in hypothesis testing (p <0.05): h0: unregistered variable rate = 20%; h1: proportion of unregistered variables <20% implementation conditions are not met. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 80 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 table 5: utilization of assessment scales and implementation of the standardized care plan. variable %c1 no n inference z-test* ic 95% x2 fisher barthel i. 0 days 100 328 328 f un ct io na l a bi lit y (b ar th el i nd ex ). barthel i. 3 days 97.37 220 228 <0,01 0-5.5 barthel i. 6 days 93.98 125 133 <0,01 0-10.3 barthel i. 9 days 91.36 74 81 <0,01 0-15.2 barthel i. 12 days 91.38 51 58 0.015 0-16.6 barthel i. 15days 91.67 33 36 0.04 0-19.1 barthel i. 18 days 88.46 23 26 0.14 0-25.6 0.08 barthel i. 21 days 88.89 14 16 0.172 0-28.6 0.078 barthel i. 24 days 88.71 12 14 0.296 0-35.2 0.197 barthel i. 27 days 85.71 7 7 0.352 0-45.2 0.428 im pl em en ta ti on o f c ar e pl an implementation 0 day 92.99 305 328 <0,01 0-70 implementation 3 days 63.6 145 228 1 0-40 <0,01 implementation 6 days 56.39 75 133 1 0-50.7 0.062 implementation 9 days 55.56 45 81 1 0-53 0.16 implementation 12 days 63.69 37 58 1 0-46 0.5 implementation 15 days 52.78 19 36 1 0-47.2 0.81 implementation 18 days 42.31 11 26 1 0-72.6 0.502 implementation 21 days 27.78 5 16 1 0-85 0.82 implementation 24 days 28.57 4 14 1 0-86 0.64 implementation 27 days 28.57 2 7 0.99 0-88% 0.6 3.3 assessment of the quality of electronic records. the average score obtained by students in the evaluation of records was 7.22 (s = 0.6, cv = 0.083), with a minimum score of 6 and a maximum of 8, with statistically significant differences based on the units in which students undertake supervised clinical practice (p <0.05). (1) proportion of completion (*) z-test results for a sample. p-value in hypothesis testing (p <0.05): h0: unregistered variable rate = 20%; h1: proportion of unregistered variables <20% implementation conditions are not met. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 81 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 table 6: analysis of the assessment tool items used by the faculty. assessment items 1 s p0 p25 p50 p75 p100 p-value * identification of the reason and type of admission 0.96 0.08 0.75 1 1 1 1 1 record of complete health history 0.84 0.15 0.5 0.75 0.75 1 1 0.99 accurate record of health history 0.68 0.23 0 0.68 0.75 0.75 1 0.41 performance of physical examination 0.54 0.16 0.25 0.5 0.5 0.75 0.75 <0,01 utilization of assessment tools 0.9 0.12 0.75 0.75 1 1 1 1 patient's evolution per shift 0.68 0.19 0.25 0.5 0.75 0.75 0.75 0.39 record of planned activities 0.62 0.12 0.5 0.5 0.62 0.75 0.75 0.017 accurate record of discharge report 0.78 0.22 0.5 0.5 0.75 1 1 0.91 identification of care diagnosis 0.48 0.32 0 0.3 0.5 0.75 1 <0,01 use of a professional and technical language 0.98 0.06 0.75 1 1 1 1 1 table 6 shows the results of the descriptive and inferential analysis of the 10 items included in the clinical practice report. the items with a score lower than 0.7 (p-value <0.5) are the following: "performance of physical examination" (=0.54, s=0.16), "record of planned activities" (1= 0.62, s =0.12), and "identification of care diagnosis" (=0.48, s = 0.32). 4. discussion. there is little evidence on the use of electronic records systems as a tool for learning and assessment in nursing studies. this study presents the results of the utilization and evaluation of an electronic registration system (loreto record system) after the first year of implementation in second year student nursing degree. the results provide a dichotomy between quantity and quality of the records. on one hand, the number of records issued is away from the amount of records estimated. in this sense, the novel presence of nursing students using an electronic record system during (*) t-student test results for a group in hypothesis testing (p-value): h0: mu = 0,7; h1: mu <0,7 gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 82 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 clinical internships with mobile devices, may cause resistance in the implementation process and hinder student access to electronic records (kowitlawakul, chan, pulcini, wang, 2014; baillie, chadwick, mann, brooke-read. 2013). other aspects that explain the small number of records are due to problems related to interoperability of mobile devices and the central database, the quality of wi-fi networks of hospitals and programming errors detected during the collection period data. in addition, there are problems related to the difficulty of using the software, according to kowitlawakul, wang and chan (2013), despite previous training students have received. the quality of the records seems appropriate in the studied variables, except the variables barthel index score every three days, implementation of the standardized care plan every three days and continuity of care. although lrs incorporates a method of prescribing care based on functional capacity (barthel index), these results reflect a reality in which nurses, following the initial assessment, use the method of solving problems and meet the needs sought by patients, according to urquhart, currell, grant and hardiker (2009), without carrying out a prescription of care required. moreover, papers that use the results of the evaluation of an electronic record system designed for nursing students in order to identify areas for improvement in the learning process have not found in the review of the literature. in our study, evaluation of the records reveals that the physical examination, the importance of patient's evolution, record of planned activities and identification of care diagnosis are issues that must be reinforced in the classroom, owing to their direct relation to nursing process (assessment, planning, implementation and evaluation) (habermann, 2006) and this process is crucial in the formation of future graduates. despite these results, the use of software and similar tools can improve nursing students' clinical skills and encourage their learning and implementation of the nursing process in gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 83 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 clinical practice (sayadi, rokhafroz 2012, saba and feeg 2005) and the patient safety (jones, donelle 2011). furthermore, the use of electronic records systems allows the development of generic skills related to the use of ict, being fundamental in the future nursing professionals (kowitlawakul, wang, chan 2013; baillie, chadwick, mann, brooke-read 2013) . in another way, the learning environment and tutors practices play a key role in clinical learning (papastavrou et al 2010). they influence the learning opportunities and skills acquisition. according to results of our study, there may be differences in the support for the students according to the clinical practice units, according to bailie and curzio (2009) and baillie, chadwick, mann and brooke-read (2013) because nursing documentation requires thought, planning and skills in making decisions that should be monitored (sverinsson, sanda 2010). furthermore, the lrs is newly established and tutors need to develop confidence with this tool based on new ways of learning. the information technologies have strongly irrupted in nursing studies. proof of this is the development of e-learning (button, harrington, belan 2013) and m-learning tools (skiba 2011), including application for mobile devices (app) and educational games (stanley, latimer 2011). the lrs has been developed as a tool for m-learning with web technology, but the results of this study suggest that develop innovations in software to enhance its educational role is necessary. currently apps are considered as new learning tools that are familiar to students (phillippi, wyatt 2011), in which principles of gamification can be applied. gamification is a relatively recent concept in the world of education that involves applying the basic elements that make games fun and engage in things that typically are not considered as games (gamify.com) concept. apply principles of gamification to lrs may improve outcomes for students and their learning. moreover, the results obtained in this study should be treated with caution because it has been carried out after the first year of implementation of the software and this limits the gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 84 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 size of the sample. in addition the tool used for the evaluation of records has not undergone to a study of validity and reliability. despite these limitations, the results are useful to initiate improvement actions that directly impact the quality of student learning and continue the evaluation and improvement of lrs. 5. conclusions. the lrs used by students during clinical practice is a useful learning and assessment tool for skill acquisition, and offer areas for improvement within the learning process. students' skills related within physical exploration, use of autonomy assessment scales and identification of care diagnosis should be reinforced in the classroom and records may require further supervision from mentors during supervised clinical practice. the use of the lrs by students can be considered adequate in terms of quality, although further improvements in terms of interoperability and programming are needed. the development of an app version of the lrs for the android operating system and the application of gamification principles are aspects that are being considered in to improve the software. 6. acknowledgements this work has been supported by the universitat jaume i in the frame of the mobility research staff program (ref e-2014-39). 7 references baillie l., curzio j. 2009. a survey of first year student nurses’ experiences of learning blood pressure measurement. nurse education in practice, 9: 61e71. baillie l, chadwick s, mann r, brooke-read m. 2013. a survey of student nurses’ and midwives’ experiences of learning to use electronic health record systems in practice. nurse education in practice, 13:437-441. doi: 10.1016/j.nepr.2012.10.003 gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 85 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 bogossian fe, kellett sem, mason b. 2009. the use of tablet pcs to access an electronic portfolio in the clinical setting: a pilot study using undergraduate nursing students. nurse educ today, 29(2),246-53. doi: 10.1016/j.nedt.2008.09.001 button d, harrington a, belan i. 2013. e-learning & information communication technology (ict) in nursing education: a review of the literature. nurse educ today. in press clay ca. 2011. exploring the use of mobile technologies for the acquisition of clinical skills. nurse educ today, 31(6),582-6. doi: 10.1016/j.nedt.2010.10.011 dearnley c, haigh j, fairhall j. 2008. using mobile technologies for assessment and learning in practice settings: a case study. nurse educ pract, 8(3),197-204. doi: 10.1016/j.nepr.2007.07.003 papastavrou e, lambrinou e, tsangari h, saarikoski m, leino-kilpi h. 2010. student nurses experience of learning in the clinical environment. nurse education in practice 10: 176–182 doi: 10.1016/j.nepr.2009.07.003 gamify.com. what is gamification? retrieved from the gamification encyclopedia:http://gamification.org/wiki/gamification gardner c, jones s. 2012. utilization of academic electronic medical records in undergraduate nursing education. ojni, 16 (2) http://ojni.org/issues/?p=1702 garrido bartolome a, sánchez martinez v, muñoz izquierdo a, ruiz hontangas a, ferrandis campos. 2007. estación de enfermería, primeros pasos en la informatización de cuidados. investigación y cuidados, 5(11), 45-55 http://www.bibliotecadigitalcecova.com/contenido/revistas/cat4/pdf/investigacion_y_cui dados_n11.pdf gonzález sánchez ja, cosgaya garcía o, simón garcía mj, blesa malpica al. 2004. registros de enfermería: convencional frente a informatizado. unidad de cuidados críticos. enferm intensiva, 15(2),53-62. doi: 10.1016/s1130-2399(04)78145-6 gordon j, mcnew r, trangenstein p. 2007. the development of an online clinical log for advanced practice nursing students: a case study. stud health technol inform, 129,14326. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 86 http://ojni.org/issues/?p=1702 http://www.bibliotecadigitalcecova.com/contenido/revistas/cat4/pdf/investigacion_y_cuidados_n11.pdf http://www.bibliotecadigitalcecova.com/contenido/revistas/cat4/pdf/investigacion_y_cuidados_n11.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 habermann, m. the nursing process: a global concept. elsevier health sciences, 2006 kowitlawakul y, wang l, chan w-cs. 2013. development of the electronic health records for nursing education (ehnre) software program. nurs educ today, 33, 152935. doi: 10.1016/j.nedt.2012.12.001 kowitlawakul y, chan w-cs, pulcini j, wang l. factors influencing nursing students' acceptance of electronic health records for nursing education (ehrne) software program. nurse education today. 2014 (in press). doi: 10.1016/j.nedt.2014.05.010 law 16/2003, de 28 de mayo, de cohesión y calidad del sistema nacional de salud. spain. boletín oficial del estado, núm128 de 29.05.2003. http://www.boe.es/buscar/doc.php?id=boe-a-2003-21840 maciá soler l, orts cortés i, galiana sánchez m, ors montenegro a. 2013. implantación simultánea de los niveles de grado, máster y doctorado en enfermería en la universidad jaume i. castellón de la plana, españa. investigación y educación en enfermería, 31(2). http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/rt/printerfriendly/14121/14036 medina-valverde, mj; rodriguez-borrego, ma; luque-alcaraz, o; de la torre-barbero, mj; parra-perea, j; moros-molina, mp. 2012. estudio evaluativo sobre una herramienta informática de gestión enfermera en el periodo 2005-2010. enferm clin, 22,3-10. doi: 10.1016/j.enfcli.2011.06.001 ministerio de sanidad y política social. 2010. el sistema de historia clínica digital del sistema nacional de salud. instituto de información sanitaria. agencia de calidad del sistema nacional de salud. spain. http://www.msc.es/organizacion/sns/plancalidadsns/docs/hcdsns_castellano.pdf ministerio de sanidad, servicios sociales e igualdad y ministerio de industria, energía y turismo. 2012. las tic en el sistema nacional de salud. el programa sanidad en línea. actualización de datos 2011. red.es. spain. http://www.red.es/media/201207/1343654162527.pdf navarro armedo, jm; orgier uranga, pe; de haro marín, s. 2007. aurora 2003: gestor de informes de alta de enfermería en cuidados intensivos. enfermería intensiva, 18(2), 78-89. doi: 10.1016/s1130-2399(07)75740-1 parra-estrada, aj; guerra-arévalo, j; domínguez guerrero, f; benítez de la calle, mc. gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 87 http://www.boe.es/buscar/doc.php?id=boe-a-2003-21840 http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/rt/printerfriendly/14121/14036 http://www.msc.es/organizacion/sns/plancalidadsns/docs/hcdsns_castellano.pdf http://www.red.es/media/2012-07/1343654162527.pdf http://www.red.es/media/2012-07/1343654162527.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 2010. proceso de implementación del módulo de cuidados de diraya (azahar) en el hospital de jerez. vii congreso nacional de formación continuada en salud innovación eimpacto. http://www.juntadeandalucia.es/agenciadecalidadcanitaria/formacionsalud/gestor/es/recur sos_formativos/material_congreso_malaga/index2.html phillippi jc, wyatt th. 2011. smartphones in nursing education. cin, 29(8):449-54. real decreto 1093/2010, de 3 de septiembre, por el que se aprueba el conjunto mínimo de datos de los documentos clínicos en el sistema nacional de salud. spain. boletín oficial del estado núm 225, 16.09.2010. http://www.boe.es/buscar/doc.php?id=boe-a2010-14199 saba vk, feeg v. 2005. a bedside pc charting system for nursing students using clinical care classification (ccc) system. libro de congresos amia http://www.ncbi.nlm.nih.gov/pmc/articles/pmc1560569/ sánchez garcía ab, lópez montesinos mj, fernández alemán jl. 2013. dispositivos inalámbricos en la educación enfermera. investigación y educación en enfermería, 31(1) http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/article/view/11681/13292 sánchez ros, n; reigosa gago, lf. 2006. selene. informatización de la historia clínica electrónica: implicación sobre el proceso de enfermería. enfermería global, 6, 1-10. http://revistas.um.es/eglobal/article/view/458 sayadi n, rokhafroz d. 2012. nursing students’ perspective about a mobile software on nursing process for bedside use. iranian journal of medical education, 12(2) http://ijme.mui.ac.ir/browse.php?a_id=2198&sid=1&slc_lang=en skiba dj. 2011. on the horizon mobile devices: are they a distraction or another learning tool? nurs educ perspect, 32(3):195-7. doi: 10.5480/1536-5026-32.3.195 severinsson, e., sanda, a., 2010. evaluation of the clinical supervision and professional development of student nurses. journal of nursing management, 18:669-77. doi: 10.1111/j.1365-2834.2010.01146.x stanley d, latimer k. 2011. “the ward”: a simulation game for nursing students. nurse education in practice, 11:20-5. doi: 10.1016/j.nepr.2010.05.010 gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 88 http://www.juntadeandalucia.es/agenciadecalidadcanitaria/formacionsalud/gestor/es/recursos_formativos/material_congreso_malaga/index2.html http://www.juntadeandalucia.es/agenciadecalidadcanitaria/formacionsalud/gestor/es/recursos_formativos/material_congreso_malaga/index2.html http://www.boe.es/buscar/doc.php?id=boe-a-2010-14199 http://www.boe.es/buscar/doc.php?id=boe-a-2010-14199 http://www.ncbi.nlm.nih.gov/pmc/articles/pmc1560569/ http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/article/view/11681/13292 http://revistas.um.es/eglobal/article/view/458 http://ijme.mui.ac.ir/browse.php?a_id=2198&sid=1&slc_lang=en multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2224 social and technological sciences eissn: 2341-2593 urquhart c, currell r, grant mj, hardiker nr. 2009. nursing record systems: effects on nursing practice and healthcare outcomes. cochrane database of systematic reviews. http://www.bibliotecacochrane.com/pdf/cd002099.pdf woodill g, udell ch. 2011. the future of mobile learning in medicine and healthcare. float mobile learning. http://floatlearning.com/ideas-and-insights/mobile-learningresearch/the-future-of-mobile-learning-in-medicine-and-healthcare/ wu ph, hwang gj, tsai cc, chen yc, huang ym. 2011. a pilot study on conducting mobile learning activities for clinical nursing courses based on the repertory grid approach. nurse educ today, 31(8), e8-15. doi: 10.1016/j.nedt.2010.12.001 gonzález-chordá et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 70-89 | 89 http://floatlearning.com/ideas-and-insights/mobile-learning-research/the-future-of-mobile-learning-in-medicine-and-healthcare/ http://floatlearning.com/ideas-and-insights/mobile-learning-research/the-future-of-mobile-learning-in-medicine-and-healthcare/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 72 fundamentals of logic, reasoning, and argumentation: an evidence-supported curriculum targeting scientific literacy to increase public understanding and engagement in science la shun l. carroll university at buffalo graduate school of education, buffalo, ny 14260, usa correspondence: lcarroll@buffalo.edu received: 1 december 2019; accepted: 5 february 2020; published: april 2020 abstract the purpose of this article is to present an evidence-supported curriculum covering the fundamentals of logic, reasoning, and argumentation skills to address the emphasized basic knowledge, skills, and abilities required to be scientifically literate, which will prepare the public to understand and engage with science meaningfully. an analytic-synthetic approach toward understanding the notion of public is taken using a theoretical biomimetics framework that identifies naturally occurring objects or phenomena that descriptively captures the essence of a construct to facilitate creative problem solving. in the present case, the problem being solved is how to reconcile what is meant by public, how it ought to be interpreted, determining the diverse levels of confidence in science that exist, and various understandings of science all with one another. the results demonstrate there is an inherent denotative-connotative inconsistency in the traditional notion of public that can be explicated through the concept of a fractal allowing for comprehension of the relationship between public confidence in, and understanding of, science. keywords: curriculum; scientific literacy; science and the public; public understanding of science (pus); public engagement with science (pes) https://doi.org/10.4995/muse.2020.12787 mailto:lcarroll@buffalo.edu https://orcid.org/0000-0003-4132-6392 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 73 1. introduction in a report by the american academy of arts and sciences entitled “perceptions of science in america” (aaas, 2018), it was suggested that additional research needs to be conducted, and it should expand upon the definition of science literacy in a manner that emphasizes the importance of understanding the scientific process and the ability to evaluate conflicting scientific evidence. conflicting scientific evidence abounds, and there exists a plethora of examples in the media of both faulty reasoning and unfounded claims (diethelm, 2009). despite all of the deficiencies in the public understanding of science (pus) on display in the media, the nature of the attitudes of the public toward science tends to be positive (aaas, 2018). also, for example, research has demonstrated that although various levels of confidence in science do exist, the majority of people believe that the benefits of science outweigh any potential risks (aaas, 2018). moreover, the public thinks the highest priority for science should be given to improving educational outcomes, reducing poverty, and finding cures for disease and illness (aaas, 2018). as admirable as the hopes for, and belief in, science may be, it is vital to understand that the publics’ trust or confidence is a function of demographics and the particular issue in question (aaas, 2018). in fact, not only are members of the public found to have diverse levels of confidence in science but if asked to explain their understanding of the term “science,” one would discover that science means something different to each member (aaas, 2018). that there exists no consensus concerning the public understanding of science as an enterprise should not astonish the reader because the state in which we find ourselves is a logical consequence of the notion of a public that i argue may be responsible for the issues that everyone, including students, ultimately has to face. these issues may be more readily appreciated by providing a potential framework for understanding with which we begin. https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 74 2. theoretical biomimetics as a framework for understanding: a fractal public whether constructing a skyscraper or building an airplane, humankind’s most significant source of inspiration has been nature and it will always be its ultimate guide. similar to the manner in which the principles found in natural things, or those underlying naturally occurring phenomena, are used to drive the development of human-made tangibles such as tall buildings or planes, carroll (carroll, 2017) has developed an approach to creating intangibles such as solutions in problem-solving, which compares and contrasts principles or phenomena as they exist naturally with realworld problems to gain insight and is referred to as theoretical biomimetics (tb). as a tool for analyzing the present issue of understanding what is meant by "public," applying theoretical biomimetics as a framework for recognizing in which way the problem or issue resembles objects or phenomena that are found around us in nature has been incredibly enlightening. through the tb lens, we discuss what a public is, attempt to reconcile it with public confidence in science, and lastly consider the role that public understanding of science plays. the dictionary defines “public” as people or community (stevenson, 2011). additionally, synonyms for the word public according to the same source include "citizens, subjects, the general public, electors, electorate, voters, taxpayers, ratepayers, residents, inhabitants, citizenry, population, populace, society, country, nation, and the world" (stevenson, 2011). if we are to accept the definition of the word public as true on authoritative grounds, then, according to the analysis of the synonyms, interpretation of the definition, and understanding of the word, what makes a public is a group of people who have at least one thing in common. with a clear understanding of the public, we must consider the notion of logical consequence, as it pertains to such a notion. something is said to be a logical consequence of another just in case it is impossible for the former to be true without the latter being the case (barker-plummer et al., 2012). what concerns us now is how the lack of uniform agreement is a logical consequence of the notion of public. the primary issue is that the notion of public tends to be somewhat misleading when considering the stance on science and related issues while referring collectively to a group of different individuals as members of the same public. besides, relying on this understanding of public as a https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 75 starting point to refer to the people as such exposes the conceptually inherent flaw of internal inconsistency or self-contradiction. as a result of the contradiction, anything may be derived, which is something to be avoided at all costs. therefore, to avoid such a logical contradiction, we ought to abandon the traditional notion of public and declare that there can be no singular public; there are only multiple publics (aaas, 2018). the notion of "public" that i hold would be best understood as analogous to “myself watching this video of myself watching this video, ad infinitum.” as it so happens, my analogy may be conceptually represented more succinctly by the notion of a fractal. mathematically, a fractal is a macroscopic shape or object in which the overall abstract figure, pattern, or phenomenon comprises progressively smaller nested versions of itself when viewed microscopically (stevenson, 2011). much like the yield of a high-powered microscopic analysis of a frozen frame of the video of “myself watching this video of myself watching this video,” or the continuous yet slow zoom examination of any fractal would reveal: there exist multiple successively smaller identical versions of itself. it is for this reason that, given the word “public” by definition must be comprised of ever-successively smaller publics, i used fractal to characterize the essence of public and adopted a fractal framework in an effort to comprehend them more fully. if i construct an argument to support the claim that there are many publics, as mentioned and begin with the fact there is a lack of consensus among the public, then it may be more convincing to the reader. for example, borrowing just one potential issue of contention among the public in which there may only be two possible positions for members comprising a public to adopt, either everyone agrees, or they do not. so, unless there was complete unanimity (i.e., 100% agreement), then the people could be designated according to the position they supported, which would result in two groups. however, if each of the groups consisted of people who share a common position, then, according to the definition provided previously, each group would be considered its own public. in other words, we have discovered that the first public comprises at least two distinct publics. furthermore, what was true of the first public must be true of each of the two new publics. of course, continuing along this same line of reasoning would result in each subsequent public containing yet others, and so on. https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 76 the takeaway message from this thought experiment is that for each property or issue of contention there exist two or more sides; thus, necessarily there is more than one public. furthermore, if the issue of contention were “what does science mean” or “how does one interpret science,” or even “what is your level of confidence in science,” then the results would be identical: the existence of a fractal public. additionally, having multiple publics is consistent with the reality of varying interpretations of science as well as distinct levels of confidence in science. given the aforementioned differences in confidence and meaning of science that exist in each public, it may be concluded that the relationship between confidence and public understanding of science is not merely one of association; the relation is one of cause-and-effect. nonetheless, the cause and effect are not what most would believe. it is not the case that levels of confidence in science cause various meanings of science or how science is interpreted; i contend that the differing levels of confidence in science can be attributed to the different interpretations and meanings that science has to different people. if there is to be any hope of society moving beyond behavior in the form of flawed reasoning, then we must strive for, and engage in, rational discussion. moreover, if rational discussion requires there to exist a possibility of establishing inference-warrants, then all parties involved in the engagement must be clear about the sort of problem or issue at hand (toulmin, 2003). such clarity concerning a particular problem may only be achieved if there can be improvements made in the public understanding of science (pus). thus, the process of improving pus must begin with a concerted effort to remediate the faulty reasoning and unsubstantiated claims that have become the norm. despite the legitimacy of science and overwhelming evidence supporting climate change and anthropogenic global warming (agw), many of the american public remain either skeptical or in complete denial of its truth (dunlap, 2013). the degree of skepticism is much higher in segments of the public on the lower end of ses and educational achievement concerning the purported benefits of science and research and the astounding rate at which technological advancement occurs (aaas, 2018). nevertheless, despite the fact that skepticism is a qualitative characteristic that is both native to, and necessary, for science (dunlap, 2013), in the face of existing evidence, complete denial is not. https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 77 that the public can remain in denial suggests a lack of appropriate knowledge, skill, and attitude for making judgments regarding such issues of scientific concern. whether skeptical but willing to accept evidence, or skeptical and unwilling to accept any evidence, in order to legitimately claim improvements in the pus have been accomplished, all members of the public ought to be equipped with basic scientific skills requisite for evaluating issues of concern. thusly equipped with improvements concerning the pus, there would genuinely exist a competent public comprised of informed individuals each of who are more likely to participate in fulfilling the role of citizen scientists (mejlgaard & stares, 2009). public engagement with science (pes) refers to opportunities for mutual learning and growth that comes about when scientists and members of the public meaningfully and deliberately interact (aaas, 2018). that notwithstanding, pus is a prerequisite for pes. therefore, to remediate the deficiencies in knowledge (i.e., pus) concerning the basic principles of logic, reasoning, and argumentation necessary for participation in rational scientific discussion (i.e., pes), i have designed a course curriculum addressing them. by availing the public of a seminar entitled “fundamentals of logic, reasoning, and argumentation for public engagement with science (pes),” individuals who enroll and complete my course will acquire the acumen, ability, and attitude that are essential to contributing to decision-making related to issues of scientific concern. it is through the acquisition of such essential tools that, not only will learners possess what is needed in order to evaluate issues of scientific interest and concern, they will be capable of forming their own opinions and appropriately supporting their respective stances. 3. student factors: potential threats to learning everyone at one time or another has had a learning experience, which consisted of them either mentally or physically doing things that led to changes in their knowledge, skills, or attitudes (jones, noyd, & sagendorf, 2015). regardless of the experience, many factors that influenced the outcome of their learning efforts. the previous learning efforts may have been either positive or negative depending on the individual and how they perceived it. no matter how they may have been perceived, such outcomes that comprise the backgrounds and experiences related to learning, culture, https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 78 family, self-esteem, and confidence, which provide the context within which learners exist. in addition, as a contextual framework within which learners live, background and experience also play a role in constructing or forming what they know, the attitudes they adopt, and any skills that they possess. for any student learner, naïve conceptions, one’s level of maturity, and his or her tendency to challenge authority are all products of their experience that may be categorized under knowledge, skills, and attitudes. additionally, individuals of greater socioeconomic means may have more learning opportunities and exposures to learning than the less fortunate, which affords them chances to improve their attitude, correct knowledge deficiencies, or perfect a skill. nonetheless, regardless of student ses and despite the many potential factors that have the potential to detrimentally impact the learning outcomes, there is one key aspect of related to my course offering that i claim effectively neutralizes most if not all the other potential student factors: the elective nature ultimately in my class. this class was deliberately designed according to principles of a learner-centered curriculum (jones, noyd, & sagendorf, 2015) because it ensures that the final design is self-contained, which facilitates the process of learning when the course is offered by allowing every student enrolled to accomplish the primary course goals without requiring that attendees rely on prior experiences. 4. factor forms: intentional versus unintentional indeed, there can be no way to know with absolute certainty who shall attend my course. as an unknown, it becomes even more clear why as course designer i insisted upon the learner-centered curriculum: without knowing the “who” (i.e., persons who will enroll) it would be nonsensical to assume knowing “what” in the form of science background as prior experiences anyone will have. nevertheless, whomever these students will one day be, along with them will come various challenges that may hinder the learning process. although many potential types of challenges may exist and could justifiably be worthy of our attention, i feel that it is necessary to categorically https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 79 distinguish between at least two main varieties that will present as challenges due to their potential to affect students’ ability to achieve an educational goal. dichotomizing the forms of challenge results in what i refer to as 1) intentional factors, and 2) unintentional factors. unintentional factors i claim would be those circumstances, outcomes, or experiences over which students exert little to no control. an example of an unintentional factor would be having been raised in poverty or a low ses as a child or suffering from a developmental disability. as opposed to unintentional ones as we have already mentioned, an intentional factor would be something over which one does have legitimate control, yet he or she fails to exert it for whatever reason. it is crucial to understand that, in such cases, a factor at play need not result from the commission of an act in order to qualify as intentional; because intentional/unintentional refer to the relation between an individual and factors with the potential to disrupt the process of learning or self-fulfillment, we may. now include the omission of an act, which might be a direct result of intentional behavior and rightfully categorized under intentional factors. 5. the case of obstinacy as an intentional factor as an example of an act of omission, for instance, were a student to exhibit obstinacy related to a naïve conception held just discovered by him in class to be impossible, the student’s refusal to cooperate by responding with the appropriate response (i.e., inaction) would be considered intentional. now, unlike unintentional factors, such deliberate stubbornness by a student in class allows me to infer that a reason –or more accurately, an absence of reason– for such behavior exists. i would argue that the absence of reason would be related to a lack of motivation, interest, or incentive. in other words, if the student had the motivation to acknowledge the naïve conception was incorrect, interest in it being incorrect or determining what is correct, or incentive to produce the correct response, then the intentional factor of stubbornness would cease to exist. as a science educator, knowing that i would be unable to repair or replace any of the horrible experiences some students endured that may serve as unintentional factors (e.g., poverty) impeding both the efforts of students and myself would be disheartening. however, in the case of my course, https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 80 since any student will either have intentional, unintentional, or both factors to contend with while learning in my course, fortunately, by virtue of there being no enrollment costs, registration being entirely voluntary, and made available on a first-come-first-served basis, not only will the majority of unintentional factors be directly and effectively neutralized (e.g., ses), but given the voluntary signup, each enrollee had to be already motivated, interested, and have the incentive to do so. in this fashion, my course offering also attempts to indirectly ensure that the potential for any remaining factors to be present – intentional or otherwise– is significantly reduced before the course even begins. while not guaranteed to eliminate all the potential challenges in the form of factors, efforts were put into all aspects of the course that are guaranteed to eliminate some factors. upon their elimination, these student factors no longer threaten to detrimentally impact the material being taught or learned in the class. 6. learning goals learning goals may be understood as what students should be able to accomplish upon completing a course of study (jones, noyd, & sagendorf, 2015). the visible result of completing my course would be that attendees will be capable of performing their own independent assessment concerning scientific issues relevant to society using logical reasoning and argumentation. a thorough understanding of science basics and an ability to employ logic, reasoning, and argumentation routinely to facilitate learning. ultimately, successful students will find that education becomes a process of self -propagation. for instance, individuals claiming to know a, b, and c, for example, should be able to employ the skills that they have learned and, relying on them as premises, derive d; then, with a, b, c and d, conclude that e is the case. possessing the ability to determine for themselves what justification –if any– there may be for believing something, whether that something is, in fact, real, and claiming to know that something is hugely empowering. moreover, from this starting point, they may metacognitively engage themselves deducing from what is already known to add to their knowledge base as well as assess whether new claims are consistent with what is already known and why https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 81 general learning goals (glgs) for the course include the following: 1) ability to make logical decisions relying on available evidence concerning scientific issues; 2) be capable of forming their own opinions regarding public policy and contribute to dialogue on critical issues in public discourse; 3) employ the acquired knowledge and skill to determine the best course of action with regard to their behavior and of those for whom they are responsible including both relatives and the public. 7. goal descriptions glg 1 entails the use of logic, reasoning, and argumentation, to increase knowledge and to aid in decision-making according to a rational process that relies on available sources of evidence concerning particular scientific issues. glgs 2 and 3 may be understood as giving students the tools needed to allow them to construct their own opinions regarding private (i.e., personal ethics) and public policy, thereby to equipping them with the ability to contribute to dialogue and discourse in the public forum. of the glgs, it is glg 1 that serves as the primary learning goal for participants and is what guided curriculum design. 8. summative assessment a summative assessment is used to evaluate the level of student comprehension and is given at the end of a course. like the formative assessment, the summative assessment was aligned with the goal of the course. aligning the summative assessment in such a fashion is akin to a method of validating that the course content, experiences, formative assessment, and proficiencies measure or reflect that students learned what they were supposed to learn. the summative assessment for my course will be the following project: given the available evidence regarding the phenomenon of global warming and climate change, conduct a review of the literature and choose 5 research papers claiming to support and 5 claiming to refute its occurrence. then, analyze the evidence presented and using logic and reasoning, determine your stance on the issue and present an argument in support of it. https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 82 9. learning proficiencies in order to achieve the goals of this course, attendees must possess specific knowledge, skills, and attitudes (ksas). without these ksas, students will be unable to move closer toward the ultimate learning goals of the course. each step of the way students will acquire these ksas to the point of proficiency so as to progress; thus, since progression hinges on students’ becoming proficient through “the acquisition of ksas,” the ksas are referred to as learning proficiencies. table 1 comprises the learning proficiencies (i.e., ksas) for this course. the proficiencies in table 1 have been categorized according to the type by row and by column, which allows for a coordinate system of designation to describe each one. table 1 descriptive learning proficiencies for fundamentals of logic, reasoning, & argumentation type knowledge skills attitude principle participants will be able to recognize the basic structure of an argument that comprises scientific evidence. construct a basic scientific argument. students will be self-efficacious (i.e., believe in their own abilities). concept students will be able to distinguish between valid argument and invalid argument. critical thinking skills (e.g., critical reading, evaluation of evidence) students will be motivated to use logic, reasoning, and argumentation as a tool for selfeducation and decision-making concept identify characteristic parts of a given scientific argument. critique a scientific argument of others and evaluate for validity, soundness. students will be comfortable fact students will be able to define and describe the characteristics of an argument, its premises, its assumptions, its conclusions, validity, soundness, deduction rules, derivable rules. metacognitive skills (e.g., monitoring their own progress) critique their own scientific argument and evaluate for validity and soundness. https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 83 for instance, the facet of proficiency requiring one to recognize the basic structure of an argument comprising scientific evidence is located in row 2, column 2: row 2 (including header row) reflects that principles are necessary for proficiency while column 2 reveals that knowledge is required to be proficient, as well. the basis for the coordinate system i decided upon is the following. an ability to recognize an argument requires knowledge of its general structure. recognition alone is the least that can be done and calls for neither skill nor does it involve attitude. moreover, though this proficiency may rely on knowledge that is derived from the cohesiveness of facts that yield concepts, it is neither factual nor itself a concept. since success in demonstrating proficiency, therefore, comprises principles and knowledge, which requires that one be capable, i propose that proficiency be conceived of as an ability that results from a thorough understanding of the concepts derived from the knowledge of isolated facts. although it does a wonderful job organizationally, table 1 is descriptive. the descriptive elements in the table may be improved upon, in my opinion, by the adoption of an alternate framework. instead of rightfully viewing learning proficiencies as merely being descriptive of the outcomes of a metaphoric “course goal equation,” i wondered whether altering perspectives on either the goal, the equation itself, or both would lead to greater insights. ultimately, it would be changing my perspective on the overall equation that made the most sense. table 2 reorganization and mapping of table 1’s learning proficiencies for fundamentals of logic, reasoning, & argumentation type beginner intermediate advanced attitude students will be motivated to use logic, reasoning, and argumentation students will be comfortable students will be self-efficacious (i.e., believe in their own abilities). knowledge participants will be able to recognize the basic structure of an argument that comprises scientific evidence. students will be able to distinguish between valid argument and invalid argument. students will be able to identify parts of a scientific argument. students will be able to define and describe an argument, its premises, its assumptions, its https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 84 by viewing each of the learning proficiencies as components of an overall goal equation that are individually necessary yet only sufficient together, the proficiencies have been transformed into what students ought to know, what they should be able to perform, and the attitude they should possess to allow them to realize the outcomes of the course. in this fashion, table 2 displays the results obtained from revising my perspective on the information contained in each cell of table 1, which are mapped to the theoretical level of a successful student outcome. conclusions, validity, soundness, natural deduction rules of logic, derivable rules of basic system of logic (system k) skill students will be able to transcribe real-world scientific issues into the arguments that comprise them participants will be able to apply basic rules of logic in the assessment of simple scientific arguments. critical thinking (e.g., critical reading, evaluation of evidence) critique scientific arguments of others, evaluate them for validity and interpret the results. judge scientific arguments as evidence, determining any consequences and their impact, recommending courses of action to be taken based on the judgment, and suggest public policy concerning scientific issues. metacognitive (e.g., monitoring their own progress) critique their own scientific argument and evaluate for validity and soundness. https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 85 table 3 angelo & cross’ (1993) classroom assessment techniques (cats) among those to implement in the assessment of course-related knowledge, skills, and attitudes (ksas). type beginning intermediate advanced attitude students will be motivated to use logic, reasoning, and argumentation students will be comfortable 39. process analysis: students outline the process they take in completing a specified assignment. 31. everyday ethical dilemma: students respond to a case study that poses a disciplinerelated ethical dilemma knowledge 1. background knowledge probe: short, simple questionnaires prepared by instructors for use at the beginning of a course or at the start of new units or topics; can serve as a pretest; typically elicits more detailed information than cat2. . 2. focused listing: focuses students’ attention on a single important term, name, or concept from a lesson or class session and directs students to list ideas related to the “focus.” 3. misconception or preconception check: focus is on uncovering prior knowledge or beliefs that hinder or block new learning; can be designed to uncover incorrect or incomplete knowledge, attitudes, or values skill (analytic and critical thinking) 8. categorizing grid: student complete a grid containing 2 or 3 overarching concepts and a variety of related subordinate elements associated with the larger concepts 9. defining features matrix: students categorize concepts according to presence or absence of important defining features 10. pro and con grid: students list pros/cons, costs/benefits, advantages/disadvantages of an issue, question or value of competing claims skill (synthetic and creative 13. one-sentence summary: students answer the questions “who does what to whom, when, where, how, and why?” (wdwwwwhw) about a given topic and then creates a single informative, grammatical, and long summary sentence 15. approximate analogies: students simply complete the 2nd half of an analogy—a is to b as x is to y; described as approximate because rigor of formal logic is not required 16. concept maps: students draw or diagram the mental connections they make between a major concept and other concepts they have learned https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 86 16. concept maps: students draw or diagram the mental connections they make between a major concept and other concepts they have learned 10. conclusions the effort was made to present the case for a course that was designed to address the need for improved scientific literacy through teaching the fundamentals of logic, reasoning, and argumentation for public engagement with science. the targeted audience includes adult students who are members of the public who are not considered scientists by profession. in addition to providing justification for the offering, the approach that will be taken toward educating students who enroll has been provided. among the more compelling reasons creating such an enriching experience in the form of this course is the fact that interest in science is growing while scientific literacy is decreasing (suleski & ibaraki, 2010). because science exists in the form of scientific theories, logic, reasoning, and argumentation are necessary in achieving scientific literacy. thus, scientific literacy is required for understanding, as well as engagement with, issues of scientific concern in any meaningful capacity. with neither understanding nor engagement by the public (comprising yet other publics, which are themselves made up of still more publics), unfortunately, policies cannot be shaped by citizens. that notwithstanding, through the provision of opportunities for enrichment such as the course herein presented or others that are similar, those individuals who do attain a level of scientific literacy that allows them to understand and engage in matters of scientific concern place themselves in the unique position of being among both society’s benefactors and its beneficiaries of public policy simultaneously. while everyone may not actively participate in the process of policymaking, one https://doi.org/10.4995/muse.2020.12787 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 87 this is certain: policies made concerning scientific issues cannot be said to truly benefit those who fail to understand them. references american academy of arts and sciences (aaas). america; 2018. diethelm p, mckee m. denialism: what is it and how should scientists respond? european journal of public. 2009;19(1):2–4. https://doi.org/10.1093/eurpub/ ckn139. carroll lsl. theoretical biomimetics: a biological design-driven concept for creative applied to the optimal sequencing of active learning techniques in educational theory. multidisciplinary journal for education, social and technological sciences, 4(2):80–96. https://doi.org/10.4995/muse.2017.7078. stevenson a, waite m. concise oxford english dictionary: luxury edition. oup oxford; 2011. https://market.android.com/details. barker-plummer d, barwise j, etchemendy j, et al. language, proof, and logic (vol. 2). stanford, ca: csli publications; 2011. toulmin se. the uses of argument. 2003. cambridge university press. isbn: 13 978-0-511-06271-1. dunlap re. climate change skepticism and denial: an introduction. the american behavioral scientist. 2013;57(6):691–98. https://doi.org/10.1177/ 0002764213477097. mejlgaard n, stares s. participation and competence as joint components in a cross-national analysis of scientific citizenship. public understanding of science. 2009; 19(5):545–561. https://doi.org/10.1177/0963662509335456. jones sk, noyd rk, sagendorf ks. (2015).building a pathway to student learning: a how-to guide to course design. https://market.android.com/ details. suleski j, ibaraki m. scientists are talking, but mostly to each other: a quantitative analysis of research represented in mass media. public understanding of science. 2010; 1(115–125). https://doi.org/10.1177/0963662508096776. https://doi.org/10.4995/muse.2020.12787 https://doi.org/10.1093/eurpub/%20ckn139 https://doi.org/10.4995/muse.2017.7078 https://market.android.com/details https://doi.org/10.1177/%200002764213477097 https://doi.org/10.1177/0963662509335456 https://market.android.com/%20details https://doi.org/10.1177/0963662508096776 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12787 social and technological sciences issn: 2341-2593 carrol (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 72-88 | 88 appendix a. author biography la shun l. carroll is a full member of sigma xi, the scientific research honor society. he received his doctoral degree, cum laude, from the university at buffalo school of dental medicine. subsequently, dr. carroll earned his ed.m. graduate degree from the university at buffalo graduate school of education. as an undergraduate, he graduated with a b.a. from baruch college, magna cum laude, majoring in both philosophy and natural science. his publications include "theoretical biomimetics: a biological design-driven concept for creative problem-solving as applied to the optimal sequencing of active learning techniques in educational theory" in the multidisciplinary journal for education, social and technological sciences (october 2017), and “a comprehensive definition of technology from an ethological perspective” (mdpi, 2017). research interests include metaphysics, logic, science, technology, and education. dr. carroll was an adjunct professor at saint michael’s college. https://doi.org/10.4995/muse.2020.12787 10. conclusions the effort was made to present the case for a course that was designed to address the need for improved scientific literacy through teaching the fundamentals of logic, reasoning, and argumentation for public engagement with science. the targeted audi... with neither understanding nor engagement by the public (comprising yet other publics, which are themselves made up of still more publics), unfortunately, policies cannot be shaped by citizens. that notwithstanding, through the provision of opportun... multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 85 students’ civic engagement in ukraine and canada: a comparative analysis oksana zabolotnaa , anna pidhaietskab "a pavlo tychyna uman state pedagogical university, 2 sadova str., uman, 20300, ukraine" "b pavlo tychyna uman state pedagogical university, 2 sadova str., uman, 20300, ukraine" * correspondence: o.zabolotna@udpu.edu.ua received: 18 january 2021; accepted: 10 march 2021; published: april 2021 abstract in this article, the authors have carried out a comparative analysis of students’ civic engagement in ukraine and canada. they have surveyed the students at pavlo tychyna uman state pedagogical university and compared the findings with the results of a study done by the canadian researcher catherine broom at british columbia university. based on the research findings, the authors have identified ukrainian students’ personal political and civic experience levels and compared them with the canadian results. the study reveals ukrainian students’ attitudes towards political and civic participation, democracy, the government in general and in comparison with canadian data. the research results have identified the following key factors that influence ukrainian students’ civic activity: students’ free time activities their attitudes and beliefs. according to the survey, gender, religious involvement, personality type, and family’s political involvement do not directly influence the students’ civic engagement. the survey has not reported any influence of school social study courses on civic engagement, stressing the importance of real-life experiences that result in attitudes and intrinsic motivation. the authors have also revealed examples of motivations and barriers for youth civic involvement. keywords: civic participation; ukraine; canada; students; youth engagement. to cite this article: zobolotna, o., pidhaietska, a.. (2021). students’ civic engagement in ukraine and canada: a comparative analysis. multidisciplinary journal for education, social and technological sciences, 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0003-4901-1053 https://orcid.org/0000-0002-3647-9880 https://doi.org/10.4995/muse.2021.14962 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 86 1. introduction under current conditions of democratic principles realisation – society renovation and world integration process, increasing attention of scholars is paid to the problems of civil society. building democracy in ukraine is impossible without the formation of active citizens, who have a sense of social responsibility. thus, students’ active engagement in civic life is the key to our society’s success and progress, whereas young people comprise a considerable part of the ukrainian population. furthermore, civic education contributes to students’ self-awareness, critical thinking, freedom of choice, comprehensive development, commitment to universal values, respect for the views of others, constructive relations and peaceful resolution of conflicts, which is crucial for both – personal development of a democratic citizen and a democratic society in general. bright examples of youth civic activity in the last decades are student protests – the “orange revolution” (2004) and “revolution of dignity” (2014) in ukraine and “maple spring” (2012) in canada. these student protests reaffirm that young people’s active civic position is formed when they feel and respond to their country’s needs and citizens. nevertheless, the participation rates may rise if the youth realise that a political issue directly impacts them. only under these circumstances, a young person is encouraged to seek change for a better life and make this world a better place to live. these political issues in light of the latest social processes gave us the idea of comparing the ukrainian and canadian students’ civic engagement, investigating their lived experiences, youth beliefs, and civic life actions. 2. theoretical background much research has been done in this field in ukraine and canada. for instance, some ukrainian authors have focused on the issues of ukrainian youth citizenship and civic education (tereshchenko, 2010; kyrylovych, 2012) and teaching students citizenship through selfgovernment in higher educational institutions of canada (vasylenko, 2009). however, we should point out that the given above works have a more speculative character and they are rarely based on empirical research. on the contrary, most canadian researchers mainly rely on the data received from surveys with the use of different methodologies. canadian researchers’ findings illustrate the https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 87 complexity of connections between youth civic views and behaviour (broom, 2016). other scholars studied relationships between youth engagement and civic action. muddiman, taylor, power and moles (2019), hooghe and stiers (2020), white and mistry (2016) examined the importance of families fostering youth civic participation, martínez et al. (2016) explored how social and academic facilitators and engagement predict the performance of university students. our study has investigated students’ free time activities, and their attitudes and beliefs towards civic issues; the influence that gender, religious involvement, personality type, and family’s political involvement have on students’ civic engagement. other researchers who have worked in this context are barrett and brunton-smith (2014), cohen and chaffee (2013), cicognani (2011), yeung (2017), schugurensky and wolhuter (2020). as broom (2016) sums up, “they refer citizenship education to the planned development of individuals’ civic knowledge, attitudes, and skills, with the aim of developing actions towards the collective life of the community that attempt to improve that life for all” (broom, 2016, p. 5). that explains the importance attached to the place of citizenship in teacher education. citizenship education has been viewed from a cross-cultural perspective in the recent book “global citizenship education and teacher education” (schugurensky & wolhuter, 2020) that has brought together researchers from different countries. along with that, the comparative pedagogical analysis of students’ civic engagement in ukraine and canada has been carried out for the first time. so far, in ukraine, few researchers have devoted their studies to answering the research question of why some youth are civically involved while others are not. 3. methods we decided to carry out the research similar to the one done at canadian british columbia university by catherine broom in 2014 and published in 2016. in her research, broom (2016) illustrated the complexity of connections between youth civic views and behaviour, paying attention to the dynamic interaction between individuals and their lived experiences and considering citizenship education curriculum. consequently, the objectives of our research were to compare canadian and ukrainian students’ in the following aspects: https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 88  students’ civic background characteristics, their free time activities and attitudes towards civic issues  the way the characteristics mentioned above influence the students’ attitudes towards civic engagement and civic activities the students participate in the research hypothesis was that the youth in canada and ukraine with similar background characteristics and ways of spending free time and attitudes towards civic issues demonstrate similar civic engagement and participate in similar civic activities. thus, the research questions are:  do the students’ civic background characteristics in ukraine and canada influence their civic engagement?  do the students’ free time activities in ukraine and canada have an impact on their civic engagement?  do the students’ attitudes towards civic issues in ukraine and canada relate to their civic engagement? the findings illustrate the complexity of connections between youth civic views and behaviour. paying more attention to the dynamic interaction between individuals and their lived experiences and considering citizenship education curricula and teaching practices concerning this complexity may improve citizenship education. hence, in 2018 we conducted similar research at pavlo tychyna uman state pedagogical university students to compare the findings with canadian study, investigating the influence of internal and external factors on youth civic engagement. the survey was conducted with students of different departments, who were the same age with canadian peers and in the same quantity as in canadian research. the participants in both countries were youth between the ages of 19 and 28. ukrainian students and canadian ones were instructed to fill out a paper form of the questionnaire that included demographic information (age/race/gender/class) and identify which free time and civic processes they are engaged in. open-ended questions asked participants their views of government/politics, civic society/democracy and public participation, their ideal political state, and how they envisioned general public and their participation in civic life (broom, 2016). https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 89 3.1. ethical issues we have written a letter to the canadian researcher catherine broom (phd, associate professor, okanagan school of education, and the university of british columbia, an editor of citizenship education research journal) with a request to use some tools of her study. dr catherine broom expressed her deep interest in conducting comparative analysis and gave us her kind permission to do that. 3.2. the survey participants since the research is comparative, to ensure its relevance, we have selected the same number of ukrainian participants in the categories defined by broom (2016) (for more information see table 1). table 1 the survey participants participants canadian participants ukrainian participants male female male female number and percentage 18 (43%) 24 (57%) 5 (11%) 37 (89%) 4. the research findings the survey data from british columbia university (canada) collected by catherine broom (2016) and from pavlo tychyna uman state pedagogical university received by the authors provided the possibility to compare the students’ civic background characteristics: including studying the subject “citizenship education” in the high school curriculum, the students’ personal political experience, the students’ family political involvement, the students’ religious involvement, the students’ free time activities (for more information see table 2). https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 90 table 2 students’ civic background characteristics positions canadian participants ukrainian participants yes no do not remember yes no do not remember the students who have learnt the subject “citizenship education” in high school and who consider this subject useful 28 (67%) 9 (21%) 5 (12%) the students who have had personal political experience 11 (26%) 27(64%) 4 (10%) 10 (24%) 26 (63%) 6 (13%) the students’ politically involved families 21 (50%) 16 (38%) 5 (12%) 30 (71%) 7 (16%) 5 (13%) often sometimes never often sometimes never the students with religious background involvement 9 (21%) 12 (29%) 21 (50%) 5 (13%) 28 (66%) 9 (21%) additionally, the questionnaire contained questions about the students’ free time activities concerning their spending time with friends, interacting with social media, engaging with pop culture, patriotic activities, and political activities (see table 3). these findings also established the links between the students’ civic engagement and their free time activities. table 3 the surveyed students’ free time activities positions canadian participants ukrainian participants spending time with friends 19% 21% interacting with social media 18% 21% engaging with pop culture 12% 18% patriotic activities 4% 2% political activities 1% 4% the authors consider the research of the students’ attitudes important for understanding their motives for their civic engagements within the investigated field. thus, some survey questions concerned the students’ attitudes towards political participation, democracy, and government system in their country with three answer variants: supportive (important); neutral; unimportant (do not care) (see table 4). https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 91 table 4 the surveyed students’ attitudes towards civic issues positions canadian participants ukrainian participants supportive / important neutral unimportant (do not care) supportive/ important neutral not supportive/ negative students’ attitudes towards political participation 37 (88%) 3 (7%) 2 (4%) 32 (76%) 10 (24%) students’ attitude towards democracy 25 (59%) 14 (34%) 3 (7%) 23 (55%) 14 (34%) 5 (11%) students’ attitude towards government system 21 (50%) 15 (35%) 6 (15%) 2 (5%) 18 (42%) 22 (53%) all the mentioned above issues (students’ civic background characteristics, their free time activities, and students’ attitudes towards civic issues) influence their attitude towards civic engagement and the choice of the civic activities the students participate in (for the former see table 5; for the latter see table 6). table 5 the students’ attitude towards civic engagement position canadian participants ukrainian participants active somewhat active rarely active active somewhat active rarely active students’ attitudes towards civic engagement 4 (10%) 18 (43%) 20 (47%) 8 (19%) 23 (55%) 11 (26%) table 6 civic activities the students participate in the choice of civic activities the students participate in positions: canadian participants ukrainian participants volunteering 34% 18% voting 31% 35% following political news 10% 31% boycotting 9% using social media to interact politically 6% 10% protesting 3% 2% https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 92 discussion. the survey findings have proved the hypothesis that the youth in canada and ukraine with similar background characteristics, ways of spending free time and attitudes towards civic issues demonstrate similar civic engagement and participate in similar civic activities. however, the extent of the similarity may be different. consequently, in canada and ukraine among students surveyed, there were more females than males, who identified themselves as middle class. however, compared with canadian research where there was a slight ratio difference between men and women (men 43%, women 57%), a considerable gender gap was observed (men 11%, women 89%). in our opinion, the fact that female students predominate in ukrainian pedagogical universities is owing to the widespread belief that a teacher's profession is unpromising and not prestigious. another reason for such discrepancy between the number of girls and boys at ukrainian and canadian universities, in our opinion, is that ukrainian educational institution is single disciplinary unlike canadian university, which is multidisciplinary. however, more research is needed in this area. the majority of canadian students had not had a significant lived experience related to community or governmental issues that had affected their views of government. those who had had experiences described issues with religious members of parliament, the northern gateway pipeline, noise, zoning, transit, and politicians they did not like (broom, 2016, p. 7). moreover, half the young adults stated that they had grown up in politically active families in some way (voting, talking about politics or other political events). the findings that emerged from canadian study showed that youth look at political and civic activity through the prism of volunteering, voting, following political news, boycotting and protesting (broom, 2016). it should be noted that canadian youth who stated that they voted considered themselves civically active members of society. it meant that canadian students identified voting as the significant civic activity, as those youth whose parents were active stated that their parents were primarily active by voting and some students connected their parents’ actions to their levels of civic activity or their views of democratic government (broom, 2016). furthermore, ukrainian students who volunteered defined themselves as civically active citizens. the beliefs of ukrainian students about the civic engagement of young people fully coincide with the views of canadian students, such as for them civic activism means: to take part in elections, various protests, and rallies; engage https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 93 in volunteering (help children from orphanages, homeless animals, etc.); to be a member of a public association; to attend city council meetings and classes at “young leader’s school” etc. the survey conducted in ukraine showed that ukrainian families were more actively involved in political activities (71%) compared to canadian families (50%). however, this factor did not significantly affect the practical experience of students’ political activity. low rates of students’ personal political experience in ukraine and canada (24% and 26% respectively) were explained by the fact that most of them had never involved in political activity in their country. a small percentage of students engaged in political activity canadian students related to such reasons: lack of time, they did not find political life relevant for themselves, they had other priorities at their life stage, or they felt powerless and believed no one was interested in youth (broom, 2016). nevertheless, 88% canadian and 76% of ukrainian students recognised political activity as a vital task for every conscious citizen, but students contradict themselves and their own beliefs in practice. here are some quotes of canadian students about the importance of engaging young adults in politics: “it’s our government, we should run it, and our voice can’t be heard unless we participate, vote, protest, volunteer, make yourself represented”; “so we can change the way how politics works, and so the government does something good” (broom, 2016, p.8). such lack of students’ interest in political activity in both countries, in our opinion, affected the low percentage of students (10% canadians) and (19% ukrainians) involving in civic activity. the vast majority of students were only occasionally active (43% canadians, 55% ukrainians) and the rest (47% canadians, 26% ukrainians) only sometimes joined community events. the most popular among ukrainian and canadian students’ types of civic engagement were voting and volunteering. based on the findings of our survey, ukrainian students as compared to canadian students are more active in voting participation (35% ukrainians; 31% canadians), are more aware of the political life of the country as they follow political news (31% ukrainians; 10% canadians), they are more likely to discuss politics using social networks (10% ukrainians; 6% canadians). meanwhile, canadian students more than ukrainian youth volunteer (34% canadians; 18% ukrainians), boycott (9% canadians; 0% ukrainians), protest (3% canadians; 2% ukrainians), https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 94 which means that canadian students more do than talk. among ukrainian students, only (2%) are members of a political party or community organisation. though canadian students are less likely to follow political news and do not actively discuss politics in social media, we conclude that canadian youth, in contrast to ukrainian students, are more focused on solving practical problems. a small percentage of student engagement to civic activity ukrainian students account for other priorities, lack of time and experience, and despair of a change for the better. here are some students’ quotes of ukrainian higher educational establishment concerning their civic activity: “in our country, participation in civic activities is complicated, unprofitable and sometimes even dangerous”; “i had no opportunity to join one”; “we are too young and inexperienced”; “i don’t believe in change for the better”; “i don’t find it interesting”; “i don’t think it’s necessary”, “i engage in civic activity at our university whenever there is a need”; “occasionally i join a public protest or attend rallies or community events”; “i am not invited to join”, “lack of time and even money”. thus, students in ukraine do not believe that civic activity can change something in their country; they do not benefit from it for them and their country. some students do not know how to engage in a civic activity or wait for a personal invitation to join it. ukrainian youth, as well as canadian, consider their young age and inexperience as a disadvantage that prevents them from creating a better future and being agents of change. however, 19% of ukrainian students’ answers turn out to be quite the opposite to the previous ones: “i participate in voting, provide temporary shelters for homeless pets and seek them owners”, “attend young leader’s school”, “i believe in change for the better and i start working in the political sphere”, “i attend city council meetings”, “i am actively engaged in volunteering because i love animals and try to help them”. the findings that emerged from the survey conducted in ukraine revealed that half of the students (53%) had a negative or not supportive attitude towards ukrainian government, most of the rest (42%) had neutral views, and (5%) of students had a positive attitude towards political leaders of the country. on the contrary, in canada, half of the students (50%) had positive views towards the canadian government and another half (50%) had a neutral attitude towards it. consequently, we found out that common for both countries were a high percentage of students who had a neutral attitude towards the political situation and government in their country. although https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 95 youth participation in civic and political life was limited, students of both countries stated that it was important for people to participate in government state-creative and public process. also deserved our attention the attitude of ukrainian and canadian students towards democracy – (55%) of canadian and (55%) of ukrainian students had a positive attitude towards democracy. recent research (sunil & verma, 2018) found that moral identity internalisation significantly predicted civic engagement attitude and moral identity symbolisation significantly predicted civic engagement behaviour. (41%) of canada youth and (34%) of their ukrainian peers had a neutral or indifferent attitude to the democratic system. it should be noted that 8% of surveyed ukrainian students mentioned their negative attitude towards democracy, arguing that democratic system was not also perfect: “democracy is a utopia, it exists only on paper"; students even quoted winston churchill: “democracy is the worst form of government, except for all the others”. (3%) of ukrainian adults stated that they did not care about democracy. meanwhile, there were no students among respondents at the canadian university with a negative attitude to the democratic system. although half of the ukrainian respondents (53%) had a negative attitude towards government, they still had a positive view of democracy (55%). we can conclude that youth in both countries value democratic forms of government and support democracy; however, they are not eager to be involved in political or civic activity, though they understand their importance. in response to the question – “is it necessary to engage in the civic activity?” – just over threequarters (76%) of ukrainian university students gave a positive answer, while nearly one quarter (24%) confirmed their neutral attitude. here are some student quotes: “it is important not only to complain about the political system in the country but also to do something in order to improve the situation”; “it is crucial to everyone to engage in civic activities, because if we do not do that, what kind of state we will have?”; “i believe in changes for the better, and i consider it necessary to involve in civic activity as much as one can, because the fate of our state is in our hands”; “if we want changes in the country we should start with ourselves first”, “it’s easy to say but hard to do”; “people themselves build their happiness”; “everything depends on a person maybe someone cannot live without civic or political activity, and someone does not see any point in it”; “citizens should live like in one family where all members help and support each other”; “active citizenship determines our future and the future of our state”; “if we sit on our https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 96 hands, life in our country won’t improve”; “if social activity solved something, then it would be important to me”. when students of both countries were asked to depict their ideal government, many young people described features that were often included as crucial elements of democracy: equality, fairness, responsibility, patriotism, representation of people, and listening to the people and freedom. however, despite such disappointment with the government, 66% of ukrainian students believed that student youth could change their country’s life for the better: “absolutely, because we are the country’s future”; “we can, for sure, we are a generation of new priorities, thoughts, ideas”, “students are the future of our nation, who bring new ideas that will help change the life of our country for the better”; “yes, we can, as we are the young generation with modern views and only when we get together we can change our country’s life for the better”; “of course we can, students are full members of the society”; “young people are the future! we should always keep abreast of the developments in order not to allow deception, manipulation, and humiliation”. nevertheless, 18% of surveyed ukrainian students considered impossible for them to bring radical social change: “it’s impossible, students don’t get any levers of power”; “students decide anything”. 16% of ukrainian participants were not sure that students could be agents of change, they doubted or found it possible only under certain circumstances: “if not only the youth defend their rights, but the older generation as well”; “if youngsters engage in volunteering or participate in voting”; “everyone can try to change something, but a good result is possible only with the officials’ support”; “in an ideal world – yes, but in real-world society is governed by oligarchic structures”; “with state support that helps youth”; “if students understand what they are doing”; “it’s rather a controversial question”. thus, we have concluded that ukrainian students want to live in a democratic society. however, they are no eager to build it, such as they do not see a connection between democracy and civic initiative. a study suggests that ukrainian youth do not realise that democracy is the people’s government; it implies the broadest possible involvement of all citizens in state decisionmaking processes. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 97 in our opinion, the low rates of students’ involvement in political and civic activities are also related to the ways students spend their free time. according to our study’s findings, a few students engage in political or civic activity in their free time. the survey showed that the vast majority of ukrainian (21%) and canadian students (19%) spent their free time with friends and interacting with social networks (21%) of ukrainians youths, (18%) of canadians peers. moreover, 18% of surveyed ukrainian students and 12% of canadian youth liked engaging with pop culture. the least students wanted to spend their spare time on political (4%) ukrainian adults, (1%) canadian youths and patriotic activity – (2% ukrainians), (4% canadians) respondents. it meant that surveyed ukrainian students more than their canadian peers spent their free time with friends, interacted on social media, and talked about the political situation in their country. nevertheless, canadian students, unlike ukrainian youths, engaged more in patriotic activity in their free time. the study results carried out in ukraine and canada have not confirmed any relationship between gender, personality type, learning school subjects and the level of a person’s civic activity. while some studies have found associations between personality type or personality traits and forms of civic engagement, and there is emerging interest in this area (dinesen, norgaard & klemmensen, 2014). according to molyneux (2017), “television viewers and those who pay attention to breaking news and crime are less civically engaged”. banks (2017) describes the schools’ role in reducing failed citizenship and helping marginalised groups become efficacious and participatory citizens in multicultural nation-states. evans, marsicano, and lennartz (2019) examine the missions, infrastructure, activities, and outcomes related to civic engagement across postsecondary institutional characteristics and conclude that a residential student population is strongly associated with an increased emphasis on civic engagement. in contrast to the canadian study (broom, 2016), we have not revealed any link between religious activity and students’ volunteering. hence, we have concluded that common for both countries are: low percentage of students’ civic and political involvement, which are related to the internal factor beliefs (concerning government, democracy), which have a direct impact on students’ motivation and efficacy to bring change. the findings that have emerged from our study prove ballard’s idea that: “youth have various beliefs, concerns, desires, and interests leading them toward or away from civic involvement. knowing what motivates youth, especially those with varying civic opportunity levels, adds to the understanding of civic development processes and has https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 98 potential practical implications for facilitating civic involvement among youth” (ballard, 2014). our study confirms the theory that “experiences are real learning events through which we build key concepts that structure our thinking. experiences have the potential to be civically empowering” (dewey, 1916) based on canadian (broom, 2016) findings and our research, family political involvement has not been statistically significant. muddiman et al. (2019) theory has contradicted our study results: “parents seem to play a key role in providing a route into civic participation and encouraging our young participants to get involved – even more so than positive experience at school or through friendships with peers”. our survey supports other researchers’ work (bennett, freelon and wells, 2010) according to whom young people are primarily active in the traditional activities of voting and volunteering and this way they become dutiful citizens. conclusions. the research demonstrates the complexity of youth views, multiple factors that influence citizenship attitudes, and do not always coincide with students’ behaviours. the empirical study conducted at canadian and ukrainian universities found that in both countries there has not been an established clear relationship between civic background characteristics such as the studying of school subject “citizenship education”, class, gender, age, temperament, religious activity, and civic engagement of young people. unlike canadian researcher catherine broom, we have not discovered the apparent connection between religious activity and student volunteering or religious activity and civic engagement. nevertheless, the study has confirmed that young people whose parents are politically active citizens are more likely to be socially active. the study confirms that the students’ free time activities in ukraine and canada have had a limited impact on their civic engagement. it appears that youth have been limited in civic engagement in everyday life and spending their free time as they consider civic involvement and political life as they are not relevant to themselves at their ages. students think that they lack experience, knowledge, and cannot change the life of their country and its citizens significantly. these are the main reasons why canadian and ukrainian students have not participated much in civic duties and activities, especially while spending their own free time. the conducted study indicates that ukrainian students have been more active than canadian students in voting. in comparison with canadian students, ukrainian students have been more knowledgeable about their country’s political life, and they have been more likely to discuss politics on social https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 99 media. in contrast, canadian students have been more active in volunteering, boycotting, and protesting, which means they have been more focused on practical problem-solving. we suppose that having life experience related to public or civic affairs only contributes to a better understanding of the community’s needs and civic activity development. in our comparative study, the theory that having personal experience of civic activity will only contribute to the citizen’s active civic engagement has had some practical application, but only under the circumstance that this gained experience is positive. it has been proved that students’ attitudes toward government and democracy have had the most significant impact on students' civic engagement, even greater than studying the subject “civic education” at school. students have firmly believed that civic activities involvement will only contribute to the development of democracy in their country. overall, canadian students have expressed a positive attitude toward government, democracy, and civic engagement. it has been found out that 50% of the surveyed ukrainian students, on the contrary, have had a negative attitude towards the government. ukrainian and canadian students’ views on the main features of an “ideal government” have coincided since they are as follows: justice, honesty, competence, responsibility, patriotism. a distinctive feature is that ukrainian students’ responses demonstrate some hopelessness and despair related to the authorities. most ukrainian students have dreamt of an incorruptible government that will care about its citizens, the government that will consist of competent officials who know their country’s history and will not repeat the mistakes. understanding why youths are not civically active can help teachers provide opportunities for their civic engagement and help them understand that government affairs are relevant to them. teaching the subject “citizenship education” for high school ukrainian students was approved by the ministry of education and science of ukraine (order № 1407) only in 2017. scholars and undergraduate educators need to develop curricula that build upon the ways students currently participate in democracy (nelson, lewis & lei, 2017). an underexplored area that has emerged from this study is how to engage and motivate youth’s civic, political and volunteer involvement. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 100 acknowledgements: the authors are thankful to catherine broom for the permission to use the questionnaire designed by her. author contributions: the article is co-authored by oksana zabolotna (o.z.) and anna pidhaietska (a.p). the author individual contributions are as follows: conceptualization, o.z. and a.p.; methodology, o.z. and a.p.; data collection, a.p.; formal analysis, o.z. and a.p.; writing original draft preparation, a.p.; writing review and editing, o.z.; supervision, o.z. all authors have read and agreed to the published version of the manuscript. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references ballard, p. (2014). what motivates youth civic involvement? journal of adolescent research, 29(4), 439– 463. https://doi.org/10.1177/0743558413520224 banks, j. (2017). failed citizenship and transformative civic education. educational researcher, 46(7), 366–377. https://doi.org/10.3102/0013189x17726741 barrett, m. & brunton-smith, i. (2014). political and civic engagement and participation: towards an integrative perspective. journal of civil society, 10(1), 5-28. https://doi.org/10.1080/17448689.2013.871911 bennett, w., freelon, d., & wells, c. (2010). changing citizen identity and the rise of a participatory media culture. in l. sherrod, j. torney-purta, & c. flanagan (eds.). handbook of research on civic engagement (pp. 393–423). hoboken, nj: john wiley & sons. https://doi.org/10.1002/9780470767603.ch15 broom, c. (2016). exploring youth civic engagement and disengagement in british columbia, canada. journal of international social studies, 6(1), 4–22. https://files.eric.ed.gov/fulltext/ej1149181.pdf cicognani, e., zani, b., fournier, b., gavray, c., & born, m. (2011). gender differences in youths’ political engagement and participation. the role of parents and of adolescents’ social and civic participation. journal of adolescence, xxx. https://doi.org/10.1016/j.adolescence.2011.10.002 cohen, a.k., & chaffee, b.w. (2013). the relationship between adolescents’ civic knowledge, civic attitude, and civic behavior and their self-reported future likelihood of voting. educ citizsh soc justice, 8(1), 43–57. https://doi.org/10.1177/1746197912456339 dewey, j. (1916). democracy and education. new york, ny: the free press. retrieved from https://archive.org/details/democracyeducati1916dewe dinesen, p., norgaard, a., & klemmensen, r. (2014). the civic personality: personality and democratic citizenship. political studies, 62(s1), 134–152. https://doi.org/10.1111/1467-9248.12094 evans, b., marsicano, c., & lennartz, c. (2019). cracks in the bedrock of american democracy: differences in civic engagement across institutions of higher education. educational researcher, 48(1), 31–44. https://doi.org/10.3102/0013189x18809053 hooghe, m., & stiers, d. (2020). political discussion begins at home. household dynamics following the enfranchisement of adolescent children. applied developmental science, pages 1-14. https://doi.org/10.1080/10888691.2020.1712207 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1177/0743558413520224 https://doi.org/10.3102/0013189x17726741 https://doi.org/10.1080/17448689.2013.871911 https://doi.org/10.1002/9780470767603.ch15 https://files.eric.ed.gov/fulltext/ej1149181.pdf https://doi.org/10.1016/j.adolescence.2011.10.002 https://doi.org/10.1177/1746197912456339 https://archive.org/details/democracyeducati1916dewe https://doi.org/10.1111/1467-9248.12094 https://doi.org/10.3102/0013189x18809053 https://doi.org/10.1080/10888691.2020.1712207 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 zabolotna and pidhaietska (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 85-101. https://doi.org/10.4995/muse.2021.14962 101 kyrylovych, o. (2012). teoretychni zasady formuvannia hromadianskoi pozytsii studentiv pedahohichnykh vnz. [theoretical principles of active civil position of pedagogical universities students` formation]. nova predahohichna dumka, 1, 162–164. martínez, i. m., peñalver, j., meneghel, i. (2016). take care of well-being: how facilitators and engagement predict performance of university students. multidisciplinary journal for education, social and technological sciences, 3(1), 100-117. http://dx.doi.org/10.4995/muse.2016.3751 molyneux, l. (2017). multiplatform news consumption and its connections to civic engagement. journalism. https://doi.org/10.1177/1464884917730216 muddiman, e., taylor, c., power, s. & moles, k. (2019).young people, family relationships and civic participation. journal of civil society, 15(1), 82–98. https://doi.org/10.1080/17448689.2018.1550903 nelson, j., lewis, d., & lei, r. (2017). digital democracy in america: a look at civic engagement in an internet age. journalism & mass communication quarterly, 94(1), 318– 334. https://doi.org/10.1177/1077699016681969 schugurensky, d. (ed.), wolhuter, c. (ed.). (2020). global citizenship education in teacher education. new york: routledge, https://doi.org/10.4324/9781351129848 sunil, s., & verma, s. (2018). moral identity and its links to ethical ideology and civic engagement. journal of human values, 24(2), 73–82. https://doi.org/10.1177/0971685818754547 tereshchenko, a. (2010). ukrainian youth and civic engagement: unconventional participation in local spaces. journal of youth studies, 13(5), 597–613. https://doi.org/10.1080/13676261.2010.487518 vasylenko, i. (2009). vykhovannia hromadianskosti studentiv zasobamy samovriaduvannia u vyshchykh navchalnykh zakladakh kanady [bringing citizenship education by means of student self-government in canadian higher education institutions]. phd dissertation, kyiv, ukraine. white, e., & mistry, r. (2016). parent civic beliefs, civic participation, socialisation practices, and child civic engagement. applied developmental science, 20(1), 4460. https://doi.org/10.1080/10888691.2015.1049346 yeung, j.w.k. (2017). religious involvement and participation in volunteering: types, domains and aggregate. voluntas, 28, 110–138. https://doi.org/10.1007/s11266-016-9756-6 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://dx.doi.org/10.4995/muse.2016.3751 https://doi.org/10.1177/1464884917730216 https://doi.org/10.1080/17448689.2018.1550903 https://doi.org/10.1177/1077699016681969 https://doi.org/10.4324/9781351129848 https://doi.org/10.1177/0971685818754547 https://doi.org/10.1080/13676261.2010.487518 https://psycnet.apa.org/doi/10.1080/10888691.2015.1049346 https://doi.org/10.1007/s11266-016-9756-6 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 54 how to teach grammar to adult esp learners at technical university more communicatively: task-based approach i. lytovchenko1* , y. lavrysh1 , v. lukianenko1 , o, ogienko2 1 department of english for specific purposes. national technical university of ukraine “igor sikorsky kyiv polytechnic institute”, 37, peremohy ave., kyiv, 03056, ukraine. 2 department of pedagogy. a. s. makarenko sumy state pedagogical university, 87, romenska st., sumy, 40002, ukraine. correspondence: irinalyt@ukr.net received: 15 october 2019; accepted: 03 march 2020; published: april 2020 abstract the article is devoted to comparison of the effectiveness of grammar acquisition by adult learners in two types of instruction – based on the traditional ppp (presentation, practice, production) approach and tbl (task-based learning) approach – in esp university classes. we conducted a qualitative research in which we tried to incorporate grammar instruction in communicative language teaching and for that purpose selected two groups of students (the ppp group and the tbl group). they were taught five topics, each with a focus on particular grammar items. the comparison of the results we obtained showed that students’ achievement in the tbl group increased more substantially than in the ppp group. also, the majority of students in the tbl group liked learning grammar at the point of need, which we consider particularly important, since it stimulated their motivation and contributed to the acquisition of grammatical structures. when considering the positive aspects of our experience of using tbl and task-based grammar learning in particular, we should first of all point out its learner-centeredness which provides a number of advantages in teaching adult students: the focus on real communication, addressing the students’ needs by putting them into authentic communicative situations, encouragement of students’ responsibility for their own learning, exposure of students to a wide variety of language, including grammar, ability to motivate students. https://doi.org/10.4995/muse.2020.12419 https://orcid.org/0000-0002-8578-3985 https://orcid.org/0000-0001-7713-120x https://orcid.org/0000-0003-3748-2616 https://orcid.org/0000-0002-3089-6288 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 55 keywords: ppp approach; tbl; esp grammar instruction; adult learner; university 1. introduction the on-going changes in business environment and quickly growing demands in professional communication set new challenges before esp learners and teachers, because at present english is a global lingua-franca and predominant language in business worldwide. since a high level of linguistic competence, which is now as important as excellent professional competence of employees, is difficult to achieve without knowledge of grammar, many scientist and practitioners debate on how grammar is best acquired and taught. the issue is even more polemic when adult learners are concerned. as lifelong learning is now a global trend and more and more people study to acquire new qualifications, the mean age of university students is constantly growing. in view of this it is particularly important to understand what makes adults different from children as learners, what learning needs they have, what learning techniques are appropriate for meeting their needs. in our article we aim to compare the effectiveness of grammar acquisition by adult learners in two types of instruction – based on the traditional ppp (presentation, practice, production) approach and tbl (task-based learning) approach – in esp university classes. we consider this study particularly important for a technical university like ours because a considerable number of students who enter it have quite a low level of english language proficiency and the least developed skill is grammar. 2. theoretical background 2.1. adult learner and his/her active role in esp instruction since the professional education of future specialists at university level is shaped by the specific characteristics of adults as learners, it is important to define the term “adulthood” which is essential for the understanding of specifics and needs of an adult learner. one of the earliest and most comprehensive definitions of adulthood belongs to knowles (1980) who viewed it from four https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 56 perspectives – physiological (“achievement of the ability to reproduce” (p. 24)), legal (the age that gives the right to vote, drive a vehicle, drink alcohol beverages, etc.), social (performance of “social roles typically assigned by our culture to those it considers to be adults – the role of worker, spouse, parent, responsible citizen, soldier, and the like” (p. 24)) and psychological (taking responsibility for their own life (p. 24)). it is notable that in adult education he suggested using the last two definitions, because the most distinguishing features of a mature individual are the ability to perform adult roles and self-concept. in the theory of education numerous researches have been dedicated to the study of specifics of an adult learner. as stated by merriam and caffarella (1999), “it is … the nature of adults as learners and the distinguishing characteristics of the adult learning process that differentiate adult education from other kinds of education. to facilitate the process of learning, it is especially important to know who the adult learner is…” (p. xi). six characteristics of adults as learners were formulated by knowles (1990) as andragogical principles of learning and are “inarguably the best known set of principles explaining learning in adulthood” (merriam, 1993, p. 1): 1. adults need to know why learning something is necessary for them and what benefits they will have from it (knowles, 1990, p. 57-58); 2. adults have a self-concept of being independent individuals responsible for their own lives and capable of self-directed learning (knowles, 1990, p. 58); 3. adults have big and varied life and professional experience which is a rich source for learning. hence knowles (1990) makes an emphasis on experiential techniques, group work and peer-helping activities (p. 59); 4. “adults become ready to learn those things they need to know and be able to do in order to cope effectively with their real-life situations” (p. 60); 5. “in contrast to children’s … subject-centered orientation to learning…, adults are lifecentered (or task-centered or problem-centered) in their orientation to learning” (p. 61); 6. adults’ motivation for learning is mainly caused by internal factors (the desire to increase the satisfaction from the job, raise the self-esteem, quality of life, etc.) (p. 63). https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 57 knowles (1990) also shows how the adult learning process can be organized based on these principles and how interactive teaching methods and group work can be used to engage learners and motivate them to be active participants of educational activities (lytovchenko, 2016). as a discipline which is learnt by adult learners, esp is based on the principles of adult learning developed by knowles (1990). this implies that learning should be meaningful to students. their knowledge and experience should be actively used in the process of instruction. in classroom, materials and tools should be used which students may really use in their professional activity. english language skills should be developed simultaneously with content learning (global talent bridge, 2018, p. 3-3). studies show that integrated learning improves interdisciplinary connections, intensifies motivation to study a foreign language, develops cognitive and practical skills as well as increases students’ professional competence (kuzminska et al., 2019). considering that the learner, his/her needs, aims and motives are at the center of the learning process, he/she is not just a passive recipient of information, but an active participant of learning whose interaction and collaboration with the teacher and other learners are the major motivating force of the learning process. the realization of such interaction and collaboration is best provided by the use of interactive learning methods which most fully involve the learners into the learning process. 2.2. tbl and teaching grammar in esp one of communicative methods which most organically fit into esp learning is tbl or tblt (task-based language teaching). it is “an approach based on use of “tasks” as the core unit of planning, instruction, research and assessment in language teaching” (knapp and seidlhofer, 2009, p. 352). nunan (2004) defines a task as “a piece of classroom work that involves learners in comprehending, manipulating, producing or interacting in the target language while their attention is focused on mobilizing their grammatical knowledge in order to express meaning” (p. 4). ellis (ellis, 2009) identifies the following key criteria of a task: 1. the primary focus must be on “meaning”. https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 58 2. there must be a kind of “gap”. 3. learners have to rely on their own linguistic and non-linguistic resources. 4. there is a clear outcome other than the use of language (p. 223). the key criterion from ellis’s (2014) point of view is no 3 since it most vividly makes a distinction between a task and an exercise. when doing an “exercise”, students have to manipulate with a text, e.g. fill in gaps, perform a scripted dialogue, substitute words in sentences. when performing a task students have to create a text in order to achieve the outcome of the task. we can add that tasks are always focused on solving problems which “come in many forms, such as naturally occurring events” (la shun l. carroll, 2017, p. 81) and thus brings in more authenticity to learning. in the context of our research it should be noted that some scientists (e.g. sheen, 2003; swan, 2005) believe that in tblt there is no room for grammar instruction, since they find that in this approach grammar is taught mainly in the form of brief corrective feedback. however, other researchers (e.g. long, 2015; ellis 2015a; nunan, 2004; esfandiari, 2018) emphasize the importance of incorporating grammar instruction in communicative language teaching, especially tblt. we totally agree with their view and think that, as a method which most fully meets the needs of adult learners, tblt can be used not only for teaching listening, reading, speaking and writing skills, but also for grammar instruction. the work on a task creates a communicative context which is necessary for making a focus on grammatical form. while working primarily on content in performing a task, students may also focus their attention on grammar, which will help them solve the problem. long (1992), particularly, claims that focus on form in tblt should arise in the process of interaction and be reactive (made when there is need to solve a communication problem) and brief. however, he does not think that focus on form in tblt should always be implicit and admits that teaching grammar rules explicitly may also be appropriate if needed in particular communicative situations. furthermore long (2015) points out that learning grammar in tblt needs not be always https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 59 incidental. on the contrary, he finds intentional learning useful as well for better language acquisition by students. ellis (2016) does not agree with long (1992) in that the focus on form should occur only as response to a problem. he finds it too narrow and notes that there are occasions in communicative esl classes when the teacher pre-empts a problem, for example, he/she may hint students that they can use a particular grammatical form when performing the task. or sometimes the learners preempt as when they begin to ask questions about linguistic forms. in these cases, focus on form helps to avoid rather than fix a linguistic problem, but it is still clearly problem oriented. he argues that “interactive focus on form, then, can be defined as the pre-emptive or responsive attention to form that occurs during an activity that is primarily meaning focused and that addresses either a communicative or linguistic problem” (ellis, 2016, p. 410). it is notable that emphasizing the importance of incorporating focus on form in communicative language teaching ellis (2015b) does not agree with the view that the teacher should not interfere while students are working on a communicative task. he argues that correcting students’ errors during their communication is highly efficient for drawing students’ attention to form. furthermore ellis (2016) considers explicit corrective feedback more effective, since it is more likely to guarantee a switch of attention to a particular form. in general he views communicative and traditional grammar teaching as complementary rather than opposed approaches to teaching and points out that it is possible to combine them by including planned form-focused lessons into tbl with the aim to address particular linguistic features which are problematic to learners. he also makes an emphasis on the efficiency of task-repetition and argues that when students are asked to repeat the same task, they are better prepared to conceptualize on the content and choose the language necessary to convey it. thus, focus on linguistic form is closer when students perform the task for the second time (ellis 2016). summarizing a number of studies ellis (2015b) concludes that “learners who have received form-focused instruction learn more rapidly and generally advance further along the interlanguage continuum than naturalistic learners” (p. 22). https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 60 3. methods to compare the effectiveness of grammar acquisition by adult learners in the traditional ppp and communicative tbl approaches while learning esp at technical university we conducted a qualitative research, since our aim was not collecting statistical data, but rather making an attempt to share the experience of incorporating grammar instruction in communicative language teaching. 3.1. participants the participants of the study were 32 first year bachelor degree students of institute of mechanical engineering of national technical university of ukraine “igor sikorsky kyiv polytechnic institute” (kyiv, ukraine). they studied esp and had predominantly b1 english proficiency level on cefr scale. the students were divided into two groups – the ppp group (16 persons) and the tbl group (16 persons). 3.2. materials and procedure five topics were studied by both groups, each with a focus on particular grammar items: robotics (present simple (active and passive)), innovations in technology (relative clauses), environmental engineering (past simple), properties of plastics (modal verbs), renewable energy (passive + to; modals + passive). the same learning materials were used in both groups. the ppp group received traditional grammar instruction consisting of explanation of the rules (presentation), controlled oral and written production exercises (practice) and an essay about the sources of energy which are most commonly used in their country, the purposes they are used for, the sources of energy that have good prospects for being used in the future (free production). the focus on grammatical form was explicit. in the tbl group, students’ attention was drawn to grammatical form while they were performing the task and thus primarily focused on meaning. the instruction in this group was learner-centered. the students worked in groups and performed the tasks which were close to those https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 61 they may have to perform in real life. the teacher focused the students’ attention on grammatical structures, explained their meaning and use during the performance of the task (at the stage of analysis), which allowed the students to improve their works and use the target language where appropriate. we present two plans of the task-based grammar lessons conducted in the tbl group which were designed using creative ideas suggested in the mooc course “teaching grammar communicatively” (world learning, 2019). lesson 1. topic: robotics target grammar: present simple (active and passive) stage procedure interaction introduction to topic teacher asks students: what are robots? where are they used at present? t-s then he/she shows pictures of robots and asks students to work in groups and decide what the robots displayed in them could be used for. students compare their answers with others in their group and provide reasons to support their answers. s-s task students listen to a recording about different applications of robots and, in groups, discuss the benefits and dangers of using robots in various spheres of people’s lives. s-s planning students work in small groups and have to describe three most important or extraordinary uses of robots they know about. then, each group prepares a three-minute ted talk about these applications. teacher acts as facilitator and helps students to correct their presentations. s-s report groups give their talks and record themselves on smart phones. students comment on the presentations, ask questions, decide which of the robot uses are most important/ extraordinary and explain why. teacher provides feedback on content and form. s-s analysis language focus: present simple (active and passive). teacher asks students to look at the script of the recording they listened to at the beginning of the lesson and find sentences with the present simple (active and passive) used to describe applications of robots. t-s s-t https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 62 then, teacher writes them on the board, elicits from the students what the present simple means and when it is used and gives explanations. t-s practice in the same groups, the students go over their ted talks, underline the present simple (active and passive) and correct mistakes if needed. then, they report to the rest of the class what changes they have made in their ted talks. s-s evaluation and reflection teacher makes notes of what students say and gives feedback at the end of the lesson. t-s on a piece of paper students write down one thing they liked and one thing they did not like about the lesson. s lesson 2. topic: renewable energy target grammar: expressing purpose, ability stage procedure interaction introduction to topic teacher asks students what they know about the energy crisis on our planet and to brainstorm the problems caused by the use of fossil fuels – coal, oil and gas – for energy production. then, the teacher shows pictures and asks students to identify the sources of renewable energy on them. t-s students work in groups and discuss how each source can be used to replace fossil fuels. s-s task students work in pairs. teacher distributes a text about different sources of energy and asks students to read it. students study the text and, in pairs, discuss the advantages and problems of each, also adding their own ideas. s-s planning students work in small groups and have to decide: • what sources of energy are most commonly used in their country; what purposes they are used for; • what sources of energy have good prospects for being used in their country in the future and why. then, each group prepares a poster illustrating their ideas. teacher acts as facilitator, helps students to correct their reports. s-s report all posters are displayed on a wall. representatives of each group read the information on the posters to the rest of the class. students find similarities and differences. teacher provides feedback on content and form. s-s analysis language focus: expressing purpose; ability. teacher asks students to look at the posters and the text they read at the beginning of the lesson and find grammatical structures which are used to express purpose; ability. t-s s-t https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 63 teacher focuses students’ attention on these structures and explains their meaning and use: purpose: passive + to: the wind is used to turn the blades of a wind mill. ability: modals + passive: solar energy can be stored during day-time and used at night. t-s practice in the same groups students revise their posters including ideas suggested by other groups (up to three ideas) and using the target language (passive + to; modals + passive). then, they report to the rest of the class what changes they have made in their posters. s-s evaluation and reflection teacher makes notes of what students say and gives feedback at the end of the lesson. t-s on small pieces of paper students write down what they learnt in the lesson and what they want to improve. s to compare the results of the study in two groups the students in each of them were asked to do the same test before the first lesson (a pre-test) and after the study of five topics (a post-test). the test was focused on the target grammar of these topics: present simple (active and passive), relative clauses, past simple, modal verbs, passive + to; modals + passive. it consisted of multiple choice and fill in the gaps tasks and was assessed using the following grading scale: a – 100-95; b – 94-85; c – 84-75; d – 74-65; e – 64-60; scores less than 60 were considered as failing scores. in the tbl group we also used an open-ended questionnaire to find out the students’ attitudes to task-based grammar instruction which included three questions: 1. did you like learning grammar at the point of need? why / why not? 2. did you feel it would be appropriate to do additional grammar exercises at the end of each lesson? if yes, what particular grammar items needed further practice? 3. did you have any difficulties learning grammar at the point of need? if yes, what kind of difficulties? https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 64 4. obtained results the results of the pre-test which was conducted before the study did not reveal any significant differences between the groups. after the study, as shown by the post-test, positive dynamics was observed in both groups. however the number of students with high grades – a, b and c – in the tbl group increased more substantially than in the ppp group (by 12.5 % vs 6.25 %; 6.25 % vs 0 % and 18.75 % vs 6.25 %, respectively). also the number of students with low grades – d and e – decreased more substantially in the tbl than in the ppp group (by 18.75 % vs 6.25 % and 12.5 % vs 6.25 %, respectively). the results of both tests are summarized in tables 1 and 2. table 1. results of the pre-test and post-test in ppp group grades ppp group pre-test post-test difference students % students % % a 2 12.50 3 18.75 +6.25 b 3 18.75 3 18.75 0.00 c 5 31.25 6 37.50 +6.25 d 4 25.00 3 18.75 -6.25 e 2 12.50 1 6.25 -6.25 total 16 100 16 100 table 2. results of the pre-test and post-test in tbl group grades tbl group pre-test post-test difference students % students % % a 3 18.75 5 31.25 +12.50 b 2 12.50 3 18.75 +6.25 c 4 25.00 7 43.75 +18.75 d 4 25.00 1 6.25 -18.75 e 3 18.75 1 6.25 -12.50 total 16 100 16 100 the analysis of the students’ answers to the questionnaire in the tbl group showed that almost all of them (15 students (93.75 %)) liked learning grammar at the point of need. the most common reasons were that: https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 65 • they understood how to use these grammar structures to perform the task; • they had clear examples of the use of these structures in the text; • they did not have to learn rules; • they could remember these structures because they used them to perform the task; • they did not have to do grammar exercises; • in the group they could get help from peers. only one student preferred working with a grammar book, because, as he wrote, he wanted to have “a whole picture” of each grammar topic. in answer to the question about the appropriateness of doing additional grammar exercises at the end of the lessons six students (37.50 %) answered that it would be appropriate, because grammar exercises: • give them more practice in the use of the grammar structures; • help them better understand the grammar structures; • help them use the structures more confidently; • help them prepare for tests. four of these students (25 %) thought that grammar exercises were appropriate when studying all grammar items, one student (6.25 %) felt they were needed when studying relative clauses and modal verbs, one student (6.25 %) found modal verbs more difficult than other topics and wanted to do additional grammar exercises to learn them better. when answering the question about the difficulties in learning grammar at the point of need, one student wrote that he needed more time for learning grammar in class. one student had problems with the use of modal verbs and needed more practice. one student said he sometimes was not sure if he used the grammatical structures correctly, but since he worked in a group, he asked his peers and they helped him. https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 66 5. discussion the results of our study give ground to believe that, being an innovative method which helps to make language learning more communicative, tbl provides many possibilities for teaching grammar to adult esp learners at technical university. we conducted a qualitative research in which we tried to incorporate grammar instruction in communicative language teaching and for that purpose selected two groups of students (the ppp group and the tbl group) and taught them five topics, each with a focus on particular grammar items. the comparison of the results we obtained showed that students’ achievement in the tbl group, as demonstrated by the post-test, increased more substantially than in the ppp group. also the majority of students in the tbl group, as can be seen from their answers to the questionnaire, liked learning grammar at the point of need, which we consider particularly important, since it stimulated their motivation and contributed to the acquisition of grammatical structures. another important result of our study showed that more than a third of the students in the tbl group thought it would be appropriate to do grammar exercises in addition to teaching grammar at the point of need as it would give them more practice in the use of grammar structures, help better understand and more confidently use them. we should also point out that as a whole the students did not find it difficult to learn grammar at the point of need while working on a task. all this gives us ground to believe that the results of our study are in line with those obtained in a whole number of other researches (ellis, 2018; long 2015; esfandiari, 2018; herrin, 2009; viriya, 2018; i-chen chen, 2018; lytovchenko, 2009; prabhu, 1999; recatalá, 2016) which suggest that tbl with its focus on problem-solving has evident advantages for both adult and child language learners and is particularly appropriate for teaching grammar. in our opinion, one of the most important advantages of tbl is that it creates the context for learning a language form and thus, as stated by nunan (1998), shows learners “how to use grammar to get things done, socialize, obtain goods and services, and express their personality through language” (p. 103). it should be noted that in our study, the instruction of grammar was both reactive and preemptive. the reactive teaching took place in the form of corrective feedback, mainly recasts, on content and form at the report stage. we found it effective, since it helped the students to improve their performance and acquisition of the grammatical forms. hence we totally agree with studies https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 67 (e.g. li, 2010; ellis, 2016; lyster, saito, & sato, 2013; sheen, 2004) which confirmed the positive impact of corrective feedback on language learning. when teaching one of the topics (“environmental engineering”) feedback was provided by the students and not the teacher. we used peer assessment because it fosters students’ cognitive abilities, enhances their responsibility for the learning outcomes and thus promotes better language learning (lavrysh, 2016). preempting grammar problems took place at the stage of analysis, when the teacher drew the students’ attention to grammatical forms, elicited from students the meaning and use of these forms, gave explanations. in our opinion, both reactive and preemptive focuses on grammatical form should be used as complementing each other. as stated by ellis, basturkmen and loewen (2001), “reactive focus on form addresses a performance problem (which may or may not reflect a competence problem) whereas preemptive focus on form addresses an actual or a perceived gap in the students’ knowledge” (p. 414). in our opinion, it is important that the preemptive focus on form should be explicit and followed by the repetition of the task. in our study the task was repeated at the practice stage, which provided students with the possibility to correct their mistakes, improve their work and draw the whole classes’ attention to the changes they had made. thus the results of our research are in agreement with those of bygate (2001) and sheppard (2006) who believe that there should be an input between the first and second performances of the task to prompt the enhanced focus on form. we also agree with ellis (2016) in that the post-task activities can also be of the focus on form kind, e.g. grammar exercises. in our study the students were not asked to do additional grammar exercises, but more then 1/3 of them, as shown by the survey, thought they would be useful for practicing grammatical structures more thoroughly. when considering the positive aspects of our experience of using tbl and task-based grammar learning in particular, we should first of all point out its learner-centeredness which, as stated by lytovchenko et al. (2018), provides a number of advantages in teaching adult students: the focus on real communication, addressing the students’ needs by putting them into authentic communicative situations, encouragement of students’ responsibility for their own learning, exposure of students to a wide variety of language, including grammar, ability to motivate students. it is particularly important that learning grammar is integrated into learning other skills, since https://doi.org/10.4995/muse.2020.12419 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 68 “when students apply a variety of skills and perspectives, they admit the sense and value of what they are studying” (lavrysh, & lytovchenko, 2019, p, 741). the integrated approach to learning allows acquiring “new knowledge and also new skills that serve the students to face the events of life in a successful way, … so that they can handle the new situations that are presented to them and develop their maximum potential as people” (gómez-ejerique, & lópez-cantos, 2019, p. 48). our research was limited to learning of only five topics but, based on our experience, we believe that tbl is a universal approach and there are no restrictions for its use in the learning of any topic and any grammatical structure in esp. 6. conclusions the comparison of the effectiveness of grammar instruction of adult learners in the traditional ppp and the communicative tbl approach in esp university classes showed that the latter approach provides better learning and is more motivating for adult students. the primary feature of tbl which promotes its efficiency is the creation of the context for learning a grammar form which shows learners how to use it for different purposes in the real-world situations. the combination of both reactive and preemptive grammar instruction provides more comprehensive learning as it addresses both the performance problems and the gap in students’ knowledge. the addition of the stages of explicit grammar teaching and the task repetition to the format of the task prompts the enhanced focus on a grammatical form. the prospects for further research may include the study of the kinds and specifics of tasks in tbl. 7. references bygate, m. (2013). effects of task repetition on the structure and control of oral language. in m. bygate, p. skehan, & m. swain (eds.), researching pedagogic tasks, second language learning, teaching and testing (pp. 23–48). london; new york: routledge. https://doi.org/10.4324/9781315838267 https://doi.org/10.4995/muse.2020.12419 https://doi.org/10.4324/9781315838267 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 69 chen, i-c. (2018). incorporating task-based learning in an extensive reading programme. elt journal, 72(4), 405–414. https://doi.org/10.1093/elt/ccy008. ellis, r. (2009). task-based language teaching: sorting out the misunderstandings. international journal of applied linguistics, 19(3), 221-246. https://doi.org/10.1111/j.14734192.2009.00231.x. ellis, r. (2014). taking the critics to task: the case for task-based teaching. in proceedings of the sixth cls international conference clasic “knowledge, skills and competencies in foreign language education”, singapore, december 4-6, 2014. retrieved may 1, 2019, from https://www.fas.nus.edu.sg/cls/clasic/clasic2014/proceeding.htm. ellis, r. (2015a). understanding second language acquisition. oxford: oxford university press. ellis, r. (2015b). the importance of focus on form in communicative language teaching. eurasian journal of applied linguistics, 1(2), 1–12. https://doi.org/10.32601/ejal.460611. ellis, r. (2016). focus on form: a critical review. language teaching research, 20(3), 405– 428. https://doi.org/10.1177/1362168816628627. ellis, r. (2018). reflections on task-based language teaching. bristol; blue ridge summit, pa: multilingual matters. https://doi.org/10.21832/ellis0131. ellis, r., basturkmen, h. and loewen, s. (2001). preemptive focus on form in the esl classroom. tesol quarterly, 35(3), 407–432. esfandiari, m. (2018). task-based learning applied: satisfactory grammar coverage. international journal of english and cultural studies, 1(1), 68 – 73. https://doi.org/10.11114/ijecs.v1i1.3104. esfandiari, m. (2018). task-based learning applied: satisfactory grammar coverage. international journal of english and cultural studies, 1(1), 68 – 73. https://doi.org/10.11114/ijecs.v1i1.3104. global talent bridge. (2018). building opportunities for skilled immigrants: a toolkit for esl practitioners. retrieved may 12, 2018, from www.globaltalentbridge.org gómez-ejerique, c., & lópez-cantos, f. (2019). application of innovative teaching-learning methodologies in the classroom. coaching, flipped classroom and gamification. a case study of success. multidisciplinary journal for education, social and technological sciences, 6(1), 46 – 70. https://doi.org/10.4995/muse.2019.9959. https://doi.org/10.4995/muse.2020.12419 https://doi.org/10.1093/elt/ccy008 https://doi.org/10.1111/j.1473-4192.2009.00231.x https://doi.org/10.1111/j.1473-4192.2009.00231.x https://www.fas.nus.edu.sg/cls/clasic/clasic2014/proceeding.htm https://doi.org/10.32601/ejal.460611 https://doi.org/10.1177/1362168816628627 https://doi.org/10.21832/ellis0131 https://doi.org/10.11114/ijecs.v1i1.3104 https://doi.org/10.11114/ijecs.v1i1.3104 http://www.globaltalentbridge.org/ https://doi.org/10.4995/muse.2019.9959 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 70 herrin, j.m. (2009). six (6) tasks for task-based learning (tbl). workshop, usa. knapp, k., and seidlhofer, b. (2009). handbook of foreign language communication and learning. gottingen: walter de gruyter. https://doi.org/10.1515/9783110214246 knowles, m. s. (1980). the modern practice of adult education: from pedagogy to andragogy. chicago: associated press follett publishing company. knowles, m. s. (1990). the adult learner: a neglected species. houston, london, paris, zurich, tokyo: gulf publishing company. kuzminska, n., stavytska, i., lukianenko, v. & lygina, o. (2019). application of clil methodology in teaching economic disciplines at university. advanced education, 11, 112–117. https://doi.org/10.20535/2410-8286.167150. la shun l. carroll (2017). theoretical biomimetics: a biological design-driven concept for creative problem-solving as applied to the optimal sequencing of active learning techniques in educational theory. multidisciplinary journal for education, social and technological sciences, 4(2), 80 – 96. https://doi.org/10.4995/muse.2017.7078. lavrysh, y. (2016). peer and self-assessment at esp classes: case study. advanced education, 6, 60 – 68. https://doi.org/10.20535/2410-8286.85351. lavrysh, y. and lytovchenko, i. (2019). the case of education for sustainable development approaches implementation at english language classes at the technical university in ukraine. pedagogy, 91(5), 736 – 749. li, s. (2010). the effectiveness of corrective feedback in sla: a meta-analysis. language learning, 60(2), 309–365. https://doi.org/10.1111/j.1467-9922.2010.00561.x long, m. (2015). second language acquisition and task-based language teaching. malden, ma: wiley blackwell. long, m., and crookes, g. (1992). three approaches to task-based syllabus design. tesol quarterly 26(1): 27–56. https://doi.org/10.2307/3587368. lyster r., kazuya s., and masatoshi s. (2013). oral corrective feedback in second language classrooms. language teaching, 46(1), 1–40. https://doi.org/10.1017/s0261444812000365. lytovchenko, i. (2009). how to make upper-level university classes more interactive. english teaching forum, 47(2), 24-29. lytovchenko, i. (2016). role of adult learning theories in the development of corporate training in the usa. future human image, 3(6), 67–80. https://doi.org/10.4995/muse.2020.12419 https://doi.org/10.1515/9783110214246 https://doi.org/10.20535/2410-8286.167150 https://doi.org/10.4995/muse.2017.7078 https://doi.org/10.20535/2410-8286.85351 https://doi.org/10.1111/j.1467-9922.2010.00561.x https://doi.org/10.2307/3587368 https://doi.org/10.1017/s0261444812000365 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12419 social and technological sciences issn: 2341-2593 lytovchenko et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 54 -71 | 71 lytovchenko, i., ogienko, o., sbruieva, a., and sotska, h. (2018). teaching english for specific purposes to adult learners at university: methods that work. advanced education, 10, 69 – 75. https://doi.org/10.20535/2410-8286.149741. merriam, s. b. (1993). an update on adult learning theory. san francisco: jossey-bass publishers. merriam, s. b., and caffarella, r. s. (1999). learning in adulthood: a comprehensive guide. san francisco: jossey-bass publishers. nunan, d. (1998). teaching grammar in context. elt journal, 52(2), 101–109. https://doi.org/10.1093/elt/52.2.101 nunan, d. (2004). task-based language teaching: a comprehensively revised edition of designing tasks for the communicative classroom. cambridge: cambridge university press. https://doi.org/10.1017/cbo9780511667336 prabhu, n. s. (1999). second language pedagogy. oxford: oxford university press. recatalá, d. (2016). using active learning methodologies in physical chemistry in clil contexts. multidisciplinary journal for education, social and technological sciences, 3(1), 71 – 83. https://doi.org/10.4995/muse.2016.3696 sheen, r. (2003). focus on form – a myth in the making? elt journal, 57(3), 225–233. https://doi.org/10.1093/elt/57.3.225. sheen, y.h. (2004). corrective feedback and learner uptake in communicative classrooms across instructional settings, language teaching research, 8(3), 263–300. https://doi.org/10.1191/1362168804lr146oa sheppard, c. (2006). the effects of instruction directed at the gaps second language learners noticed in their oral production. unpublished phd thesis, university of auckland, auckland, new zealand. swan, m. (2005). legislation by hypothesis: the case of task-based instruction. applied linguistics, 26(3), 376–401. https://doi.org/10.1093/applin/ami013 viriya, c. (2018). using task-based learning with students of academic english. arab world english journal, 9(4), 337–346. https://dx.doi.org/10.24093/awej/vol9no4.25. world learning. (2019). module 4 summary. in “teaching grammar communicatively” [mooc]. https://doi.org/10.4995/muse.2020.12419 https://doi.org/10.20535/2410-8286.149741 https://doi.org/10.1093/elt/52.2.101 https://doi.org/10.1017/cbo9780511667336 https://doi.org/10.4995/muse.2016.3696 https://doi.org/10.1093/elt/57.3.225 https://doi.org/10.1191/1362168804lr146oa https://doi.org/10.1093/applin/ami013 https://dx.doi.org/10.24093/awej/vol9no4.25 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 51 effects of business combinations on the competitive environment moisescu florentina, university "dunarea de jos" of galati florym2003@yahoo.com gоlоmоz аnа-mаriа, university "dunarea de jos" of galati golomoz_ana@yahoo.com received: 2018-01-18; accepted: 2018-05-28 abstract this paper deals with the results of the businesses combinations and the advantages felt in solving the problems of certain entities, the extension on other markets or obtaining increased quotas on the market. also, businesses combinations can generate disadvantages regarding the access to credits or negative effects on the salary. a business combination is based on taking risks an has advantages as well as disadvantages, on both the short and the long term, while the decision of making it has in view the development strategy of the organisation. the aim of the paper is to analyze and balance the benefits and disadvantages of business combinations. it is questioned the need to use this method and its use only for the advantages obtained despite the existing disadvantages. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 52 1. introduction the economic progress, the competitive environment, the economic and financial problems determine the entities to identify the best solutions in order to cope with conditions, often difficult, in which they unfold their activity. among the strategies to which the managers often resort, having effect the development of the entity they manage or at least its survival on the dynamic market, is the restructuring strategy, an important part of these being represented by the entities combinations. international financial reporting standards use the term of the business combination, which is defined as being a merger of separate entities into о single enterprise reporting (iasplus 2017). the accounting treatment of business combinations from the perspective of the international financial reporting standards was an issue much debated, polemics continuing even after the changes made in the content of the ifrs 3 "business combinations" (elimination of the method of interests pooling). even if the acquisition method remained the only recognized method by international standards, its practical application has undergone changes over time, focusing on the identification and evaluation of assets, liabilities, contingent liabilities, goodwill, the subsequent treatment of these elements and information that are required to be presented in the financial statements. another important issue that creates accounting controversy is resorting to the depreciation of the value of goodwill, as a result of the completion of the test of compliance with the standard ias 36 impairment of assets. some entities might be tempted to use the goodwill as a tool of "embellishing" the image of the performance, they impairing the asset in question whenever it would be in the advantage of the management or would improve the image towards third parties. as a result, systematic amortization has the advantage of а not being practiced in relationship with the size of the performance of the buyer (raffournier, 2005, р.476). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 53 according to ifrs 3 a business combination represents the grouping of several separate entities in a single reporting entity (iasplus 2017). the following transactions meet the definition of a group of entities: • the acquisition of all the assets and liabilities of an entity • the acquisition of assets, liabilities and rights to the activities of an entity that meet the definition of a business • the establishment of a new legal entity which will take over the assets, liabilities and business of the grouped entities a business combination may be structured in different ways for legal, tax or other reasons. the transaction can take place between an entity and the shareholders of the other or between shareholders of the resulted entity. the counterparty can be represented by the actions of the buyer, money or other assets, taking over debts (grigoroi, 2013, р.158). it can be inferred from here that in the scope of action of ifrs 3 enters the merger by absorption, merger by acquisition, partial contribution of assets under certain conditions. all business combinations entering into the scope of action of ifrs 3 are managed according to the acquisition method. a transaction may be accounted for according to ifrs 3 if the purchased items meet the definition of a business. a business is an integrated set of assets and activities managed to ensure a gain, lower costs or other economic benefits to the participants. in the scope of this standard, the following do not enter (iasplus 2017): • business combinations in which individual enterprises or entities are brought together in order to form joint ventures; • business combinations between entities or businesses under common control; • business combinations which comprise two or more mutual entities; multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 54 • business combinations in which the entities or individual businesses are grouped into one reporting entity only on the basis of a contract, without obtaining equity interest. an entity may apply ifrs 3 to manage groups excluded from the sphere of action of the standard or to use another policy following the provisions of the іаѕ 8. the competition is one of the most important factors that affect the competitive strategies of organisations. in order to develop strategies, we must analyze both the existing competition on the market, and the new companies entering the market. strategies need to be adjusted permanently taking into account the dynamics of the competition as the competition changes its strategies according to market trends and requirements. organizations need to plan their strategies taking into account possible reactions from the competition, such as business combinations that would change considerately dynamics of the market and of the competitive environment (markgraf, 2017). a business combination often means obtaining a larger share of the market, which, in many cases, puts the competition in a weaker position by obtaining a dominant position on the market. groups of businesses can lead to obtaining economies of scale, which means lower unit costs, resulting in greater profits (wiley, 2012). a business combination, in order to be profitable, must abide by the following three rules (gomes-casseres, 2015): 1) grouping of the competition on the market, the existence of entry barriers on the market, obtaining an important share which would represent a monopoly, the existence of similar products, the innovation within a certain market, as well as whether the benefits of combination overpass the negative impact on competition (mehta, 2012, p.97) must have the potential to create more value than each separate business would create. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 55 2) the grouping must be structured and conducted in a correct manner to create the desired value, and to achieve the proposed objectives. 3) the profit made by each entity of the group must motivate the parties to participate in the business and to achieve the objectives. in order to establish whether a business combination would affect competition, different factors are taken into account such as the level of competition on the market, existence of entry barriers on the market, obtaining an important share which would represent a monopoly, the existence of similar products, the innovation within a certain market, as well as whether the benefits of combining overpass the negative impact on competition (mehta, 2012, p.97). the purpose of this paper is to analyze and compare the advantages and disadvantages of business combinations, the impact of the combination on the acquiring entity and if this method is required or is used for the advantages obtained, in spite of the existing disadvantages. 2. managing business combinations by acquisition method managing business combinations, which fall within the scope of the standard ifrs 3, is carried out on the basis of the acquisition method and the use of fair value. all business combinations, except in rare cases of international standard exclusion from the field of use, must be handled according to the acquisition method (iasplus 2017). ifrs 3 standard prohibits the method of pooling interests for both the combinations which enter into its field of application, as well as for groupings between mutual entities and those based on the single contract, without making financial stakes (feleagă & feleagă, 2006, p.54). the acquisition method is applied and in the case of true mergers or mergers of equals. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 56 the acquisition method is at the basis of managing to take control through partial provision of assets or mergers since the operation affects one or more activities, such as those defined by the ifrs 3 standard, and the method is not specifically excluded from its scope of application. the acquisition method involves the following stages: 1. identifying the buyer; 2. determining the соѕt of the group; 3. the allocation of the cost of the group between the assets and liabilities acquired (including contingent liabilities) at the date of the group. figure 1: reсognizing assets / liabilities in a business соmbinаtion the assets disposed and liabilities undertaken by the buyer are assessed at fair value. if the transaction is carried out through the issue of shares, they are assessed at the market price at the date of exchange. the acquirer assesses the acquired identifiable assets and liabilities assumed at their fair values at the date of acquisition, as shown in figure 2. recognition •must match the definition of assets/liabilities at the date of acquisition •must be changed as part of the acquisition classificatio n and designation •it is made at the purchase date, regardless of the classification made by the entity acquired •exception: in case of leasing contracts and insurance contracts purchasedexceptions from the recognition principle •contingent liabilities exceptions from the principle of recognition and evaluation •income tax •employee benefits •compensation assets multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 57 figure 2. assessing at fair value in a business combination direct costs related to acquisition include fees paid to accountants, assessors, legal advisors. administrative costs and other costs that have no direct connection with the acquisition are period costs. the costs of issue of shares are accounted for according to іаѕ 32 (they are deducted from shareholders ' equity), and the costs related to the issuance of debt instruments affects the value of the debt (according to іаѕ 29). the contractual provisions, in specific situations, suppose an additional counterperformance based on certain subsequent events. its value influences the cost of acquisition at the date of the combination if it is likely to be supported and if it can be assessed reliably and requires recognition of a liability assessed at fair value for the contingent counterperformance. if subsequent events no longer exist, or the value of the counterperformance is amended, the purchase cost will be adjusted. the allocation of the acquisition cost of the acquired assets and liabilities. the buyer will allocate, at the acquisition date, the cost of the combination over the fair values of assets, liabilities and identifiable contingent liabilities. exception to the rule of the evaluation at fair value, are the assets owned in view of transfer under ifrs 5. fair value is the amount at which an asset can be traded or a liability settled between the parties concerned and knowledgeable in a transaction conducted under objective conditions. market approach income-based approach cost-based approach exceptions fromt the evaluation principle reacquired rights share-based payment transactions assets held for sale multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 58 the elements that require to be accounted for separately from goodwill are assets, debts and liabilities, if any, of the acquired company that meet certain conditions. the general conditions for the identification of assets and liabilities of the acquired company are (iasplus 2017): (і) it is probably that future economic benefits relating to the asset to provide benefits to the buyer or the latter should pay the outflows of resources required for obligations settlement; (іі) there is a reliable assessment of fair value. in case of an intangible asset or of any liability, only the condition of reliable assessment of fair value must be fulfilled. the buying entity shall recognize in the moment of the combination a contingent liability of the bought entity if its fair value can be determined reliably. after initial recognition, the buyer will assess the contingent liability to the maximum between the amount that would be recognised in accordance with іаѕ 37 and the amount initially recognised, less cumulative amortisation recognised according to the іаѕ 18. ifrs 3 prohibits the buyer to recognize at the date of the grouping a debt for future losses and other costs estimated to be incurred. a restructuring plan of the purchased company conditioned by the grouping is not, before grouping, an obligation of the purchased entity and neither its contingent liability. there is a period of one year during which the assets and liabilities recognised at the acquisition date can be derecognised on the basis of information relating to the circumstances existing at the time of purchase. any adjustments affect the goodwill. 3. the recognition and assessment of goodwill in a combination of enterprises according to ifrs 3, goodwill is defined as being "the unidentifiable part of the payment made by the buyer with the occasion of a cluster of enterprises" (feleaga & feleaga, 2006, р.54). goodwill is included in the consolidated balance sheet at an amount equal to multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 59 the difference between the acquisition cost and the share belongs to the acquirer in the fair value of assets, liabilities and future identifiable acquired liabilities (charles, stanley & clark, 2004). the international body of normalization (iasb) considers that the goodwill is indicated to be stated fully at assets as it is constituted, mainly, from the nucleus of the goodwill, and its recognition as asset is recommended than its accounting as expenditures (yuan, herve & jacques, 2007). according to the romanian legislation, о.m.f.р. no. 1802/2014 specifies that goodwill "recognized in a business combination is an asset representing the future economic benefits arising from other assets acquired in a business combination that are not individually identified and recognised separately"(nafa, 2014, р.31). in the case of a business combination, the buyer who purchases the assets, takes over the goodwill created by the salesman internally which can be defined as supervalue which helps and influences the development of a profitable business. however, ias 38 "intangible assets" prohibits the recognition of goodwill created internally. goodwill is distinguished from intangible assets by the fact that the latter may be transfered, sold, changed, rented in exchange for future economic benefits which can be distributed, but without affecting the other assets of the same activity. positive goodwill occurs as a result of a positive difference between the two, but a negative difference leads to a negative goodwill. goodwill does not create cash flows independently from other assets or groups of assets, but contributes to generating cash flows through its allocation to each cash generating unit of the buyer, from the date of the purchase. after initial recognition, the purchaser shall evaluate the goodwill as a cost of which the losses from cumulative write-downs are deducted. after the initial accounting, according to ifrs 3, the positive goodwill is not depreciated. however, it is subject of an impairment test before the end of the acquisition multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 60 exercise and subsequently annual tests or more frequent, in case there are clues regarding the loss of value. ias 36, 'impairment of assets', stipulates that the loss associated with the depreciation is the positive result of the difference between the carrying amount and the recoverable amount. the recoverable amount is regarded as the minimum between the net fair value and value in use (mititelu, caraconcea & dinu, 2014, p.5). 4. determining the acquisition cost of an entity in order to illustrate the calculation of the acquisition cost, we assume that the company x purchased 75% of the shares of firm y at a cost of 60,000 u.m. at the date of acquisition, the balance sheet of the companies x and y, is as follows: elements company x company y -historical values company y -fair values intangible assets 10.800 15.200 29.200 tangible assets 20.000 7.400 7.000 other fixed assets 100.000 4.000 3.800 stocks 33.000 27.800 claims commercial 10.000 30.000 30.000 receivables deferred tax 2.800 available 18.000 10.000 10.000 total assets 158.800 99.600 110.600 equity 80.000 58.000 64.000 provisions for risks and charges 18.800 1.000 6.600 debt deferred tax 600 other liabilities 60.000 40.000 40.000 total equity 158.800 99.600 110.600 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 61 for this acquisition, the company x had the following expenses: fees for auditing financial situations of the company y8.250 u.m. legal fees for acquisition – 7.500 u.m. fees for strategic advisors – 10.000 u.m. the acquisition cost of the company y: element vаlue (u.m.) the acquisition price 60.000 the fee for the auditor 8.250 legal fees 7.500 fees for strategic advisors 10.000 tоtаl 85.750 at the date of acquisition, the buyer will allocate the cost of the group over the fair values of assets, liabilities and contingent liabilities. ifrs 3 prohibits the buyer to recognize at the date of the grouping a debt for future losses and other costs estimated to be incurred. as a result, the net accounting asset at the acquisition date, assessed at fair value, is: total value of assets 110.600 provisions for risks and charges 6.600 other liabilities 40.000 64.000 the accounting treatment of the difference resulting from the acquisition goodwill is determined as the difference between the acquisition cost of securities and the share that belongs to the purchaser in the fair value of assets, debts and liabilities eventually purchased, as thus: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 62 company x paid 85.750 u.m. to get 75% of the net accounting asset of the company y, which is : 75% x 64.000 = 48.000 u.m. the difference between the acquisition cost and the share in the fair value of net asset value of the company y (85.750 – 75% x 64.000 = 37.750 u.m.) represents a goodwill and is registered in the asset consolidated balance sheet of the group x. the part of the mother-company is eliminated in counterparty with the titles y owned by company x. the acquisition of the company y is found in the consolidated balance sheet of the group x as such: elements company x company y consolidated balance sheet after the grouping operation goodwill 37.750 intangible assets 10.800 29.200 40.000 tangible assets 20.000 7.000 27.000 other fixed assets 100.000 3.800 18.050 stocks 27.800 27.800 claims commercial 10.000 30.000 40.000 receivables deferred tax 2.800 2.800 available 18.000 10.000 28.000 total assets 158.800 110.600 221.400 equity 80.000 64.000 80.000 minority interests 16.000 provisions for risks and charges 18.800 6.600 25.400 other liabilities 60.000 40.000 100.000 total equity 158.800 110.600 221.400 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 63 the success of the business combination can be analysed either by financial indicators that are found in the accounting (financial indicators, evolution of results, costs) or by the indicators used in marketing (client portfolio, sales evolution, market share, etc.) (hramei, 2013). the advantages which have been analyzed throughout the paper, as well as strengthening the market position, achieving higher market shares or expanding on other markets, cannot be analyzed through the financial indicators from the consolidated balance sheet, but by the indicators used in marketing. the balance sheet presented is useful in the event of contracting a credit. 5. discussions in the current economic situation, organizations adopt different strategies in order to maximise profit, but also to strengthen the market position. thus, one of the methods by which organisations can expand on other markets or to obtain larger quotas, is represented by the business combination by the acquisition method. one of the benefits in the case of the business combination is obtaining higher quotas on the market. but this has certain limitations, as the competition council investigates any purchases that would put in danger the competitive environment and would lead to the obtaining of a monopoly within a particular market. for example, in march 2018, the competition council has started an analysis of the acquisition of cemacon s.a and cemacon real estate s.r.l by dedeman s.r.l and fip industrial s.r.l. to determine the compatibility of the acquisition with the normal competitive environment (the competition council, 2018). the acquisition of an enterprise consolidates the position of the buyer on the market, by obtaining higher market shares, but also through the expansion on other markets. an example is the acquisition of the online store fashiondays by emag in 2015. according to data from the ministry of finance, the turnover of emag in 2015 was 1.7 billion lei, and in 2016 after the acquisition of fashiondays, the turnover was of 2.47 billion lei (ministry of public finances, 2017). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 64 european legislation, as well as article 101, 102 and 103 of the treaty on the functioning of the european union, provides strict rules on unfair competition and abusive practices regarding the competition. in the case of a business combination, one of the disadvantages is the more difficult access to bank loans and auctions, because on the short term, there is not a cumulative turnover and a common history. however, this disadvantage can turn into long-term advantage as the turnover will increase and the history of the joint venture will generate trust. from the data presented above about the emag, you can see the increase in the turnover of emag, which strengthens both the market position, but also can help the company in the future in the event of contracting a credit higher than the period before the acquisition. but on the short term it is possible that the organization should not have access to larger loans due to the uncertainty in the result of the combination. another disadvantage, as a result of business combination carried out according to ifrs 3, is represented by the salary conditions, as the entity that buys must recognize and assess the salary benefits in accordance with ias 19. according to ifrs 3, all assets and liabilities following an acquisition of enterprises are recognised at fair value, but one of the exceptions is the remuneration packages which can be superior to those which are already offered to employees of the buying company. for example, in china a study has been carried out in the market of acquisitions and mergers, which showed that companies in the media business consider the merger as an opportunity to enhance competitive advantage, thus taking over some assets that would help a development at the productive and technical level, through cost reduction (deng, 2010). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 65 figure 3: the number of transactions of mergers and acquisitions in romania between 2011 and 2017 (statista, 2018) analyzing the data in figure 3, it is noted that the number of transactions of merger and acquisitions has an upward trend from 2015 to 2017, which indicates that although the business combinations have certain disadvantages, they are superseded by the advantages which determine the companies to do the acquisition of other companies or of certain assets. the grouping resulting from business combinations influence the competitive environment, as it increases the market share, it is possible to decrease the cost of production and its positioning in a posture in which it can influence prices or demand and supply. the competition may respond by changing strategies, either by reducing prices, more aggressive marketing, or finding ways so that their market share is improved. a combination of enterprises is analyzed and may be prohibited by the competent authorities, such as the competition council, if determined the competing companies to leave the market or to put them in the obvious drawback to the new group. it can be concluded that although the groupings of businesses can influence considerately the competitive environment, european legislation analyzes, in order to prevent obtaining unfair competitive advantages, or monopolies. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 66 at the international level, mergers and acquisitions have had an increasing global dynamic. in the year 1990, the volume of international acquisitions and mergers was $151billion, reaching in 2007 to a maximum of $1.700 billion. acquisitions and mergers at the global level have registered a decrease in 2013 as compared to 2012, from $2.29 trillion to $2.22 trillion. acquisitions and mergers in the technology sector, media and telecommunications (tmt) have been on the first place in 2013, the sector encouraged significant transactions such as the verizon/vodafone, liberty global/virgin media and vodafone/kabel germany (hen, 2014, p.238). the total value of acquisitions and mergers in 2017, was $4,740.97 billion, the sectors with the highest values being energy, mining and utilities. most transactions were conducted in the us, in total value of $1,460 billion, while in europe, the value of acquisitions and mergers was € 93.3 million (statista, 2017). figure 4: global value of mergers and acquisitions between 2012 and 2017 (statista, 2017) in romania, in terms of business combinations, according to o.m.f.p. 1802/2014, on individual and consolidated financial statements, the legislation stipulated that the acquisition process submit the final result by adding up each asset and debt account respectively. the law 571/2003 regarding the fiscal code with subsequent modifications and completions, stipulates that the acquisition to be highlighted by a document of multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 67 transfer which contains information on companies data, the address and the value of the acquisition, and the assets to be purchased, to be assessed individually, at fair value. in terms of the companies that apply the o.m.f.p. 2844/2016, regarding the accounting regulations compliant with international financial reporting standards, they must have in view the recognition of deferred taxes represented by the differences which arise between the book values and tax amounts of assets and liabilities of the branch of activity, provided for by ias 12 income tax and accounting for positive or negative goodwill according to ifrs 3 business combinations. in the uk, accounting for business combinations, is done according to the standard of financial reporting frs 102 (financial reporting standards), as well as according to ifrs 3. although frs 102 is based on ifrs 3, the difference between the two consists in the fact that the deferred tax will be recognised at fair value adjustments for assets and liabilities, but the goodwill is calculated as the difference between the acquisition cost and the fair value of the assets and liabilities acquired. the cma (competition and market authority) in the uk, carries out an activity similar to that of the competition council, by which the it investigates acquisitions or mergers which would affect competition through the possession of the monopoly. article 101 of the treaty on the functioning of the european union, provides also exceptions on the cancellation business combinations which could affect the competitive environment, even if the group holds at least 25% of the market share, or has a turnover above 10 million euro. these exceptions apply when the grouping is aimed at the development of technologies for the benefit of consumers, however, without totally limiting the competition, when there are entry barriers on the market, or when the consumer benefits from the advantages from the result of combining (official journal of the european union, 2016, p.88). for example, the grouping of the two pharmaceutical companies to get a particular medicine which would be of benefit to consumers would represent an exception even if the market share of the two would be greater than 25%. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 68 romania uses the continental accounting system, while the uk uses the anglo-saxon accounting system. the difference between them lies in the fact that in romania the accounting system is influenced by the legal form of transactions and tax regulations, relying on the principle of 'prudence', while the anglo-saxon accounting system is based on the principle of 'true and fair view', drawing up transparent financial statements necessary to potential investors or other users. 6. conclusions according to ifrs 3, the acquisition method is the only accepted method for accounting for business combinations. this method involves the identification of the buyer, the evaluation at fair value of the assets and liabilities of the entity acquired at the time of purchase, regardless of the method of remuneration of the acquisition (shares or cash) and the legal form of combination. the iasb approves the fact that most business combinations are represented by purchases, but do not deny the probability of the existence of “real” interest unions in which the buyer cannot be identified, although he did not have success in finding some objective criteria for defining them. the acquisition method requires identification of the buyer after evaluation of the circumstances related to the transaction, a decision which has a significant effect on the financial statements prepared after the combination. according to ifrs 3, all intangible assets identifiable of the entity acquired are distinctly recognized and assessed at fair value. thus, the mode of their assessment affects the size of the goodwill and of the possible expenses in the future with the amortisation or depreciation. intangible assets with indefinite life are attractive, because they are not amortized, but are subjected to an annual test for impairment. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 69 through the use of the ifrs 3, it is used a international financial reporting common language which allows users to more easily understand the necessary information, so that the interpretation of the accounting to be relevant and consistent. taking into consideration the information presented, business combinations presents both advantages and disadvantages to the acquirer, but also has an influence on the competitive environment which must be taken into account. although studies and statistics on this subject have not reached a final result, each organization may be influenced more or less by the business combinations depending on the number of competitors, the market in which it operates the success of the group, the type of products, as well as others. business combinations affect in some extent the competitive environment, but not to exceed certain limits, the eu legislation prohibits and restricts any purchases which would jeopardize the competitive environment by obtaining the monopoly, or the practice of abusive strategies. analyzing the data presented, it can be seen that the advantages obtained from business combinations far outweigh the disadvantages, this fact being supported by the increasing number of acquisitions in recent years. reference nafa. (2014). оmfр 1802/2014 – reglementările соntаbile рrivind ѕituаţiile finаnсiаre аnuаle individuаle şi соnѕоlidаte. available at: httрѕ://ѕtаtiс.аnаf.rо/ѕtаtiс/10/аnаf/legiѕlаtie/оmfр_1802_2014.рdf. аcсessed: 10 december 2017. сhаnсe, с. (2014) оur inѕightѕ intо m&а trendѕ 2014. glоbаl dynаmiсѕ. available at: httр://glоbаlmаndаtооlkit.сliffоrdсhаnсe.соm/dоwnlоаdѕ/сс-mаndа-trendѕ-july2014.рdf. ассeѕsed: 12 december 2017. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 70 сhаrleѕ, e., ѕtаnley, j. & сlаrk, j. (2004). big bаth eаrningѕ mаnаgement: the саѕe оf gооdwill imраirment under ѕfаѕ nо. 142 – jоurnаl оf аррlied buѕineѕѕ reѕeаrсh. available at: httр://www.jоurnаlѕ.сluteоnline.соm/index.рhр/jаbr/аrtiсle/view/2206/2183. ассeѕsed: 15 december 2017. the competition council. (2018). consiliul concurenţei analizează tranzacţia prin care dedeman preia cemacon. available at: http://www.consiliulconcurentei.ro/uploads/docs/items/bucket13/id13147/analiza_ded eman_cemacon_mar_2018.pdf. accessed: 12 april 2018. deng, p. (2010). what determines performance of cross-border m&as by chinese companies? an absorptive capacity perspective. thunderbird international business review 52 (6), 509-524. feleаgă, n. & feleаgă, l. (2006). соntrоverѕe рrivind роlitiсile de evаluаre, deрreсiere şi соntаbilizаre а fоndului соmerсiаleсоnоmie teоretiсă şi арliсаtă. available at: httр://ѕtоre.eсtар.rо/аrtiсоle/111.рdf. ассeѕsed: 8 december 2017. găină, а.m. (2012) glоbаlizаreа și сrizele finаnсiаre, buсharest: ed. рrоuniverѕitаriа. găină, а.m. (2014) . internаtiоnаl mergerѕ аnd асquiѕitiоnѕ under the finаnсiаl mаrketѕ glоbаlizаtiоn. available at: httр://uрm.rо/ссi3/ссi-03/eсо/eсо%2003%2023.рdf. ассeѕsed: 10 december 2017. gomes-casseres, b. (2015). making mergers, acquisitions, and other business combinations work – business harvard review. available at: https://hbr.org/2015/08/making-mergersacquisitions-and-other-business-combinations-work. accessed: 12 april 2018. grigоrоi, l. (2013). орţiuni şi роlitiсi соntаbile рrivind соmbinările de întreрrinderi. аѕem, 11th edition. no.1 / 2013. grоѕu, v. & hоrgа, р. (2009) соnѕiderаtiоnѕ regаrding ifrѕ 3 buѕineѕѕ соmbinаtiоn. available at: httрѕ://ѕѕrn.соm/аbѕtrасt=1325367. ассeѕsed: 05 deсember 2017. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 71 hromei, a.s. (2013). motive de realizare a fuziunii societăţilor comerciale şi impactul acestor operaţuni. management intercultural journal. vol. xv. nr.1 (27), 41-47. iаѕрluѕ. (2017) ifrѕ 3buѕineѕѕ соmbinаtiоnѕ. available at: httрѕ://www.iаѕрluѕ.соm/en/ѕtаndаrdѕ/ifrѕ/ifrѕ3. ассeѕsed: 02 deсember 2017. markgraf, b. (2017). factors that affect a firm’s competitive strategy. available at: http://smallbusiness.chron.com/factors-affect-firms-competitive-strategy-70012.html. accessed: 12 april 2018. mehta, p.s. (2012). evolution of competition laws and their enforcement: a political economy perspective. london:routledge. ministry of public finances. (2017). agentul economic cu codul unic de identificare 14399840. available at: http://www.mfinante.gov.ro/infocodfiscal.html. accessed: 12 april 2018. mititelu, c.f., caraconcea, e.c. & dinu, m.i. (2014). deprecierea fondului comercial – posibilitati si implicaţii în manipularea rezultatelor. available at: http://www.conta.ase.ro/media/default/pdfs/ses2014/pdfuri/1%20deprecierea%20fondului%20comercial.pdf. accessed: 03 may 2018. official journal of european union. (2016). consolidated versions of the treaty on european union and the treaty on the functioning of the european union. available at: https://www.ecb.europa.eu/ecb/legal/pdf/oj_c_2016_202_full_en_txt.pdf. accessed: 12 april 2018. rаffоurnier, b. (2005). leѕ nоrmeѕ соmрtаbleѕ internаtiоnаleѕ, 2e éditiоn, раriѕ: eсоnоmiса. рр. 476-477. statista. (2017). value of mergers and acquisitions (m&a) worldwide from 2012 to 2017 (in billion u.s. dollars). available at: https://www.statista.com/statistics/267369/volume-of-mergers-and-acquisitionsworldwide/. accessed: 12 march 2018. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10164 social and technological sciences eissn: 2341-2593 moisescu and golomoz (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 51-72 | 72 statista. (2018). number of merger and acquisition (m&a) deals in romania from 2011 to 2017. available at: https://www.statista.com/statistics/414971/m-and-a-romanianumber-of-deals/. accessed: 04 may 2018. wiley. (2012). introduction to business combinations. available at: https://www.wiley.com/college/bline/0471327751/samplechapter/ch01.pdf. accessed: 13 april 2018. wiley. (2014). ассоunting fоr buѕineѕѕ соmbinаtiоnѕ. available at: httрѕ://www.wiley.соm/соllege/jeter/0471218529/ѕаmрleсhарter/сh02.рdf. ассeѕsed: 02 deсember 2017. yuаn, d., herve, ѕ. & jасqueѕ, r. (2007). tоwаrdѕ аn underѕtаnding оf the рhаѕeѕ оf gооdwill ассоunting in fоur weѕtern сарitаliѕt соuntrieѕ: frоm ѕtаkehоlder mоdel tо ѕhаrehоlder.. available at: httр://ideаѕ.reрeс.оrg/р/ebg/heссаh/0872.html#рrоvider. ассeѕsed: 12 december 2017. << /ascii85encodepages false /allowtransparency false /autopositionepsfiles true /autorotatepages /none /binding /left /calgrayprofile (dot gain 20%) /calrgbprofile (srgb iec61966-2.1) /calcmykprofile (u.s. web coated \050swop\051 v2) /srgbprofile (srgb iec61966-2.1) /cannotembedfontpolicy /error /compatibilitylevel 1.4 /compressobjects /tags /compresspages true /convertimagestoindexed true /passthroughjpegimages true /createjobticket false /defaultrenderingintent /default /detectblends true /detectcurves 0.0000 /colorconversionstrategy /cmyk /dothumbnails false /embedallfonts true /embedopentype false /parseiccprofilesincomments true /embedjoboptions true /dscreportinglevel 0 /emitdscwarnings false /endpage -1 /imagememory 1048576 /lockdistillerparams false /maxsubsetpct 100 /optimize true /opm 1 /parsedsccomments true /parsedsccommentsfordocinfo true /preservecopypage true /preservedicmykvalues true /preserveepsinfo true /preserveflatness true /preservehalftoneinfo false /preserveopicomments true /preserveoverprintsettings true /startpage 1 /subsetfonts true /transferfunctioninfo /apply /ucrandbginfo /preserve /useprologue false /colorsettingsfile () /alwaysembed [ true ] /neverembed [ true ] /antialiascolorimages false /cropcolorimages true /colorimageminresolution 300 /colorimageminresolutionpolicy /ok /downsamplecolorimages true /colorimagedownsampletype /bicubic /colorimageresolution 300 /colorimagedepth -1 /colorimagemindownsampledepth 1 /colorimagedownsamplethreshold 1.50000 /encodecolorimages true /colorimagefilter /dctencode /autofiltercolorimages true /colorimageautofilterstrategy /jpeg /coloracsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /colorimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000coloracsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000colorimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasgrayimages false /cropgrayimages true /grayimageminresolution 300 /grayimageminresolutionpolicy /ok /downsamplegrayimages true /grayimagedownsampletype /bicubic /grayimageresolution 300 /grayimagedepth -1 /grayimagemindownsampledepth 2 /grayimagedownsamplethreshold 1.50000 /encodegrayimages true /grayimagefilter /dctencode /autofiltergrayimages true /grayimageautofilterstrategy /jpeg /grayacsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /grayimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000grayacsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000grayimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasmonoimages false /cropmonoimages true /monoimageminresolution 1200 /monoimageminresolutionpolicy /ok /downsamplemonoimages true /monoimagedownsampletype /bicubic /monoimageresolution 1200 /monoimagedepth -1 /monoimagedownsamplethreshold 1.50000 /encodemonoimages true /monoimagefilter /ccittfaxencode /monoimagedict << /k -1 >> /allowpsxobjects false /checkcompliance [ /none ] /pdfx1acheck false /pdfx3check false /pdfxcompliantpdfonly false /pdfxnotrimboxerror true /pdfxtrimboxtomediaboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxsetbleedboxtomediabox true /pdfxbleedboxtotrimboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxoutputintentprofile () /pdfxoutputconditionidentifier () /pdfxoutputcondition () /pdfxregistryname () /pdfxtrapped /false /createjdffile false /description << /ara <feff06270633062a062e062f0645002006470630064700200627064406250639062f0627062f0627062a002006440625064606340627062100200648062b062706260642002000410064006f00620065002000500044004600200645062a064806270641064206290020064406440637062806270639062900200641064a00200627064406450637062706280639002006300627062a0020062f0631062c0627062a002006270644062c0648062f0629002006270644063906270644064a0629061b0020064a06450643064600200641062a062d00200648062b0627062606420020005000440046002006270644064506460634062306290020062806270633062a062e062f062706450020004100630072006f0062006100740020064800410064006f006200650020005200650061006400650072002006250635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e0635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e> /bgr <feff04180437043f043e043b043704320430043904420435002004420435043704380020043d0430044104420440043e0439043a0438002c00200437043000200434043000200441044a0437043404300432043004420435002000410064006f00620065002000500044004600200434043e043a0443043c0435043d04420438002c0020043c0430043a04410438043c0430043b043d043e0020043f044004380433043e04340435043d04380020043704300020043204380441043e043a043e043a0430044704350441044204320435043d0020043f04350447043004420020043704300020043f044004350434043f0435044704300442043d04300020043f043e04340433043e0442043e0432043a0430002e002000200421044a04370434043004340435043d043804420435002000500044004600200434043e043a0443043c0435043d044204380020043c043e0433043004420020043404300020044104350020043e0442043204300440044f0442002004410020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200441043b0435043404320430044904380020043204350440044104380438002e> /chs <feff4f7f75288fd94e9b8bbe5b9a521b5efa7684002000410064006f006200650020005000440046002065876863900275284e8e9ad88d2891cf76845370524d53705237300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /cht <feff4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef69069752865bc9ad854c18cea76845370524d5370523786557406300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /cze <feff005400610074006f0020006e006100730074006100760065006e00ed00200070006f0075017e0069006a007400650020006b0020007600790074007600e101590065006e00ed00200064006f006b0075006d0065006e0074016f002000410064006f006200650020005000440046002c0020006b00740065007200e90020007300650020006e0065006a006c00e90070006500200068006f006400ed002000700072006f0020006b00760061006c00690074006e00ed0020007400690073006b00200061002000700072006500700072006500730073002e002000200056007900740076006f01590065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f007400650076015900ed007400200076002000700072006f006700720061006d0065006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076011b006a016100ed00630068002e> /dan <feff004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000620065006400730074002000650067006e006500720020007300690067002000740069006c002000700072006500700072006500730073002d007500640073006b007200690076006e0069006e00670020006100660020006800f8006a0020006b00760061006c0069007400650074002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /deu <feff00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f006300680077006500720074006900670065002000500072006500700072006500730073002d0044007200750063006b0065002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /esp <feff005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f00730020005000440046002000640065002000410064006f0062006500200061006400650063007500610064006f00730020007000610072006100200069006d0070007200650073006900f3006e0020007000720065002d0065006400690074006f007200690061006c00200064006500200061006c00740061002000630061006c0069006400610064002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /eti <feff004b00610073007500740061006700650020006e0065006900640020007300e4007400740065006900640020006b00760061006c006900740065006500740073006500200074007200fc006b006900650065006c007300650020007000720069006e00740069006d0069007300650020006a0061006f006b007300200073006f00620069006c0069006b0065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740069006400650020006c006f006f006d006900730065006b0073002e00200020004c006f006f0064007500640020005000440046002d0064006f006b0075006d0065006e00740065002000730061006100740065002000610076006100640061002000700072006f006700720061006d006d006900640065006700610020004100630072006f0062006100740020006e0069006e0067002000410064006f00620065002000520065006100640065007200200035002e00300020006a00610020007500750065006d006100740065002000760065007200730069006f006f006e00690064006500670061002e000d000a> /fra <feff005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f0075007200200075006e00650020007100750061006c0069007400e90020006400270069006d007000720065007300730069006f006e00200070007200e9007000720065007300730065002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /gre <feff03a703c103b703c303b903bc03bf03c003bf03b903ae03c303c403b5002003b103c503c403ad03c2002003c403b903c2002003c103c503b803bc03af03c303b503b903c2002003b303b903b1002003bd03b1002003b403b703bc03b903bf03c503c103b303ae03c303b503c403b5002003ad03b303b303c103b103c603b1002000410064006f006200650020005000440046002003c003bf03c5002003b503af03bd03b103b9002003ba03b103c42019002003b503be03bf03c703ae03bd002003ba03b103c403ac03bb03bb03b703bb03b1002003b303b903b1002003c003c103bf002d03b503ba03c403c503c003c903c403b903ba03ad03c2002003b503c103b303b103c303af03b503c2002003c503c803b703bb03ae03c2002003c003bf03b903cc03c403b703c403b103c2002e0020002003a403b10020005000440046002003ad03b303b303c103b103c603b1002003c003bf03c5002003ad03c703b503c403b5002003b403b703bc03b903bf03c503c103b303ae03c303b503b9002003bc03c003bf03c103bf03cd03bd002003bd03b1002003b103bd03bf03b903c703c403bf03cd03bd002003bc03b5002003c403bf0020004100630072006f006200610074002c002003c403bf002000410064006f00620065002000520065006100640065007200200035002e0030002003ba03b103b9002003bc03b503c403b103b303b503bd03ad03c303c403b503c103b503c2002003b503ba03b403cc03c303b503b903c2002e> /heb <feff05d405e905ea05de05e905d5002005d105d405d205d305e805d505ea002005d005dc05d4002005db05d305d9002005dc05d905e605d505e8002005de05e105de05db05d9002000410064006f006200650020005000440046002005d405de05d505ea05d005de05d905dd002005dc05d405d305e405e105ea002005e705d305dd002d05d305e405d505e1002005d005d905db05d505ea05d905ea002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e05d005de05d905dd002005dc002d005000440046002f0058002d0033002c002005e205d905d905e005d5002005d105de05d305e805d905da002005dc05de05e905ea05de05e9002005e905dc0020004100630072006f006200610074002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e> /hrv (za stvaranje adobe pdf dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. stvoreni pdf dokumenti mogu se otvoriti acrobat i adobe reader 5.0 i kasnijim verzijama.) /hun <feff004b0069007600e1006c00f30020006d0069006e0151007300e9006701710020006e0079006f006d00640061006900200065006c0151006b00e90073007a00ed007401510020006e0079006f006d00740061007400e100730068006f007a0020006c006500670069006e006b00e1006200620020006d0065006700660065006c0065006c0151002000410064006f00620065002000500044004600200064006f006b0075006d0065006e00740075006d006f006b0061007400200065007a0065006b006b0065006c0020006100200062006500e1006c006c00ed007400e10073006f006b006b0061006c0020006b00e90073007a00ed0074006800650074002e0020002000410020006c00e90074007200650068006f007a006f00740074002000500044004600200064006f006b0075006d0065006e00740075006d006f006b00200061007a0020004100630072006f006200610074002000e9007300200061007a002000410064006f00620065002000520065006100640065007200200035002e0030002c0020007600610067007900200061007a002000610074007400f3006c0020006b00e9007301510062006200690020007600650072007a006900f3006b006b0061006c0020006e00790069007400680061007400f3006b0020006d00650067002e> /ita <feff005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f00620065002000500044004600200070006900f900200061006400610074007400690020006100200075006e00610020007000720065007300740061006d0070006100200064006900200061006c007400610020007100750061006c0069007400e0002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /jpn <feff9ad854c18cea306a30d730ea30d730ec30b951fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e305930023053306e8a2d5b9a306b306f30d530a930f330c8306e57cb30818fbc307f304c5fc59808306730593002> /kor <feffc7740020c124c815c7440020c0acc6a9d558c5ec0020ace0d488c9c80020c2dcd5d80020c778c1c4c5d00020ac00c7a50020c801d569d55c002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /lth <feff004e006100750064006f006b0069007400650020016100690075006f007300200070006100720061006d006500740072007500730020006e006f0072011700640061006d00690020006b0075007200740069002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b00750072006900650020006c0061006200690061007500730069006100690020007000720069007400610069006b007900740069002000610075006b01610074006f00730020006b006f006b007900620117007300200070006100720065006e006700740069006e00690061006d00200073007000610075007300640069006e0069006d00750069002e0020002000530075006b0075007200740069002000500044004600200064006f006b0075006d0065006e007400610069002000670061006c006900200062016b007400690020006100740069006400610072006f006d00690020004100630072006f006200610074002000690072002000410064006f00620065002000520065006100640065007200200035002e0030002000610072002000760117006c00650073006e0117006d00690073002000760065007200730069006a006f006d00690073002e> /lvi <feff0049007a006d0061006e0074006f006a00690065007400200161006f00730020006900650073007400610074012b006a0075006d00750073002c0020006c0061006900200076006500690064006f00740075002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006100730020006900720020012b00700061016100690020007000690065006d01130072006f00740069002000610075006700730074006100730020006b00760061006c0069007401010074006500730020007000690072006d007300690065007300700069006501610061006e006100730020006400720075006b00610069002e00200049007a0076006500690064006f006a006900650074002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006f002000760061007200200061007400760113007200740020006100720020004100630072006f00620061007400200075006e002000410064006f00620065002000520065006100640065007200200035002e0030002c0020006b0101002000610072012b00200074006f0020006a00610075006e0101006b0101006d002000760065007200730069006a0101006d002e> /nld (gebruik deze instellingen om adobe pdf-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. de gemaakte pdf-documenten kunnen worden geopend met acrobat en adobe reader 5.0 en hoger.) /nor <feff004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d00200065007200200062006500730074002000650067006e0065007400200066006f00720020006600f80072007400720079006b006b0073007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /pol <feff0055007300740061007700690065006e0069006100200064006f002000740077006f0072007a0065006e0069006100200064006f006b0075006d0065006e007400f300770020005000440046002000700072007a0065007a006e00610063007a006f006e00790063006800200064006f002000770079006400720075006b00f30077002000770020007700790073006f006b00690065006a0020006a0061006b006f015b00630069002e002000200044006f006b0075006d0065006e0074007900200050004400460020006d006f017c006e00610020006f007400770069006500720061010700200077002000700072006f006700720061006d006900650020004100630072006f00620061007400200069002000410064006f00620065002000520065006100640065007200200035002e0030002000690020006e006f00770073007a0079006d002e> /ptb <feff005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020006d00610069007300200061006400650071007500610064006f00730020007000610072006100200070007200e9002d0069006d0070007200650073007300f50065007300200064006500200061006c007400610020007100750061006c00690064006100640065002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /rum <feff005500740069006c0069007a00610163006900200061006300650073007400650020007300650074010300720069002000700065006e007400720075002000610020006300720065006100200064006f00630075006d0065006e00740065002000410064006f006200650020005000440046002000610064006500630076006100740065002000700065006e0074007200750020007400690070010300720069007200650061002000700072006500700072006500730073002000640065002000630061006c006900740061007400650020007300750070006500720069006f006100720103002e002000200044006f00630075006d0065006e00740065006c00650020005000440046002000630072006500610074006500200070006f00740020006600690020006400650073006300680069007300650020006300750020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e00300020015f00690020007600650072007300690075006e0069006c006500200075006c0074006500720069006f006100720065002e> /rus <feff04180441043f043e043b044c04370443043904420435002004340430043d043d044b04350020043d0430044104420440043e0439043a043800200434043b044f00200441043e043704340430043d0438044f00200434043e043a0443043c0435043d0442043e0432002000410064006f006200650020005000440046002c0020043c0430043a04410438043c0430043b044c043d043e0020043f043e04340445043e0434044f04490438044500200434043b044f00200432044b0441043e043a043e043a0430044704350441044204320435043d043d043e0433043e00200434043e043f0435044704300442043d043e0433043e00200432044b0432043e04340430002e002000200421043e043704340430043d043d044b04350020005000440046002d0434043e043a0443043c0435043d0442044b0020043c043e0436043d043e0020043e0442043a0440044b043204300442044c002004410020043f043e043c043e0449044c044e0020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200431043e043b043504350020043f043e04370434043d043804450020043204350440044104380439002e> /sky <feff0054006900650074006f0020006e006100730074006100760065006e0069006100200070006f0075017e0069007400650020006e00610020007600790074007600e100720061006e0069006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b0074006f007200e90020007300610020006e0061006a006c0065007001610069006500200068006f0064006900610020006e00610020006b00760061006c00690074006e00fa00200074006c0061010d00200061002000700072006500700072006500730073002e00200056007900740076006f00720065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f00740076006f00720069016500200076002000700072006f006700720061006d006f006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076016100ed00630068002e> /slv <feff005400650020006e006100730074006100760069007400760065002000750070006f0072006100620069007400650020007a00610020007500730074007600610072006a0061006e006a006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b006900200073006f0020006e0061006a007000720069006d00650072006e0065006a016100690020007a00610020006b0061006b006f0076006f00730074006e006f0020007400690073006b0061006e006a00650020007300200070007200690070007200610076006f0020006e00610020007400690073006b002e00200020005500730074007600610072006a0065006e006500200064006f006b0075006d0065006e0074006500200050004400460020006a00650020006d006f0067006f010d00650020006f0064007000720065007400690020007a0020004100630072006f00620061007400200069006e002000410064006f00620065002000520065006100640065007200200035002e003000200069006e0020006e006f00760065006a01610069006d002e> /suo <feff004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f00740020006c00e400680069006e006e00e4002000760061006100740069007600610061006e0020007000610069006e006100740075006b00730065006e002000760061006c006d0069007300740065006c00750074007900f6006800f6006e00200073006f00700069007600690061002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a0061002e0020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /sve <feff0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400200073006f006d002000e400720020006c00e4006d0070006c0069006700610020006600f60072002000700072006500700072006500730073002d007500740073006b00720069006600740020006d006500640020006800f600670020006b00760061006c0069007400650074002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /tur <feff005900fc006b00730065006b0020006b0061006c006900740065006c0069002000f6006e002000790061007a006401310072006d00610020006200610073006b013100730131006e006100200065006e0020006900790069002000750079006100620069006c006500630065006b002000410064006f006200650020005000440046002000620065006c00670065006c0065007200690020006f006c0075015f007400750072006d0061006b0020006900e70069006e00200062007500200061007900610072006c0061007201310020006b0075006c006c0061006e0131006e002e00200020004f006c0075015f0074007500720075006c0061006e0020005000440046002000620065006c00670065006c0065007200690020004100630072006f006200610074002000760065002000410064006f00620065002000520065006100640065007200200035002e003000200076006500200073006f006e0072006100730131006e00640061006b00690020007300fc007200fc006d006c00650072006c00650020006100e70131006c006100620069006c00690072002e> /ukr <feff04120438043a043e0440043804410442043e043204430439044204350020044604560020043f043004400430043c043504420440043800200434043b044f0020044104420432043e04400435043d043d044f00200434043e043a0443043c0435043d044204560432002000410064006f006200650020005000440046002c0020044f043a04560020043d04300439043a04400430044904350020043f045604340445043e0434044f0442044c00200434043b044f0020043204380441043e043a043e044f043a04560441043d043e0433043e0020043f0435044004350434043404400443043a043e0432043e0433043e0020043404400443043a0443002e00200020042104420432043e04400435043d045600200434043e043a0443043c0435043d0442043800200050004400460020043c043e0436043d04300020043204560434043a0440043804420438002004430020004100630072006f006200610074002004420430002000410064006f00620065002000520065006100640065007200200035002e0030002004300431043e0020043f04560437043d04560448043e04570020043204350440044104560457002e> /enu (use these settings to create adobe pdf documents best suited for high-quality prepress printing. created pdf documents can be opened with acrobat and adobe reader 5.0 and later.) >> /namespace [ (adobe) (common) (1.0) ] /othernamespaces [ << /asreaderspreads false /cropimagestoframes true /errorcontrol /warnandcontinue /flattenerignorespreadoverrides false /includeguidesgrids false /includenonprinting false /includeslug false /namespace [ (adobe) (indesign) (4.0) ] /omitplacedbitmaps false /omitplacedeps false /omitplacedpdf false /simulateoverprint /legacy >> << /addbleedmarks false /addcolorbars false /addcropmarks false /addpageinfo false /addregmarks false /convertcolors /converttocmyk /destinationprofilename () /destinationprofileselector /documentcmyk /downsample16bitimages true /flattenerpreset << /presetselector /mediumresolution >> /formelements false /generatestructure false /includebookmarks false /includehyperlinks false /includeinteractive false /includelayers false /includeprofiles false /multimediahandling /useobjectsettings /namespace [ (adobe) (creativesuite) (2.0) ] /pdfxoutputintentprofileselector /documentcmyk /preserveediting true /untaggedcmykhandling /leaveuntagged /untaggedrgbhandling /usedocumentprofile /usedocumentbleed false >> ] >> setdistillerparams << /hwresolution [2400 2400] /pagesize [612.000 792.000] >> setpagedevice multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 25 encouraging engineering undergraduates to voice their ideas worth sharing arazazu garcía-pinar universidad politécnica de cartagena email: arancha.garcia@cud.upct.es orcid: 0000-0003-4287-6521 received: 2019-12-01; accepted: 2019-04-18 abstract ted talks have these days become a valuable tool for online information dissemination in a wide range of areas of expertise. the use of ted talks in a course of technical english offers numerous advantages. ted teaches how to communicate by linking different modes (i.e. the visual, gestural, verbal, written and spatial) to technological production. students can construct communication when they attentively observe and make meaning from this ensemble of modes which go beyond the verbal. ted talks might also give rise to different tasks that entail some type of critical multimodal analysis, by which students can study the aptness of modes. they can explore why the speaker says something visually and not verbally, or which mode is best for which purpose. yet, ted and its zeal for sharing and transmitting ideas to a wide audience should not be regarded as a means incompatible with more traditional models of information. as jewitt highlights (2005), rather than asking what is best, the book or the screen”, it seems more reasonable to ask “what is best for what purpose”. keywords ted talks, mode, engineering, communication, multimodal analysis. mailto:arancha.garcia@cud.upct.es https://doi.org/10.4995/muse.2019.11370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 26 1. introduction this article outlines an overview of inspirational and influential online ted talk videos, and how these can be considered by lecturers as challenging and valuable tools not only to be viewed and listened to, but also to be analysed in terms of multimodality. ted talks might be regarded as optimal artefacts of visual media, and as new forms of speech delivery to be emulated. additionally, these talks might find their way into the course of technical english as a source of ideas and information that engineering undergraduates can use to delve into a wide range of topics of their interest, and to propel and inspire their course essays. this article mainly aims to draw attention to the multimodal nature of ted talks. ted talks are multimodal to the extent that the speakers on stage need fluency in different verbal and nonverbal modes (i.e. the power of speech, visual design, gesture, facial expressions and proxemics). verbal and non-verbal communication co-occur in these talks, and students construct meaning by tracking speakers’ gestures, posture and real and natural language. l2 lecturers have the challenging role of showing students the ways these talks can be deconstructed, so as to introduce them to the ways semiotic resources interact and function in digital multimodal texts. in this way, they can learn how speakers use linguistic, visual and audio resources to involve and captivate their audiences. this article will detail the multimodal character of ted talks. it will also offer an overview of some empirical studies that have researched the effect of ted talks on student motivation. the last part of the article will explain how a ted talk can be analysed in the course of technical english so that students realise the salience and aptness of both verbal and nonverbal modes. the common pattern shared by different ted events is that all of them must spread thought-provoking ideas. the mission of ted is, as surgimoto and thelwall point out (2013: 664), “one of change and engagement”. ideas are understood as influential patterns of information that can shape human culture. ted curator chris anderson (2017: 12-13) gives ideas a relevant role in the following terms: the only thing that truly matters in public speaking […] is having something worth saying. […] an idea is anything that can change how people see the world. if you can multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 27 conjure up a compelling idea in people’s minds, you have done something wondrous. you have given them a gift of incalculable value. anderson’s words, therefore, might be encouraging for anyone aspiring to speak in public, as the key prerequisite seems to be having an idea worth sharing. lecturers’ experience may lead them to think that other elements beyond the possession of a compelling idea are necessary. the next section is concerned with non-verbal modes onstage. if aptly combined with verbal modes, non-verbal modes can give compelling and original ideas greater emphasis, while at the same time capturing the attention of the audience. this modal interweaving is precisely what one finds in many ted talks. 2. multimodal ted talks ted talks are multimodal insomuch as these confer a prominent role to not only the verbal mode. ted speakers have been coached to be fluent in different modes beyond the verbal. in many talks, the emotional load is communicated with a specific tone of voice, a hand gesture, or an impacting visual. while words build and develop ideas, explain complex concepts or simply narrate, visuals, gestures and voice qualities transmit emotion and can arouse the audience’s curiosity and infuse a speech with variety. 2.1 verbal model the verbal mode is certainly the most important mode in ted talks. as anderson highlights, “language works its magic only to the extent that it is shared by speaker and listener”. words are basically the tool that ted speakers use to recreate their ideas in the minds of their audiences. it is a tool that, if cautiously used, can ensure the talk’s success. speakers have the difficult task of turning their sets of ideas into words, and in this process, they must ensure that their audiences understand the logical relationships among their sentences. upgrading the audience’s mental model of the world through the masterful use of explanation in any kind of talk is relevant. yet, many undergraduates find it difficult to establish effective connections between sentences in order to express similarity, contrast, multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 28 cause and effect or exemplification. cognitive psychologist and harvard professor steven pinker, also a ted speaker on several occasions, highlights that to achieve full understanding, the whole hierarchical structure of an idea must be clearly communicated. in his work ‘the sense of style: the thinking person’s guide to writing in the 21st century’ (2014: 39), he offers some guidance on how to use language more efficiently, and to communicate ideas hierarchically: no writer can leave the macroscopic organization of a text to chance (…) no sentence is an island; nor is a section or a chapter. all of them contain links to other chunks of a text. a sentence may elaborate, qualify, or generalize the one that came before (…) people, places and ideas may make repeat appearances, and the reader must keep track of them as they come and go. though clearly focusing on writing, pinker’s guidance might well apply to oral communication with writer being replaced by speaker and reader by listener. speakers need to join their ideas and make sure that these are linked in a logical manner. another relevant aspect with regard to the verbal mode in ted talks has to do with the simplicity and explicitness of the language used. most ted speakers deliver their talks in simple language, totally devoid of technical terms or acronyms that might cause the audience to be confused, and in consequence, to switch off. for this reason, ted speakers often start their talks using their audiences’ language, and mentioning their assumptions and concepts, and from there they start building new ideas inside their audiences’ minds. speakers, as anderson notes (2017: 85), must be able to explain: you can’t give a powerful new idea to an audience unless you can learn how to explain […]. each step builds on what the listener already knows. metaphors and examples are essential to revealing how an idea is pieced together. metaphors and examples, as evidenced by anderson’s quote, play a relevant role in illuminating and simplifying more sophisticated and complex technical talks. although student oral presentations might be addressed to a specific audience (i.e. their peers), it might be appropriate to encourage them to resort to examples whenever these can enhance multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 29 understanding. likewise, students must be cautious with acronyms and make sure these are explained. the manner in which specialised talks are communicated is of great relevance, and students should be aware of the fact that they might be required to disseminate knowledge not just to a specialised public in their future careers, but also to a lay public. 2.2 non-verbal modes while one cannot underestimate the efficacy of language, nor can its supreme status be denied in any oral interaction, it seems quite legitimate to state that on some occasions, different modes beyond the verbal can fulfil different purposes. language, therefore, in some instances might be less resourceful and have less potential for making meaning than, for instance, the use of a beat gesture, a specific posture, different voice qualities or a carefully considered image. paralinguistics includes the features of spoken communication that do not involve words. the common european framework of reference (cefr) (2001: 89-90) makes a distinction between paralinguistic body language (i.e. gesture, posture, facial expression, proxemics and eye contact) and prosodic qualities (i.e. voice quality, pitch, loudness and length). all these paralinguistic features form what is known as non-verbal communication, and often play a key role in speaking, as they can add layers of significance to spoken words. young and travis (2012: 58) identify four different functions of non-verbal communication: it can replace a verbal message: pointing to something instead of explaining where it is using words. it can highlight a verbal message: the use of hand gestures is a clear example of a way to enhance a verbal message. it can express the opposite of a verbal message: particular care should be taken to not contradict verbal with nonverbal communication. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 30 it can direct people’s conversations: noticing speakers’ intentions to make some kind of eye contact, for example, can guide people in recognising their turns to speak. it seems appropriate and necessary to make students aware of the fact that different nonverbal modes (i.e. gestures, facial expressions and prosodic features) may contribute to the whole communication process (campoy-cubillo, 2016) and to listening comprehension (sueyoshi and hardison, 2005). 2.2.1 gestures gestures play a pivotal role in communication, as these can enhance listening comprehension, clarify verbal meanings and convey additional information. gestures can contribute to listeners’ comprehension by building an internal cognitive simulation or mental representation of the message (hostetter and alibali, 2010). mcneill’s (1992) classification of hand gesture has been frequently used to study gestures in communication. iconic, metaphorical, deictic and beats are used by speakers to facilitate communication so that listeners can focus their attention on the relevant information in the message, and in consequence, derive meaning. a quick look at a representative sample of ted talks shows that speakers recurrently resort to the use of hand gesture throughout their talks. the most viral talks have been found to have been given by speakers who extensively use gestures (van edwards, 2015). ted speakers often resort to beat gestures (i.e. a rapid movement of the hand or the finger, produced to the rhythm of the accompanying speech which does not communicate semantic content) and to deictic gestures (i.e. pointing gestures commonly used to refer to specific objects, events, notions and ideas). these types of gestures help speakers punctuate important stretches of their discourses, regulating the flow of speech and coherently integrating the overall discourse. beat gestures also perform a social pragmatic function (weinberg, fukawa-connelly & wiesner, 2013) that contributes to enhancing audience awareness of the key ideas in speakers’ talks. speakers address their audiences with a beat to get them involved with an issue that concerns all of them (i.e. environment, technology, multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 31 politics, poverty). ted speakers also use deictic gestures extensively to call attention to some specific points in any visual that accompanies their talks. these gestures perform a referential and indexical function (i.e. indicate a position), as they refer to the image, concept, figure which is being described. head movements (i.e. nods, lateral head sweeps and head shake) is another type of gesture closely connected to speech in the process of communication (briñol & petty, 2003; kendon, 1980; 2002). research on head nods and shakes (briñol & petty, 2003), for instance, has focused on the influence these can have on people’s confidence in their thoughts when having to give response to a message. the authors contend (p.1135) that “manipulation of head movements affects confidence in the validity of one’s thoughts, not confidence in the validity of the persuasive message per se”. lateral sweeps can ‘co-occur with concepts of inclusivity such as the words ‘everyone’ and ‘everything’ (mcclave, 2000: 860). the narrative functions include marking direct quotes, portraying mental images of characters, performing deixis, and indexing items on lists. according to the cognitive function, head movements might be involved in cognitive processing. head movements, in relation to the interactive function, can be used for backchannelling purposes (i.e. reaction to your interlocutor, showing interest). head movements, though often culturally specific, are, as mcclave notes (2000), quite valuable, since listeners are greatly sensitive to them. ted speakers’ habitual use of lateral sweeps is worth highlighting, particularly in talks that delve into topics of major global concern (i.e. activism, social change, the environment, education, recycling, technological advance). the lateral sweep concurs with concepts of inclusivity (i.e. everyone, everything) and express intensification when they co-occur with words such as “very”, “great”, “a lot”, among others. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 32 2.2.2 facial expressions regular eye contact with members of the audience can also enhance positive connections while reflecting interest, and transmitting caring and courtesy (young and travis 2012: 60). the audience, in turn, by tracking the speaker’s gaze, might predict what the speaker’s intentions are, and what he or she is planning to convey next. additionally, eye contact accompanied with specific facial expressions (e.g. a smile, nod, or frown) can even have a captivating effect. speakers’ facial expressions are another channel for successful communication. these expressions allow the audience to sense speakers’ feelings and moods. changes in facial expression can anticipate changes in tone and mood. examples of ted talks that combine eye contact and facial expressions are countless. 2.2.3 proxemics proxemics is the study of the ways in which people organise and use their space (norris, 2004: 19). people might adopt different distances both in relation to others and to concrete objects. it is the specific distance of a person from others in different interactions that may allow one to deduce the person’s understanding of the formality or informality of the situation. hall (1966) established four distinct distances: (1) intimate distance (15-45 cm) denotes a close relationship; (2) personal distance (45-120 cm) takes place among friends and family members; (3) social distance (1.20m-3.50 m) occurs among acquaintances, and (4) public distance, which is the distance frequently used in public speaking (i.e. classroom lectures, ted talks). it is relevant to note how proxemic behaviour can contribute to understanding the type of social interaction that is taking place. regarding this, speakers at ted know how to arrange and use their space to enhance meaning-making. even though they keep their distance from the public (i.e. public distance), and the size of the stage indicates their high status, they know how to get closer to their audience while maximising their stage presence. they rarely stay put on stage. ted speakers have carefully studied how to move around a stage multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 33 with the flow of their speeches, and when they should stop to emphasise meaningful parts in their talks. 2.2.4 prosodic qualities “conventionalised meanings that are related to attitudes and states of mind” (cefr, 2001: 89). prosodic qualities, also referred to as nonverbal voice qualities (poyatos, 1983) include: pitch range, loudness and prosody. in the 1960s and 1970s, voice quality was conceived as an arbitrary mark of individual or social identity, and was merely depicted in articulatory and acoustic terms. van leeuwen (1999), still drawing on phonetics and linguistic work, semioticised and theorised that voice quality might be used to convey meaning. he built on lakoff and johnson’s (1980) metaphor theory, according to which metaphors can be understood on the basis of concrete experiences. the authors postulated (p.19) that “no metaphor can ever be comprehended or even adequately represented independently of its experiential basis”. this assertion refers to physical, bodily experience, such as tensing the voice. voice tension is commonly associated with situations of nervousness, anxiety or threat, and is therefore a voice quality with meaning and metaphor potential that can signal certain states of mind. other resources for vocal meaning are different components and features that shape the quality of voice, lead to the construction of meaning, and effect how people might perceive the speaker’s message: pitch range and degrees of loudness. leeuwen notes (2011: 71) that pitch range, for instance, with men using higher regions of their pitch range to convey dominance or assertion and women using the lower end of their pitch range to be assertive, can be deliberately modified to convey other types of intentions. some men who tend to speak with a low pitch might not aim to dominate but to make themselves small. some women might opt to speak softly, using a low pitch to evoke the ‘dangerous woman’ stereotype or loudly and in a high pitch, which might invoke the stereotype “of the shrill and strident fishwife” (kress and van leeuwen, 2001: 84). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 34 the loudness range of the voice is significant to conveying distance. at close range, people’s voices are associated with intimacy and confidentiality. vocal loudness might relate to power and dominance, and it is this quality of voice that is pervasive in ted talks. speakers are centre stage and generally give talks to a considerable number of people whose attention they try to capture. prosody encompasses the group of speech properties that influence more than one sound segment (i.e. stress, intonation, rhythm) (valeiras-jurado, 2017). the fact that prosody can help listeners process and understand a message has made research on this topic particularly fruitful. it has been widely acknowledged that many of the difficulties l2 students face have to do with these non-verbal characteristics of communication (chun, 2002). prosody has also been researched in connection with public presentations and how this might affect the audience’s perception of vivacity (hincks, 2009: 46): it is vital to use one’s voice well when speaking in public. it is the channel of communication, and when used poorly, communication can be less than successful. if listeners either stop listening, or fail to perceive what is most important in a speaker’s message, then all actors in the situation are in effect wasting time. rhythm can play a significant role in spoken interactions, creating structure in communicative situations. rhythm and the alternation of accented and unaccented moments articulate meaning. rhythm, according to van leeuwen (2005), is along with layout in composition in space, a major resource to create cohesion in any communicative event. rhythm and layout, the author also highlights (p.181), create the link between semiotic articulation and the body: human action is by nature rhythmically coordinated, and, as micro-analytical studies have shown, so are human interactions (…). rhythm does not just provide some kind of formal structure, some kind of scaffolding to keep the text from collapsing, or some kind of cement to hold it together. it also plays an indispensable part in getting the message across. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 35 rhythm, therefore, comes to join and integrate all the different modes (e.g. body movement, language, gesture) involved in the communicative event as these unfold in time. rhythm can also be achieved with stylistic devices such as parallel structures (i.e. the repetition of a series of words and phrases which share a grammatical form and length). it is quite relevant to observe how some ted talk speakers resort to parallel structures to achieve a specific cadence. ted speaker simon sinek resorted to parallel structure repeatedly in his talk (how great leaders inspire action). he used this device accompanied with stress on specific words and with gestures to finish his talk: we follow those who lead, not because we have to, but because we want to. we follow those who lead, not for them, but for ourselves. paralinguistic features may enhance the passionate and enthusiastic style of delivery many speakers at ted have. lecturers cannot underestimate the relevance of all these features, and must call attention to the fact that it is not so much the ‘what’ of the idea they want to transmit but the ‘how’. the next section focuses on the role of visuals in oral presentations and on how these can often make the difference between success and failure. different types of visual support are frequently found in many ted talks. 2.2.5 visuals an extensive number of ted talks use photographs, graphs, tables, and illustrations to upgrade the verbal message and to strengthen the aesthetic appeal of the speaker’s oral performance. it might then be useful to direct engineering undergraduates to the opportunities offered by these visuals and to raise their curiosity regarding the specific reasons that might have led a speaker to choose a specific type of visual over another. this might, in turn, lead students to consider some important issues when they design the power points for their oral presentations. issues such as the salience of some resources and the aptness (kress 2005: 19) of representing different types of content, the type of roles different illustrations might play, the kind of content that might be expected from headings multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 36 and illustrations, the type of information images facilitate, and where the visuals they use make a complex phenomenon easier are all relevant issues and must be carefully attended. a common practice among students is to load their slides with more than one idea. regarding this, tom rielly, one of the members of the coaching group in ted, warns (anderson, 2017: 116) about the dangers of cognitive load in presentation slides: with a talk and slides you have two streams of cognitive output running in parallel. (…). talking about theoretical physics has a high cognitive load (…). in these circumstances, the audience member’s brain has to decide whether to focus on your words, your slides, or both, and it’s mostly involuntary. so you must design where attention is going and make sure a high cognitive load on a slide doesn’t fight with what you’re saying. the choice of a particular visual in student oral presentations can be, as stated above, an issue that should be carefully considered. multimodal concepts such as modal affordance (i.e. different modes can present different potentials for making meaning), aptness (i.e. some modes may be more suitable for a specific purpose than others) and visual salience (i.e. the specific ways different elements in a visual layout such as colour, size and contrast appear in order to capture the viewer’s attention) are key concepts students should be aware of when designing their power points, as these will have a determining role in the overall performance (van leeuwen & kress, 1996: 183). 3. the influence of ted talks on student motivation relevant studies have also incorporated these online talks in different educational contexts to motivate students. rubenstein (2012) focused on the study of ted talks that can upgrade teachers’ understanding of student motivation and teaching procedures. throughout her article she proposes different ted talks that can be used in the classroom for different purposes; to promote student motivation and to initiate teachers into novel instructional practices. her selection of talks about motivation, as she suggests, can lead educators to raise important questions. the first ted talk she discusses is dan pink’s multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 37 (2009). this talk, which revolves around workplace motivation, can promote meaningful debate and reflection among teachers (p.263): how can teachers encourage autonomous learning? how can teachers ask questions in class so that students are freer to research content? how can teachers guide students to recognise problems and work to find their solutions? another ted talk she analyses is psychologist csikszentmihalyi’s. this talk focuses on the psychological movement of flow, and explains the contributing role that performing a challenging task may have on facilitating one’s happiness. drawing on this talk, rubenstein urges teachers to promote flow in the classroom in order to both provide motivation and transform the classroom into an enjoyable context. takaesu’s (2013) study explored how the extensive use of ted talks as listening resources affected the listening skills of 468 tertiary japanese students in a course of english for academic purposes (eap). qualitative data obtained through surveys and journal entries showed that students positively assessed the effectiveness of the listening activities designed from ted talks. this fact contributed to raising their feelings of self-efficacy and encouraged them to research the topics included in the talks. elk’s article (2014) describes the way she designed different listening tasks from ted talks, with the aim of focusing attention on the shortcomings she and her students had found in an eap course book. one of these deficiencies had to do with the lack of authentic materials in eap textbooks. the regular use of ted talks in the classroom provided authentic listening tasks, and allowed the researcher to improve strategies for “processing visual input that is synchronous, but not identical to aural input” (p.219). students listened to these talks, wrote about they had heard, verified comprehension with the help of the talk transcription, and finally wrote about the listening difficulties they had encountered. these tasks were mainly intended to promote students’ autonomy and to draw their attention to the errors they made in bottom-up and top-down processing. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 38 4. ted talks multimodal analysis the website www.ted.com has launched 3,000 online talks to date. the process of analysing all the modes at play in these talks is unattainable. this section focuses on one of the most viewed ted talks to date (do schools kill creativity? by sir ken robinson), and intends to offer a closer look at how speakers achieve impact through efficient handling of the modes detailed above. different gestures, facial expressions and appealing slides when working in synergy with words might enhance comprehension and achieve emphasis. the speaker transmits passionately, persuasively and emphatically, key educational notions and concepts with the help of features such as rhetorical questions, repetition, parallelisms, intensifying adverbs and punctual hand gestures, different types of facial expression and a particular rhythm. do schools kill creativity? (19’22’’). sir ken robinson link: https://www.ted.com/talks/ken_robinson_says_schools_kill_creativity about the speaker: sir ken robison is a british author, speaker and international advisor on education, non-profit organisations, and art institutions. he was director of the arts in schools project (1985–89) and professor of arts education at the university of warwick (1989–2001). he is currently professor emeritus at the same institution. in 2003, he was appointed knight bachelor by queen elizabeth ii for services to the arts. year of the talk: 2006 about the talk: in this talk, sir ken robinson challenged the way current educational institutions are educating children, and urges a radical transformation of school systems into those where creativity is cultivated and multiple types of intelligence are acknowledged. this educational reform discourse is sprinkled with anecdotes and humour, which conveys informality to robinson’s presentation. his talk is full of poignant points, http://www.ted.com/ https://www.ted.com/talks/ken_robinson_says_schools_kill_creativity https://en.wikipedia.org/wiki/university_of_warwick https://en.wikipedia.org/wiki/emeritus multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 39 and he achieves his intended emphatic tone through various modes: verbally, through hand gestures, and with facial expressions. modal interplay: students’ attention can be particularly directed to the way sir ken robinson repeatedly resorts to the use of hand gestures when he wants to highlight important parts of his discourse. on several occasions, he also tilts his head to raise his audience’s awareness of important educational issues. table 1. modal interplay in the ted talk ‘do schools kill creativity?’ minute transcript of talk non-verbal mode rhetorical strategy 5:22 i don't mean to say that being wrong is the same thing as being creative. what we do know is, if you're not prepared to be wrong, you'll never come up with anything original -if you're not prepared to be wrong beat gesture: the speaker uses a beat gesture with an extended finger that entails two phases of movement. when the finger goes downwards, it points to the discourse flow, emphasising keywords that receive prosody stress as well. the speaker uses the rhetorical strategy of repetition of the phrase “if you are not prepared to be wrong”. this repetition helps sir robinson to reinforce his key idea. he also uses paralinguistic stress on the word “wrong”, repeated three times. gesture function: social to attract the audience’s attention and to give emphasis to keywords 8:16 every education system on earth has the same hierarchy of subjects. every one. doesn't matter where you go. you'd think it would be otherwise, but it isn't. at the top are mathematics and languages, then the humanities, and at the bottom are the arts. everywhere on earth. beat gesture: the speaker uses a finger beat with fast flicks whose downward movements fall on the words ‘same’, ‘hierarchy’, ‘subjects’ rhetorical strategy the speaker resorts, in this statement, to the use of repetition. this rhetorical device helps him to hold his ideas cohesively and make the message clearer. he repeats the adjective ‘every’ twice. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 40 gesture function: social. the speaker intends to highlight the importance of his idea the use of adverbs ‘everywhere’ and ‘on earth’ contributes to emphasising the part he wants to focus on. ‘every’ and ‘everywhere’ are also given prominence. 17:28 our education system has mined our minds in the way that we strip-mine the earth: for a particular commodity. and for the future, it won't serve us. we have to rethink the fundamental principles on which we're educating our children. beat gesture / lateral head sweep the speaker resorts to the finger beat gesture with a repetitive downward movement that falls on we, have, rethink, punctuating important stretches in his discourse. finger beat gesture is also accompanied by a lateral head sweep which might convey inclusivity: education concerns all of us. rhetorical strategy he uses the inclusive pronouns ‘we’ and ‘us’ to make his talk more intimate and inclusive. this strategy helps him to raise his audience’s awareness about the current educational system. 5. conclusion this article has focused on the multimodal character of ted talks and on how the accurate construction of communication from the interweaving of modes might have a strong influence on how students receive, interpret, and ultimately, prepare and deliver their classroom technological oral presentations. speakers at ted stand out because they transmit creativity and brilliant ideas. the way they disseminate knowledge has no equal. ted talks, therefore, can provide suitable communication guidelines for undergraduates of engineering, and can help them to transmit their complex innovations and ideas. to this end, the last section of this article focused on one of the most viewed ted talks to date, in order to offer a closer look at how the speaker achieves impact through efficient handling of the modes detailed above. different gestures, facial expressions and appealing slides, when working in synergy with words, might enhance comprehension and achieve emphasis. if students’ attention is directed towards the modal salience and aptness in these talks, they might be encouraged to incorporate a set of modes in their classroom oral presentations and to voice their ideas worth sharing. this, in turn, can make this speaking activity less daunting, and might encourage students to visualise their l2 speaking selves. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 41 5. references anderson, c. (2017). ted: the official ted guide to public speaking. boston: houghton mifflin harcourt. brazil, d. (1985). the communicative value of intonation in english. birmingham: the university printers, university of birmingham. briñol, p. & petty, r. e. (2003). overt head movements and persuasion: a self-validation analysis. journal of personality and social psychology, 84, 1123-1139. campoy-cubillo, m.c. (2016). multimodal listening skills: issues in assessment and implementation. in v. bonsignori & b. crawford camiciottoli (eds.), multimodality across communicative settings, discourse domains and genres (pp. 14-36). cambridge: cambridge scholars publishing. chun, d. m. 2002. discourse intonation in l2: from theory and research to practice. j. benjamins. council of europe. (2001). common european framework of reference for languages: learning, teaching, assessment. cambridge, u.k: press syndicate of the university of cambridge. elk, c. k. (2014). beyond mere listening comprehension: using ted talks and metacognitive activities to encourage awareness of error. international journal of innovation in english language teaching and research, 3(2), 215. hadar, u., steiner t.j., grant e.c. & rose, f.c. (1984). the timing of shifts of head postures during conservation. human movement science, 3(3), 237-245. doi:10.1016/0167-9457(84)90018-6 hall, e.t. (1966). the hidden dimension. new york: doubleday. http://dx.doi.org/10.1016/0167-9457(84)90018-6 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 42 halliday, m.a.k. & greaves, s. (2008). intonation in the grammar of english. london: equinox. hostetter, a. b., & alibali, m. w. (2010). language, gesture, action! a test of the gesture as simulated action framework. journal of memory and language, 63, 245-257, doi:10.1016/j.jml.2010.04.003 hincks, r., & edlund, j. (2009). promoting increased pitch variation in oral presentations with transient visual feedback. language learning & technology, 13(3), 32–50. kress, g., & van leeuwen, t. (1996). reading images: the grammar of visual design. london: routledge. kress, g., & van leeuwen, t. (2001). multimodal discourse: the modes and media of contemporary communication discourse. london: arnold. kress, g. (2005). gains and losses: new forms of texts, knowledge, and learning. computers and composition, 22(1), 5-22, doi:10.1016/j.compcom.2004.12.004 lakoff, g. & johnson, m. (1980). metaphors we live by. chicago and london: university of chicago press. lee, a. (2013, april). angela lee: grit: the power of passion and perseverance [video file].www.ted.com/talks/angela_lee_duckworth_grit_the_power_or_passion_and_perseverance. loya, m. & klemm, t. (2016). teaching note-using ted talks in the social work classroom: encouraging student engagement and discourse. journal of social work education, 52, 4, 518-523, doi: 10.1080/10437797.2016.1198291 mcclave, e. (2000). linguistic functions of head movements in the context of speech. journal of pragmatics, 32, 855-78. mcgregor, a., zielinski, b., meyers, c., & reed, m. (2016). an exploration of teaching intonation using a ted talk. in j. levis, h. le., i. lucic, e. simpson, & s. vo (eds.), https://doi.org/10.1080/10437797.2016.1198291 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 43 proceedings of the 7th annual conference of pronunciation in second language leaning and teaching (psllt) (pp.143-159). ames ia: iowa state university. mcneill, d. (1992). hand and mind: what gestures reveal about thought. chicago: university of chicago press. mcneill, d. (2005). gesture and thought. chicago: university of chicago press. norris, s. (2004). analysing multimodal interaction: a methodological framework. new york: routledge. kendon, a. (1980). gesticulation and speech: two aspects of the process of utterance. in mary. r. key (ed). the relationship of verbal and non-verbal communication (pp 20727).the hague: mouton. kendon, a. (2002). some uses of the head shake. gesture, 2(2), 147-82. pink, d. (2019, july). dan pink: the puzzle of motivation are [video file]. retrieved from https://www.ted.com/talks/dan_pink_on_motivation. pinker, s. (2014). the sense of style: the thinking person’s guide to writing in the 21st century. new york: penguin books. poyatos, f. (1983). language and nonverbal sign systems in the structure of face to face interaction. language and communication, 3(2), 129-40. robinson, k. (2006, february). ken robinson: do schools kill creativity? [video file]. retrieved from https://www.ted.com/talks/ken_robinson_says_schools_kill_creativity. rubenstein,l. (2012) using ted talks to inspire thoughtful practice. the teacher educator, 47(4), 261-267, doi: 10.1080/08878730.2012.713303. sinek, s. (2009, september). simon sinek: how great leaders inspire action [video file]. retrieved from www.ted.com/talks/simon_sinek_how_great_leaders_inspire_action. https://www.ted.com/talks/ken_robinson_says_schools_kill_creativity https://doi.org/10.1080/08878730.2012.713303 http://www.ted.com/talks/simon_sinek_how_great_leaders_inspire_action multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 44 sueyoshi, a., & hardison, d. (2005). the role of gestures and facial cues in second language listening comprehension. language learning, 55(4), 661-699, doi: 10.1111/j.0023-8333.2005.00320.x sugimoto, c., & mike t. (2013). scholars on soap boxes: science communication and dissemination via ted videos. the journal of the american society for information science & technology, 64(4), 663-74. takaesu, a. (2013). ted talks as an extensive listening resource for eap students. language education in asia, 4(2), 150-162, doi:10.5746/leia/13/v4/i2/a05/takaesu valeiras-jurado, j. (2017). a multimodal approach to persuasion in oral presentations : the case of conference presentations, research dissemination talks and product pitches. universitat jaume i ; ghent university. faculty of arts and philosophy, castellón de la plana, spain ; ghent, belgium van edwards v. (2015). 5 secrets of a successful ted talk. retrieved from: http://www.scienceofpeople.com/2015/03/secrets‐of‐a‐successful‐ted‐talk/. van leeuwen, t. (1999). speech, music, sound. london: macmillan. van leeuwen, t. (2005). introducing social semiotics. london/new york: routledge. van leeuwen, t. (2011). the language of colour. oxon/ new york: routledge. van lier, l. (2007). action-based teaching, autonomy and identity. international journal of innovation in language learning and teaching, 1(1), 46-65, doi: 10.2167/illt42.0 weinberg, a., fukawa-connelly, t., & wiesner, e. (2013). instructor gestures in proofbased mathematics lectures. in m. martinez, & a. castro superfine (eds.), proceedings of the 35th annual meeting of the north american chapter of the international group for the psychology of mathematics education (p. 1119). chicago: university of illinois at chicago. http://dx.doi.org/10.1111/j.0023-8333.2005.00320.x http://www.scienceofpeople.com/2015/03/secrets-of-a-successful-ted-talk/ http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.134.3208&rep=rep1&type=pdf multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11370 social and technological sciences issn: 2341-2593 garcía-pinar (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 25-45 | 45 young, k. & travis, h. (2012). oral communication: skills, choices and consequences. long grove, il: waveland press, inc. multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 61 the level of psychological and social adjustment among a sample of persons with learning disabilities in asir region in light of some variables mohamad ahmad saleem khasawneh assistant professor, special education department, king khaled university, saudi arabia, mkhasawneh@kku.edu.sa received: 30 july 2020; accepted: 05 september 2020; published: 0ctober 2020 abstract this study aimed at investigating the level of psychological and social adjustment in relation to the grade and type of learning disability among persons with learning disabilities in the asir region. the study was applied to a sample of (350) students studying in schools in the department of education in asir. the sample responded to the psychological and social adjustment scale developed as a tool of the study. the results showed that the level of psychological and social adjustment came with a medium degree, as well as the absence of statistically significant differences in the level of psychological and social adjustment students in the basic stage. in light of the results, the researcher recommended preparing awareness programs for teachers of learning disabilities to identify the various indicators that accompany the phenomenon of psychological and social adjustment for persons with learning disabilities. the study also recommends conducting more studies on psychological and social adjustment and its relationship to other variables, such as emotional intelligence. keywords: learning disabilities, psychological and social adaptation https://creativecommons.org/licenses/by-nc-nd/4.0/ mailto:mkhasawneh@kku.edu.sa https://orcid.org/0000-0002-1390-3765 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 62 1. introduction learning disabilities are among the serious problems experienced by children at school. psychological and academic pressures facing students with learning disabilities lead to an imbalance in their psychological and social adaptation. this category of students shows clear patterns of difficulties in psychological and social adaptation, as a result of their academic problems (mahmoud, 2017). attention to social aspects and emotions of persons with learning disabilities is of vital importance because the social experiences that children go through play a major role in influencing their feelings of competence, their awareness of themselves, or their expectations of themselves (pearl & bryan, 1992). social problems prevail among persons with learning disabilities. researchers argue that they have serious problems in acquiring social skills, which leads to creating difficulties in social relations and adjustment with others, isolation, and lack of good communication with others (al-khasawneh, 2017). children who have learning disabilities show many behavioral and emotional disturbances such as withdrawal, depression, and poor self-esteem. they also have a tendency to external attribution, as they attribute their success to causes beyond themselves. they expect failure and underestimate themselves and believe that they will not achieve success regardless of what they do (al-khasawneh, 2018). the question here arises regarding whether social disorders are part of learning disabilities, or they are a form of misalignment as a result of learning disabilities. the answer might be in the unclear link between the two, as social disorders may not be directly related to the level of learning disability, and appropriate learning is often linked to good performance, even in the absence of psychological or behavioral therapy (ray & elliott, 2006). many factors hinder the development of the social aspect of children with learning disabilities, such as factors related to the existence of problems in the stage of processing information, immaturity, poor attention, excessive activity, lack of self-esteem, and difficulty expressing feelings. therefore, the current study attempts to reveal the level of psychological and social adjustment of persons with learning disabilities in the asir region in relation to the variables of the grade and type of learning disability. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 63 1.1. research questions the present study attempts to answer the following questions: • the first question: what is the level of psychological and social adjustment of students with learning disabilities in the asir region? • the second question: are there any statistically significant differences in the level of psychological and social adjustment due to the variables of the type of learning disability (reading, writing, math) and the grade (third, fourth, fifth, sixth) and the interaction between them? 1.2significance of the study the significance of this study stems from two aspects. first, the scientific aspect, where the value of the results of the study in identifying the level of psychological and social adjustment of students with learning disabilities in the region. this study links psychological and social adjustment to their impact on students with learning disabilities. the practical aspect, as this study opens a field for future research that is interested in alleviating psychological and social anxiety among students with learning disabilities. the results are expected to benefit interested researchers by providing them with a theoretical background on the relationship between psychological and social anxiety and learning disabilities. results will provide the literature with an important measure to determine the level of psychological and social adjustment to be used by researchers and specialists to identify the problems experienced by their students. 1.3 delimitations of the study the generalization of findings is highly related to the size of the study sample. the present study is limited to students with learning disabilities. the selected students' age ranges from (8-11) years and no other ages have been selected. the sample of this study is limited to students, who enrolled in the rooms of learning difficulties of schools that belong to the department of education in the asir region in the school year 2018-2019. the study is also limited by the psychometric characteristics of the used instrument. the results of this study are determined by the ability of teachers of learning disabilities rooms to measure the responses of students with learning disabilities on the scale that was prepared for this purpose. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 64 1.4 definition of operational terms social adjustment is "a process or result of organic changes in the social organization, where the group contributes to achieving survival or continuity in achieving the goal that the organism seeks" (al-anani, 2015, 32). it is defined procedurally as the degree obtained by students on the social adjustment scale prepared by the researcher for the purposes of this study. physiological adjustment is the harmonization between the person and himself, which means that the person is satisfied with himself and receptive to it with the relative liberation from the conflicts and tensions that are associated with feelings of guilt, lack, anxiety, distress, and personal objectives (greenberg & murray, 2006). it is defined procedurally as the degree obtained by students on the psychological adjustment scale prepared by the researcher for the purposes of this study. learning disabilities students: "they are students who demonstrate disturbances in one or more of the basic psychological processes that include understanding and using the written or spoken language, which appears in the disorders of hearing, thinking, speech, reading, spelling, and arithmetic, which are due to reasons concerning simple functional brain injury, but it is not due to causes related to mental, auditory or visual disabilities or other disabilities”(al-khasawneh, 2018). as for the procedural definition, these students are officially diagnosed by the ministry of education are considered to have learning disabilities, and they are enrolled in learning difficulties rooms in schools of the department of education in the asir region. 2.literature review social and psychological adjustment among persons with learning disabilities educational institutions play an important role in supporting psychological and social adaptation among students. all educational institutions seek to eliminate the impacts of psychological and social anxiety among students of different levels and ages. the failure to satisfy the needs of students and meet their desires only means failure to achieve academic adjustment with their educational environment (darwish, 2010). students who face problems and difficulties in their academic and educational lives suffer from a lack of psychological adaptation to themselves, in addition to problems in social adjustment in their schools and their local environment. these circumstances affect their mental health and psychological security and result in the emergence of https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 65 feelings of dissatisfaction with the self and others, which increases the severity of their problems. eventually, this may lead to the feeling that they are socially rejected because of these difficulties (sultan, 2009). ray & elliott (2006) argued that students with educational and behavioral disabilities tend to exhibit lower levels of social adjustment than those of their peers with normal academic achievement. students with learning disabilities also suffer from problems in the field of social communication, forming friendships with others, and accepting criticism directed at them. the child's ability to acquire social skills is one of the vital actors in the learning process, as evidence indicates that many children with learning disabilities lack these skills, and they do not enjoy good acceptance from their peers. they are also less controlled socially compared to their regular peers. they fail to fulfill the basic social requirements of daily life, although they have the standard child's level in many aspects such as verbal intelligence (ali and hassan, 2009). therefore, studying the psychological and social factors of students with learning disabilities is of crucial importance. this category is characterized by a number of social and academic features that affect their adaptive ability compared to their regular peers. however, there is a scarcity of studies and research that have tried to study psychological and social adjustment among students with learning disabilities in the asir region, which gives a clear indication of the significance of conducting this study. the literature shows many studies conducted to investigate the psychological and social adjustment among students with learning disabilities. wolodiger and pope (2019) investigated the reciprocal relationships between parenting stress and psychosocial adjustment of children with congenital craniofacial anomalies (cfas) at 2-time points: school entry and approximately 2.5 years later after children had time to adjust to school. the study sample consisted of 42 parents of children. this study was based on a retrospective review of medical charts of children with congenital cfas who were patients of a department of reconstructive plastic surgery at a large urban medical center serving a regional population. the study showed that compared to norms, more parents scored in the clinical range on parenting stress both at time 1 and time 2. parenting stress remained stable across the 2-time points. the study concluded that early school years may be a period that is particularly stressful for parents of children with cfas. there appears to be a transactional https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 66 relationship between parenting stress and child psychosocial adjustment during the early school years. 2.1 previous studies metsala et al (2017) examined emotion recognition, understanding, and regulation in a clinicreferred group of young children with nld (nonverbal learning disabilities). these processes are related to social competence and psychological adjustment in td (typically developing) children. the study followed the cognitive method and consisted of 10 children with nld between 6.3 and 9.6 years old and 4 td. the nld was recruited through a children’s mental health center serving children with learning disabilities and also through local pediatricians. the results of the study showed that the young children with nld scored lower than the td comparison group on tasks assessing recognition of happy and sad facial expressions and tasks assessing understanding of how emotions work. adeyemi (2017) investigated the correlational indicators of psychosocial adjustment among senior secondary school students in ogun state, nigeria. the study adopted the correlational research design. a sample of 700 students was randomly selected. a reliable instrument was used to elicit the data. the data were analyzed using pearson correlation and multiple linear regression analysis. the result revealed that the independent variables significantly correlated with students’’ psychosocial adjustment. there was a significant joint contribution of the independent variables (sense of coherence, optimism, and self-efficacy) to the prediction of psychosocial adjustment. fakolade and oyedokun. (2017) examined different psychosocial variables as predictors of school adjustment of 40 gifted students with learning disabilities from four schools from 13 junior secondary schools in ikenne local government council area of ogun state, nigeria. the study applied six assessment instruments to the participants. results demonstrated that both emotional intelligence and self-concept were potent factors that could predict school adjustment of gifted students with learning disabilities. socioeconomic status was a significant factor in influencing school adjustment of gifted students with learning disabilities. gender also had a differential effect on school adjustment of students with learning disabilities. the study concluded that the adjustment of school children with learning disabilities is determined by their gender, self-concept, emotional https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 67 intelligence, the class in which they study, their school’s medium of instruction, school management, and their parent’s education and occupation. khataibeh (2013) identified the level of psychological and social adjustment among children with learning disabilities and slow learning students in amman. the study sample consisted of (222) male and female students; (182) students with learning disabilities and (40) students with slow learning, who were chosen randomly. the study used a psychological and social adjustment scale and administered to the sample of the study. the results revealed that the level of psychological and social adjustment for children with learning disabilities and slow learning students was average. the results also showed significant differences between the responses of the sample in relation to the gender variable. there were also significant differences due to the type of learning disability. villegas et al. (2012) conducted a study to reveal the severity of the risks of social anxiety and depression on social adjustment between a group of mexican students with learning disabilities and another group of students without learning disabilities. the study was applied to a sample of (260) students from the fourth and fifth grades aged 9-12 years old. after distributing a scale to measure anxiety and depression, the results yielded high rates of anxiety and depression among children with learning disabilities compared to their normal peers. the results showed no statistical differences attributed to gender between the participants of the study in relation to social adaptation. the study showed that working to increase awareness among students with learning disabilities leads to a reduction in the rate of depression and anxiety, as well as improving their social adaptation. reed et al. (2011) investigated the effect of a set of friendship-related variables between third and fifth graders with learning disabilities and students without learning disabilities with their social adaptation. the study sample consisted of (30) students from the third and fifth-grade students in an elementary school in the northeast of the united states of america. the study used a self-esteem scale and self-efficacy survey as instruments of the study. the results revealed that students with learning disabilities enjoy a good level of self-efficacy and such students tend to make friends with other students with learning disabilities but they do not choose one of these students as their favorite friend. the results showed that students who have learning disabilities had a high level of behavior problems and a low level of integration, social adaptation, and academic performance compared to their peers who do not have learning disabilities. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 68 yezreel & israel (2010) conducted a study aimed at detecting social anxiety and social adaptation among arab adolescents with learning disabilities and their normal peers without learning disabilities. the sample of the study consisted of (70) arab christian students in israel from the seventh-grade students aged (12-13) years. the study used the social anxiety scale and the social adjustment scale as instruments. the results revealed that the level of social anxiety and social adaptation of students with learning disabilities, who received special educational services, is high compared to students without learning difficulties. the results also showed that the percentage of social anxiety and social adaptation among males was higher than females. greenberg and murray (2006) identified the social adjustment of children with learning disabilities through their relationships with their parents, peers, teachers, and friends at school. the study sample consisted of (96) fifth and sixth graders receiving education services. the study was conducted in schools in the northwest of the united states of america. the results showed that the positive and negative relationships and ties of children with learning disabilities are closely related to social, behavioral, and emotional adaptation. the different characteristics of the ties and relationships of children with learning disabilities are related differently to the adaptation variables. the results showed no statistical differences in social adjustment among children due to the variable of attendance to school. wainright et al (2004) investigated the associations among family type (same-sex vs. opposite-sex parents); family and relationship variables; and the psychosocial adjustment, school outcomes, and romantic attractions and behaviors of adolescents. participants included 44 12to 18-year-old adolescents parented by same-sex couples and 44 same-aged adolescents parented by opposite-sex couples, matched on demographic characteristics and drawn from a national sample. data employed in this study were collected through the in-home interviews and surveys, as well as inschool surveys of students (collected from 1994 to 1995) and through the in-home questionnaires of parents. normative analyses indicated that, on measures of psychosocial adjustment and school outcomes, adolescents were functioning well, and their adjustment was not generally associated with family type. the results of this study showed that on nearly all of a large array of variables related to school and personal adjustment, adolescents with same-sex parents did not differ significantly from a matched group of adolescents living with opposite-sex parents. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 69 sorensen et al. (2003) investigated the problems of psychological and social adjustment for children with learning disabilities. the study used the psychological and social adjustment scale to assess the relationship between the psychological and social adjustment and changes in academic skills and environmental factors among (100) students with ld aged (7-11) years old. the results of the study indicated that environmental factors have a clear role in the different levels of psychological and social adjustment of students with learning disabilities. the results also showed that the academic skills of students with learning disabilities are better when they have a better level of psychological and social adjustment. these previous studies tackled the psychological and social adjustment among students with learning disabilities from different perspectives. the current study comes in line with the literature to investigate this issue. however, and to the best knowledge of the researcher, no studies conducted in the kingdom of saudi arabia, especially in the asir region, attempted to reveal the level of psychological and social adaptation among students with learning disabilities, thus arise the significance of this study. 3.research methodology this section provides information on the methodology of the present study which includes: population and sample, research instrument, and validity and reliability of the instrument 3.1 population and sample the study population consisted of 370 students enrolled in resource rooms with learning disabilities in schools of the department of education in the asir region from the elementary grades: third, fourth, fifth and sixth. the study sample consisted of the same study population. the researcher distributed (370) questionnaires to measure the level of psychological and social adjustment. information was obtained with the assistance of the learning disabilities teacher, arabic teacher, and physical education teacher; this is because of the difficulties of reading and writing that students with learning disabilities have. table (1) shows the distribution of the study sample according to the grade and the type of learning disability. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 70 table (1): distribution of study sample according to grade and type of learning disability grade grade no. third 60 fourth 104 fifth 116 sixth 90 type of learning disability type of learning disability no. reading 158 writing 182 math 30 total 570 3.2 ethical considerations the official ethical consent was obtained from the ministry of education. participants were first contacted and invited to participate in the study, and then were briefed on the objectives of the study. consent of the participants was obtained before conducting the study and they were assured that their identity and responses stay confidential. 3.3 instrumentation: 3.3.1 survey to achieve the objectives of the current study and to identify the level of psychological and social adjustment among students with learning disabilities in asir, the researcher developed a scale of psychological and social adjustment by referring to theoretical literature and previous studies (alhajri, 2002; sorenson et al 2003; al-qudah; 2004); awwad and sharbat, 2004; khazaleh and alkhatib, 2011; al-miqdad et al, 2011; reid et al, 2011). the paragraphs of the scale were built according to the following steps: • clarifying the goals and purpose and preparing sources and references that were used in developing the scale. • selecting the paragraphs that indicate psychological and social adjustment, developing them, and defining the responses according to the likert five-point scale. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 71 in light of this, the dimensions and paragraphs of the questionnaire were prepared, which consisted in its final form (40) paragraphs (see appendix 1). 3.3.2 validity the validity of the content was achieved by presenting it to (10) judges with expertise and specialization from the faculty members at king khaled university. the consensus of 80% of the judges was the criterion for accepting any paragraph. their observations and suggestions were taken into consideration to amend, delete, or add some paragraphs, and as the scale was in its final form (40) paragraphs. 3.3.3 reliability to ensure the reliability of the scale, the test-retest method was used by applying it to a pilot sample of (40) students with learning disabilities, and with a two-week interval between the first and second applications. pearson's correlation coefficient between the scores on the two applications was calculated. the correlation coefficient of the scale as a whole was (0.835), and the stability coefficient was calculated using the cronbach alpha, as the coefficient of stability by the method of the internal consistency of the scale as a whole was (0.816), which indicated that the scale has an appropriate level of reliability as shown in table (2). table (2): reliability coefficient using the internal consistency "cronbach alpha" and test-retest for the overall score of the scale method reliability coefficient test-retest 0.835 cronbach alpha 0.816 https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 72 4. data analysis the returned questionnaire was recorded and tabulated with the assistance of statistical package for social sciences (spss) for windows 17.0 in order to identify the correlated relationships of variables concerning the psychological and social adjustment. different statistical methods were used to achieve the main objectives of the present investigation. these methods included descriptive statistics and independent sample t-test. descriptive statistics, including mean scores, standard deviation, and frequencies, were employed to calculate the demographic data of the psychological and social adjustment in terms of grade and type of learning disability. an independent sample t-test is a statistical method employed to demonstrate the variations among the mean scores of two groups of a variable. in the current study, this statistical method was used in order to identify the significant differences between the levels of psychological and social adjustment among students with learning disabilities and their grade and type of learning disability. an analysis of variance (anova) is a method of statistical analysis used to determine differences among the means of more than two groups of a variable (corbin, strauss, & strauss, 2014). in the present study, this statistical method was used to determine the relationship between the psychological and social adjustment among students with learning disabilities and their grade and type of learning disability. 5. findings and discussion this section provides the findings obtained from the present study. these findings are presented based on the research questions that guide the present research. first: results related to answering the first question: “what is the level of psychological and social adjustment among students with learning disabilities in the asir region?” to answer this question, mean scores and standard deviations for the level of psychological adjustment have been extracted as shown in table (3). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 73 table (3): mean scores and standard deviations for the level of psychological adjustment of students with learning disabilities in the asir region as arranged descendingly according to the mean score par. no. order paragraph mean score st. dv level 6 1 he tends to befriend his younger peers 3.38 0.949 medium 18 2 he needs a friend 3.29 0.967 medium 1 3 he likes his classmates 3.23 0.860 medium 17 4 his classmates mock him 3.22 0.767 medium 4 5 he feel sadness 3.21 0.983 medium 9 6 he feels wronged 3.20 0.937 medium 2 7 he feels embarrassed when he fails 3.13 0.849 medium 10 8 he is stubborn 3.12 1.081 medium 8 9 he makes an effort to excel in his studies 3.11 0.743 medium 11 10 he seems self-confident 3.09 1.067 medium 12 11 he frequently misses school 2.97 1.059 medium 16 12 he is irritable 2.95 1.130 medium 7 13 he is discharged during class 2.94 0.738 medium 19 14 he quickly feels bored 2.90 0.970 medium 20 15 he makes his decision without help 2.83 1.007 medium 13 16 he has temperament 2.81 1.063 medium 5 17 he seems worried 2.79 1.000 medium 3 18 he assaults others 2.72 1.006 medium 14 19 he feels good when his homework ends 2.70 1.065 medium 15 20 he performs his homework even if it is difficult 2.66 1.030 medium table (3) shows that the individuals of the sample possess a medium level of psychological adjustment, as the mean score of psychological adjustment as a whole was (3.01) and a standard deviation of (0.751). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 74 to reveal the level of social adjustment among students with learning disabilities, mean scores, standard deviations, and the relative importance of each of the paragraphs of the field of social adjustment were used for students with learning disabilities, and table (4) illustrates the results table (4): mean scores and standard deviations for the level of social adjustment of students with learning disabilities in the asir region ordered descendingly according to the mean scores par. no. order paragraph mean score st. dv level 7 1 he share ideas with classmates 3.53 0.833 medium 10 2 he tends to be isolated 3.47 0.946 medium 20 3 he establishes a positive relationship with his classmates 3.46 0.796 medium 8 4 he accepts the mood swings of his classmates 3.44 0.805 medium 19 5 he shares games with classmates 3.43 0.794 medium 9 6 he organizes his study times 3.41 0.960 medium 18 7 he speaks the words correctly 3.35 0.897 medium 16 8 he tends to be independent 3.34 0.825 medium 2 9 he has fun during the school break 3.28 0.809 medium 4 10 he is keen to please his teachers 3.25 0.833 medium 3 11 he disagrees with his classmates 3.24 0.841 medium 1 12 he has the power to persuade his colleagues 3.16 0.823 medium 13 13 he respects his classmates 2.86 0.801 medium 5 14 he stays away from his classmates 2.70 0.877 medium 11 15 he lacks many friendships at school 2.62 0.861 medium 15 16 he likes to be close to his teachers 2.60 0.864 medium 6 17 he happily follows the teacher's instructions 2.59 0.814 medium 17 18 he is confronted with irony because of his appearance 2.55 0.834 medium 12 19 he is happy when participating in school trips 2.53 0.946 medium 14 20 his friends depend on him 2.50 0.786 medium https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 75 table (4) shows that the individuals of the sample possess an average level of social adjustment, as the mean score of social adjustment as a whole was (3.07) and a standard deviation of (0.694). the following table (5) summarizes the values of the mean scores, the standard deviations, and weights for both the psychological adjustment and social adjustment for students with learning disabilities. table (5): mean scores and standard deviations of differences in psychological and social adjustment for students with learning disabilities no. dimension mean score st.dv. weight level 1 psychological adjustment 3.01 0.751 60.2% متوسط 2 social adjustment 3.07 0.694 61.4% متوسط total score 3.04 0.637 60.8% متوسط table (5) reveal that the level of psychological and social adjustment of students with learning disabilities was average, as the mean score was (3.04) and the total weight was (60.8%). the mean scores ranged between (3.01 3.07), and the dimension of social adjustment came first with a mean score of (3.07) and a weight of (61.4%) while the dimension of psychological adjustment had a mean score of (3.01) and a weight of (60.2%). second: the results related to answering the second question: are there any statistically significant differences in the level of psychological and social adjustment due to the variables of the type of learning disability (reading, writing, math) and the grade (third, fourth, fifth, sixth) and the interaction between them? to answer this question, mean scores and standard deviations for the level of psychological and social adjustment have been extracted according to their grades and the type of learning disability as shown in table (6). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 76 table (6): mean scores and standard deviations for the level of psychological and social adjustment according to their grades and the type of learning disability scale type of ld reading writing math grade mean score st. dv mean score st. dv mean score st. dv psychological adjustment third 3.04 0.862 3.01 0.942 2.98 0.866 fourth 3.02 0.931 3.03 0.695 3.03 0.791 fifth 3.05 0.792 2.94 0.772 3.06 0.725 sixth 2.97 0.864 2.94 0.824 3.05 0.864 social adjustment third 3.08 0.772 3.09 0.894 3.11 0.965 fourth 3.12 0.862 3.07 0.795 3.04 0.874 fifth 3.11 0.759 3.05 0.886 3.08 0.825 sixth 2.95 0.902 3.09 0.697 3.05 0.877 table (6) shows that there are apparent differences between the mean scores of students on the psychological adjustment scale according to the study variables. to identify the significance of these differences, two-way anova has been conducted as shown in table (7). table (7): results of the two-way anova analysis source of variance sum of squares degree of freedom squares average f value sig. grade psychological adjustment 0.078 3 0.026 0.216 0.885 social adjustment 0.206 3 0.069 0.369 0.776 type of ld psychological adjustment 0.220 2 0.110 0.915 0.401 social adjustment 0.204 2 0.102 0.547 0.579 grade*type of ld psychological adjustment 0.295 4 0.074 0.613 0.653 social adjustment 0.234 4 0.058 0.313 0.869 error psychological adjustment 42.108 350 0.120 social adjustment 65.279 350 0.187 total psychological adjustment 4830.740 360 social adjustment 4337.438 360 https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 77 it appears from the previous table that the values of the significance levels were greater than (0.05) for all variables, which indicates that there are no statistically significant differences between the mean scores students on the psychological and social adjustment scale due to the grade or type of learning disability. the result could be attributed to the great interest of the ministry of education represented by the department of education in asir region to the instructional programs and the availability of specialists in the field of learning disabilities in various schools to address the various psychological and behavioral disorders experienced by students with learning disabilities. these efforts contributed to addressing many of the students' behavioral disorders, which helped to raise the level of psychological and social adjustment among the sample of the study. the result of this question may be attributed to the existence of modern educational methods that are far from corporal punishment in various schools. perhaps this is due to the fact that students with learning disabilities succeed in learning some skills, and fail to learn other skills, which indicates that they have a variation in educational capabilities and reflects the difference between achievement and intelligence. the presence of the ability of these students to learn and their educational levels helped them with psychological and social adjustment. students with learning disabilities may have skills and mental abilities that do not exist in others, which gives them popularity among their classmates and increases their psychological and social adjustment. 6.conclusion and recommendations this study found that the level of psychological and social adjustment of students with learning disabilities came with a medium degree. there were no statistically significant differences in the degree of psychological and social adjustment among students with learning disabilities in the basic stage due to their grade and type of ld. the study emphasizes the urgent need for educational programs to support raising the level of psychological and social adjustment of this category of students. in light of the results, the study recommends the following: • preparing awareness programs for teachers to identify various indicators (educational and demographic) that accompany the low level of psychological and social adjustment of students with learning disabilities. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 78 • supporting the provision of counseling programs for students in primary schools in general, and learning disabilities rooms in particular. • conducting more studies on the level of psychological and social adjustment among students with learning disabilities and its relationship to other variables, such as social anxiety and emotional intelligence. 7. acknowledgments this study was sponsored by the research group program of the scientific deanship at king khalid university, kingdom of saudi arabia (r.g.p.il34l40). references adeyemi, a. o. (2017). correlational indicators of psychosocial adjustment among senior secondary school students in ogun state, nigeria. ife psychologia: an international journal, 25(1), 532-555. al-anani, h. a. (2015). psychological health. amman: dar al-fikr publishers and distributors. ali, i. and hassan, h. (2009). learning disabilities between theory and practice. amman: al-manhaj house for publishing and distribution. al-khasawneh, m a. (2017). developmental learning disabilities (evaluated book). amman: dar al-fikr publishers and distributors. al-khasawneh, m. a. (2018). introduction to learning disabilities (evaluated book). amman: dar al-fikr publishers and distributors. al-khazaleh, a. k and al-khatib, j m. (2011). the social and emotional skills of students with learning disabilities and their relationship to some variables. journal of educational sciences studies, university of jordan, 38(1) pp. 372-389. al-miqdad, q. batayneh, o. and al-jarrah, a. (2011). the level of social skills among ordinary children and children with learning disabilities in jordan from the teachers’ point of view. journal of educational sciences, university of jordan. 7(3): 253-270. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 79 al-qudah, m. (2004). psychosocial adjustment among students with learning disabilities in the city of amman and its relationship to academic achievement, gender, and school type. unpublished master thesis, amman arab university, amman, jordan. al-sultan, i. (2009). social support and stressful life events. amman: safa house for publishing and distribution. darwish, z. (2010). social psychology: principles and applications. cairo: dar al-fikr al-arabi. fakolade, o. a., & oyedokun, s. o. (2017). psychosocial variables as predictors of school adjustment of gifted students with learning disabilities in nigeria. editorial staff, 34. greenberg, m & murray, c. (2006). examining the importance of social relationships and social contexts in the lives of children with high-incidence disabilities, the journal of special education, 39(4), pp, 220-233. https://doi.org/10.1177/00224669060390040301 khataibeh, j. f. (2013). psychological and social adjustment among students with learning disabilities and slow learning students in amman governorate. unpublished master thesis, amman arab university, amman, jordan. mahmoud, m. i. (2017). upnormal children. amman: arab society library for publishing and distribution. metsala, j. l., galway, t. m., ishaik, g., & barton, v. e. (2017). emotion knowledge, emotion regulation, and psychosocial adjustment in children with nonverbal learning disabilities. child neuropsychology, 23(5), 609-629. pearl, r., & bryan, t. (1992). students' expectations about peer pressure to engage in misconduct. journal of learning disabilities, 25(9), 582-585. https://doi.org/10.1177/002221949202500907 ray، c. & elliott, s. (2006). social adjustment and academic achievement: a predictive model for students with diverse academic and competencies. school psychology review, 35 (3): 493501. doi: 10.1080/02796015.2006.12087980 reed, f., mcintyre, l., dusek, j., quintero, n.(2011). preliminary assessment of friendship, problem behavior, and social adjustment in children with disabilities in an inclusive education setting, j dev phys disabil, 23, pp, 477–489. https://doi.org/10.1007/s10882-011-9236-2 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1177%2f00224669060390040301 https://doi.org/10.1177%2f002221949202500907 https://doi.org/10.1080/02796015.2006.12087980 https://doi.org/10.1007/s10882-011-9236-2 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 80 sorensen, l., g., forbes, p., w., bernstein, j., h., michael, m., w., mitchell, and waber, d., p. (2003). psychological adjustment over two-year period in children referred learning problems: rish, resilience, and adaptation. learning disabilities research and practice, 18(1), 1-21. https://doi.org/10.1111/1540-5826.00054 villegas, d., langley, a., gallegos, j. (2012). anxiety, depression, and coping skills among mexican school children: a comparison of students with and without learning disabilities, learning disability quarterly,35(1), pp, 54–61. https://doi.org/10.1177/0731948711428772 wainright, j. l., russell, s. t., & patterson, c. j. (2004). psychosocial adjustment, school outcomes, and romantic relationships of adolescents with same-sex parents. child development, 75(6), 1886-1898. wolodiger, e. d., & pope, a. w. (2019). associations between parenting stress at school entry and later psychosocial adjustment: a longitudinal study of children with congenital craniofacial anomalies. the cleft palate-craniofacial journal, 56(4), 487-494. yezreel, e., israel. (2010). social anxiety among arab adolescents with and without learning disabilities in various educational frameworks, british journal of guidance & counselling, 39(2), pp, 161_177. https://doi.org/10.1080/03069885.2010.547053 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1111/1540-5826.00054 https://doi.org/10.1177%2f0731948711428772 https://doi.org/10.1080/03069885.2010.547053 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 81 appendix (1) the psychological and social adaptation scale dear teachers, greetings, the researcher is conducting a study aiming to reveal the psychological and social adjustment among students with learning disabilities in the asir area in light of some variables. please add (x) in the box that represents the level of your psychological and social adaptation. the results of the current study will be used only for scientific and academic purposes only and be used objectively. please fill in the following information: age: ( ) grade: ( ) the type of learning disability: simple ( ) moderate ( ) severe ( ) best regards, the researcher https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 82 first: psychological dimension no. items measuring psychological adjustment agree strongly agree i don’t know disagree disagree d strongly 1. he likes his classmates 2. he feels embarrassed when he fails 3. he assaults others 4. he feel sadness 5. he seems worried 6. he tends to befriend his younger peers 7. he is discharged during class 8. he makes an effort to excel in his studies 9. he feels wronged 10. he is stubborn 11. he seems self-confident 12. he frequently misses school 13. he has temperament 14. he feels good when his homework ends 15. he performs his homework even if it is difficult 16. he is irritable 17. his classmates mock him 18. he needs a friend 19. he quickly feels bored 20. he makes his decision without help https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14111 social and technological sciences issn: 2341-2593 khasawneh (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 61-83 | 83 second: social adjustment no. items measuring social adjustment agree strongly agree i don’t know disagree disagreed strongly 1. he has the power to persuade his colleagues 2. he has fun during the school break 3. he disagrees with his classmates 4. he is keen to please his teachers 5. he stays away from his classmates 6. he happily follows the teacher's instructions 7. he share ideas with classmates 8. he accepts the mood swings of his classmates 9. he organizes his study times 10. he tends to be isolated 11. he lacks many friendships at school 12. he is happy when participating in school trips 13. he respects his classmates 14. his friends depend on him 15. he likes to be close to his teachers 16. he tends to be independent 17. he is confronted with irony because of his appearance 18. he speaks the words correctly 19. he shares games with classmates 20. he establishes a positive relationship with his classmates https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 74 adoption of e-learning at higher education institutions: a systematic literature review rushna khalil awan1, gul afshan1 , atia bano memon2 1 business administration department, sukkur iba university, sukkur, pakistan 2 department of computer science at the university of sindh, jamshoro, pakistan correspondence: rushna.msmgt18@iba-suk.edu.pk; gul.afshan@iba-suk.edu.pk; atia.memon@usindh.edu.pk received: 02 february 2021; accepted: 13 september 2021; published: october 2021 abstract the concept of e-learning offers several benefits; however, the effective adoption of e-learning systems at higher education institutes (heis) is a relatively new concept and thus a challenging task. the current paper provides a comprehensive review of the extant literature on adopting e-learning systems at heis. using the prisma search technique, owing to the widespread adoption of e-learning since 2005, the relevant articles published from 2005 to 2020 are selected. the paper identifies and puts forward the level of compatibility and readiness of students and teachers in adopting e-learning, factors that motivate and hinder the adoption of e-learning respectively, benefits of adopting an e-learning system, and the strategies for implementing elearning at heis. in this realm of covid-19 and e-learning, this paper also envisages different strategies, policies, and recommendations for effectively implementing e-learning at heis. keywords: e-learning; digital learning; higher education institutions; higher education; ict in education to cite this article: khalil awan, r., afshan, g., bano memon, a. (2021). adoption of e-learning at higher education institutions: a systematic literature review. multidisciplinary journal for education, social and technological sciences, 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0003-0016-5721 https://orcid.org/0000-0001-6893-0931 https://doi.org/10.4995/muse.2021.15813 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 75 1. introduction with the continuous development of new and improved information and communication technology (ict) tools and applications, ict has changed all aspects of everyday life (kjellsdotter, 2020). it has drastically changed the way people undertake their personal and professional activities and has transformed the world into a global village wherein there is no geographic barrier for interpersonal and inter-organizational communication (memon & meyer, 2017). similarly, ict has brought significant reforms with positive effects and introduced a new trend in the education system. implementation of ict tools in the teaching-learning process offers a wide range of benefits that collectively lead to a better and more efficient understanding of complex scientific concepts and procedures that are difficult to understand otherwise. research shows that the adoption of ict tools in the teaching practices increases the students’ learning output, equips students with 21st-century skills of digital literacy, aids in teachers’ training, makes the course interesting, and prepares students well for competing in the global marketplace of information-rich society (sayaf et al., 2021). in addition, incorporating ict tools in delivering lectures aid teachers in adopting innovative teaching practices and simultaneously enhances learners’ curiosity and understanding, resulting in improved learning capacities and personal development (akçayır et al., 2016; al shuaili et al., 2020). furthermore, integrating advanced ict such as simulation and augmented reality in education enables a better understanding of the scientific and complex phenomenon. it thus improves students’ understanding as human nature and behavior are inclined to comprehend concrete aspects more readily than abstract ones (akçayır et al., 2016). given that the integration of technology enables the modification of human nature and cognition, in the current technological and digital literacy era, ict tools are often regarded as artificial organs along with the biological ones in the core of human beings that allow the humans to continue transforming the world and also transforming themselves (hasan et al., 2020; sánchez-sordo, 2019). it is believed that for some specific courses, it is challenging to keep students motivated by just delivering lectures in a traditional teaching style. in this way, sometimes, these courses are criticized as overly theoretical, uncreative, and non-innovative. thus students find it boring and difficult to develop understanding to the level they should do. therefore, in such instances, it becomes essential and beneficial to aid the lectures with advanced ict https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 76 instruments such as gamification, simulations, scenography, and problem-solving and simulating practices (erdmann & torres marín, 2019; gerbic, 2011). increasingly, the advancements in ict have introduced the concept of electronic learning (elearning). e-learning, varyingly termed as virtual learning, online education, digital learning, and distance education (singh & thurman, 2019), refers to the teaching-learning process in the online environment. with e-learning, geographic barriers to the sharing of knowledge sharing have dissolved. thus, it has become a flexible and affordable way of providing quality education to people with limited or no access to education facilities otherwise (wu et al., 2010). along with an array of benefits of e-learning, such as flexibility in terms of time and space, widespread sharing of knowledge resources, and support for social mobility, e-learning brings considerable challenges as well. the key challenges include the absence of a traditional classroom environment, lack of teacher’s control, unavailability of adequate ict facilities (e.g., computers, internet connectivity), and shortage of financial resources for the ict implementation (tarhini et al., 2014). the problems in implementing an e-learning system become even more in developing countries because of insufficient infrastructure. thus such countries are generally lagging in the race of e-learning adoption. during the current scenario of a global pandemic of covid-19, whereby the usual routine is disturbed in all sectors, the education sector has moved away from its regular operations. many parts of the world are experiencing complete or partial lockdown. at the same time, education institutions are closed, examinations are suspended, the teaching-learning process is interrupted, and students are deprived of learning due to restricted mobility. in such circumstances, it has become a need of the hour to envisage and discover the solutions that can help overcome the education loss and prepare the teachers and students for such situations arising in the future. as the pandemic was sudden and education stakeholders were unprepared for such a situation, it has been challenging to adopt any alternative education system such as e-learning. however, the developed countries with ample ict resources and technology-based society have been much able to resume their educational activities early in the pandemic without facing any significant obstacles or technological transformations. however, the developing countries lacking enough resources for this sudden shift to online mode face substantial issues in resuming their academic activities. sooner or later, many have now https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 77 switched to online mode to some level, but still, they are struggling with turning this into a very smooth, effective, and fruitful way. in this realm, this paper undertakes a systematic review of the extant literature on e-learning to understand the opportunities and challenges for adopting an e-learning system and envisage different strategies, policies, and recommendations for effectively implementing e-learning at higher levels education institutions. as a result, the paper offers a comprehensive guide for the government, administrators, and other education policymakers for conceptualizing, implementing, evaluating, and improving the e-learning systems at higher education institutions. the remainder of this paper is structured as follows. section 2 explains the methodological approach of this study. the results of the review are presented in section 3. the findings are further discussed in section 4, and the paper is concluded in section 5. 2. methodological approach 2.1. search strategy the searching of relevant literature was carried out following the systematic literature search methods employed in existing review articles (e.g., paul & criado, 2020; paul et al., 2021; williams et al., 2020). the search process began with the identification of relevant journals. the database used for searching the relevant journal and related articles was scopus. the search for relevant journals was performed first rather than directly exploring the pertaining literature to confine the study scope to some journals with high scores. the journal search yielded 18 relevant journals, then screened and filtered their respective journal scores and citation percentage. as a result, three journals with good scores (i.e., one or near one) were selected. table 1 presents the details of three selected journals included in this study. subsequently, the selected journals were searched for relevant articles published from 2005 to 2020, owing to the widespread adoption of e-learning since 2005. the scopus database was searched using the publish or perish software. the search keywords used for searching relevant articles were ‘e-learning in higher education institutes,’ ‘effective e-learning,’ ‘e-learning,’ and ‘technology in learning.’ all results were limited to the english language only. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 78 table 1 selected journals for inclusion in the review journal title database source cite score percentage cited international journal of educational technology in higher education https://educationaltechnologyjournal.springeropen.com/ scopus 2.77 71% australasian journal of educational technology https://ajet.org.au/index.php/ajet scopus 2.13 80% international journal of emerging technologies in learning https://online-journals.org/index.php/i-jet scopus 0.97 45% 2.2. inclusion and exclusion criteria the initial search with the keywords mentioned above yielded 87 possibly relevant articles that were successively screened and filtered for inclusion in this study based on four criteria: a) the article must be a peer-reviewed article published in one of the three selected relevant journals, b) the article must be in english language, c) the article must be published between 2005 and 2020, and d) the article must discuss e-learning as a central topical theme. the screening is done by going through abstracts only, and as a result, 50 articles were shortlisted. following this, full text of these identified articles was retrieved. subsequently, the second screening of these articles was done based on full text concerning the four inclusion criteria. the articles were excluded if they did not meet the inclusion criteria in their abstract, results, or discussion section. as a result of this screening, 15 articles were finally selected to be included in the review. the general search and screening process of the study is illustrated in figure 1. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://educationaltechnologyjournal.springeropen.com/ https://ajet.org.au/index.php/ajet https://online-journals.org/index.php/i-jet multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 79 figure 1. overall article search and screening process of the study 2.2. data analysis following the search and selection of relevant literature for the review, the next step involved undertaking thematic analysis of identified articles. to do so, a well-organized way was employed, similar to fereday and muir-cochrane (2006). accordingly, the data were extracted, summarized, and recorded in the spreadsheet. subsequently, the main themes regarding the broad concept of elearning were identified. the study identified five different themes that these articles are dealing with; i.e., students’ compatibility with e-learning (5 included articles), teachers’ compatibility with e-learning (2 included articles), factors motivating the adoption of e-learning environment (7 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 80 included articles), factors hindering the adoption of e-learning environment (3 included articles), and strategies for implementing effective e-learning (5 included articles). it is essential to mention that a particular article was found dealing with more than one central theme. table 2 provides details of the selected papers regarding study details, the number of citations, and the central theme that the study is focused on. table 2 details of the articles included in the study s.no. study no. of citations (as of may, 2020) main theme 1 kennedy et al. (2008) 1590 students' compatibility with e-learning 2 mtebe and raphael (2018) 42 factors motivating e-learning environment 3 (al-azawei et al., 2017) 106 students' compatibility with e-learning teachers' compatibility with e-learning 4 birch and burnett (2009) 193 factors motivating e-learning environment strategies for implementing effective elearning 5 holt et al. (2013) 24 strategies for implementing effective elearning 6 kasim and khalid (2016) 105 strategies for implementing effective elearning 7 naveed et al. (2017) 52 factors hindering the adoption of e-learning 8 huda et al. (2018) 142 factors motivating e-learning environment big data 9 kamba (2009) 78 factors hindering the adoption of e-learning strategies for implementing effective elearning 10 el-seoud et al., 2014 104 factors motivating e-learning environment students' compatibility with e-learning 11 (ngampornchai & adams, 2016) 88 students' compatibility with e-learning https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 81 12 ibrahim and nat (2019) 19 teachers' compatibility with e-learning factors motivating e-learning environment 13 (kim et al., 2019) 27 factors motivating e-learning environment 14 lange and costley (2020) 5 factors hindering the adoption of e-learning strategies for implementing effective elearning 15 pham et al. (2019) 45 students' compatibility with e-learning factors motivating e-learning environment 3. results 3.1. factors motivating e-learning environment studies have found that e-learning systems such as moodle, sakai, and blackboard are widely used as a new teaching method in heis (costa et al., 2012; ouatik & ouatik, 2021). e-learning is the software for managing the online distribution of knowledge (islam & azad, 2015; moore et al., 2011). the key features of e-learning, including online discussions, chatrooms, online quizzes, polls, and more access to educational content, are the main reasons students' increasing interest in this new technology (grimus, 2020). the study conducted at carnegie mellon university revealed that e-learning techniques result in better academic performance of students. while stressing the importance of e-learning, it is suggested that online classrooms make the student-teacher relationship more formal and less personal. this can reduce the chances of nepotism and favoritism that might occur otherwise in the traditional classroom, and hence this can motivate institutions to adopt more e-learning technologies. while discussing the global pressure of e-learning technologies, bates (2005) posits that technological advancement is crucial for higher education institutes to compete internationally. therefore it would not be illogical to state that one of the significant factors that motivate heis to adopt distance learning technologies is the international pressure. wu et al. (2010) assert that the considerable benefits of using e-learning technologies are the flexibility and accessibility of the information. other motivators behind using e-learning technologies is identified as saving learning https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 82 time and efforts, promoting students' intellectual abilities and skills, improving teacher-student interaction, and expanding students' knowledge (al-azawei et al., 2017). 3.2. students’ compatibility with e-learning as the world is moving towards digitalization at a breakneck pace, it is indispensable for every person to get hands-on the technology. particularly, the students belonging to the new generation are surrounded by digital gadgets like computers, mobile phones, video games, etc. (cha et al., 2020). these young students exposed to technology every moment are becoming pro at using the technology and can actively reap the benefits of e-learning tools. khan and umair (2017) mentioned that students (termed as digital natives) are gradually utilizing the technology for educational purposes due to their exposure to digital gadgets daily. it can be concluded that students are at ease while using elearning tools and can easily manage to get through the distance learning technology. at the same time, students can easily use general technologies but find difficulties in using specialized technologies (jones et al., 2010). it was revealed in the study that although students are pretty techsavvy, they still feel less comfortable while using technology in the classroom. similarly, kennedy et al. (2008) also revealed that students' inclination towards general technology does not reflect their willingness or expertise in using e-learning tools. comparing these studies with some relatively recent studies (e.g., al‐taweel et al., 2021; popoola & olajide, 2021) (it can be suggested that this impact is positive vice versa, as claimed in these recent studies that using the e-learning tools enhances students' computer expertise. while discussing factors contributing towards students' satisfaction from using e-learning, a survey by pham et al. (2019) conducted in vietnam suggests that e-learning system quality, e-learning instructor and course material quality, and e-learning administrative support quality enhance the students' satisfaction with the e-learning system. with the advancement in technology, students are getting more compatible with the elearning environment. if an effective way of utilizing the media and technology is employed, it can contribute to the better cognition of the students (mayer, 2014). 3.3. teachers’ perception of e-learning since we are moving from the teacher-centered model of learning to the learner-centered model, we need to empower the teachers more in every manner. to implement the e-learning system https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 83 effectively at heis, it is necessary to take the teachers on board. rasheed et al. (2020) posits that if a blended learning system (including e-learning and traditional learning) is used, it can help the heis function better cost-effectively. according to a study by birch and burnett (2009), while teaching through distance learning systems, teachers find it challenging to cater to the needs of generation y students, second language students, and students with disabilities. in a qualitative study by leo et al. (2021), teachers claimed that they had put an extra effort while teaching through e-learning systems during the recent pandemic of covid-19 and have to teach the students how to utilize these tools matching their principal learning style. the teachers further stated in a study that effective implementation of e-learning could only be achieved when students are thoroughly trained regarding the use of the technology being employed by heis (birch & burnett, 2009). 3.4. barriers affecting the implementation of e-learning the e-learning methodology of teaching is gradually gaining the edge, and soon its prevalence will surpass the traditional teaching methodology. still, there are specific barriers that are affecting the successful implementation of e-learning systems. these barriers are manifold; barriers from the students' side, instructors', institutional, and technological barriers. studies (mehall, 2021; wu et al., 2010) have shown that the absence of face-to-face interaction and lack of classroom environment are the major obstacles that hinder the successful implementation of e-learning systems. similarly, a study (naveed et al., 2017) conducted on heis in saudi arabia identified 16 significant obstacles affecting e-learning systems' implementation. the instructors' barriers included the lack of ict skills, e-learning knowledge, resistance to change, lack of time to develop e-learning courses and motivation. the obstacles related to technology and infrastructure included inappropriate infrastructure, low bandwidth internet, and lack of technical support. the barriers related to management included the lack of financial support, e-learning training, inadequate policies, and instructional design; while the obstacles from the students' side included the lack of ict skills, elearning knowledge, proficiency in the english language, and motivation. another study (kamba, 2009) also proposes somewhat similar barriers towards the implementation of e-learning. it posits that lack of trained instructors in terms of ict and lack of technical support hinder effective e-learning systems. these barriers have a more severe effect on elearning implementation when it comes to developing countries. olum et al. (2020) conducted a https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 84 study in uganda universities that attributed the less popularization of e-learning to a low level of awareness regarding e-learning, low computer literacy, the huge financial cost of implementation, and severe connectivity issues. while, students in pakistani universities face challenges of low proficiency in the english language, cultural beliefs, and students' inadequate access to the internet and computer (ahmed et al., 2017; mumtaz et al., 2021). this low proficiency in the english language and lower technological skills result in a lack of self-motivation among students regarding using e-learning systems. students' low confidence in using technology might also be why students' low motivation to adopt e-learning and ultimately become an obstacle in implementing e-learning systems at heis (abou el-seoud et al., 2014; maldonado et al., 2011). 3.5. strategies, policies, and procedures for implementing effective e-learning the barriers hindering the successful implementation of e-learning are almost common among all the regions. a lot of research has been conducted to identify the solutions to overcome these barriers or challenges. some studies (e.g., au et al., 2019; panda & mishra, 2007; saeed al-maroof et al., 2021) mentioned the absence of comprehensive strategic plans, policies, and procedures was the reason behind the lack of distance learning popularity. hence, there is a need for formulating effective strategies for the successful implementation of the e-learning system. birch and burnett (2009) suggest that the management at heis needs to design clear strategic plans, policies, and procedures that mainly address the issues related to e-learning adoption. one of the most critical concerns while formulating the strategies for e-learning is the selection of suitable e-learning systems that the institute is going to install. the essential characteristics that need to be considered are accessibility, ease to operate, flexibility, and user-friendliness (kasim & khalid, 2016). the study further suggested that the selection of e-learning systems must be based on the learning skills tools, communication tools, and productivity tools that the particular system offers. it is important to identify the strategies to popularize the e-learning system usage among university students and thereby enhance their satisfaction. mtebe and raphael (2018) also suggested that the system quality and the instructor quality play a crucial role in improving learners' satisfaction in an e-learning system. therefore, the e-learning strategic development process must include decisions related to the primary system needs of the institute. learning is not a one-way process; for the successful implementation of e-learning, the students need to be well trained and motivated enough. to enhance https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 85 the student's awareness and interest regarding e-learning, self-regulatory learning strategies need to be employed (richardson et al., 2012). kamba (2009) also suggested conducting training and tutorials to train and aware students regarding the efficient utilization of e-learning. along with training the students, teachers must look out for the factors that can motivate students to use elearning. maldonado et al. (2011) suggest that teachers should assess students' motivation for using e-learning technology through an online form and formulate the strategies accordingly for enhancing their motivation. holt et al. (2013) stress that to bring effectiveness in an online learning environment at heis, continuous change in the system following the external and the internal environment is necessary. this study conducted by five australian universities suggested that an outdated system can deteriorate the entire distance learning environment. hence, an improvement continually needs to be carried out to manage the e-learning system's quality effectively. similarly, while stressing advancement in learning technology, kobusińska et al. (2018) and huda et al. (2018) also suggested using big data technology in heis will provide students with a more interactive online learning environment. ultimately, students will be able to make the most of online learning resources. while one of the most recent papers on e-learning topics (i.e., lange & costley, 2020) suggests that improving the quality of media used in delivering online lectures can help in the successful implementation of e-learning. putting it together, most of the papers reviewed in the present study extend that the quality of the e-learning system being used affects the efficiency and effectiveness of the distance learning environment. therefore, the management of heis must give due diligence towards the quality of the system they are using and must train their staff and the students regarding its use. 4. discussion and proposed research today, when we are trying to digitize everything along with our education system, it is high time to put extra effort into improving our technology. particularly, when it comes to the dissemination of knowledge, the use of technology has become indispensable. while discussing the importance of e-learning, we must include the current pandemic situation in our discussion due to its massive disruption. the outbreak of coronavirus or covid-19 has confined everyone to their homes, https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 86 including the students. this situation has further highlighted the importance of e-learning or distance learning (al-okaily et al., 2020; alqahtani & rajkhan, 2020; azlan et al., 2020). students are taking online lectures and taking online exams and have cleared the semesters from higher education institutions. overall, the papers under consideration in this review article suggest that distance learning is a modern and beneficial way of teaching students. it helps save time, provides flexibility to students who cannot physically attend the institutions or who cannot attend at some particular time, provides flexibility in teaching methods, provides access to more information and knowledge, etc. meanwhile, the online learning system has its drawbacks as well if it is not implemented correctly. mainly, when it comes to the developing countries, the problems of low internet bandwidth, non-availability of internet and even computers, low computer literacy rate, and low english language proficiency make the situation even worse. but, still, the current scenario has compelled even these countries to opt for the online learning system. academic leaders' role as knowledge-oriented is highly important for information sharing for better implementation of quality education (sahibzada et al., 2021). given the literature review conducted and the current situation, some literature gaps are identified. it was found that a lot of research has been undertaken on barriers faced by the students taking online classes in developing countries, and those barriers were compared with those of developed countries. but no research until now has been undertaken where obstacles faced by the students following their characteristics are compared. therefore, it is recommended to conduct a study that may explore the comparative barriers faced by the working students in taking online classes versus the obstacles faced by the students who are not working and are taking online courses. similarly, comparative studies can also be conducted based on other students' characteristics. for instance, students living in rural areas versus students living in urban areas, students having proficiency in the english language versus students having low ability in the english language, and likewise. 5. conclusion https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 87 based on the previous studies, it can be concluded that e-learning is the future of higher education institutions. all the stakeholders will be reaping benefits from this particular use of technology. institutions can save resources and time and promote creativity by adequately utilizing the saved resources and time. similarly, students can get the benefits of time flexibility, more access to information, and can manage both studies and work more efficiently in case they are working students. instructors can deliver lectures from their homes, benefit from time flexibility, and introduce innovative learning methods to the students. for the successful implementation of e-learning, the management of the institutions must take timely and wise measures. training staff and students regarding the usage of e-learning systems are important for motivating students and teachers to accept and use this new technological breakthrough. it is significant to have a sound and user-friendly e-learning system installed at their institutions and maintain and update the system's quality. funding: this research received no external funding conflicts of interest: the authors declare no conflict of interest. references abou el-seoud, m. s., taj-eddin, i. a., seddiek, n., el-khouly, m. m., & nosseir, a. (2014). e-learning and students' motivation: a research study on the effect of e-learning on higher education. international journal of emerging technologies in learning, 9(4), 20-26. https://doi.org/10.3991/ijet.v9i4.3465 ahmed, s. s., khan, e., faisal, m., & khan, s. (2017). the potential and challenges of moocs in pakistan: a perspective of students and faculty. asian association of open universities journal. https://doi.org/10.1108/aaouj-01-2017-0011 akçayır, m., akçayır, g., pektaş, h. m., & ocak, m. a. (2016). augmented reality in science laboratories: the effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. computers in human behavior, 57, 334-342. https://doi.org/10.1016/j.chb.2015.12.054 al-azawei, a., parslow, p., & lundqvist, k. (2017). investigating the effect of learning styles in a blended e-learning system: an extension of the technology acceptance model (tam). australasian journal of educational technology, 33(2). https://doi.org/10.14742/ajet.2741 al-okaily, m., alqudah, h., matar, a., lutfi, a., & taamneh, a. (2020). dataset on the acceptance of e-learning system among universities students' under the covid-19 pandemic conditions. data in brief, 32, 106176. https://doi.org/10.1016/j.dib.2020.106176 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.3991/ijet.v9i4.3465 https://doi.org/10.1108/aaouj-01-2017-0011 https://doi.org/10.1016/j.chb.2015.12.054 https://doi.org/10.14742/ajet.2741 https://doi.org/10.1016/j.dib.2020.106176 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 88 al-taweel, f. b., abdulkareem, a. a., gul, s. s., & alshami, m. l. (2021). evaluation of technologybased learning by dental students during the pandemic outbreak of coronavirus disease 2019. european journal of dental education, 25(1), 183-190. https://doi.org/10.1111/eje.12589 al shuaili, k., al musawi, a. s., & hussain, r. m. (2020). the effectiveness of using augmented reality in teaching geography curriculum on the achievement and attitudes of omani 10th grade students. multidisciplinary journal for education, social technological sciences, 7(2), 20-29. https://doi.org/10.4995/muse.2020.13014 alqahtani, a. y., & rajkhan, a. a. j. e. s. (2020). e-learning critical success factors during the covid19 pandemic: a comprehensive analysis of e-learning managerial perspectives. 10(9), 216. https://doi.org/10.3390/educsci10090216 au, o. t.-s., li, k., & wong, t. (2019). student persistence in open and distance learning: success factors and challenges. asian association of open universities journal. https://doi.org/10.1108/aaouj-12-2018-0030 azlan, c. a., wong, j. h. d., tan, l. k., huri, m. s. n. a., ung, n. m., pallath, v., tan, c. p. l., yeong, c. h., & ng, k. h. (2020). teaching and learning of postgraduate medical physics using internetbased e-learning during the covid-19 pandemic–a case study from malaysia. physica medica, 80, 10-16. https://doi.org/10.1016/j.ejmp.2020.10.002 bates, a. t. (2005). technology, e-learning and distance education: routledge. birch, d., & burnett, b. (2009). bringing academics on board: encouraging institution-wide diffusion of e-learning environments. australasian journal of educational technology, 25(1). https://doi.org/10.14742/ajet.1184 cha, h., park, t., & seo, j. j. s. (2020). what should be considered when developing ict-integrated classroom models for a developing country? , 12(7), 2967. costa, c., alvelos, h., & teixeira, l. (2012). the use of moodle e-learning platform: a study in a portuguese university. procedia technology, 5, 334-343. https://doi.org/10.1016/j.protcy.2012.09.037 erdmann, a., & torres marín, a. j. (2019). can we improve academic performance and student satisfaction without additional time cost for teachers? evidence from a blended methodology in microeconomics. multidisciplinary journal for education, social technological sciences, 6(2), 54-91. https://doi.org/10.4995/muse.2019.11869 fereday, j., & muir-cochrane, e. (2006). demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. international journal of qualitative methods, 5(1), 80-92. https://doi.org/10.1177/160940690600500107 gerbic, p. (2011). teaching using a blended approach–what does the literature tell us? educational media international, 48(3), 221-234. https://doi.org/10.1080/09523987.2011.615159 grimus, m. (2020). emerging technologies: impacting learning, pedagogy and curriculum development. in emerging technologies and pedagogies in the curriculum (pp. 127-151): springer. hasan, r., palaniappan, s., mahmood, s., abbas, a., sarker, k. u., & sattar, m. u. (2020). predicting student performance in higher educational institutions using video learning analytics and data mining techniques. applied sciences, 10(11), 3894. https://doi.org/10.3390/app10113894 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1111/eje.12589 https://doi.org/10.4995/muse.2020.13014 https://doi.org/10.3390/educsci10090216 https://doi.org/10.1108/aaouj-12-2018-0030 https://doi.org/10.1016/j.ejmp.2020.10.002 https://doi.org/10.14742/ajet.1184 https://doi.org/10.1016/j.protcy.2012.09.037 https://doi.org/10.4995/muse.2019.11869 https://doi.org/10.1177/160940690600500107 https://doi.org/10.1080/09523987.2011.615159 https://doi.org/10.3390/app10113894 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 89 holt, d., palmer, s., munro, j., solomonides, i., gosper, m., hicks, m., sankey, m., allan, g., & hollenbeck, r. (2013). leading the quality management of online learning environments in australian higher education. australasian journal of educational technology, 29(3). https://doi.org/10.14742/ajet.84 huda, m., maseleno, a., atmotiyoso, p., siregar, m., ahmad, r., jasmi, k., & muhamad, n. (2018). big data emerging technology: insights into innovative environment for online learning resources. international journal of emerging technologies in learning, 13(1), 23-36. https://doi.org/10.3991/ijet.v13i01.6990 ibrahim, m. m., & nat, m. (2019). blended learning motivation model for instructors in higher education institutions. international journal of educational technology in higher education, 16(1), 1-21. https://doi.org/10.1186/s41239-019-0145-2 islam, a. n., & azad, n. (2015). satisfaction and continuance with a learning management system: comparing perceptions of educators and students. the international journal of information learning technology. https://doi.org/10.1108/ijilt-09-2014-0020 jones, c., ramanau, r., cross, s., & healing, g. (2010). net generation or digital natives: is there a distinct new generation entering university? computers education, 54(3), 722-732. https://doi.org/10.1016/j.compedu.2009.09.022 kamba, m. (2009). problems, challenges and benefits of implementing e-learning in nigerian universities: an empirical study. international journal of emerging technologies in learning, 4(1), 66-69. https://doi.org/10.3991/ijet.v4i1.653 kasim, n. n. m., & khalid, f. (2016). choosing the right learning management system (lms) for the higher education institution context: a systematic review. international journal of emerging technologies in learning, 11(6). http://dx.doi.org/10.3991/ijet.v11i06.5644 kennedy, g. e., judd, t. s., churchward, a., gray, k., & krause, k.-l. (2008). first year students' experiences with technology: are they really digital natives? australasian journal of educational technology, 24(1). doi:https://doi.org/10.14742/ajet.1233 khan, a. a., & umair, s. (2017). handbook of research on mobile devices and smart gadgets in k-12 education: igi global. kim, h. j., hong, a. j., & song, h.-d. (2019). the roles of academic engagement and digital readiness in students’ achievements in university e-learning environments. international journal of educational technology in higher education, 16(1), 1-18. https://doi.org/10.1186/s41239019-0152-3 kjellsdotter, a. (2020). what matter (s)? a didactical analysis of primary school teachers’ ict integration. journal of curriculum studies, 52(6), 823-839. https://doi.org/10.1080/00220272.2020.1759144 kobusińska, a., leung, c., hsu, c.-h., raghavendra, s., & chang, v. (2018). emerging trends, issues and challenges in internet of things, big data and cloud computing. in: elsevier. lange, c., & costley, j. (2020). improving online video lectures: learning challenges created by media. international journal of educational technology in higher education, 17(1), 1-18. https://doi.org/10.1186/s41239-020-00190-6 leo, s., alsharari, n. m., abbas, j., & alshurideh, m. t. (2021). from offline to online learning: a qualitative study of challenges and opportunities as a response to the covid-19 pandemic https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.14742/ajet.84 https://doi.org/10.3991/ijet.v13i01.6990 https://doi.org/10.1186/s41239-019-0145-2 https://doi.org/10.1108/ijilt-09-2014-0020 https://doi.org/10.1016/j.compedu.2009.09.022 https://doi.org/10.3991/ijet.v4i1.653 http://dx.doi.org/10.3991/ijet.v11i06.5644 https://doi.org/10.14742/ajet.1233 https://doi.org/10.1186/s41239-019-0152-3 https://doi.org/10.1186/s41239-019-0152-3 https://doi.org/10.1080/00220272.2020.1759144 https://doi.org/10.1186/s41239-020-00190-6 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 90 in the uae higher education context. the effect of coronavirus disease on business intelligence, 334, 203. http://doi.org/10.1007/978-3-030-67151-8_12 maldonado, u. p. t., khan, g. f., moon, j., & rho, j. j. (2011). e-learning motivation and educational portal acceptance in developing countries. online information review. https://doi.org/10.1108/14684521111113597 mayer, r. e. (2014). multimedia instruction. in handbook of research on educational communications and technology (pp. 385-399): springer. mehall, s. (2021). purposeful interpersonal interaction and the point of diminishing returns for graduate learners. the internet higher education, 48, 100774. memon, a. b., & meyer, k. (2017). why we need dedicated web-based collaboration platforms for inter-organizational connectivity? a research synthesis. international journal of information technology and computer science, 9(11), 1-11. https://doi.org/10.5815/ijitcs.2017.11.01 moore, j. l., dickson-deane, c., & galyen, k. (2011). e-learning, online learning, and distance learning environments: are they the same? the internet higher education, 14(2), 129-135. mtebe, j. s., & raphael, c. j. a. j. o. e. t. (2018). key factors in learners’ satisfaction with the elearning system at the university of dar es salaam, tanzania. 34(4). mumtaz, n., saqulain, g., & mumtaz, n. (2021). online academics in pakistan: covid-19 and beyond. pakistan journal of medical sciences, 37(1), 283. naveed, q. n., muhammed, a., sanober, s., qureshi, m. r. n., & shah, a. (2017). barriers effecting successful implementation of e-learning in saudi arabian universities. international journal of emerging technologies in learning, 12(6). https://doi.org/10.3991/ijet.v12i06.7003 ngampornchai, a., & adams, j. (2016). students’ acceptance and readiness for e-learning in northeastern thailand. international journal of educational technology in higher education, 13(1), 1-13. olum, r., atulinda, l., kigozi, e., nassozi, d. r., mulekwa, a., bongomin, f., kiguli, s. j. j. o. m. e., & development, c. (2020). medical education and e-learning during covid-19 pandemic: awareness, attitudes, preferences, and barriers among undergraduate medicine and nursing students at makerere university, uganda. 7, 2382120520973212. https://doi.org/10.1177/2382120520973212 ouatik, f., & ouatik, f. (2021). learning management system comparison: new approach using multi-criteria decision making. paper presented at the international conference on business intelligence. panda, s., & mishra, s. (2007). e-learning in a mega open university: faculty attitude, barriers and motivators. educational media international, 44(4), 323-338. https://doi.org/10.1080/09523980701680854 paul, j., & criado, a. r. (2020). the art of writing literature review: what do we know and what do we need to know? international business review, 29(4), 101717. paul, j., lim, w. m., o’cass, a., hao, a. w., & bresciani, s. (2021). scientific procedures and rationales for systematic literature reviews (spar-4-slr). international journal of consumer studies. https://doi.org/10.1111/ijcs.12695 pham, l., limbu, y. b., bui, t. k., nguyen, h. t., & pham, h. t. (2019). does e-learning service quality influence e-learning student satisfaction and loyalty? evidence from vietnam. international https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://doi.org/10.1007/978-3-030-67151-8_12 https://doi.org/10.1108/14684521111113597 https://doi.org/10.5815/ijitcs.2017.11.01 https://doi.org/10.3991/ijet.v12i06.7003 https://doi.org/10.1177/2382120520973212 https://doi.org/10.1080/09523980701680854 https://doi.org/10.1111/ijcs.12695 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 khalil awan et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 74-91. https://doi.org/10.4995/muse.2021.15813 91 journal of educational technology in higher education, 16(1), 1-26. http://doi.org/10.1186/s41239-019-0136-3 popoola, s. o., & olajide, o. (2021). influence of library anxiety and computer literacy skills on use of library information resources by undergraduates in private universities in southwest nigeria. international information library review, 1-19. https://doi.org/10.1080/10572317.2021.1873051 rasheed, r. a., kamsin, a., & abdullah, n. a. (2020). challenges in the online component of blended learning: a systematic review. computers education, 144, 103701. https://doi.org/10.1016/j.compedu.2019.103701 richardson, m., abraham, c., & bond, r. j. p. b. (2012). psychological correlates of university students' academic performance: a systematic review and meta-analysis. 138(2), 353. http://doi.org/2012-04281-001 saeed al-maroof, r., alhumaid, k., & salloum, s. (2021). the continuous intention to use e-learning, from two different perspectives. education sciences, 11(1), 6. https://doi.org/10.3390/educsci11010006 sahibzada, u. f., xu, y., afshan, g., & khalid, r. (2021). knowledge-oriented leadership towards organizational performance: symmetrical and asymmetrical approach. business process management journal. https://doi.org/10.1108/bpmj-03-2021-0125 sánchez-sordo, j. m. (2019). data mining techniques for the study of online learning from an extended approach. multidisciplinary journal for education, social technological sciences, 6(1), 1-24. https://doi.org/10.4995/muse.2019.11482 sayaf, a. m., alamri, m. m., alqahtani, m. a., & al-rahmi, w. m. (2021). information and communications technology used in higher education: an empirical study on digital learning as sustainability. sustainability, 13(13), 7074. https://doi.org/10.3390/su13137074 singh, v., & thurman, a. (2019). how many ways can we define online learning? a systematic literature review of definitions of online learning (1988-2018). american journal of distance education, 33(4), 289-306. https://doi.org/10.1080/08923647.2019.1663082 tarhini, a., hone, k., & liu, x. (2014). measuring the moderating effect of gender and age on elearning acceptance in england: a structural equation modeling approach for an extended technology acceptance model. journal of educational computing research, 51(2), 163-184. https://doi.org/10.2190/ec.51.2.b williams, r. i., clark, l. a., clark, w. r., & raffo, d. m. (2020). re-examining systematic literature review in management research: additional benefits and execution protocols. european management journal. https://doi.org/10.1016/j.emj.2020.09.007 wu, j.-h., tennyson, r. d., & hsia, t.-l. (2010). a study of student satisfaction in a blended e-learning system environment. computers education, 55(1), 155-164. https://doi.org/10.1016/j.compedu.2009.12.012 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://doi.org/10.1186/s41239-019-0136-3 https://doi.org/10.1080/10572317.2021.1873051 https://doi.org/10.1016/j.compedu.2019.103701 http://doi.org/2012-04281-001 https://doi.org/10.3390/educsci11010006 https://doi.org/10.1108/bpmj-03-2021-0125 https://doi.org/10.4995/muse.2019.11482 https://doi.org/10.3390/su13137074 https://doi.org/10.1080/08923647.2019.1663082 https://doi.org/10.1016/j.emj.2020.09.007 https://doi.org/10.1016/j.compedu.2009.12.012 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 89 neurocommunicative methodologies: attention and emotion of the audiovisual story in the classroom mario rajas1, vanessa izquierdo2, maría luisa garcía3 1 universidad rey juan carlos, email: mario.rajas@urjc.es 2 garrigues, email: vanessa.izquierdo@garrigues.com 3 universidad complutense de madrid, email: mluisagarcia@ccinf.ucm.es abstract audiovisual content as a learning tool has been incorporated extensively into lecture classes. emotion-cognition is intrinsic to the functioning of the human brain, and therefore can explain the acquisition of knowledge and competencies in the educational field, and more specifically, the transformative impact on digital natives. advances in the study of the brain have allowed for quantitative measurement of attentional (edl) and emotional (edr) terms. the objective of this article is to analyze and evaluate the correlation between attention and emotion during the viewing of two videos shown in a classroom in the academic space of a university. the method consisted of recording the electrodermal activity of various groups of audiovisual communication and advertising and public relations students during the viewing of the two audiovisual stories. the main results and conclusions link characteristics of the audiovisual narrative and technicalexpressive qualities and objectives of the videos with the levels of edl and edr recorded by the device and establish advanced lines of research in the field of neuroeducation and neurocommunication. keywords neuroeducation, audiovisual didactic, teaching methodology, attention, emotion, electrodermal activity, story, audiovisual language https://doi.org/10.4995/muse.2019.10670 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 90 1. introduction and status of the question educommunication integrates two disciplines of the social and human sciences that historically have been developed in parallel, communication and education, to form a new pedagogical paradigm (freire, 1970). recognized by unesco in 1979, educommunication "includes all forms of study, learning and teaching, at all levels and in all circumstances, the history, creation, use and evaluation of communications media as practical arts and techniques” (morduchowicz, 1997). masterman refers to the possibilities of education through audiovisual content (masterman, 1985). aparici (2010), meanwhile, warns of the danger of digital technologies repeating vertical and non-participatory pedagogical models, i.e., that greater frequency of use of audiovisual resources does not assume a real change in learning processes. dejaeghere (2009) and martinez-rodrigo & segura garcia (2011) link the use of digital communications media in education to socio-cultural and personal-emotional transformations. analysis of the teaching and learning process through digital audiovisual media in the classroom can be approached from very heterogeneous perspectives, such as a study of the educational technology involved (garcia garcia & rajas, 2011), the procedures for acquiring competences (ferres & piscitelli, 2012; perez-rodriguez & delgado, 2012), media literacy in digital content (perez-tornero & cerda, 2011), or in the field in which this article is developed, the effectiveness of current technology as a methodological tool in student learning (aranda, sanchez-navarro & tabernero, 2009, gabelas & marta-lazo, 2008, garcia garcia & gertrudix, 2009, caceres, ruiz san roman & brandle, 2011, area, 2012). digital technology has boosted the production and distribution of online audiovisual content through the internet and social networks (gonzalez valles & valderrama santome, 2014). in this media context in which visual text and sound multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 91 predominate, the videographic format acquires special relevance as audiovisual content support. with quantitative and qualitative exponential growth, young people invest hours of attention in viewing videos on the web. written text gives way to image and sound as the predominant expressive substances in the learning process (buckingham, 2003). the search to optimize educational resources used in university classrooms has led to analyzing the attention that audiovisual content awakens in students. equally, the interest in understanding, from rigorous scientific approaches, the quality of audiovisual teaching methods has raised interest in new research tools that measure the effects of the use of video in the classroom. in this context, neuroeducation is a discipline that combines knowledge of neuroscience, education and psychology with the objective of understanding learning and teaching processes and their direct link with communication (salas, 2013). it can be stated that "neuroscience allows for an approach that complements traditional research methods. qualitative research offers variety and depth through interpretation, while the quantitative counterpart offers data” (garcia guardia & llorente barroso, 2014). several authors have established the direct relationship between neuroscience and education (cacioppo, 2002, battro, fischer & lena, 2008, hardiman, rinne, gregory & yarmolinskaya, 2012), and specifically between emotion and learning processes (pekrun, 1992; fernandez abascal, 1995, masson, 2015). students, as emotional beings, learn and memorize better that which involves them, that which demands their sensory participation and that which they love. information, before being processed by the cerebral cortex, passes through the limbic or emotional brain system, in whose areas of association neural networks are produced and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 92 distributed, creating the abstract, ideas and the basic elements of thought (mora, 2013). the action of paying attention is one of the processes that most concerns teaching activities (anderson, 2014). without attention from the student, the communication involved in any act of teaching-learning achieves satisfactory results, either because it does not end, or because it does not even occur. attentional, emotional and motivational processes are closely linked to the achievement of learning objectives (carew & magsamen, 2010). this article provides an approach to analyzing the effects of audiovisual media on student learning through a neuroscientific experiment: measuring the attention (edl) and emotion (edr) that a series of audiovisual resources of a narrative nature provoke in the classroom. based on research on electrodermal activity and measurement of edl and edr levels (ketterer & smith, 1982, tranel, 2000, dawson, schell & filion, 2000, martinez herrador, garrido martín, valdunquillo carlón & macaya sanchez, 2008; gomez, 2013) applied to audiovisual productions, an analysis was conducted to determine if videographic content in a didactic context can reinforce attentional and emotional levels; therefore, its use in the classroom merges the communicativedidactic process which concludes successfully with the acquisition of knowledge and competency by the student. the study of audiovisual material in the classroom from this perspective is pertinent since attention and emotion are two cognitive processes strongly related to memory and consequently play a key role in learning. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 93 2. material and methods the general objective of this research is to understand the attentional and emotional efficacy of the screening of audiovisual materials in university classrooms. specifically, if the screening of audiovisuals during a given classroom lesson causes variation in attention and emotion during viewing that results in an increase in the attentional and emotional development of the class as a whole. the specific objectives are to test diverse audiovisual materials to compare the level of attention and emotion of each, and secondly, to test their efficacy in the communicative flow in relation to the audiovisual character of the lesson taught. to achieve these objectives, we have chosen to conduct an experiment from applied neuroscientific formulations (sutil, 2013). we are aware that we are faced with flexible and generic indicators (ferres and piscitelli, 2012) and that they must be adapted to the applicable educational situation, depending on age, objectives and competencies required by the curricular content; therefore particular attention has been given to the design of the experiment for a specific situation. media communication depends to a large extent on reception spaces, therefore we have implemented a neuroeducative approach for a specific environment and experiment. in this specific case, a lecture was prepared to be taught by professors of audiovisual communication and advertising and public relations that included the screening of a video a and a video b to determine which of the two videos was more relevant in attentional and emotional terms for the students. the two audiovisual materials were selected and validated by a committee of experts in audiovisual narrative and emerging technologies consisting of 7 teacherresearchers from a general sample of 20 audiovisual productions. the videos address, in accordance with the above regarding emerging technologies and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 94 education, the digital society of social networks, and include images, animation style motion graphics, voice over and different types of musical content. the image presents strong chromatic contrasts and different playback or timewarp speeds. the videos validated by the committee of experts and screened were the social media revolution by evan kutsko (2016) video a and socialnomics by erik qualman ( 2014) video b both broadcasted on the youtube platform. the screening of the materials began with video a, longer in duration (4:25), followed by video b, shorter in duration (2:22). the sampling procedure chosen was convenient, given the commitment the method offers in cost and reliability, since the sample was non-random. the sample chosen consisted of 54 people between 18 and 30 years old the most common age of enrollment in university studies from both sexes in a similar proportion (50%): 26 students degree in audiovisual communication and 28 students degree in advertising and public relations, at the complutense university of madrid. the size of the sample has been validated in accordance with recent studies and can be considered correct and reliable with respect to the neuroscientific method performed in this research in comparison with similar research (martinez herrador, 2007; vecchiato, cherubino, maglione et al. 2014; reimann, 2012; karmarkar, yoon & plassman, 2015; orzan, zara & purcarea, 2015; tapia & martín, 2016). field work was conducted in april 2017 at the faculty of information sciences of the complutense university of madrid. the participants in this study were informed of the purpose of the research and subsequently provided the applicable verbal consent. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 95 the procedure used for recording attention and emotion of the group was the measurement of the electrodermal activity (eda) of the subjects. due to reliability and efficiency of measurement, electrodermal activity is one of the most frequently used methods in analogous experiments (martinez herrador, monge benito & valduquillo, 2012, tapia, martin & puentes, 2016). the technology used to observe electrodermal activity was developed by the scientific marketing company, sociograph (www.sociograph.es). the technology consists of a wristband with two diodes placed on the index and middle fingers, which measures electrodermal activity; subsequently, the devices send the measurements to a central calculation unit for storage and processing (aiger, palacín & cornejo, 2013). the technology measures two parameters: 1) tonic activity: related to attention (edl). the unit of measurement used is the summation of the electrodermal resistance in kilo ohms (kω) of all the participants. in this article, the inverted values are shown to facilitate reading. 2) phasic activity: related to emotion (edr). the unit of measurement is the arithmetic mean of the electrodermal resistance in kilo ohms (kω) of all the participants. as a note, the machine simply detects the presence and intensity of emotion, but not the content or quality of it, research objectives that extend beyond the methodological framework of this experiment. for the use of results, cross-sectional statistical models and techniques applicable to the study of time series were employed. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 96 over the course of the class, the videos were screened separately to ensure each viewing was independent of the other – video a was broadcast first, and after a sixty second pause, video b was broadcast. the university professor taught a class that developed the content related to the videographic materials before and after the broadcasts, content that was part of the theme of the two subjects. the sessions were also recorded with the objective of detecting potential deviations in terms of instructions received by the students from the teacher, performance of the task in general, correct broadcasting of the videos and development of the session. 3. analysis and results first, in relation to the attentional function (edl), it should be noted that the model utilized demonstrates significant autocorrelation (0.838 in 16 delays, p value 0.000), which indicates the dependence of subsequent values on previous values. however, this data is applicable to the study of attention (edl), but not to the study of emotion, as due to the sudden nature of emotion it does not demonstrate dependence. 3.1. analysis of attention (edl) the average attention level demonstrated during the screening of video a, longer in duration, maintained at an average attention measurement of -666.051 kω with a standard deviation of 16.926 kω. for the screening of video b, -671.400 kω with a standard deviation of 5.488 kω. given that the exposed value measures the inverse resistance, it can be stated, firstly, that more attention was paid on average multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 97 to the screening of the longer video than to the viewing of the shorter video. however, it is important to emphasize that the attentional dispersion was much higher in the case of longer video. i.e., more attention was paid during video a on average, but that attention showed more ups and downs, which could be a relevant factor in the learning process. the difference was also statistically significant (t test of difference between averages, p value 0.000 per p value in levene test 0.000). however, an analysis of the structure of the attention during the screening, as can be seen in the following figure (up to the red line of video a, then video b), allows extraction of other significant data. figure 1. edl recording (attention) during broadcast of the videos (-σkω ) source: compiled by the author the broadcast of video a maintains with slight ups and downs attention for the first two minutes, at which time attention drops very intensely almost until the end four minutes -, where a turning point occurs during which it increases again but without recovering the levels prior to the descent. i.e., during the broadcast of multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 98 almost half of the video from minute two to four -, the students paid a low level of attention. additionally, during this segment, the level of attention was significantly lower than during the course of the screening of video b. however, the structure of the attention given to video b demonstrates a different evolution. it increases slightly during the first 20 seconds, notably declines during the next 30 seconds, and increases again up to minute 1:20 until recovering and maintaining the levels prior to the descent until the end of the screening. analysis of the attention of both videos must also be based on the study of the rate of change of the variable. this data reveals moments of increase and decrease in attention in response to the stimuli deployed. the following figure shows the rate of change in edl of the two pieces equally, video a extends to the red line and then video b begins. figure 2. rate of change in edl (-kω) source: compiled by the author multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 99 as shown, the recording of the rate of change in edl demonstrates notable attentional alterations which are described in detail in the following table. table 1. edl increments in rate of change temporary segment ∑kω description video a 00:13-00:21 1.30 audio: electronic music. alters tones on a constant basis. image: on white background appears overlaid: “welcome to the revolution. over 50% of the population is under 30 years old. 96% of millennials have joined a social network.” 00:39-00:49 1.1480 audio: electronic music continues. image: on white background appears overlaid: “years to reach 50 millions users. radio. 38 years. tv. 13 years. internet. 4 years.” 00:59-01:10 2.0274 audio: electronic music continues. image: on white background appears overlaid: “we don´t have a choice on whether we do social media, the question is how well we do it.” 01:34-01:43 1.4315 audio: electronic music continues. a drum sample is added. image: appears overlaid: “twitter. what´s happening? ashton kutcher and britney spears have more facebook followers than the population of sweden, israel, switzerland, ireland, norway and panama.” 02:28-02:34 1.2178 audio: electronic music continues. a repetitive and unintelligible voice sound is added. image: on white background appears overlaid: “if you were paid $1 for every article posted on wikipedia you would earn $1,712.32 per hour. there are over 200,000,000 blogs.” 03:18-03:23 1.5969 audio: electronic music continues. the voice does not continue. image: on white background appears an image of a tv overlaid and the text: “only 18% of traditional tv campaigns generate a positive roi. 90% of people skip ads via tivo/dvr.” 03:36-03:41 1.3615 audio: electronic music continues. the same voice indicated above is added. image: on a search box similar to that of google appears the writing: “we no longer search for the news. the news find us. we no longer search for products and services.” 03:50-04:19 1.4053 audio: electronic music continues. the voice does not continue. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 100 image: on white background appears overlaid, “successful companies in social media act more like dale carnegie and less like mad men. listening first, selling second.” video b 00:09-00:13 0.9599 audio: electronic music. image: on blue background appears overlaid: “world population. 1. china, 2. facebook, 3. india, 4. tencent, 5. whastapp, 6. united states, 7. google+, 8. indonesia, 9. linkedin, 10. twitter.” 00:50-00:54 1.299 audio: electronic music. image: on blue background appears overlaid: “1 in 5 divorces involve social media. what happens in vegas stays in facebook.” 00:59-01:03 0.8391 audio: electronic music. image: on blue background, an image of a mobile phone. on the phone, an image of a group taking a selfie, and above in white text, it reads, "selfie is now a word in webster." 01:10-01:14 0.7393 audio: electronic music continues. a repetitive and unintelligible voice is added. image: on white background appears various shields from american universities. 01:56-02:22 0.6203 audio: electronic music continues and the voice is repetitive and unintelligible. image: on blue background oreo cookies and traces of milk can be seen. the text states: “real time marketing and newsjacking are becoming staples for savvy brands. goodbye 4 ps of marketing: product, place, price promotion.” source: compiled by the author considering times when attentional increase occurs as a whole, a certain common pattern can be seen. the background, whether white or blue, focuses attention on the graphic overlay elements. the text on the image background stands out. regarding the text, large headings stood out with data that was surprising and unknown to the audience a priori, and attracted attention due to showing very high statistics in comparison with other numbers for example, the relationship between the population of certain countries and inhabitants of the social networks. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 101 8 attentional increases above 0.5 kω were detected during the screening of video a, and 5 during the course of video b, which indicate on average an increase in attention in the case of the first video every 33 seconds and in the case of the second every 29 seconds. 3.2. analysis of emotion (edr) the average level of emotion recorded during the broadcast of video a was 0.1489 with a standard deviation of 0.08280. the average value for video b was 0.1572 with a standard deviation of 0.08724. therefore, it is important to note, in the first instance, the similarity of the emotional values of both screenings. therefore, this indicates a similar display of average emotional intensity. additionally, a t test of difference between averages was conducted, which ruled out significant differences p value 0.352 with p value 0.282 in the levene test for equality of variances. however, the fact that there are no relevant differences in terms of average intensity does not indicate the same homogeneity in emotional peaks. these are moments in which, in response to a certain stimulus, the audience responded suddenly with a peak of intensity. the responses, as shown in the following figure, are produced on four occasions during the screening of video a, and also on four other occasions during video b, the difference being that in this case the peaks are much closer to one another. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 102 figure 3. edr of the screenings (kω) source: compiled by the author the times indicated, of a sudden and independent character with respect to the previous content, are described in the auditory and visual plan in the following table. table 2. edr increases (kω) temporary segment kω description video a 01:07 0.3993 audio: electronic music; a voice repeats: “right here, right now.” image: a list appears in columns with the most populated countries in black text with the following highlighted in blue text: “3. facebook.” 02:01 0.4436 audio: electronic music; a repetitive voice, this time unintelligible. image: on black background, white and blue text that states, "instead they are distributing: ereaders, ipads, tablets. the social media revolution 2016.” multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 103 03:19 1.0307 audio: electronic music, this time without a voice. image: on white background, image of a tv and the text, "only 18% of traditional tv campaigns generate a positive roi." suddenly the text, "90% of people skip ads via tivo/dvr," appears. 03:37 0.4556 audio: electronic music; the same unintelligible voice as before is added. image: "we no longer search for the news. the news finds us," is written on a search box similar to that of google. video b 00:10 0.3987 audio: electronic music. there is the sound effect of a typewriter. image: a classification on blue background is displayed: “world population. 1. china, 2. facebook.” 00:51 0.413 audio: electronic music. there is also a sound effect of broken glass. image: on blue background appears an overlay: “1 in 5 divorces involve social media.” 01:00 0.4745 audio: electronic music. there is no voice or sound effect. image: on blue background, the image of a mobile phone is formed; a group of people taking a selfie is seen. white text states, "selfie is now a word in webster." 01:06 0.4223 audio: electronic music; a repetitive voice states, "right here, right now." image: on a white background, black text is shown that states, "every second 2 new members join linkedin." source: compiled by the author as with attentional increases, emotional increases of a more sudden and brief character are seen above the electronic music that keeps the viewer in a state of alert, in this case more frequent use of voice and sound effects that highlight or amplify the meaning of the image. the text is also shown as a large headline intended to provoke surprise in the audience. it is important to note also, coinciding with the above, that in the case of video a, two of the four times during which emotional content was generated took place in multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 104 sections of increased attention. the same situation is found in three of the four times during the screening of video b. therefore, emotional content was generated more frequently in sections of increased attention. however, a related issue is whether there is a significant relationship between emotional and attentional increase, or formulated in another way, if attentional increase indicates emotional increase or not – and vice versa. the tests performed -pearson correlations do not allow for this relationship to be significantly validated (p value 0.517), therefore this link is not emphasized as significant. 3.3. proposal for the development of an audiovisual performance factor. as mora (2013) emphasizes, attention and emotion are two fundamental cognitive processes in the learning process. following this approach, we propose the establishment of a comparative performance index for audiovisual materials viewed in the classroom. to accomplish this, the average rate of change and the average edr at n(0.1) are normalized, as shown in the following table. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 105 table 3. average edl rate of change and average edr normalized to n(0.1). edl (average rate of change) edr (average edr) data source standardized rate n(0.1) data source standardized rate n(0.1) video a average -0.1058 -0.0314 0.1489 -0.0348 standard deviation 0.5163 0.9792 0.828 0.9812 video b average -0.059 0.057 0.1572 0.0634 standard deviation 0.5473 1.038 0.8724 1.0339 source: compiled by the author these indexes are considered in terms of attention and emotion equivalence. likewise, the value of the standard deviation is studied to indiscriminately consider the intrinsic variation in the average values of each of the videos, the iperformance correlation coefficient. iperformance = edl (average + 2*std. dev.) + edr (average + 2*std. dev.). in this way, the performance for video a would obtain a value of 3.8546 and 4.2642 for video b, which would indicate a higher general value for video b. video b achieves a higher score in edl where it decreases, but less than video a -, and it also obtains a somewhat higher value in average edr although it also has greater dispersion -, which definitively describes why video b obtained a significantly higher score. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 106 4. conclusions when comparing the attention (edl) of students during a university class on two videos, the social media revolution by evan kutsko (2016) and socialnomics by erik qualman (2014), the following conclusions were drawn. in relation to the objectives proposed in the research, in the sense of understanding attentional and emotional efficacy of the screening of audiovisual materials in the university classroom by comparing the levels of attention and emotion of the different materials used, there is less attention regarding video b, shorter in duration, although the attentional dispersion was greater. video a, longer in duration, maintains attention with slight ups and downs during the first two minutes. subsequently, there is a moment in which attention decreases very intensely until the end, with lower attention in the final two minutes than throughout the course of video b. this fact indicates that the order of the videos and their total duration influenced the attention process, since they deal with two subjects that are very similar in content and form, which excludes other audiovisual language and narrative variables in the case of having used different recording or post-production techniques. on average, attentional increases differed depending on the duration of the video: during the long duration video every 33 seconds and during the short duration video every 29 seconds. in particular, there is an overall increase in attention when text overlaid on an image appears on a blue or white background. i.e., this type of expressive resource increases the attention of the students, equal to what was observed in the tables multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 107 above, a variety of surprising data due to the unsuspected nature both quantitatively and qualitatively, producing the same result where more attention is concerned. on the other hand, when analyzing emotion, the values of both screenings were similar: the pearson correlation with a value of 0.517 confirms that there is no relationship between attention and emotion in the two videos. similar to that which occurred with attention, sudden and short increases in emotion, which coincided with the use of electronic music, kept the students in a state of alert, in this case the use of voice and sound effects was more common, as explained in the experiment, to highlight the different informative and emotional messages intended to reach the spectators through the story. finally, it is important to note that iperformance demonstrates that video b obtains a higher value than video a. according to this data, it can be concluded that neuroeducational analysis can yield significant results regarding which audiovisual materials may be the most effective in provoking attention and emotion in the classroom, and when its efficacy or measurement is positive or negative. after comparing and contrasting the results of this research, we propose to open discussion regarding the influence of the order of the screening of the videos by performing an experiment which begins with the screening of video b, shorter in duration, followed by video a, longer in duration, and likewise, if the merging of different expressive substances audiovisual resources that bring into play other languages of image and sound improves attention and emotion. similarly, the possibility of crossing data with other research methods will be assessed in multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 108 addition to video recording of the expressions made by the students such as the use of a survey or focus group and other neuroscientific measurement systems including eye-tracking and microexpression analysis software. the results obtained in this work refer to a given context, therefore it would be of scientific interest to create the experience with other types of learners in other knowledge areas, with different ages or other geographical environments, as well as to measure the delivery of content type lecture classes given in conjunction with audiovisual materials. this study presents relevant results given the lack of previous analysis regarding this type of experiment with audiovisual content in university classrooms, but its weakness is in the context; it is essential not to over emphasize the results, and to propose new experiments that result in greater global knowledge of the subject of study. therefore, neuroscientific experiments applied to the use of audiovisual resources in the classroom is a field of study that can provide relevant data and interpretations in the scope of teaching methodologies: attention and emotion are two fundamental aspects of the process of teaching a classroom or virtual class, and knowledge and use of these concepts can be applied to the development of audiovisual content and to the improvement of the way in which the communicative act of teaching and learning is produced and shared, the didactic story, ultimately. thus, for example, content analysis facilitates obtainment of valuable data when evaluating the narrative and aesthetic structure of videos, or in another area of equal interest, a comparison of edl and edr of the same content taught exclusively orally -lecture-, with the support of visual presentations, or in audiovisual form with a video, among other fascinating lines of research in the interrelated field of education, communication and neuroscience. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 109 5. references aiger, m., palacín, m. & cornejo, j.m. (2013). electrodermal signal by sociograph: methodology to measure the group activity. revista internacional de psicología social: international journal of social psychology, 28 (3), 333-347. doi:http://dx.doi.org/10.1174/021347413807719102 anderson, o.r. (2014). progress in application of the neurosciences to an understanding of human learning: the challenge of finding a middle-ground neuroeducational theory. international journal of science and mathematics education, 12 (3), 475-492. doi:http://dx.doi.org/10.1007/s10763-013-9455-3 aparici, r. (2010). la construcción de la realidad [the construction of reality]. in r. aparici (coord.). la construcción de la realidad en los medios de comunicación [the construction of reality in communication media] (pp. 11-22). madrid: uned. aranda, d., sanchez-navarro, j., & tabernero, c. (2009). jóvenes y ocio digital: informe sobre el uso de las herramientas digitales por parte de adolescentes en españa [youth and digital leisure: report on the use of digital tools by adolescents in spain]. barcelona: uoc. area, m. (2012). sociedad líquida, web 2.0 y alfabetización digital [liquid society, web 2.0 and digital literacy]. aula de innovación educativa, 212, 55-59. battro, a.m., fischer, k.w & lena, p.j (2008) the educated brain: essays in neuroeducation. cambridge: cambridge university press, doi:http://dx.doi.org/10.1017/cbo9780511489907 buckingham, d. (2003). media education: literacy, learning and contemporary culture. cambridge: polity press. carew, t.j & magsamen, s.h, (2010). neuroscience and education: an ideal partnership for producing evidence-based solutions to guide 21st century learning. neuron, 67 (5), 685-688. doi:http://dx.doi.org/10.1016/j.neuron.2010.08.028 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 110 caceres, m.d., ruiz san roman, j.a., & brandle, g. (2011). el uso de la televisión en un contexto multipantalla: viejas prácticas en nuevos medios [the use of television in a multi-screen context: old practices in new media]. análisis, 43, 2144. cacioppo, j.t. (2002). foundations in social neuroscience. new york: mit press. dawson, m.e.; schell, a.m. & filion, d.l. (2000). the electrodermal system. in: cacioppo, j.t.; tassinary, l.g.; bernstom, g.c. handbook of psychophysiology. new york: cambridge university press, 200-223. dejaeghere, j. (2009). critical citizens education for multicultural societies. interamerican journal of education and democracy, 2, 223-236. fernandez abascal, e.g. (1995). manual de motivación y emoción [manual of motivation and emotion]. madrid: centro de estudios ramón areces. ferres, j., & piscitelli, a. (2012). la competencia mediática: propuesta articulada de dimensiones e indicadores [media competence. articulated proposal of dimensions and indicators]. comunicar, 38, 75-86. doi: http://dx.doi.org/10.3916./c38-2012-02-08. freire, p. (1970). pedagogía del oprimido [pedagogy of the oppressed]. mexico: siglo xxi. garcia garcia, f., & gertrudix, m. (2009). el mare nostrum digital: mito, ideología y realidad de un imaginario sociotécnico [the digital mare nostrum: myth, ideology and reality of a sociotechnical imaginary]. icono14, 7(1), 7-30 http://www.icono14.net/ojs/index.php/icono14/article/view/331 (date of access: 10/15/2016). garcia garcia, f., & rajas, m. (2011). narrativas audiovisuales: mediación y convergencia [audiovisual narratives: mediation and convergence]. madrid: icono14 editorial. garcia guardia, m.l., & llorente barroso, c. (2014). neurocommunication, neuroshopping and efficacy. en j. timoteo alvarez (ed.), social neurocommunication. applying the findings from neurosciences and network multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 111 theory to the science and communication industry (pp. 217-237). porto: media xxi. gonzalez valles, j.e., & valderrama santome, m. (2014). comunicación actual: redes sociales y 3.0 [current communication: social networks and 3.0]. madrid: mcgraw-hill. karmarkar, u., yoon, c., & plassmann, h. (2015). marketers should pay attention to fmri. harvard business review. goo.gl/5tdxbj (date of access: 10/6/2016). ketterer, m.w., & smith, b.d. (1982). lateralized cortical/cognitive processing and electrodermal activity: effects of subject and stimulus characteristics. psychophysiology, 19, p. 328-356. hardiman, m., rinne, l., gregory,e., & yarmolinskaya, j. (2012). neuroethics, neuroeducation, and classroom teaching: where the brain sciences meet pedagogy. neuroethics, 5 (2), 135-143. doi:http://dx.doi.org/10.1007/s12152-011-9116-6 martinez herrador, j.l. (2007). la medida de la atención y la emoción de grupos sociales mediante una nueva técnica: sociograph. [the measure of attention and emotion of social groups through a new technique: sociograph]. xi conferencia española y i encuentro iberoamericano de biometría, salamanca [11th spanish conference and 1st ibero-american meeting of biometrics, salamanca]. martinez herrador, j.l., garrido martin, e., valdunquillo carlon, m.i., & macaya sanchez, j. (2008). análisis de la atención y la emoción en el discurso político a partir de un nuevo sistema de registro psicofisiológico y su aplicación a las ciencias políticas [analysis of attention and emotion in political discourse based on a new psychophysiological record system and its application to political science]. dpsa. documentos de trabajo del departamento de psicología social y antropología, 2 [working documents of the department of social psychology and anthropology, 2]. martinez herrador, j.l., monge benito, s., & valdunquillo carlon, m.i. (2012). medición de las respuestas psicofisiológicas grupales para apoyar el análisis de discursos políticos [measurement of group psychophysiological responses to support the analysis of political discourse]. tripodos, 29. 53-72. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 112 martinez-rodrigo, e., & segura garcia, r., (2011). jóvenes digitales. la dinámica de las emociones en el uso de las tecnologías [the dynamics of emotions in the use of technologies]. in e. martinez-rodrigo y c. marta-lazo (coords.), jóvenes interactivos: nuevos modos de comunicarse [interactive youth: new ways of communicating] (pp.19-35). la coruña: netbiblo. masson, s. (2015). how neuroeducation learning can contribute to educational practice: from neuromyths to current research findings. anae approche neuropsychologique des apprentissages chez l'enfant, 27 (134), 11-22. masterman, l. (1985). la enseñanza de los medios de comunicación [the teaching of communications media]. madrid: ediciones de la torre. mora, f. (2013). neuroeducación [neuroeducation]. sólo se puede aprender aquello que se ama [you can only learn what you love]. madrid: alianza editorial. morduchowicz, r. (1997). la escuela y los medios. un binomio necesario [school and media. a necessary binomial]. buenos aires: editorial aique. orzan, g., zara, i., & purcarea, v.l. (2015). neuromarketing techniques in pharmaceutical drugs advertising. a discussion and agenda for future research. journal of medicine and life, 5 (4), 428-432. id pubmed: 23346245 pekrun, r. (1992). the impact of emotions on learning and achievement: towards a theory of cognitive/motivational mediators. applied psychology, 41, 359–376. doi: 10.1111/j.1464-0597.1992.tb00712.x perez-rodriguez, m.a., & delgado, a. (2012). de la competencia digital y audiovisual a la competencia mediática: dimensiones e indicadores [from digital and audiovisual competence to media competence: dimensions and indicators]. comunicar, 39, 25-34. doi: http://dx.doi.org/10.3916/c39-2012-02-02 perez tornero, j.m., & cerda, j.f.m. (2011). hacia un sistema supranacional de indicadores mediáticos [towards a supranational system of media indicators]. políticas de alfabetización en la unión europea [literacy policies in the european union]. infoamérica: iberoamericanan communication review, 5, 39-57. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 113 rienda gomez, j.j. (2013). técnicas de diagnóstico en neuromanagement [diagnostic techniques in neuromanagement]. en l. sutil (coord.), neurociencia, empresa y marketing, 165-176. madrid: esic. salas, c. (2013). neurocomunicación [neurocomunicacion]. in l. sutil (coord.), neurociencia, empresa y marketing, 195-213. madrid: esic. sutil, l. (2013). herramientas del neuromanagement [neuromanagement tools]. in l. sutil (coord.), neurociencia, empresa y marketing, 101-116. madrid: esic. tapia, a., & martin, e. (2016). neurociencia aplicada a la televisión: medición de la atención y la emoción de la serie forever [neuroscience applied to television: measuring the attention and emotion of the forever series]. revista de comunicación vivat academia · 19 (134), 69-82· doi: http://dx.doi.org/10.15178/va.2016.134.69-82 tapia, a., martin, e., & puentes, j.e. (2016). neurociencia y publicidad. atención, emoción y premios obtenidos en el festival internacional de publicidad de cannes [neuroscience and advertising. attention, emotion and awards obtained at the cannes international advertising festival]. analisi, 54. tranel, d. (2000). electrodermal activity in cognitive neuroscience: neuroanatomical and neuropsychological correlates. in: lane, r.d.; nadel, l. (eds.). cognitive neuroscience of emotion. new york: oxford university press, 192-224. vecchiato, g., cherubino, p., maglione, a.g., ezquierro, m.t.h., marinozzi, f., bini, f., trettel, a., & babiloni, f. (2014). neurophysiological tools to investigate consumer´s gender differences during the observation of tv commercials. computational and mathematical methods in medicine, 91, 29-81. doi: http://dx.doi.org/10.1155/2014/912981. videography kutsko, e. (2016). the social media revolution. https://www.youtube.com/watch?v=n4znqdyz038 (date of access: 10/30/2016). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.10670 social and technological sciences issn: 2341-2593 rajas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci vol. 6 nº 1 (2019): 89-114 | 114 qualman, e. (2014). socialnomics. goo.gl/nwrtlw (date of access: 10/30/2016). 6. key ideas the study interrelates the use of audiovisual resources in the classroom with the analysis of attention and emotion in a group of communication sciences (advertising and public relations and audiovisual communication) students. the introduction of various visual and sound stimuli such as key words in motion graphics and specific references in the voice over -, as well as the temporal construction of stimuli displayed by the videos produced significant changes in edl and edr values. therefore, we highlight the possibility that neuroscientific experiments can be applied to the use of audiovisual resources, in the narrative and aesthetic construction of these materials to obtain higher levels of attention and emotion, as well as in the integration in university classrooms with other types of content and how it can provide relevant data and interpretations in the field of teaching methodologies, both in the classroom and in virtual classes, with the intention of improving the teaching-learning process in the era of digital content. multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 103 a criteria catalog of innovation facilities at higher education institutes: an observational approach atia bano memona,b,* , shakeel ahmed memona auniversity of sindh, jamshoro, pakistan buniversity of leipzig, germany * correspondence: atia.memon@usindh.edu.pk received: 13 may 2022; accepted: 26 august 2022; published: october 2022 abstract the present paper deals with designing a criteria catalog for the definition and characterization of academic innovation facilities established within premises of higher education institutes. the indepth content analysis of available scientific literature on the topic and web description of existing innovation facilities from their official websites is carried out following a purposive observational approach. as a result, a criteria catalog of 12 aspects including 6 functional and 6 structural aspects along with their possible field configurations is devised and proposed herein. the proposed catalog should assist in defining a particular innovation facility in more standardized manner and thereby serve as a guideline for planning and establishing new academic innovation facilities. the catalog should also enable to cross-compare and distinguish multiple innovation facilities across different geographies in order to reveal their relative strengths and weaknesses. keywords: academic entrepreneurship; innovation facilities; incubation centers; higher education institutes; criteria catalog; innovation teaching and ecosystem. 1. introduction in recent years, higher education institutes (heis) such as universities are in transition. previously heis were characterized by two functionalities; i.e. education and research. lately, however, another functionality being considered integral to heis is the academic entrepreneurship to cite this article: memon, a.b., memon, s.a. (2022). a criteria catalog of innovation facilities at higher education institutes: an observational approach. multidisciplinary journal for education, social and technological sciences, 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0001-6893-0931 https://orcid.org/0000-0002-8525-4100 https://doi.org/10.4995/muse.2022.17683 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 104 and innovation (jahangir et al., 2014). academic entrepreneurship refers to the facilitation and fostering of entrepreneurial activities of students and faculty by the heis within the academic context as an attempt to address the changes in the global arena that calls for the integration of industry, government, and academia in national innovation systems (etzkowitz & leydesdorff, 2000). this orientation of universities towards becoming entrepreneurial organizations and engaging in a triple helix (university-industry-government) environment, is being nurtured by the external influences in the form of explicit policies from government aiming at increasing universities’ responsibilities in ensuring the proper communication and commercialization of research outcomes and also the internal development within universities intending to facilitate their academics and students to utilize academic learning in practical ways with the support of academic experts (clauss et al., 2018). it is believed that in order to prepare the students to face modern challenges with success and better compete in labor market, heis need to offer relevant and updated curricula with strong focus on critical thinking, creativity, flexibility, and entrepreneurship and vocational training (aguilarsantelises et al., 2015). this vocational orientation of academic activities prompts the heis to adopt a strategy for teaching-learning process that ensures the students` access to knowledge and skills for lifelong learning that include the core generic skills, skills for personal development, and specific skills for their integration in work environment (roger-monzó et al., 2015). furthermore, heis have to ensure that the students are able to recognize and apply the acquired skills in professional field adequately and in an efficient manner (canzer, 1997). the basic premise underlying the academic entrepreneurship and innovation is that the heis conduct wide range of research activities and some of research results have high potential of commercialization. academic entrepreneurship and innovation enable the utilization of such potential research results and flourishing of new ideas produced by the faculty and students of heis leading towards their commercialization in the market as a contribution in the socio-economic development of the communities (salamzadeh et al., 2011). by doing so, heis promote new startups, introduce new ways of commercialization, and prepare graduating students for recognizing and adopting self-employment opportunities (tunio, 2020) by enabling and encouraging the close collaboration between academic coordinators and those in the associations and professional bodies (roger-monzó et al., 2015). thus, the heis are believed to act as a catalyst for entrepreneurial activities and also as revenue-generating agents (wood, 2011). in order to implement academic entrepreneurship within academic environment, heis have adopted various approaches including the offering entrepreneurship teaching and training programs, http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 105 organizing entrepreneurial events such as idea competitions and entrepreneurship fairs, introducing the academia-industry linkages programs, and the like (garo et al., 2015; ndou et al., 2018). increasingly, to promote such activities in a proper and structured manner, heis have started investing in organizational structures internal to the institutes (garo et al., 2015). one prominent approach of doing so is the establishment of dedicated innovation facilities (ifs) within organizational boundaries. these facilities are varyingly termed as entrepreneurship centers, incubation centers, innovation centers, innovation laboratories, research centers, innovation units, and technology incubation centers; having a uniform objective of developing and strengthening the academic entrepreneurship ecosystem at local and national levels. these facilities are mostly not traditional or discipline-oriented departments; rather possess an interdisciplinary and transdisciplinary research scope (dolan et al., 2019). in addition to being potential sources of research commercialization and thereby bringing economic and development benefits to the societies, such facilities are also believed to be an excellent source of revenue generation for the heis through donations, external project funding and grants, technology commercialization, and open academic events (finkle et al., 2006). realizing the importance of such entrepreneurship and innovation facilities in transforming traditional universities and heis into the entrepreneurial ones and their potential in alleviating the economic situation of the nations, increasingly, the government and other national innovation policy makers have also taken various initiatives of introducing, funding and supporting the establishment of such facilities (tunio, 2020) in developed as well as in developing countries (marques et al., 2019). these facilities at the heis are increasingly considered as an indispensable component of the triple helix initiatives and for guaranteeing the competitiveness and growth in global based economy. they are believed to promote different forms of interactions between different actors to foster economic growth, and social change and development (marques et al., 2021). with the increasing number of such ifs being established at universities and other heis, research investigating their structural and functional embodiments has also grown proportionally. however, the extant literature has remained fragmented because that the different authors have investigated the phenomenon from different perspectives. the present studies can be seen focusing on four different directions of the phenomenon. the first set of studies (e.g. dolan et al., 2019) have focused on the exploration of the role and effectiveness of such facilities in achieving the entrepreneurial objectives of heis in particular and regional and national communities in general. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 106 these studies look at the external evaluation and offer much importance in determining the functional value of such facilities. these studies, however, do not look at their internal design and further development. the second set of studies (e.g. finkle et al., 2006; finkle et al., 2013; salamzadeh et al., 2011; yıldırım & aşkun, 2012) have focused on their financial and strategic dimensions and have explored the benefits and challenges that the heis encounter while establishing such facilities. the third strand of studies (e.g. jahangir et al., 2014; karimi et al., 2010) have focused more on policy aspects and institutional embodiments and factors that affect the establishment and success of such facilities in particular regions. these two group of studies mainly focus on structural embodiments of these facilities in terms of their external and physical factors and do not elaborate their internal and functional factors that may also influence their working and effectiveness in achieve their targeted goals. the last set of studies (e.g. finkle et al., 2006; ndou et al., 2018) have elaborated the stakeholders of these facilities and determined their relevant activities while participating in these academic ifs. these studies also focus on external factors of participants and do not cater the internal attributes and activities of the facilities that also play an important role. increasingly, to the best of our knowledge, no any study has yet dealt with the phenomenon holistically and captured a complete picture of the paradigm. consequently, research investigating innovation facilities hosted at heis has remained fragmented and thereby a conclusive description of their structural and functional attributes is still missing in scientific literature. in other words, it is still ambiguous that “what an innovation facility located at hei may comprise of, how it can be structured, and what it may offer in terms of its structural and functional propositions?” in order to address above mentioned research gap in scientific literature and offer a comprehensive description of the paradigm, the current paper aims to elaborate structural and functional manifestations of ifs located at heis aggregately from the available literature on the topic and the web description of existing ifs at heis worldwide. the main research question guiding this study is: “how an innovation facility established within hei can be defined and characterized in its structural and functional terms?” more specifically, the study intends to discover a conclusive description of such ifs and determine: a) “what an innovation facility at hei might structurally comprise of?” b) “what objectives an innovation facility at hei might target to achieve?” c) “what entrepreneurial activities an innovation facility at hei might undertake?” and d) “what functionalities and services an innovation facility at hei might offer to its stakeholders?” http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 107 as a result, the paper offers a comprehensive conceptual framework of innovation facilities at heis embodying their several structural and functional key attributes together with their proper possible field configurations. the criteria catalog of ifs envisaged herein should offer several implications. firstly, as the literature review discussed above indicated the lack of all conclusive description of such innovation facilities, the proposed catalog will fill the research gap and advance the understanding of structural and functional attribution of ifs hosted at heis. on the other hand, this would practically assist the hei administrators and other policy makers in planning, initiating, and exercising the establishment of such new facilities in a standardized manner. secondly, it will offer a criteria catalog that will help the administrators and heis to define a particular innovation facility in more standardized manner. similarly, this would facilitate the cross-comparing of multiple facilities and thereby enable to uncover relative strengths and shortcomings of different ifs hosted at different heis. thirdly, it will establish the reference grounds for measuring the comprehensiveness and effectiveness of particular innovation facility at heis and thereby reveal individual challenges that these facilities encounter while undertaking and promoting the entrepreneurial activities within academic environment. the remainder of the paper is structured as follows. section 2 describes the methodological approach of the study. section 3 explains the proposed criteria catalog for the ifs established within the premises of the heis. section 4 concludes the paper with an outlook to future research agenda. 2. research approach this is a conceptual paper wherein understanding of the topic and relevant data is derived from two sources: the available scientific literature on the topic, and the description of existing academic innovation facilities from their official websites. the overall research approach is given in figure 1. initially, related academic literature (in english language) was searched with relevant keywords such as ‘entrepreneurship/innovation/incubation at/and heis/universities’, ‘entrepreneurial university’, ‘academic entrepreneurship and innovation’, ‘student entrepreneurship’, and ‘academic startup/business incubation’ using google scholar search engine. subsequently, the title and abstract of searched articles were reviewed to identify relevant articles and successively full text of selected articles was retrieved. following this, the list of url of innovation facilities located at heis was http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 108 generated by searching the web with google search engine with same keywords as used for literature search. all search hits were manually evaluated to filter out irrelevant hits and duplicate entries. furthermore, the retrieved urls were analyzed to identify cross-links to other such ifs. after that, the urls were examined one by one and all relevant information was retrieved for further analysis. following this, the relevant literature and web description retrieved herein was analyzed following a thematic data analysis approach (bryman, 2012) through the stages of familiarization, coding, theme generation, theme revision, themes definition, and write-up as in ullah and ullah (2021). as a result, the list of several structural and functional dimensions/constructs of the ifs at heis along with their possible field configurations (possible values) was retrieved. finally, a criteria catalog of ifs at heis is constructed based on identified dimensions/constructs. figure 1research approach of the study http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 109 3. criteria catalog of innovation facilities at heis the content analysis of existing literature and web description of existing ifs located within heis undertaken in this study has revealed that there are 12 essential attributes that collectively frame the definition and characterization of ifs at heis. the identified attributes can be distinguished between primary/functional criteria (6 attributes) and secondary/structural criteria (6 attributes). figure 2 presents the identified attributes along with their possible field configurations. it is important to note here that whilst the field values for identified criteria are not disjoint, therefore, with respect to a particular criterion, a specific facility may qualify for more than one possible field value. 3.1. primary / functional criteria the six identified functional criteria for defining and distinguishing ifs include strategic focus, service portfolio, business incubation process coverage, methodological maturity, technology integration, and success indicators. i. strategic focus: the first and most important criterion for defining a particular if is its strategic focus. the content analysis carried out in this study shows that the ifs are aimed at achieving three strategic objectives; i.e. entrepreneurship education and research, industry linkage and networking activities, and business incubation and research commercialization. while some of ifs offer all-inclusive services ranging from entrepreneurship education and learning to the incubation and commercialization activities; the others focus only on the entrepreneurship learning and knowledge creation (maas & jones, 2015). it can be concluded that the most prioritized strategic focus which is integral to all aecs is the entrepreneurship education and research with the vision to encourage students’ inclinations and intentions to act in an enterprising and/or entrepreneurial manner (maas & jones, 2017). it is believed that entrepreneurship process and capabilities can be taught to students by professional before, during, and after the commencement of entrepreneurial activities (matlay, 2008). in this regard, the ifs are focused on designing a suitable entrepreneurship curriculum and executing and practicing the designed modules and programs (maas & jones, 2017). herein, most ifs leverage combination of theoretical and practical approaches for describing the entrepreneurial challenges and business plans together with understanding of case and field studies. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 110 whilst, traditionally entrepreneurial education has been limited to business schools; recently it is considered as relevant for other disciplines too and viewed as an effective way to engage larger part of students (wright et al., 2009) from the engineering, life sciences, and computer science and technology disciplines who are more inclines to produce innovations (secundo & passiante, 2007). in this way, ifs play twofold role of promoting entrepreneurship programs and activities, and also engaging in joint/multidisciplinary programs with other faculties and departments. in regard of entrepreneurial education, the ifs engage in different activities that could be defined as either ‘educating about entrepreneurship’ including the programs aimed at creativity stimulation, selfdependence and personal development, and initiative orientation and entrepreneurial mindset; as ‘education for entrepreneurship’ including the programs aimed at identification of business opportunities, and venture creation and being startup/entrepreneur, or as ‘educating through entrepreneurship’ including the programs aimed at providing abilities and skills for business survival and progress (kirby, 2004). the second strategic focus of ifs at heis could be recognized as industry linkage and networking activities. this focus is supported with the vision of the university to participate in triple helix alliances and to open up the pathways to its research activities and their output for the industry and government agencies. by doing so, the facilities broadens its boundaries from in-house development to the open innovation ecosystem whereby academia (faculty and students) are put in contact with industry through different networking events and activities. as a result, entrepreneurs at university gain insights on market attributes and opportunities whereas industry gain knowledge on new research and developments produced from university. one particular activity in this regard are the networking events that allow the students to present their ideas before interested funding agencies from government and industry and thereby acquire finance for the further development and configuration of their entrepreneurial ideas. the funding partners, often regarded as angels, finance the entrepreneurial activities of students for developing a proof of concept of selective innovative ideas and successively adopt them to their business. the networking events organized by ifs also serve as an open platform for the academia and industry to share their knowledge, competencies, experiences, and skills with each other in order to configure new innovation projects individually as well as in collaboration with each other. ifs, also organize various internal and external idea pitching and competition events that allow the students to present their ideas, solicit feedback from teachers, experts, and peers, and refine and transform them into more viable ones with higher reliability and http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 111 effectiveness. the interactions enable the expansion of networks beyond academic fields and thus the clarification of the benefits and difficulties bound up with the commercialization of research output (fritsch & krabel, 2012; perkmann et al., 2013). the third and most advanced strategic focus of ifs at heis is the business incubation and commercialization to get the research and the innovative products and services produced therein actually implemented in the market. in this regard, ifs in addition to organizing the networking and knowledge sharing events, also facilitate the students throughout the incubation process of their entrepreneurial ideas by providing the expert advice, tangible resources, financing opportunities, and network mediating services. in doing so, these facilitate the shared co-working spaces along with usual amenities and field specific office accessories for the students and faculty to transform their research results and innovative ideas into useful products, and also offer the financing either from internal funds on the profit sharing basis or by acquiring external funding through transfer and commercialization process with external funding agencies. thus, the ifs supporting business incubation and commercialization process encompasses several stages including the discovery, evaluation, dissemination, and registering of research results, and then their marketing and supply to the industry through licensing and formal commercialization (lopes et al., 2018; siegel et al., 2004). the later may take place through oral communication of intangible research results, physical transfer of tangible products, or through an intellectual property licensing program (parker & zilberman, 1993). ii. service portfolio: the second characterizing attribute of an if is its service portfolio. as the innovation process is a multifaceted activity; supporting it in a successful and effective way requires an array of diverse services (memon & meyer, 2020). in this regard, all the services offered by existing ifs can be delineated in three types of services: human facilitation, resources provision, and financial support. human facilitation in the form of teachers and experts is the most integral component of ifs. the facilitators are supposed to offer consultation, mediation, and guidance to the students throughout the innovation process with their methodological and technical expertise and skills. this enables to achieve the objectives of the facility, bring in the group/team dynamics and process management, and get most out of the innovative ideas of students and other participants. the second array of services offered by ifs is the resource provision. the existing ifs are offering a wide array of resources ranging from the availability of a creativity stimulating environment to the usual http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 112 amenities to the field specific technical tools. the dedicated physical environment offered by ifs is supposed to bring a dislocation effect and promote the out-of-the-box thinking of the participants and thereby enhance the novelty of the ideas (lewis & moultrie, 2005; memon et al., 2018). besides the dynamic and vibrant physical space, ifs offer a set of necessary tools to be dynamically configure dependent on the requirements of different innovation projects. this includes the usual work spaces along with office amenities, and the state-of-the-art technology of not only one kind but often the competing technologies for the generation, discussion, assessment, and implementation of the ideas. the combination of methodological skills available from human facilitators and cross cutting technology assists the students in assessing the potential of their ideas, predicting any future obstacles, and thus reducing the risk factor involved in the commercialization of ideas. also, by offering access to all required resources, ifs address and alleviate the dearth of necessary resources and therefore guarantee more fruition of potential ideas in the market. the third service category offered by ifs is the financial support. in this regard, as already discussed, ifs arrange programs and events to interconnect the students with potential investors from industry and government agencies (often regarded as angels) who provide seed money for the actual implementation and actualization of proposed ideas and then are willing to commercialize them together. the second way which the ifs financially aid the students is with the prize money through various idea competitions. in such events, students present their ideas and compete with each other to win cash prizes as well as free services or resource facilities provided by ifs. another way the ifs offer finance to students is through loans or partnership in the selected innovation project. in this regard, ifs either generate their own funding sources or acquire grants from government and other funding agencies and invest that money in potential ideas. iii. process support: the third functional aspect that the ifs can be characterized are the extent and scope of innovation process supported by their offered services. as discussed above, ifs are offering an array of varying services; certain services target to facilitate certain aspects of entrepreneurship and innovation process and therefore bring different functional benefits (memon & meyer, 2020). in this regard, while few ifs offer all-inclusive support throughout the innovation process from idea generation and evaluation to the idea implementation and commercialization. whereas, many ifs only support a part of innovation process and thus their service portfolio can be delineated as pre incubation, incubation, and post incubation. in the context of pre-incubation services, the ifs facilitate front-end of innovation and entrepreneurship process through consultation, http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 113 moderation, and idea generation and evaluation stages. during this process, ifs offer expert advice, brainstorming sessions, and group as well as individual consultation meetings, idea generation and hatching techniques, tools, and methods, and networking events for feedback acquisition and idea refinement. as a result, such ifs leave students with a well-developed idea with strong potential of fruitful implementation whereby the rest of the process is solely relied on students. whereas, in the context of incubation services, the ifs also support the development of proof of concept, actual implementation, and market-launch or industry handover of the resultant product/service. in this regard, ifs facilitate all required operational resources methodological skills and expert advice, tangible resources (office space, raw material, and technological tools, etc.) and finance. herein, the students and if moderators (teachers and administrators) collectively develop the actual innovation prototype, present to potential partners, and get it actually implemented in the market. once, the product/service is implemented in the market as a startup business by the students or in collaboration with an established business, the ifs are no more part of it and students manage it further and carry on their business on their own. in the end, some ifs support innovation process even beyond the commercialization. in doing so, ifs remain in long-term contact with students (mostly while investing in said business on partnership basis) and also engage in the evaluation of offered innovation in the market, soliciting user feedback, and supporting continuous improvement of product or service based on user feedback and changes in market. iv. methodological maturity: the fourth functional aspect characterizing the ifs at heis is the maturity level of their methodological approaches that they apply in intermediating innovation process. as innovation process is not a defined process and thus every project has its own requirements and procedures, there is no any single methodology used by all ifs in assisting their students through the innovation process. furthermore, the intermediation process for the successful innovation development is dependent on several factors such as target market and customer community, involved stakeholders, nature of innovation, time frames for development and commercialization, etc. therefore, in terms of methodological approaches levied by ifs in different projects, they can be either defined as ifs with fixed methodology, varying methodologies, or no defined methodology. the ifs working with a single fixed and very well defined methodological approach are very rare; yet examples exist who adopt same methodological approach in all projects. they usually have their own theoretical or practical methods developed as a result of their own research which they find most handy and effective in undertaking activities that they are engaged in. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 114 the structure and undertaking of such an approach is very standardized and always disclosed to all involved parties and to the public. the other ifs can be defined as ifs with varying methodological approaches. this group of ifs can be regarded as most advanced ones with respect to this particular criteria because they have availability and command over several possible methodological approaches. they have understanding of several existing methodologies that can be applied to a particular situation for accomplishing a particular task and always adopt the most suitable one for a particular project depending on its nature and requirements, and suitability with and preference of the students and other stakeholders who have to work with it. yet another group of ifs exist who have no defined methodology. these ifs are very much in incepting stage and have not devised or decided any particular methodological approach. they always adopt varying approaches on ad hoc and trial basis and often apply more than one to the same task simultaneously. v. technology integration: the fifth functional aspect characterizing the ifs at heis is the maturity level of their technology adoption and integration in achieving their goals. in this regard, the level of technology integration can be considered as low if the if only works with basic usual technological tools and applications for digital content presentation, processing, and storage, and teaching/learning tools. beyond this, technology integration can be recognized as medium if the if, in addition, also leverages the technology for idea generation and evaluation, developing and simulating prototypes, and evaluating market and business plan. at the advanced level, technology integration can be regarded as high if the if works with very specific, sophisticated, and field specific technological tools. it is important to note here that this characterization with reference to technology integration is very relative and can be identified differently in conjunction with stated goals of if, actual activities undertaken therein, and the possible availability of relevant technological tools for various tasks. within the present highly technology-induced era wherein more and more attention is being given to developing technological and technology enabled innovations; leveraging technology to ease the innovation development, and to ensure the effectiveness and applicability of new product or service is very much crucial. therefore, technological maturity of ifs directly relates to and predicts the chances of successful outcome of the innovations being developed therein. also, availability of technological tools enables to produce more viable ideas and aids in testing and developing the proof of concept of abstract ideas which are very difficult to actualize otherwise. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 115 figure 2 criteria catalog of innovation facilities at heis vi. success indicators: the sixth functional aspect characterizing ifs at heis are the factors that ifs perceive as their success indicators. according to this criteria, ifs can be organized in three groups. the first ones are those ifs who mainly focus on providing entrepreneurial education and mentorship. these ifs consider the acceptance and attitude of their students towards the offered services as their success indicators. they assess their effectiveness in terms of students’ learning and efficacy. if students are found satisfied with the knowledge and mentoring received therein, the ifs believe they are working well and achieving their defined target. the second group, in addition to students’ satisfaction with their offered services, also perceive that if their mentored innovative products/services are actually implemented in the market successfully. the;6y follow the ideas up to the launch in the market, collect market feedback, and also engage in the refinement/improvement http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 116 process iteratively until the product/service is fully accepted by the customers. these ifs measure their success in terms of the number and performance of successful innovations actually introduced in the market. the third group of ifs in this regard measures the success in broader aspect. they do not only strive to introduce the innovations in the market, but also assess how the developed innovations are generating revenue and what socio-economic growth they are bringing to the owner, region, and the nation at a larger level. 3.2. secondary / structural criteria the six identified structural criteria for defining and distinguishing ifs at heis include expression of strategic intention, management structure, business model, funding sources, thematic focus, and stakeholder engagement. i. expression of strategic intention: the first and foremost structural aspect of an if is the expression of strategic intention (yıldırım & aşkun, 2012) meaning the entrepreneurial strategy, mission and objectives that the facility is established to pursue and achieve through its dedicated innovation support activities. it has been observed that ifs are established to target entrepreneurial intentions specified at three different levels. first, the entrepreneurial objectives and motivations are encompassed within university’s strategy, mission, and vision and objective/policy statement. such heis target entrepreneurial achievement at larger level and are often regarded as new form of universities; the entrepreneurial universities. these ifs established to achieve university’s objectives are usually university owned and mostly working in multidisciplinary fields wherein faculty and students of different academic departments are involved. at the middle level, ifs are established to streamline the achievement of mission, vision, policy and objectives of individual academic units within heis such as business-related departments or centers. this is the more traditional and more common form of ifs being established worldwide. these department owned ifs are mostly working on single thematic projects in alignment of thematic orientation of the owning department. at the lower and more independent level, the ifs are established with their own agenda and entrepreneurial objectives to achieve. such ifs work independently and are often focused on multidisciplinary entrepreneurial activities. ii. management structure: the second distinguishing structural aspect of ifs is the management structure / positioning of the facility within hei. in this aspect, the facility can be either http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 117 owned by the university and working independent of any academic unit/department, or the facility could be embedded within an academic unit and be controlled by same administration who is responsible for taking care of the owning department. the management structure of ifs aligns with the level and type of entrepreneurial intention and objectives that the facility is established to achieve, and the overall corporate strategy of the hei (maas & jones, 2015). it is observed that within department owned facilities the entrepreneurship goals of the students and faculty are pursued, while in university owned facilities the interdisciplinary projects involving people from various faculties including humanities, life sciences, engineering, and business are undertaken. on the contrary, there also exist examples of ifs which are established within hei premises but are controlled with independent administration. such self-administered facilities are either established with external funding/grant acquired within the context of some externally funded research/development project or are established to achieve a designated specific task and thus have a shallower, but an interdisciplinary and multidisciplinary, focus and entrepreneurial objectives. iii. business model: the third aspect that can distinguish existing ifs is their business model. it has been revealed from the content analysis that ifs are either working on a not-for profit model with public/private finance. these facilities are offering their services as part of public services for their students, faculty or society and are more focused on developing entrepreneurial skills and enabling participants in undertaking their entrepreneurial projects. the students, as part of their educational stay at heis, are trained and guided to transform their startup ideas into well-framed and established businesses. typical services of such centers include credit based entrepreneurial education, idea and business plan competitions, and student internships and training programs. whereas, the ifs might be established with profit-oriented business model whereby they, besides facilitating entrepreneurial activities of students and faculty, also sale their services to external people form industry and society in order to earn revenue that can be used to run the facility or engage in other entrepreneurial development work. such facilities work more as business units and are not only engaged in training and teaching soft skills but also on more specialized services for the business community and often support business incubation or innovation development throughout the process. examples of external offerings of such ifs include executive education programs, practical startup mentoring and incubation services, industry startup competitions, and paid seminar/workshops on various entrepreneurship related topics such as business planning, strategic planning and management, and fund acquisitions and utilizations. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 118 iv. funding sources: the next structural aspect that the ifs can be differentiated is their funding source. the ifs can be seen operating on internal finance provided by the hei itself as part of its broader vision towards entrepreneurial activities. such ifs operate under direct control of the hei or business-related academic unit and are mostly offering their services for the students as part of their standard curriculum and teaching and training modules. on the other hand, ifs are also being operated with external finance wherein they engage in several fund raising activities and seek finance from grants, donations, and funded projects. these are the facilities which are not established by the heis to comprehend their overall entrepreneurial intentions, rather they are established as part of external project or as an advance step of the hei in achieving a particular milestone without any proper budget from internal finance sources. besides, there exist ifs which are running with share of both internal and external finance. these facilities are initially setup with internal finance from hei and continue receiving some fixed or variable amount from internal sources, but also engage in external fund raising programs in order to support usual activities and further development and enhancement of service portfolio. according to finkle et al. (2013), the internal fund raising activities that the ifs engage in might include idea/business plan competitions, internships/students clubs, technology parks/incubators, technology transfer and venture capital fund, and distance learning programs. while the external activities that ifs engage in to raise funds might include seminar/workshops, grants, executive education programs, entrepreneur competitions and venture capital fund, and incubators and practical entrepreneur/startup programs. v. stakeholder engagement: another defining attribute of ifs is the stakeholder engagement. while students and faculty are basic stakeholders who are facilitated by all ifs in working on their entrepreneurial ideas and cultivating and effectively implementing them in the market. in this regard, the department owned ifs seem to be open to students of respective department only; whereas the university controlled ifs tend to be open to students and faculty of all departments and thus are mostly multidisciplinary in their thematic orientation. on the other hand, some ifs are also triggering collaborative linkages with other stakeholders that includes the industry partners, other heis, and also the wider user communities and regional startups from civil society. this allows such ifs to extend the impact of their activities, encompass wide array of ideas and evaluations, and thereby develop most effective product/service systems. these ifs, besides usual internal and external entrepreneurial activities, are also holding various networking events such as conferences, open idea competitions, idea or business plan fairs and exhibitions, etc. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 119 vi. thematic scope: the last structural attribute defining a particular if is its thematic scope which refers to the domains/areas wherein it supports the entrepreneurial activities and business incubation. the ifs established within academic units seem to have specific focus related to the department that they are part of. for example, the if established within business department concentrate towards entrepreneurship process in a general way, whereas an if established within it department supports the hatching and fruition of technology oriented ideas. as opposed to these department owned ifs, the ifs operating under the management of university and not any particular academic unit are more open in their scope and therefore support entrepreneurial ideas in multiple domains often described as multidisciplinary/interdisciplinary innovations. 4. conclusion the current paper deals with the concept of innovation facilities which are increasingly being established within premises of higher education institutions in order to adopt the triple helix of industry-academia-government linkages for ensuring the proper utilization of research results of heis into the market. research indicates that the existing ifs at heis are varying in their structure and functioning and thereby literature on the topic has explored them from different perspectives. in this regard, the objective of this paper is to offer a comprehensive criteria catalog of ifs at heis for understanding the paradigm in a holistic manner and enabling the comparison and differentiation of various facilities with each other. the current paper based on the purposeful observation and review of available scientific literature on the topic and the web description of existing ifs at heis worldwide proposes a criteria catalog of 12 attributes (cf. figure 2). in terms of functional manifestations, ifs can be differentiated in terms of their strategic orientation and objectives that they aim to focus, array of services they offer, stages of innovation process they support through their offered services, maturity of their methodological approaches, extent of their technology leverage, and the factors that they consider as indicators of their success. whereas, in terms of structural manifestations, ifs can be differentiated with respect to the expression of their strategic intention that they strive to achieve, management structure, business model, funding sources, types of stakeholders who are engaged therein, and the thematic focus of innovation that they support to develop. http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 120 the proposed criteria of ifs envisaged herein offers several implications. firstly, it will aid in understanding the overall structure and functioning of ifs hosted at heis and thereby help in conceptualizing and establishing new facilities at heis in a standardized manner. secondly, it will help to define and assess the comprehensive and effectiveness of a particular if and also crosscompare multiple ifs for discovering their relative strengths and shortcomings. the current paper has proposed a criteria catalog which opens research directions for further pursuit. one, empirical research on existing ifs at heis within and across different regions and economies might be undertaken to cross compare and thereby identify their similarities and differences. this would enable to reveal their relative strengths and shortcomings as well. two, extended qualitative and quantitative studies may be undertaken to further explore and verify the completeness and generalizability of the proposed criteria catalog of ifs at heis. author contributions: this paper is co-authored by atia bano memon (a.b.m) and shakeel ahmed memon (s.a.m). conceptualization, a.b.m. and s.a.m.; methodology, a.b.m.; investigation and formal analysis, a.b.m. and s.a.m.; validation, s.a.m.; writing original draft preparation, a.b.m.; writing review and editing, s.a.m. all authors have read and agreed to the published version of the manuscript. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references aguilar-santelises, l., corona-ortega, m. t., cruz-millán, m., rojas-fernández, m., aguilar-santelises, m., & garcía-del valle, a. (2015). keeping teachers up-to-date is essential to reach and maintain a high quality university education. multidisciplinary journal for education, social and technological sciences, 2(1), 1–14. https://doi.org/10.4995/muse.2015.2204 bryman, a. (2012). social research methods: oup oxford. canzer, b. (1997). marketing education on the internet: a world wide web based introductory marketing course design for the virtual-u project in distance education at simon fraser university. journal of marketing education, 19(3), 56–65. https://doi.org/10.1177/027347539701900306 clauss, t., moussa, a., & kesting, t. (2018). entrepreneurial university: a stakeholder-based conceptualisation of the current state and an agenda for future research. international journal of technology management, 77(1-3), 109–144. https://doi.org/10.1504/ijtm.2018.091726 dolan, b., cunningham, j. a., menter, m., & mcgregor, c. (2019). the role and function of cooperative research centers in entrepreneurial universities. management decision. advance online publication. https://doi.org/10.1108/md-10-2018-1172 etzkowitz, h., & leydesdorff, l. (2000). the dynamics of innovation: from national systems and “mode 2” to a triple helix of university‐industry‐government relations. research policy, 29(2), 109–123. https://doi.org/10.1016/s0048-7333(99)00055-4 http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4995/muse.2015.2204 https://doi.org/10.1177/027347539701900306 https://doi.org/10.1504/ijtm.2018.091726 https://doi.org/10.1108/md-10-2018-1172 https://doi.org/10.1016/s0048-7333(99)00055-4 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 121 finkle, t. a., kuratko, d. f., & goldsby, m. g. (2006). an examination of entrepreneurship centers in the united states: a national survey. journal of small business management, 44(2), 184–206. https://doi.org/10.1111/j.1540-627x.2006.00163.x finkle, t. a., menzies, t. v., kuratko, d. f., & goldsby, m. g. (2013). an examination of the financial challenges of entrepreneurship centers throughout the world. journal of small business & entrepreneurship, 26(1), 67–85. https://doi.org/10.1080/08276331.2012.761803 fritsch, m., & krabel, s. (2012). ready to leave the ivory tower? academic scientists’ appeal to work in the private sector. the journal of technology transfer, 37(3), 271–296. https://doi.org/10.1007/s10961-0109174-7 garo, e., kume, v., & basho, s. (2015). determinants of entrepreneurial intention among university students: case of albania. multidisciplinary journal for education, social and technological sciences, 2(2), 176– 190. https://doi.org/10.4995/muse.2015.3691 jahangir, y. f., modarresi, m., motavaseli, m., & salamzadeh, a. (2014). institutional factors affecting academic entrepreneurship: the case of university of tehran. economic analysis, 47(1-2), 139–159. karimi, s., chizari, m., biemans, h. j. a., & mulder, m. (2010). entrepreneurship education in iranian higher education: the current state and challenges. european journal of scientific research, 48(1), 35–50. kirby, d. a. (2004). entrepreneurship education: can business schools meet the challenge? education+ training. advance online publication. https://doi.org/10.1108/00400910410569632 lewis, m., & moultrie, j. (2005). the organizational innovation laboratory. creativity and innovation management, 14(1), 73–83. https://doi.org/10.1111/j.1467-8691.2005.00327.x lopes, j. n. m., farinha, l.\’\i. m. c., ferreira, j. j. m., & ferreira, f. a. f. (2018). peeking beyond the wall: analysing university technology transfer and commercialisation processes. international journal of technology management, 78(1-2), 107–132. https://doi.org/10.1504/ijtm.2018.093936 maas, g., & jones, p. (2015). an overview of entrepreneurship education. systemic entrepreneurship: contemporary issues and case studies, 22–95. https://doi.org/10.1057/9781137509802_2 maas, g., & jones, p. (2017). the role of entrepreneurship centres. in entrepreneurship centres (pp. 11–16). springer. https://doi.org/10.1007/978-3-319-47892-0_2 marques, c., marques, a. v., braga, v., & ratten, v. (2021). technological transfer and spillovers within the ris3 entrepreneurial ecosystems: a quadruple helix approach. knowledge management research & practice, 19(1), 127–136. https://doi.org/10.1080/14778238.2020.1777909 marques, h. r., oliveira, t. a. d., andrade, d. m., & zambalde, a. l. (2019). university entrepreneurship in brazil: panorama of the technological innovation centers of universities. world journal of entrepreneurship, management and sustainable development, 15(2), 149–158. https://doi.org/10.1108/wjemsd-10-2018-0091 matlay, h. (2008). the impact of entrepreneurship education on entrepreneurial outcomes. journal of small business and enterprise development. advance online publication. https://doi.org/10.1108/14626000810871745 memon, a. b., & meyer, k. (2020). towards the functional roles of an innovation laboratory as a platform for innovation: an observational approach. in disruptive technology: concepts, methodologies, tools, and applications (pp. 555–574). igi global. https://doi.org/10.4018/978-1-5225-9273-0.ch027 memon, a. b., meyer, k., thieme, m., & meyer, l.-p. (2018). inter-innolab collaboration: an investigation of the diversity and interconnection among innovation laboratories. journal of engineering and technology management, 47, 1–21. https://doi.org/10.1016/j.jengtecman.2017.11.003 http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1111/j.1540-627x.2006.00163.x https://doi.org/10.1080/08276331.2012.761803 https://doi.org/10.1007/s10961-010-9174-7 https://doi.org/10.1007/s10961-010-9174-7 https://doi.org/10.4995/muse.2015.3691 https://doi.org/10.1108/00400910410569632 https://doi.org/10.1111/j.1467-8691.2005.00327.x https://doi.org/10.1504/ijtm.2018.093936 https://doi.org/10.1057/9781137509802_2 https://doi.org/10.1007/978-3-319-47892-0_2 https://doi.org/10.1080/14778238.2020.1777909 https://doi.org/10.1108/wjemsd-10-2018-0091 https://doi.org/10.1108/14626000810871745 https://doi.org/10.4018/978-1-5225-9273-0.ch027 https://doi.org/10.1016/j.jengtecman.2017.11.003 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse social and technological sciences e-issn: 2341-2593 memon & memon (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 103-122. https://doi.org/10.4995/muse.2022.17683 122 ndou, v., secundo, g., schiuma, g., & passiante, g. (2018). insights for shaping entrepreneurship education: evidence from the european entrepreneurship centers. sustainability, 10(11), 4323. https://doi.org/10.3390/su10114323 parker, d. d., & zilberman, d. (1993). university technology transfers: impacts on local and us economies. contemporary economic policy, 11(2), 87–99. https://doi.org/10.1111/j.1465-7287.1993.tb00382.x perkmann, m., tartari, v., mckelvey, m., autio, e., broström, a., d’este, p., fini, r., geuna, a., grimaldi, r., hughes, a., & others (2013). academic engagement and commercialisation: a review of the literature on university‐industry relations. research policy, 42(2), 423–442. https://doi.org/10.1016/j.respol.2012.09.007 roger-monzó, v., guijarro-garcía, m., & martí-sánchez, m. (2015). digital signage: an experience of innovation in higher education. multidisciplinary journal for education, social and technological sciences, 2(2), 1–13. https://doi.org/10.4995/muse.2015.3769 salamzadeh, a., salamzadeh, y., & daraei, m. (2011). toward a systematic framework for an entrepreneurial university: a study in iranian context with an ipoo model. secundo, g., & passiante, g. (2007). an innovative approach to creating business leaders: evidence from a case study. international journal of management in education, 1(3), 214–230. siegel, d. s., waldman, d. a., atwater, l. e., & link, a. n. (2004). toward a model of the effective transfer of scientific knowledge from academicians to practitioners: qualitative evidence from the commercialization of university technologies. journal of engineering and technology management, 21(1-2), 115–142. https://doi.org/10.1016/j.jengtecman.2003.12.006 tunio, m. n. (2020). academic entrepreneurship in developing countries: contextualizing recent debate. in p. sinha, j. gibbs, m. e. m. akoorie, & j. m. scott (eds.), research handbooks in business and management series. research handbook on entrepreneurship in emerging economies: a contextualized approach (pp. 130–146). edward elgar publishing limited. https://doi.org/10.4337/9781788973717.00014 ullah, r., & ullah, h. (2021). teachers’ perspectives on boys’ underperformance in education in khyber pakhtunkhwa, pakistan. journal of elementary education, 14(1), 113–123. https://doi.org/10.18690/rei.14.1.113-123.2021 wood, m. s. (2011). a process model of academic entrepreneurship. business horizons, 54(2), 153–161. https://doi.org/10.1016/j.bushor.2010.11.004 wright, m., piva, e., mosey, s., & lockett, a. (2009). academic entrepreneurship and business schools. the journal of technology transfer, 34(6), 560–587. https://doi.org/10.1007/s10961-009-9128-0 yıldırım, n., & aşkun, o. b. (2012). entrepreneurship intentions of public universities in turkey: going beyond education and research? procedia-social and behavioral sciences, 58, 953–963. https://doi.org/10.1016/j.sbspro.2012.09.1074 http://polipapers.upv.es/index.php/muse https://doi.org/10.4995/muse.2022.17683 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.3390/su10114323 https://doi.org/10.1111/j.1465-7287.1993.tb00382.x https://doi.org/10.1016/j.respol.2012.09.007 https://doi.org/10.4995/muse.2015.3769 https://doi.org/10.1016/j.jengtecman.2003.12.006 https://doi.org/10.4337/9781788973717.00014 https://doi.org/10.18690/rei.14.1.113-123.2021 https://doi.org/10.1016/j.bushor.2010.11.004 https://doi.org/10.1007/s10961-009-9128-0 https://doi.org/10.1016/j.sbspro.2012.09.1074 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 87 doctoral education in ukraine: the application of digital tools and services by doctoral students under covid-19 pandemic lyudmyla khoruzhaa , olena lokshynab* , nataliia mazura , volodymyr proshkina a borys grinchenko kyiv university, 18/2 bulvarno-kudriavska st., kyiv 04053, ukraine b institute of pedagogy of the national academy of educational sciences of ukraine, 52 d sichovych striltsiv st., kyiv, 04053, ukraine * correspondence: olena.lokshyna@gmail.com received: 29 november 2021; accepted: 08 march 2022; published: april 2022 abstract the paper aims to explore the peculiarities of doctoral education under covid-19 in ukraine. the application of digital instruments and services by doctoral students has become a priority during distance training. however, this approach is mainly a new experience and, accordingly, undergoes theoretical substantiation and piloting. the focus of the study is two-fold. it theoretically discusses the digital support of dissertation research at various stages. secondly, based on the findings of the survey on the doctoral students’ application of digital tools and services the challenges have been identified. we conclude that the priority of digital education in the eu countries, especially under covid-19 pandemic highlights the task of restructuring the doctoral education in ukraine towards its dynamic digitalization. the digitization should be comprehensive and systematic. systematization should include technical, organizational, motivational and developmental components aimed at synergy of institutions, doctoral students and supervisors. the complexity should include both the digitization of the training component and independent work of the doctoral students while writing a dissertation. the groups of dt&s for application by doctoral students are recommended for the inclusion into the doctoral students’ training curriculum both in the format of a separate course and by integration into the syllabi of other disciplines. based on the research findings the targeted scientific/methodical events (seminars, master classes, doctoral schools, etc.) providing knowledge and skills on dt&s application are recommended. keywords: doctoral education, doctoral students; digital instruments and services; university; ukraine. to cite this article: khoruzha, l., lokshyna, o., mazur, n., proshkin, v. (2022). doctoral education in ukraine: the use of digital tools and services by doctoral students under covid-19 pandemic. multidisciplinary journal for education, social and technological sciences, 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://orcid.org/0000-0003-4405-4847 https://orcid.org/0000-0001-5097-9171 https://orcid.org/0000-0001-7671-8287 http://orcid.org/0000-0002-9785-0612 mailto:olena.lokshyna@gmail.com multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 88 1. introduction the perspective of europe’s digital sovereignty by building and deploying technological capabilities in a way that empowers people and businesses to seize the potential of the digital transformation is proclaimed in 2030 digital compass. digitally skilled population and highly skilled digital professionals are named as a target for achieving solidarity, prosperity, and sustainability (european commission, 2021). the eu vision of digital skills and competences is outlined in the european digital competence framework, also known as digcomp (vuorikari et al., 2016) and updated in digcomp 2.1: the digital competence framework for citizens (european commission, 2017). the digcomp 2.1 identifies the components of digital competence in the areas of information and data literacy, communication and collaboration, digital content creation, safety, problem solving. skills to locate and retrieve digital data, information and content; interact, communicate and collaborate through digital technologies; create and edit digital content; identify needs and problems, and to resolve conceptual problems and problem situations in digital environments, etc. are defined as basic for the eu citizens. making better use of digital technology for teaching and learning, improving education through better data analysis and foresight and enhancing digital skills and competences for the digital transformation are proclaimed a priority in the eu digital education action plan (20212027) (european commission, 2020). the digital literacy has been declared a necessary condition for the development of the european educational area by 2025 (european commission, 2020a). this actualizes the mission of education, respectively universities are viewed from the standpoint of leaders in generating innovations. 2017 communication on a renewed eu agenda for higher education (european commission, 2017a) has confirmed the importance of strengthening links between education, research and innovation. given this high quality doctoral education is viewed critical for the promotion and adoption of new ideas. the salzburg principles developed at the bologna seminar on “doctoral programmes for the european knowledge society” (bologna seminar, 2005) and enriched in the salzburg ii recommendations in berlin (european university association, 2010) provide universities with clear ideas to cultivate the research mindset, nurture flexibility of thought, creativity and intellectual autonomy through an original, concrete research project. however, the dynamic http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 89 development of the world, which involves the digitalization of all spheres of human life, is making changes in the organization of doctoral education. in the digital world a modern researcher is to perform many research tasks, namely to: • carry out the expanded search and use data from various databases within the research, including scientometric; • collect statistical data and use digital tools (software applications, online calculators, etc.) for scientific statistical data analysis; • adapt scientific knowledge digital spaces for the exchange of experience (conferences, seminars, scientific associations, etc.); • use digital tools to disseminate research results (web publications, blogs, etc.). the covid-19 pandemic has intensified changes of the doctoral education format. they are complex and multi-layered taking the application of digital tools and services (dt&s) by doctoral students to a new level. the study aims to explore the peculiarities of the application of dt&s by doctoral students during research in universities under covid-19 in ukraine. this paper is, therefore, intended to respond to the following research questions:  what types of dt&s correspond to the specifics of doctoral research?  what dt&s do doctoral students use at different stages of their research?  what factors inhibit doctoral student to apply dt&s while doing dissertation research? 2. literature review in order to conduct literature review, the authors carried out literature search collecting relevant papers/documents focusing on digitalisation/digital transformation, doctoral education, the http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 90 use of dt&s by doctoral students. priority was given to the papers/documents published after the covid-19 pandemic had begun. the eurodoc documents provided a holistic view of the doctoral education in europe under covid-19 pandemic. the sources analysed include also the academic journals papers, which the authors classified into three categories, namely digitalisation/digital transformation as a phenomenon of today; digitalisation of higher education; digitalisation of doctoral education. 2.1. the digitalisation/digital transformation as a phenomenon of today google scholar finds approximately 757,000 “digitization” hits, 599,000 “digitalization” hits, 115,000 “digitalisation” hits and 4 190 000 “digital transformation” hits. it confirms the importance of the scientific understanding of the holistic integration of digital technology into all areas of human life. the researchers focus on the interrelationships and interactions between digitization (as a process of conversion of analog data into digital format) and digitalization, which in randall’s et al. (2018) vision means the transformation of all sectors of our economy, government and society based on the large-scale adoption of existing and emerging digital technologies. as a result of the terminology study van veldhoven & vanthienen (2019) collected seventeen definitions of the digital transformation that emphasize various aspects of this process, i.e. use of digital technologies, society transformation, organizational transformation, digital innovation, digital economy, new business models, etc. based on the developed digital transformation framework van veldhoven & vanthienen (2019) define the digital transformation as the continuously increasing interaction between digital technologies, business, and society, which has transformational effects and increases the change process’s velocity, scope, and impact. 2.2. the digitalisation of higher education the studies on the digitalization of higher education are particularly relevant as this sector is considered to be crucial in providing the highly skilled human capital for economic growth. the studies address the relationship between digital transformations and the developments in higher education, the impact of digital transformations on academic teaching, ways to overcome barriers/gaps in digital teaching and lack of digital teaching/ learning resources to increase the potential of higher education. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 91 the problem of digital skills and competencies is no less relevant. the scholars state that digital transformation covering the development of new infrastructure and increasing use of digital technologies and resources for instruction necessitate mastering digital skills by students and teachers. garcía & ferrando (2014) underline that digital literacy requires the high level use of technological resources. araújo, cardoso, toubes & fraiz (2020) write about the importance of mastering digital competence by students in spanish university education. basing on the findings of the research the scholars recommend including curricular programme methodologies that provide skills to younger generations to make the use of new technologies expedient. sales, cuevas-cerveró & gómez-hernández (2020) hold a similar opinion advocating incorporation of the information and digital competence into the work methodology of all subjects in a university. on the example of borys grinchenko kyiv university khoruzha, proshkin & hlushak (2020) prove the effectiveness of digital technologies (distance learning systems, electronic libraries, scientific and metric databases, resources for communication and cooperation with mobile devices, services for real-time surveys time, online video services, resources for creating multimedia presentations and interactive testing of existing knowledge, skills, etc.) to implement the training programme. based on a systemic literature review of the extant literature on adopting e-learning systems at higher education institutions awan, afshan & memon (2021) argue the importance of such systems adoption. the paper identifies and puts forward the level of compatibility and readiness of students and teachers in adopting e-learning, factors that motivate and hinder the adoption of e-learning respectively, benefits of adopting an e-learning system, and the strategies for implementing e-learning at higher education institutions. the research of erdmann, presedo & valdes (2021) on the digital transformation of universities caused by covid-19 pandemic is especially relevant. the authors describe the process of digitalization of educational institutions and analyse students’ insights regarding the implementation of hybrid methodologies of learning and identify potential differences in perception across university degrees. 2.3. the digitalisation of doctoral education google scholar finds approximately 10,900 “digitalisation of doctoral education” hits, 30,300 “digitalization of doctoral education” hits and 640,000 “doctoral education under covid-19” hits that indicate the relevance of this problem for the research. the scholars stress the importance of http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 92 ict for raising both the efficiency of the research and the quality of the research product. at the same time, the research of sanchez-macias & veytia-bucheli (2020) shows that doctoral students often do not have a higher level of digital competence solely because they belong to this level of education. this conclusion correlates with opinion of sim (2016) who is convinced that doctoral students require greater support to enhance the ict usage during their study because of low “eliteracy”. the author advocates for a shift to a new understanding of doctoral study schema by evolving a mutual understanding of ict application among institutions, phd supervisors, and phd students in the doctoral research. batool et al. (2021) have concluded that use of ict has a significant impact on the performance of the research students. the covid-19 pandemic has accelerated the digitization of doctoral education, intensifying challenges. to the common to europe, the eurodoc referrers obstacles for geographical or intersectoral mobility under lockdowns; a lack of interaction of doctoral candidates with supervisors/mentors, with obvious effects on their work; problematic access to research resources (non-digital libraries, lab physical resources, etc.) or at precise physical locations (archaeological, geological sites, etc.); lack of an adequate working environment at home; lower productivity and increased precarity; increased sense of isolation and anxiety (eurodoc, 2020). besides, the scholars underline the problem of the so called “digital divide” (persky et al., 2020), negative impact on the overall psychological health of doctoral students and lack of interaction with peers and in-person discussions (varadarajan, brown & chalkley, 2021). important for our study are the findings of lambrechts & smith (2020) relating the problem of data collection (interviews and focus groups survey) by doctoral students during covid-19 pandemic. the responders of their survey have reported about the necessity to move data collection online, which requires updating the research strategy and actualizes the availability of technical conditions. the use of multitude of online tools for the research is becoming the norm today and is assessed as enhancing student academic learning (börgeson et al., 2021). at the same time, the literature analysis shows that most research on this topic concerns the use of ds&t during doctoral training. innovativeness is that the paper offers recommendations on raising the effectiveness of the use of dt&s by doctoral students in the process of their individual work on writing a dissertation. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 93 3. methodology in the study, we follow marakovits’s (2021) understanding of the dt&s as internet-based programmes and resources which can be used to support, enhance, and facilitate the doctoral research and instruction. both qualitative and quantitate research methods have been used. the international organisations’, governmental and institutional websites; research papers at google scholar (https://scholar.google.com/); reference resources at wikipedia (https://en.wikipedia.org/wiki/main_page) have been addressed to search for various documents on education digitalisation. to map the findings, a content analysis method has been applied, with which the documents/papers were read to select keywords linked to digitalisation of a) education; b) higher education; c) doctoral students’ education. analysis, synthesis, generalization, systematization, comparison of the research findings have made it possible to develop the authors’ vision of the dt&s that meet the specifics of doctoral education. the survey on the doctoral students’ usage of digital tools and services during their research was conducted during may-june 2021. the target group comprised 35 doctoral students of the 1 st – 2 nd years and 19 doctoral students of the 3 rd – 4 th years of the borys grinchenko kyiv university (kyiv, ukraine), alfred nobel university (dnipro, ukraine), donbass pedagogical university (slavyansk, ukraine) and pavlo tychyna uman pedagogical university (uman, ukraine). the google form questionnaire was used for data collection. conducting the survey at several universities made it possible to identify common characteristics of the dt&s use of by doctoral students under covid-19 in ukraine. the data derived from the survey have been analysed to explore the peculiarities of dt&s application by doctoral students. the survey instrument delved into the following domains: doctoral students’ attitude towards the use of dt&s at different stages of the dissertation research; experience of doctoral students to apply dt&s for implementing the dissertation research; factors that inhibit the use of dt&s for the research; dt&s proficiency level of doctoral students during dissertation research; prospects to improve the efficiency of dt&s use while doing dissertation research. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://scholar.google.com/ https://en.wikipedia.org/wiki/main_page multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 94 4. findings 4.1. the authors’ vision on the dt&s application by doctoral students during dissertation research under covid-19 pandemic, the use of dt&s s becomes a necessity for the doctoral research. at the same time, such research has its own specifics, to the key features to which we referrer the phasing character of doctoral research (that traditionally covers elaboration of the theoretical basis; generalization of the theoretical research results; designing and doing an experiment; statistical processing of experimental results; preparation and publication of research articles on the research findings; etc.); the sufficient level of digital skills of most doctoral students; prevalence of the independent format of doctoral students’ work during writing dissertation. therefore, it is important to select dt&s corresponding to such specifics, the number and variety of which is very large. in the study the following criteria were developed to select such resources: the compliance with the specifics of the dissertation research; providing a researcher with an opportunity to conduct research at all stages (search for publications, compiling bibliography, doing experiment, statistical processing of quantitative data, etc.), matching with the level of digital skills of doctoral students, the possibility of application on most digital devices. according to the criteria, dt&s were grouped into clusters. electronic search and usage of information. during the writing of the theoretical part of the dissertation it is necessary to search for and analyse the scientific works of scholars who conduct research in the area relating of the doctoral student’s dissertation. electronic search and use of available information from the specialized electronic libraries, institutional repositories and online databases can significantly reduce time. as a result, a large amount of data is accumulated that need to be stored and processed correctly. the bibliographic managers, i.e. programs or online services for bibliography management are the most valid to facilitate such activities. examples of such resources are bibme (www.bibme.org), classtools.net (www.classtools.net), writinghous (www.writinghouse.org), makecitation (www.makecitation.com/index.php), endnote (https://endnote.com), zotero (www.zotero.org), diigo (www.diigo.com), etc. diigo is one of the simplest and the most effective one for these purposes. for correct and convenient work with this online service it is necessary to make the eponymous application to the browser in which the http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 http://www.bibme.org/ http://www.classtools.net/ http://www.writinghouse.org/ http://www.makecitation.com/index.php https://endnote.com/ http://www.zotero.org/ http://www.diigo.com/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 95 researcher works, and to register on a resource. next, viewing the materials of articles, books, images from various sources, you need to click on the appropriate icon of the application in the upper right corner of the browser, fill in the appropriate fields (title; search keywords; section of the study to which this material belongs; if necessary, specify a group for collective processing of the specified material) and click save. as a result, a link to the diigo will be added to the specified section or group (fig. 1). figure 1 page viewing materials to a specific section on the diigo resource source: own work the advantages of the bibliographic manager diigo include: • ability to save links to various sources: web pages, images, pdf-files; • ability to group sources by a certain criterion; • possibility of creating groups and further study of the sources of the group as a researcher and his supervisor (scientific consultant); • there is an option to add handwritten annotations and notes to each added source; • simple and user-friendly interface; • lack of unnecessary advertising that distracts from work. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 96 the disadvantages include the following: lack of support for languages other than english; in the free package there are restrictions on working in only three groups and adding links of certain types, etc. research design and preparation. during the design and preparation of a dissertation research or scientific article, it is often necessary to present the material in a concise schematic form or in the form of an image. to do this, text editors have the appropriate graphics tools (for example, smartart in a microsoft word word processor), but they have limited functionality and do not always allow the researcher to present the material in the appropriate form. therefore, it is advisable to use elements of infographics and tools for the development of mental/knowledge maps for the design of schemes, certain algorithms or generalizing images. information graphics or infographics are graphical visual representations of information, data, or knowledge designed to display complex information quickly and clearly (infographics). infographics creation services have a number of advantages: clarity of the interface and simplicity of work; a large number of ready-made templates to fill in your own data; the ability to use both static and animated elements; presentation of various types of data (analytical; description of a certain event or algorithm of actions; presentation of data chronologically, etc.); the result of the work is stored in the form of an image that is easy to place in a dissertation or scientific article. the algorithm for developing infographics is quite simple: choose a resource and register for it; among the proposed templates to choose the most appropriate for the purpose of creating such an image; highlighting any blocks in the template, they can be edited or formatted to your liking; save the received image with data on the resource (it will be possible to always return and edit something if necessary); add the finished image to the dissertation. the resources for creating infographics we propose are piktochart (https://piktochart.com), сreately (https://creately.com), visual.ly (https://visual.ly), google charts (https://developers.google.com/chart), easel.ly (https://www.easel.ly), сacoo (https://cacoo.com), etc. a mind map or knowledge map is a diagram that displays words, ideas, tasks, or other elements radially around the main word or idea (mind map). an example of a knowledge map is given in fig. 2. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://piktochart.com/ https://creately.com/ https://visual.ly/ https://developers.google.com/chart https://www.easel.ly/ https://cacoo.com/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 97 figure 2 example of a knowledge map of the structure of the discipline, developed on the resource mindomo source: own work the most popular services for developing mental maps are coogle (https://coggle.it), mindmeister (https://www.mindmeister.com), mindomo (www.mindomo.com), bubblus (https://bubbl.us) and others. some researchers attribute mental maps to infographics, but this, in our opinion, is not entirely correct. a mental map allows a researcher to focus on certain keywords or theses, while infographics allow visualizing the idea in general. the advantages of using knowledge maps when writing a dissertation or scientific article include: • simple and user-friendly interface; • quick and complete overview of a large topic (areas, problems, topics); • ability to use multimedia objects and links to other online resources; • collect and present a large amount of different data on one sheet, showing the links and distances; • save time; • stimulate imagination and problem solving by developing new ways, etc. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://coggle.it/ https://www.mindmeister.com/ http://www.mindomo.com/ https://bubbl.us/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 98 doing a pedagogical experiment. a pedagogical experiment to confirm the effectiveness of the proposed ideas with quantitative and qualitative data is an important part of a dissertation. to do this, the doctoral student is to develop appropriate tools for obtaining statistical data and conduct an appropriate analysis of the results obtained according to relevant statistical criteria. it is advisable to use various online services to develop a questionnaire/test. such services available on the internet include google form (www.google.com/intl/ru_ua/forms), kahoot! (https://create.kahoot.it), mytest (http://mytest.klyaksa.net), quizlet (https://quizlet.com) and others. the online service google form is especially popular among scholars in ukraine due to the possibility to develop a single questionnaire for different categories of survey participants. to do this, a researcher needs to develop separate sections for different groups of respondents, and then in the first question to set the triggers-transitions to the appropriate section (fig. 3). figure 3. example of a trigger question in the google form service source: own work in addition, the advantages of this resource include: • free of charge; • ease of operation and clarity of the interface; http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 http://www.google.com/intl/ru_ua/forms https://create.kahoot.it/ http://mytest.klyaksa.net/ https://quizlet.com/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 99 • availability of various questions and support for multimedia content; • viewing the answers received both on the site in the form of tables and diagrams, and the ability to export data to a spreadsheet for further processing; • joint work in the development of questionnaires and questionnaires, etc. according to the requirements to dissertation research the obtained findings of the ascertaining and formative stages of the pedagogical experiment should be analysed, applying certain statistical criteria in accordance with the chosen scale of data presentation. the pearson or kolmogorov-smirnov criteria are recommended if the results of the study are presented in an interval school, and the student's or mann-whitney criteria when using a relative scale. to do this, we consider the specialized data processing software, such as spss statistics, statistica, statplus or ms excel spreadsheet to be the most efficient for this. statistica, developed by statsoft implements procedures for data analysis, data management, data acquisition and visualization (fig. 4). figure 4 example of a trigger question in the google form service source: own work http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 100 the main features of the program are:  contains a complete set of classical methods of data analysis: from basic methods of statistics to progressive methods, which allows flexible organization of analysis;  is a means of building applications in specific areas;  the set of software includes specially selected examples that allow you to systematically master the methods of analysis;  meets all windows standards, which allows you to make the analysis highly interactive;  the system can be integrated into the internet;  supports web-formats: html, jpeg, png;  easy to learn, and as practice shows, users from all applications quickly learn the system;  statistica system data is easily converted into various databases and spreadsheets;  supports high-quality graphics, which allows you to effectively visualize data and perform graphical analysis;  is an open system: contains programming languages that allow you to extend the system, run it from other windows applications, such as excel. in addition, free online resources are available to perform statistical evaluation of data. for example, on the website psychol-ok (www.psychol-ok.ru/lib/statistics.html) a researcher can perform calculations according to any of the above criteria. however, it should be noted that the available online resources for the analysis of the obtained research data reflect only the final results and conclusions of their analysis, but do not demonstrate intermediate calculations and do not guarantee the correctness of the calculation. therefore, these tools should be used as an auxiliary tool to verify the correctness of the calculation by the researcher. examples of other online services for statistical analysis are social science statistics (https://www.socscistatistics.com/tests), http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 http://www.psychol-ok.ru/lib/statistics.html https://www.socscistatistics.com/tests/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 101 statistics kingdom (https://www.statskingdom.com), datatab (https://datatab.net/statisticscalculator/descriptive-statistics) and others. thus, under digitalization and covid-19 pandemic, the use of dt&s becomes a prerequisite for the preparation of the dissertation. today, digital resources are widely available meet ing the objectives of each stage of the dissertation research. using them in complex saves time and increases the validity of the research findings. at the same time, the motivation of the doctoral students, the level of their digital skills is the key condition for this. the availability of technical conditions (computer or mobile devices, speed of internet connection) is equally important. in order to study these aspects, a survey on doctoral students’ attitude towards the use of dt&s has been conducted. 4.2. the doctoral students’ attitude towards the dt&s application to do dissertation research a total of 35 doctoral students of the 1 st – 2 nd years and 19 doctoral students of the 3 rd – 4 th years were interviewed on their attitude towards the use of dt&s at different stages of the dissertation research: • elaboration of the theoretical basis of the dissertation research (stage 1 in the table 1); • substantiation and development of author’s know-how (pedagogical conditions, technologies, systems, forms and methods, etc.) (stage 2 in the table 1); • designing and doing an experiment (stage 3 in the table 1); • statistical processing of experimental results (stage 4 in the table 1); • generalisation of the theoretical research results (stage 5 in the table 1); • writing and publication of research articles on the research findings (stage 6 in the table 1); • approbation of research results through participation in conferences, seminars (stage 7 in the table 1); http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://www.statskingdom.com/ https://datatab.net/statistics-calculator/descriptive-statistics https://datatab.net/statistics-calculator/descriptive-statistics multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 102 • dissertation testing for compliance with scientific ethics (stage 8 in the table 1); • pre-defence and defence of the dissertation (stage 9 in the table 1). among the possible answers: “important”, “rather important”, “rather unimportant”, “unimportant”, the vast majority of graduate students chose the first two (table 1). table 1 the attitude of doctoral students to the usage of dt&s at different stages of the dissertation research stage of dissertation research phd students of the 1 st – 2 nd years phd students of the 3 rd – 4 th years answer options answer options importa nt rather importan t rather unimpor tant unimpor tant importa nt rather importan t rather unimpor tant unimpor tant 1. 30 4 1 15 4 2. 24 10 1 3. 21 10 2 1 16 3 4. 29 3 2 1 17 1 1 5. 22 11 1 1 15 2 2 6. 24 9 1 1 16 3 7. 24 9 1 1 14 5 8. 24 8 2 1 18 1 9. 21 11 2 1 15 3 1 table 1 shows that doctoral students of the 3 rd – 4 th years are more supportive of the dt&s application while among 1 st – 2 nd years students there are many who are hesitant and have chosen the answers “rather important” and “rather unimportant”. we attribute this to the fact that the 3 rd – 4 th years doctoral students have completed the educational component of the doctoral education that contains the “ict in modern research” discipline. this shaped their vision on the importance of dt&s for the research, developing necessary digital skills. noteworthy is the fact that the 1 st – 2 nd year doctoral students differently assess the importance of the dt&s application at different stages of their dissertation research. the respondents believe that dt&s should be primarily applied at the stages of elaboration of the theoretical basis of the dissertation research (86% of respondents) and statistical processing of experimental results (83%). the use of such resources at the stages of generalisation of theoretical research findings (63%), http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 103 designing and doing an experiment, pre-defense and defense of the dissertation (60%) is viewed as the least important. the next question concerned the students’ experience of using dt&s for doing dissertation research (table 2). the following resource groups were proposed for the evaluation: • resources for visualization (visme, power bi, easel.ly, google charts, piktochart, venngage, canva, mentimeter, etc.) (1 in the table 2); • resources for doing an experiment/conducting survey, etc (google forms, classtime, mentimeter, kahoot !, poll everywhere, edpuzzle, classmaker, online test pad, triventy, quizizz, etc.) (2 in the table 2); • resources for searching publications (repositories, libraries, google academy, etc.) (3 in the table 2); • resources for bibliography design (bibliographic managers) (bibme, classtools.net, bibtex, endnote, zotero, etc.) (4 in the table 2); • resources for statistical processing of quantitative data (microsoft excel, spss, mathcad, statistica, etc.) (5 in the table 2.). table 2 doctoral students experience of using dt&s for doing dissertation research resources 1 st – 2 nd years phd students 3 rd – 4 th years phd students answer options answer options yes partial no yes partial no 1 14 12 9 9 8 2 2 20 14 1 10 9 3 27 8 16 3 4 14 18 3 12 5 2 5 17 17 1 13 5 1 table 2 shows that the resources for searching publications (repositories, libraries, google academy, etc.) have been named as the most used. this is affirmatively indicated by 77% and 84% http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 104 of doctoral students of the 1 st – 2 nd and 3 rd – 4 th years, respectively. the resources for doing an experiment/conducting survey and for the statistical processing of quantitative data have been also named by doctoral students as being actively used. in order to develop recommendations for improving the quality of the dt&s application for doing the dissertation research, it was important to identify the factors that inhibit this process (table 3). table 3 factors that inhibit the process of dt&s application while doing dissertation research factors 1-2 years phd students ranks 3-4 years phd students ranks number of answers number of answers reluctance, passivity of doctoral students 12 3 11 1 reluctance, passivity of scientific advisors 2 6 4 4-6 unsatisfactory technical capabilities (low speed internet connection, limited computer or mobile devices) 21 1 8 3 insufficient knowledge of doctoral students about the usage of computer programs and online services 19 2 9 2 lack of time to use dt&s 9 4-5 4 4-6 the workload of the doctoral training programme 9 4-5 4 4-6 table 3 shows that doctoral students of the 1 st – 2 nd and 3 rd – 4 th years identify the factors inhibiting the use of dt&s differently. the 1 st – 2 nd years doctoral students rank unsatisfactory technical capabilities and insufficient level of digital skills to use dt&s as the most significant factors. the reluctance/passivity of the doctoral students is the most significant factor for the 3 rd – 4 th years doctoral students. insufficient level of digital skills to use dt&s ranks the second place in their answers. the reluctance, passivity of scientific advisors; lack of time to use dt&s and the workload of the doctoral training program are ranked as insignificant. http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 105 table 4 shows how doctoral students assess the level of their own digital competence for the implementation of dissertation research (1 – very low, ..., 5 – very high). table 4 self-assessment of the level of own digital competence for realization of dissertation research level phd students of 1 st – 2 nd years phd students of 3 rd – 4 th years number of answers number of answers 5 5 3 4 12 13 3 15 3 2 2 1 1 besides, the respondents were asked to assess their level of digital competence in accordance with the levels of the corporate standard of digital competence of educators of borys grinchenko kyiv university (borys grinchenko kyiv university, 2021) (table 5). table 5 corporate standard of digital competence of educators of borys grinchenko kyiv university levels level description mandatory а perform a simple search in scientometric databases, repositories, etc; analyse and systematize information related to the research topic; critically evaluate online resources. collect and analyse statistics using digital tools from the content of the study. usage digital tools to test the uniqueness of research texts. format, layout scientific publications according to the requirements of the publication. update your own scientist profiles. sufficient в1 carry out an advanced search and use data from scientometric databases in accordance with the problem of scientific research. collect, analyse and interpret statistics using digital tools from the content of the study. format, layout publications (including templates) using cross-references to sources and bibliographic managers. disseminate scientific ideas and research results through profiles in scientific social networks. в2 create and use databases with research materials. collect statistics and use digital tools (software applications, online calculators, etc.) for scientific statistical data analysis. analyse trends in digital tools and predict their use for research. carry out expert assessments using digital tools, provide opinions on research activities. systematically use digital tools to disseminate research results (web publications, blogs, vlogs, posts in the international community, http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 106 etc.). high с1 involve colleagues in collaboration in the database of research materials that the team is working on. adapt digital science spaces to share experiences (conferences, science schools, etc.). experiment with the introduction of new digital tools in research and involve colleagues. use innovative formats for the organization and implementation of collective research and project activities. с2 manage research activities using digital tools. develop digital scientific spaces for the exchange of experience (conferences, scientific schools, training programs, etc.). adapt / develop and implement innovative digital research tools. coordinate research in research projects using digital tools. organize and manage research teams with coverage of their research through a variety of digital tools. the corporate standard of digital competence of educators of borys grinchenko kyiv university developed and adopted within 2020 – 2022 borys grinchenko kyiv university concept of digitalization (borys grinchenko kyiv university. 2020) is aimed at increasing the level of digital competence of educators and thus enhancing the quality of education under covid-19 pandemic. it defines mandatory (a), sufficient (b1, b2) and high (c1, c2) levels of digital competence. as a result of the survey, it was found that the majority of the 1 st – 2 nd year doctoral students assess themselves at the level of b2 and c1 (77% of the respondents), the 3 rd – 4 th year students – mostly at the c1 level (68%). the last question of the survey concerned the suggestions for the improvement of the efficiency of the dt&s usage for doing dissertation research. the following suggestions were received:  organization of scientific and methodical events (seminars, master classes, doctoral schools, etc.) for doctoral students on the usage of dt&s (36 respondents);  development of teaching materials on the usage of dt&s (33 respondents);  updating the content of the academic disciplines of phd training to open opportunities for the dt&s usage to do dissertation research (25 respondents);  organization of scientific and methodical events (seminars, master classes, etc.) for scientific supervisors on the usage of dt&s (11 respondents). the findings of our research answer the research questions: http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 107 research question 1. what types of dt&s correspond to the specifics of doctoral research? in the paper the specifics of doctoral education has been substantiated, the key features of which include: the phasing character of doctoral research; the sufficient level of digital skills of most doctoral students; prevalence of the independent format of doctoral students’ work during writing dissertation. the following criteria have been developed for the dt&s selection: the compliance with the specifics of the dissertation research; providing a researcher with an opportunity to conduct research at all stages (search for publications, compiling bibliography, doing experiment, statistical processing of quantitative data, etc.), matching with the level of digital skills of doctoral students, the possibility of application on most digital devices. based on the selected criteria the groups of dt&s for application at different stages of a dissertation research have been defined: resources for visualization (visme, power bi, easel.ly, google charts, piktochart, venngage, canva, mentimeter, etc.), resources for doing experiment/conducting survey (google forms, classtime, mentimeter, kahoot !, poll everywhere, edpuzzle, classmaker, online test pad, triventy, quizizz, etc.), resources for searching publications (repositories, libraries, google academy, etc.), resources for bibliography design (bibliographic managers) (bibme, classtools.net, bibtex, endnote, zotero, etc.), resources for statistical processing of quantitative data (microsoft excel, spss, mathcad, statistica, etc.). the defined groups of the resources in complex form a digital package of methodological support for doctoral students’ training. research question 2. what dt&s do doctoral students use at different stages of their research? the resources for searching publications (repositories, libraries, google academy, etc.) have been named by the doctoral students as the most popular. resources for doing an experiment/conducting survey (google forms, classtime, mentimeter, kahoot !, poll everywhere, edpuzzle, classmaker, online test pad, triventy, quizizz, etc.) are also received the highest number of points from the responders. resources for statistical processing of quantitative data (microsoft excel, spss, mathcad, statistica, etc.) took the third place. at the same time, the resources for visualization (visme, power bi, easel.ly, google charts, piktochart, venngage, canva, mentimeter, etc.) and resources for bibliography design (bibliographic managers) (bibme, classtools.net, bibtex, endnote, zotero, etc.) are the least used by doctoral students according to their responses. summarizing the answers we can conclude that doctoral students do not take full advantage of the innovative potential of dt&s (resources for visualization, for bibliography http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 108 design, etc.) when writing a dissertation. besides, there is almost no difference in the use of dt&s by doctoral students of the 1 st – 2 nd and 3 rd – 4 th years. this shows that doctoral students do not improve their digital competence during their doctoral studies. research question 3. what factors inhibit doctoral student to apply dt&s while doing dissertation research? it is clear that unsatisfactory technical capabilities (low speed internet connection, limited computer or mobile devices) factor has an organizational and technical nature. given this, creating a modern digital environment for doctoral students is an important task for universities. at the same time, insufficient knowledge/skills of doctoral students to use dt&s factor that ranks the 2 nd place among answers allows us to draw a conclusion about the need for the targeted digital courses /programmes for the doctoral students. limitation of the study and areas of further research. the sample of the target group (doctoral students) is not representative. the inhibiting factors and proposals to improve the efficiency of the usage of dt&s for doing dissertation research reflect the common position of the doctoral students of all universities (borys grinchenko kyiv university (kyiv, ukraine). alfred nobel university (dnipro, ukraine), donbass state pedagogical university (slavyansk, ukraine) and pavlo tychyna uman state pedagogical university (uman, ukraine). the article does not highlight differences in the answers of the respondents between universities. the above limitations can be considered as areas of the further research. a comparative analysis of the use of dt&s by doctoral students of both classical (borys grinchenko kyiv university (kyiv, ukraine). alfred nobel university (dnipro, ukraine)) and pedagogical universities (donbass state pedagogical university (slavyansk, ukraine) and pavlo tychyna uman state pedagogical university (uman, ukraine)) is promising. 5. conclusion and recommendations under conditions of the european vector of ukraine’s development, the education in ukraine is being integrated into the european educational and research areas. the doctoral education is one of such areas of integration. the priority of digital education in the eu countries, especially in the http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 109 context of the covid-19 pandemic (european commission, 2020) highlights the task of restructuring the doctoral education system in ukraine towards its dynamic digitalization. the digitization should be comprehensive and systematic. systematization should include technical, organizational, motivational and developmental components aimed at synergy of institutions, doctoral students and supervisors. the technical and organizational components include the creation of a digital environment by universities, as only 86.9% of the population in the cities and 69.8% in villages has access to the fixed broadband internet at home in ukraine (ministry of digital transformation of ukraine, 2019). this is the internet that doctoral students need in the process of using the dt&s. the personal digital development of the doctoral students is no less important. such development should be constant, as digitalization is a dynamic process. equally important is the formation of his / her motivation for such development. the complexity should include both the digitization of the training component and independent work of the doctoral students while writing a dissertation. based on the findings of our study the following groups of dt&s for application at different stages of a dissertation research are recommended: resources for visualization (visme, power bi, easel.ly, google charts, piktochart, venngage, canva, mentimeter, etc.), resources for doing experiment/conducting survey (google forms, classtime, mentimeter, kahoot !, poll everywhere, edpuzzle, classmaker, online test pad, triventy, quizizz, etc.), resources for searching publications (repositories, libraries, google academy, etc.), resources for bibliography design (bibliographic managers) (bibme, classtools.net, bibtex, endnote, zotero, etc.), resources for statistical processing of quantitative data (microsoft excel, spss, mathcad, statistica, etc.). the defined groups of the resources in complex form a digital package of the methodological support for the doctoral students’ independent work in the process of writing a dissertation. such a package is recommended for the inclusion into the doctoral students’ training curriculum both in the format of a separate course and by integration into the syllabi of other disciplines. at the same time, according to the findings of the survey, the doctoral students highly appreciate the effectiveness of the targeted scientific/methodical events (seminars, master classes, http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 110 doctoral schools, etc.) providing knowledge and skills on dt&s application. this has already been confirmed by the experience of borys grinchenko kyiv university, which annually holds the winter school for doctoral students as part of which the targeted classes on dt&s application are held. author contributions: the paper is co-authored by lyudmyla khoruzha (l.k.), olena lokshyna (o.l), nataliia mazur (n.m.) and volodymyr proshkin (v.p.). conceptualization, l.k. and o.l.; methodology, l.k. and o.l.; software, v.p.; validation, n.m. and v.p.; resources, n.m.; data curation, n.m.; writing original draft preparation, l.k., v.p., n.m.; writing review and editing, o.l.; supervision, l.k. all authors have read and agreed to the published version of the manuscript. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references european commission. (2021). communication from the commission to the european parliament, the council, the european economic and social committee and the committee of the regions 2030 digital compass: the european way for the digital decade. https://eur-lex.europa.eu/legalcontent/en/txt/?uri=celex:52021dc0118 vuorikari, r., punie, y., carretero gomez s., van den brande, g. (2016). digcomp 2.0: the digital competence framework for citizens. update phase 1: the conceptual reference model. joint research centre (european commission). https://doi.org/10.2791/11517 european commission. (2017). digcomp 2.1: the digital competence framework for citizens with eight proficiency levels and examples of use. https://op.europa.eu/en/publication-detail//publication/3c5e7879-308f-11e7-9412-01aa75ed71a1/language-en european commission. (2020). digital education action plan 2021-2027. resetting education and training for the digital age. https://ec.europa.eu/education/sites/default/files/document-library-docs/deapcommunication-sept2020_en.pdf european commission. (2020a). communication from the commission to the european parliament, the council, the european economic and social committee and the committee of the regions on achieving the european education area by 2025. https://eur-lex.europa.eu/legalcontent/en/txt/?uri=celex:52020dc0625 european commission. (2017a). communication from the commission to the european parliament, the council, the european economic and social committee and the committee of the regions on a renewed eu agenda for higher education. https://eur-lex.europa.eu/legalcontent/en/txt/?uri=celex%3a52017dc0247 bologna seminar on “doctoral programmes for the european knowledge society” (2005, 3-5 february, salzburg). conclusions and recommendations. eua. https://eua.eu/downloads/publications/salzburg%20recommendations%202005.pdf http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://eur-lex.europa.eu/legal-content/en/txt/?uri=celex:52021dc0118 https://eur-lex.europa.eu/legal-content/en/txt/?uri=celex:52021dc0118 https://doi.org/10.2791/11517 https://op.europa.eu/en/publication-detail/-/publication/3c5e7879-308f-11e7-9412-01aa75ed71a1/language-en https://op.europa.eu/en/publication-detail/-/publication/3c5e7879-308f-11e7-9412-01aa75ed71a1/language-en https://ec.europa.eu/education/sites/default/files/document-library-docs/deap-communication-sept2020_en.pdf https://ec.europa.eu/education/sites/default/files/document-library-docs/deap-communication-sept2020_en.pdf https://eur-lex.europa.eu/legal-content/en/txt/?uri=celex:52020dc0625 https://eur-lex.europa.eu/legal-content/en/txt/?uri=celex:52020dc0625 https://eur-lex.europa.eu/legal-content/en/txt/?uri=celex%3a52017dc0247 https://eur-lex.europa.eu/legal-content/en/txt/?uri=celex%3a52017dc0247 https://eua.eu/downloads/publications/salzburg%20recommendations%202005.pdf multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 111 european university association. (2010). salzburg ii recommendations. european universities achievements since 2005 in implementing the salzburg principles. eua. https://eua.eu/downloads/publications/salzburg%20ii%20recommendations%202010.pdf randall, l., berlina, a., teräs, j., & rinne, t. (2018). digitalisation as a tool for sustainable nordic regional development. nordregio. http://www.nordregio.org/wpcontent/uploads/2017/04/digitalisation_discussion-paper_jan-31.pdf van veldhoven, z., & vanthienen, j. (2019). designing a comprehensive understanding of digital transformation and its impact. in a. pucihar et al. (eds.), 32nd bled econference humanizing technology for a sustainable society (pp. 745 –763). university of maribor press. https://doi.org/10.18690/978-961-286-280-0.39 girón garcía, carolina navarro ferrando, ignasi (2014). digital literacy and metaphorical models. multidisciplinary journal for education, social and technological sciences, 1(2), p. 160-180. https://doi.org/10.4995/muse.2014.2991 araújo-vila, n., cardoso l., toubes d.r. & fraiz-brea j.a. (2020). digital competence in spanish university education and its use by students. publications, 8(4), 47. https://doi.org/10.3390/publications8040047 sales, d., cuevas-cerveró, a. y gómez-hernández, j.-a. (2020). perspectives on the information and digital competence of social sciences students and faculty before and during lockdown due to covid-19. profesional de la información, 29(4). https://doi.org/10.3145/epi.2020.jul.23 khoruzha l., proshkin v., & hlushak, o. (2020). high school teachers’ competence development by means of digital technologies. information technologies and learning tools, (4), 298–314. awan, r.h., afshan, g., & memon, a.b. (2021) adoption of e-learning at higher education institutions: a systematic literature review. multidisciplinary journal for education, social and technological sciences, [s.l.], 8(2), 74 –91. https://doi.org/10.4995/muse.2021.15813 erdmann, a., presedo, a.e., & valdes, m.d. (2021) digital transformation of universities: the influence of covid-19 and students’ perception. multidisciplinary journal for education, social and technological sciences, [s.l.], 8(2), 19–41. https://doi.org/10.4995/muse.2021.16007 sanchez-macias, a., & veytia-bucheli, m.g. (2019). digital competences in phd students. a study in two mexican universities. revista academia y virtualidad, 12. https://doi.org/10.18359/ravi.3618 sim, k. n. (2016). an investigation into the way phd students utilise ict to support their doctoral research process [doctoral thesis, university of otago dunedin]. http://hdl.handle.net/10523/6263 batool, u., dahar, m.a., & akhtar, s. (2021). impact of utilization of ict on the performance of research students in the public sector universities of rawalpindi and islamabad. sir syed journal of education & social research, vol. 4, no 2, april – june 2021 (spring). https://doi.org/10.36902/sjesr-vol4-iss22021(147-159) eurodoc. (2020, july 6). the aftermath of the pandemic for early career researchers in europe. retrieved november 24, 2021, http://eurodoc.net/news/2020/the-aftermath-of-the-pandemic-for-early-careerresearchers-in-europe persky, a. m., fuller, k. a., jarstfer, m., rao, k., rodgers, jo e. and smith, m. (2020). maintaining core values in postgraduate programs during the covid-19 pandemic. american journal of pharmaceutical education, 84(6). https://doi.org/10.5688/ajpe8158 http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://eua.eu/downloads/publications/salzburg%20ii%20recommendations%202010.pdf http://www.nordregio.org/wp-content/uploads/2017/04/digitalisation_discussion-paper_jan-31.pdf http://www.nordregio.org/wp-content/uploads/2017/04/digitalisation_discussion-paper_jan-31.pdf https://doi.org/10.18690/978-961-286-280-0.39 https://doi.org/10.4995/muse.2014.2991 https://doi.org/10.3390/publications8040047 https://doi.org/10.3145/epi.2020.jul.23 https://doi.org/10.4995/muse.2021.15813 https://doi.org/10.4995/muse.2021.16007 https://doi.org/10.18359/ravi.3618 http://hdl.handle.net/10523/6263 https://doi.org/10.36902/sjesr-vol4-iss2-2021(147-159) https://doi.org/10.36902/sjesr-vol4-iss2-2021(147-159) http://eurodoc.net/news/2020/the-aftermath-of-the-pandemic-for-early-career-researchers-in-europe http://eurodoc.net/news/2020/the-aftermath-of-the-pandemic-for-early-career-researchers-in-europe https://doi.org/10.5688/ajpe8158 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16768 social and technological sciences e-issn: 2341-2593 khoruzha et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 87-112. https://doi.org/10.4995/muse.2022.16768 112 varadarajan, j., brown, a. m. & chalkley, r. (2021). biomedical graduate student experiences during the covid-19 university closure. plos one 16(9): e0256687. https://doi.org/10.1371/journal.pone.0256687 lambrechts, a & smith, k. (2020). impact of the covid19 pandemic crisis on doctoral researchers in the uk. the university of york. https://eprints.whiterose.ac.uk/168633/ börgeson, e., sotak, m., kraft, j. et al. (2021). challenges in phd education due to covid-19 – disrupted supervision or business as usual: a cross-sectional survey of swedish biomedical sciences graduate students. bmc med educ 21, 294. https://doi.org/10.1186/s12909-021-02727-3 marakovits, s. j. (2021). four barriers to facilitating 21st century competencies through digitalization. https://doi.org/10.4018/978-1-7998-6967-2.ch006 infographics. (2021, 2 september). in wikipedia. https://en.wikipedia.org/wiki/infographic mind map. (2021, 14 september). in wikipedia. https://en.wikipedia.org/wiki/mind_map borys hrinchenko kyiv university. (2020). the concept of digitalization of borys hrinchenko kyiv university for 2020 – 2022. https://kubg.edu.ua/images/stories/departaments/vdd/documenty/rozdil_7/concept_digital-20.pdf borys grinchenko kyiv university. (2021). corporate standard of digital competence of educators. https://kubg.edu.ua/images/stories/departaments/vdd/nakaz__421_11.06.21.pdf ukraine. ministry of digital transformation of ukraine. (2019). digital literacy of the population of ukraine. https://osvita.diia.gov.ua/uploads/0/585-cifrova_gramotnist_naselenna_ukraini_2019_compressed.pdf http://polipapers.upv.es/index.php/muse/ https://doi.org/10.4995/muse.2022.16768 https://doi.org/10.1371/journal.pone.0256687 https://eprints.whiterose.ac.uk/168633/ https://doi.org/10.1186/s12909-021-02727-3 https://doi.org/10.4018/978-1-7998-6967-2.ch006 https://en.wikipedia.org/wiki/infographic https://en.wikipedia.org/wiki/mind_map https://kubg.edu.ua/images/stories/departaments/vdd/documenty/rozdil_7/concept_digital-20.pdf https://kubg.edu.ua/images/stories/departaments/vdd/nakaz__421_11.06.21.pdf https://osvita.diia.gov.ua/uploads/0/585-cifrova_gramotnist_naselenna_ukraini_2019_compressed.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 83 an initiative to improve oral and written skills of engineering students maria teresa baeza-romero 1 *, fuensanta andrés abellán 2 escuela de ingeniería industrial de toledo. avenida carlos iii s/n. real fábrica de armas. 45071, toledo (españa), 925268800 ext. 5720, 925268849, 1 departamento química-física 2 departamento de matemáticas mariateresa.baeza@uclm.es received: 2013-12-15; accepted: 2014-01-28 abstract in this paper, we describe some activities to develop to written and oral skills in students of degree in electrical and electronics engineering in the school of industrial engineering in toledo. among these activities, we have designed a workshop, included in welcome activities of the school, for first year students and two learning activities included in chemistry module. in the workshop, we explained the key points to consider when an oral or written presentation is prepared. moreover, we tried to make conscious to our students of the importance of the development of written and oral skills. in addition, we have designed an assessment method for oral skills and other skills like critical thinking in the chemistry module, though an exercise that combines conventional evaluation with peer evaluation. as part of this work, an assessment rubric has been developed to mark oral presentations. keywords oral and written skills, workshop, oral presentation assessment, critical thinking, peer evaluation. mailto:mariateresa.baeza@uclm.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 84 1. introduction this work has been carried out during the last two academic years (2011-12 and 201313) in the school of engineering in toledo (university of castilla la mancha). this school has been teaching technical engineers for close to thirty years, until the academic year 2010-11, when the new bachelors´ degrees (ects credits) were introduced. nowadays, we offer two bachelors´ degrees programmes: electrical engineering and electronics and automatic engineering. to study in our school the student do not require any minimum grade in their previous studies and moreover, we have a significant number of students who join us after taking the university entry exams (“pau”) in september, or after finishing his/her advanced vocational education. from the first moment, we detected important limitations in most of our students to express their ideas orally and in written form. for example, when marking exams or homework, it is evident that they have important difficulties to express in writing their knowledge, and these difficulties are even more important when they have to express something orally. in a vast majority of our students, we have detected that public speaking is something that they do not feel able to do. at the same time, and for different reasons that are not explained here, our students do not consider it important to express in a precise and correct form, when they are speaking or writing. it would be really ambitious to claim that with this initiative all the previously described problems would be solved completely, since we are dealing with a really complex problem. however, we want at least, to try to make students conscious that they have a problem that at some point they must face. taking into account these premises and being realistic, our main goal is to call attention to the huge importance of being able to express themselves correctly in writing and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 85 orally, and to show how improving these skills will help them attain their short and long term objectives. at present, the students will have to make exams and homework, prepare laboratory notebooks, make oral presentations, and defend their ideas in group debates, etc. in the long term, when they have to prepare and present their final year dissertation, or in their professional life they will have to present a project or report and defend it in public, not having good communication skills will be limiting. the last figures available about the professional profile of our students are the ones published by universidad de castilla la mancha in the year 2010 (uclm, 2010). in this survey, it was shown that more than 85% of our students were working. the professional sectors more important were building with 10.35%, services (to other businesses, public services, administration, etc,) with a 62.07% and different branches of industrial sector with a 20%. most of the questioned students claimed to have a job with an important level of responsibilities and be well adapted to their formative profile. a relevant piece of information obtained for this study highlighted in this survey is that 60% of surveyed students consider that public speaking and to defence their ideas is very important. our curriculum includes the development of written and oral skills. these two skills are included as transversal skills to develop in most of the modules of engineering curriculum (a4: to be able to transmit information, ideas, problems and solution for a public both specialized and not specialized). (uclm, 2010b). in the academic year 2011-12 we decided to teach a workshop about oral and written skills as part of welcome activities for students of first year (romero, 2012). in this workshop we practiced writing scientific and technical documents and making oral presentation in public. moreover, during this academic year, in the module of chemistry multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 86 we included a component of assessment (laboratory classes) to assess oral skills. as it was the first try, the contribution of this assessment was minor, but it was useful to detect that this skill is not developed in general in our students, and even less when they have to explain technical or scientific concepts. this exercise was useful as starting point to develop a better method to introduce oral skills as part of the teaching and assessment methods in the module of chemistry for upcoming years. 2. a description of teaching tools used to develop oral and written skills 2.1. oral and written skills workshop to organize this workshop we searched in the literature to try to find similar workshops. most of the previous literature was about workshops for students in degrees like law, psychology, etc. (universidad nacional de la loja, 2011; gallardo, 2006). additionally, we knew that there was, in our own university, an optional module for students in their last year of civil engineering called “oral skills” (uclm, 2011). all available resources were very general and lengthy. we tried to summarize some of main aspects and highlight in those related to write and speak correctly at scientific and technical level. our workshop was taught in sessions of three hours, and it was structured in three parts: (1) introduction (30 min); (2) written skills (90 min) and (3) oral skills (60 min). the contents of each part are related below. in the first part of this workshop, we did an introduction and set the main objectives of this workshop, i.e.: to make our students conscious of the importance of expressing multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 87 themselves correctly both orally and in writing, and to show how to structure an oral presentation or a written report, in an academic or professional environment. we started with a simple exercise. we proposed to our students to write an email to their mathematics lecturer asking for an extension and after that, we selected some of these emails and we commented on them. we mentioned orthographic and syntactic mistakes, confusing or incorrect sentences, and problems of “laismo” (a grammatical mistake where “la” is used when it should not be used) that are very common in toledo. usually, we heard justifications such as that they had written in a rush, that for formal homework they would have taken more care, etc. nevertheless, we tried to make them understand that this is not true, that when someone knows how to use the language correctly, he/she will do it in a natural and unconscious way in any situation. after that, we showed them a couple of written exams of the same module, one really well presented, and another one with a very bad presentation, but both similar from technical point of view. we highlighted the aspects that we considered when we marked both exams (clarity, tidiness, correction in the exposition of ideas, etc.). our students need to understand that a clear, tidy and correct exposition of arguments is an important factor to consider when they are marked in a written exam. in the second part of the workshop, we dealt with aspects related to written skills and we started talking about plagiarism. this is a criminal offence, but in our country, it is fair to admit that there is almost a complete absence of social consciousness about it. to assure that they know what plagiarism is, we did a group activity where we described a series of situations that students in groups need to classify as plagiarism, absence of plagiarism or bad academic practice. after that, we proposed an activity where they had multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 88 to paraphrase, and we showed them different styles for references and citations. then, we explained to them, the basic rules of a written presentation, focusing in how to write a good laboratory notebook, essay, etc. we commented on the structure and format for this kind of homework, and explained in detail how they have to make graphs and tables in these written documents. we concluded this part of the workshop by making reference to their most common mistakes (they can use this list as “check list”): (1) inappropriate use of capital letters; (2) use of acronyms without defining them; (3) grammar mistakes in concordance; (4) lack or inappropriate use of punctuation; (5) orthographic mistakes; (6) inappropriate use of inverted commas; (7) not using paragraphs to separate text; (8) mistakes or absence of citation and plagiarism; (9) lack of use of scientific language, using slang, etc. in the third part, we dealt with aspects related to oral skills. we began by showing them some videos where we wanted to mark clearly the difference between a good and bad speaker. just after that, we invited them to do a test to try to measure their skills as a speaker. this test is based in the one developed by b. gallardo pauls (gallardo, 2006). the objective of this part of the workshop is that students understand the keys to make a good oral presentation. from our point of view such keys are the following: (1) to know properly the topic you are going to talk about; (2) to structure the presentation in an appropriate way; (3) to know the audience you are going to make the presentation, and to make an entertaining presentation; (4) to speak in fluid way, trying to vocalize, using voice intensity to call attention of the public when necessary and using a rich vocabulary; (5) using paralinguistic complements like body language and humour but always in an intelligent and moderate way; (6) to use resources such as power point presentation with graphs, figures, photos, etc.; (7) to repeat the presentation alone or multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 89 preferable with someone before to overcome stage fright and to be sure the presentation sticks to time limit. moreover, we noted them that the oral presentation can change depending on the type, although scientific oral presentations are normally structure in the following parts: introduction, experimental procedure or method, results, discussion, conclusions and future work. acknowledgements can be made at the beginning or the end of the presentation. finally, we proposed that they individually prepare at home an oral presentation to be recorded with a webcam and to send it via “moodle” (uclm virtual learning platform) for us to give them feedback. the topic for this presentation was “why do i want to be an engineer?” 2.2. activities in the module “chemistry” chemistry is a basic module of 6 ects credits that is taught in the semester of the first year of degree in electrical and electronics engineering (uclm, 2010b). in the academic year 2011-12, the assessment method with the weight of each component was the following: written exam (70%), homework that included solving chemistry problems (20%) and laboratory work (10%). in this academic year, 2012-13, the assessment component where students have to submit their homework has been eliminated and we have introduced a compulsory new evaluation method to promote development of oral skills between our students, and additionally we have modified the weight of each assessment component. now the written exam contributes 80% to the final mark, 10% comes from laboratory work and another 10% from an oral presentation. this new assessment component has been developed as described below. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 90 2.2.1. laboratory classes the students attended laboratory classes for two hours every two weeks. before starting laboratory work, they had to make an oral exam, where they have to answer, in couples, a series of questions that the lecturer asked related to the practical work that they have to do in this session. in the academic year 2012-2013, it was highlighted by the lecturer that students would be assessed not only by their chemical knowledge, but as well for their ability to express their knowledge in a correct, exact and scientific way. this activity took about five-ten minutes per couple at the start of the session but the contact between students and lecturer continued during the rest of the session where the lecturer continues asking questions related to what exactly the students were doing in this moment in the laboratory. the objective of continuing with this exercise during the whole length of the session was to make the students avoid using slang for the entire session. they knew they were assessed continuously, so they had to be careful with the kind of language they were using. we need to clarify that the final mark in the laboratory classes, although conditioned by the oral skills, in a large extent was determined by the theoretical-practical knowledge of the student about the laboratory work they had to carry out. however, we propitiated the environment for students to realize their limitations when they have to use technical/scientific language: lecturers observed a consistent improvement in the use of this kind of language during the semester for some students, but not for all of them. it was detected that students coming from vocational training had more limitations in using technical/scientific language. in the last laboratory session (6 sessions in total) the students worked in groups of ten. a project learning approach was used (galeana, 2006) where, instead of providing in multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 91 advance a detail descriptive protocol for the laboratory work, as it has happened in previous sessions, we only gave them a description of the objective to achieve (to demonstrate faraday law). we gave to them a series of laboratory material that they would need to demonstrate this law, and additionally we gave them some advice on how to organize their work. we told them that they had to assume that the team was a small business (with sub-teams) and they have to try to divide the tasks to be sure they can achieve the general objective. after that, we asked them to start working independently from the teacher (they could not ask questions). the teacher only helped them, once they have arranged themselves to carry on the practical work. this experience was described by the participants as fun and didactic, except by one team (from the five formed teams) that could not work together and consequently, could not achieve the objective. from an oral skills point of view, this last activity allowed teacher to establish definitely the level that students had reached using technical/scientific language, especially in those students that worked in the team as leaders. however, it was just a formative assessment since this exercise did not count for the laboratory mark. in the surveys carried out by the students of this module, we observed that although some students think that this type of exercises are stressful, most of them think that these types of teaching techniques are more effective for improving their learning experience. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 92 2.2.2. oral presentation in small groups taking advantage of the fact that the last two units in chemistry module are very descriptive (“module 12. basics of inorganic chemistry” and “module 13. introduction to carbon chemistry”), we thought that it would be interesting to ask students to prepare an oral presentation about these topics, and make this part of the module more entertaining. in the academic year 2011-12, it was not compulsory to make such an oral presentation, as the students could choose between making it or handing in homework. most of the students chose the second option. probably, they thought it was easier, and they were used to that. however, six students decided to be assessed by an oral presentation. having such a low number of students making the oral presentation, was really positive, since it allowed us to learn from this experience to prepare this exercise to make it compulsory in the following academic year. moreover, it allowed us to check how good our assessment rubric was. we made a unique group, and each student worked individually. we invited the rest of students to come to listen their classmates and give them feedback. the experience was especially interesting due to the reduced number of speakers, it was possible to provide them a detailed “feedback” by students and the lecturer, and they thought it was especially useful to improve their future oral presentations. after this first experience, in the following academic year 2012-13, it was decided that oral presentations are the only option to assess the last two units. moreover, we promoted group work by allowing oral presentations to be done in groups of four students instead of individually. in this way, tutorial groups of eight students were formed (two groups) and each group had to expose one of the two units. it was highlighted to the students to avoid content overlapping in each group and to show that multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 93 they had worked in a coordinate group. to ensure that, the mark of each student was individual, but 20% of it was determined by group work (see assessment rubric in figure 1). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 94 figure 1. oral presentation assestment rubric used in the academic year 2012-13 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 95 the objectives of this activity were the following: (1) to develop skill of speaking in public, (2) to develop research skills and be able to summarize and explain their findings to their classmates; (3) to develop critical thinking skills; (4) to develop the skills to work as part of a team; (5) to learn about industrial applications and natural resources of organic and inorganic chemical compounds; and (6) to learn how to teach classmates. the bibliography to use in this activity could be any general chemistry book or reliable sources of information on the web. in 2012-13, we used the assessment rubric developed in the previous academic year taking into account that this year rubric was the only way student could get feedback (it was not possible to give oral feedback after presentations due to the large number of students), and including group work assessment criteria. in the development of the rubric, we had to define indicators in a way that were clear enough for students, and that the content of the talk, and not just how student present information were assessed. it had five components with different weights: delivery (25%), content (40%); visual aids (5%); answering questions (10%) and teamwork (20%). the rubric was made in an excel file to make it easy to use for students, and it was available for students well in advance. about 50% of the students who did the written exam took part in oral presentations. the rest decided not to make such a presentation although this meant that they were losing one point over ten, in their final mark in this module. the final mark in this activity was weighted as follows: 70% mark given by lecturer, 20% from the averaged of all classmate marks given to the presenters, and 10% depending on the agreement of their marks in relation to lecturer mark when they were assessing their classmates. we understood that with this weighting, apart from multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 96 promoting group work and the development of oral skills, we worked in the development of critical thinking and on how to accept criticism. in consequence, the evaluation procedure was a combination of peer evaluation and conventional assessment by the lecturer, and it is fair to say that students were quite suspicious about fairness of this evaluation procedure. this small problem was sorted out demonstrating to them that really they are as good judge as lecturer is. moreover, we highlighted to them that 20% of their mark depends on how well they have assessed their classmates. consequently, their classmates, if they want to have a good mark themselves, need to mark him/her well. this method solved the typical problem in peer evaluation of producing excessive high marks (lu and bo, 2007). 3. obtained results students’ feedback in these activities was obtained using a moodle survey and asking direct questions face to face to students. oral feedback obtained immediately after the written and oral skills workshop, showed students considered very interesting the recommendations given about how to improve their oral and written skills, and they valued very positively their participation in the activities included in the workshop. in relation to plagiarism, they thought the lesson was too long, but apart from that, they did not mention any other negative aspect about the workshop. in the feedback obtained via moodle survey, carried out at the end of first semester related to oral and written skills workshop, more than 90% of the surveyed students thought this workshop was useful to prepare their laboratory notebooks and essays, and in particular, in the preparation of the oral presentation they carried out in chemistry. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 97 again, they proposed to reduce the extent related to plagiarism to promote more practical exercises to improve in how they speak and write. the marks obtained in chemistry oral presentations ranged from 5 to 9.5, with an average of 7.5. from our point of view, this result is quite positive; that is, the students got involved, and all of them passed, but we cannot forget that only 50% of the students made oral presentation. the rest of the students decided not participate in this assessed activity, and 5% of these students confessed, that their reason for not making the oral presentation was their stage fright. moreover, we need to highlight that the students, in general, marked quite well their classmates, except in some punctual case where lecturer had to apply a correction factor to a student who has been marked very low by their mates. 12.5% of surveyed students claimed that oral expositions in chemistry were the teaching activities most useful to learn from all activities carried out in all modules during their first semester, and 37.5% of them thought that about laboratory classes. students also claimed they needed ten hours of preparations for their oral presentations, while lecturers estimated they would only need three hours. moodle feedback, in relation to oral presentation, was quite positive and students asked to have more topics to choose for their oral presentations. the students think that this kind of activity is really formative and rewarding activity, but more topics related to unit 12 and 13 should be included as options. the conclusion from lecturers, involved in chemistry oral presentations, was that although students have made important advances in their use of scientific/technical multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 98 languages in relation of their skills at the beginning of semester, there was still a need for improvement there. 4. future work and links with other modules in higher levels the oral and written skills workshop will be held again during the academic year 201314, and after this study, some modifications will be done considering both lecturers and students´ opinions. the most important changes proposed are the following: (1) to divide the workshop in 2 workshops of 1.5 h each, one about oral skills and another one about written skills, held on different days; (2) to reduce the lesson related to plagiarism; (4) to introduce a practical exercise in the oral skills workshop where students have to prepare a small presentation in advance; (4) to focus more time on the use of technical/scientific language or make students more aware of the importance of this skill. in chemistry, we will continue with oral exams in laboratory classes and oral presentations assessment, but with small modifications in the rubric. additionally, we will include more topics for the presentations but always related with organic and inorganic chemistry. e.g., a possible topic for next year would be “why are isomers important for life?” we are studying possibilities to introduce more activities to promote oral skills in other modules of the second semester in the first and consecutive years of these degrees. for example, in the module of “environmental technologies” (first year, second semester), we have included an oral exam in laboratory work, too, or in other modules like “renewable energies” (third year electrical engineer degree), students have to make multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 99 oral presentations and they are thinking about modifying the rubric presented in figure 1. in addition, a rubric is currently being developed in our school to mark final year dissertations when they are presented orally. references oficina de planificación y calidad. vic. economía & planificación, uclm (2010). estudios sobre calidad e inserción laboral. uclm, (2010). plan de estudios de grado de ingeniería electrónica industrial y automática y plan de estudios de grado de ingeniería eléctrica industrial. pascual romero, f., andrés, f., baeza, m.t. (2012) xx cuieet, las jornadas de acogida como plataforma para el desarrollo de competencias genéricas y específicas para los estudiantes de nuevo ingreso, gran canaria. http://www.eiic.ulpgc.es/documentoscongresos/francisco%20pascual%20romero.pdf accessed 19th december 2013. universidad nacional de la loja, (2011), taller de expresión oral y escrita, para el módulo de las carreras de formación profesional o de pregrado del área de la educación, el are y la comunicación y la universidad de loja, ecuador. http://www.sisman.utm.edu.ec/libros/facultad%20de%20ciencias%20zoot% c3%89cnicas/carrera%20de%20ingenier%c3%8da%20en%20inform atica%20agropecuaria/05/expresion%20oral%20u%20oratoria/li bros/taller-de-expresion-oral-dos.pdf accessed 19th december 2013. b. gallardo, (2006). la enseñanza de habilidades comunicativas: un taller de expresión oral. universidad de valencia. http://www.eiic.ulpgc.es/documentoscongresos/francisco%20pascual%20romero.pdf http://www.sisman.utm.edu.ec/libros/facultad%20de%20ciencias%20zoot%c3%89cnicas/carrera%20de%20ingenier%c3%8da%20en%20informatica%20agropecuaria/05/expresion%20oral%20u%20oratoria/libros/taller-de-expresion-oral-dos.pdf http://www.sisman.utm.edu.ec/libros/facultad%20de%20ciencias%20zoot%c3%89cnicas/carrera%20de%20ingenier%c3%8da%20en%20informatica%20agropecuaria/05/expresion%20oral%20u%20oratoria/libros/taller-de-expresion-oral-dos.pdf http://www.sisman.utm.edu.ec/libros/facultad%20de%20ciencias%20zoot%c3%89cnicas/carrera%20de%20ingenier%c3%8da%20en%20informatica%20agropecuaria/05/expresion%20oral%20u%20oratoria/libros/taller-de-expresion-oral-dos.pdf http://www.sisman.utm.edu.ec/libros/facultad%20de%20ciencias%20zoot%c3%89cnicas/carrera%20de%20ingenier%c3%8da%20en%20informatica%20agropecuaria/05/expresion%20oral%20u%20oratoria/libros/taller-de-expresion-oral-dos.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2196 social and technological sciences baeza-romero and andrés (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 83-100 | 100 http://www.uv.es/pauls/habilidadescvasbgallardo.pdf accessed 19th december 2013 uclm, (2011), guía académica 2011-12. asignatura expresión oral. ingeniería de caminos http://www.uclm.es/cr/caminos/iccp/asignaturas.pdf#page=206 accesed 19th december 2013 galeana de la o. l. (2006), aprendizaje basado en proyectos. ceupromet 27 http://ceupromed.ucol.mx/revista/pdfart/1/27.pdf accessed 19th december 2013 lu, r., bo, j j. (2007). a comparison of anonymous versus identifiable e-peer review on college student writing performance and the extent of critical feedback. journal of interactive online learning, 6(2), 100-115. http://www.uv.es/pauls/habilidadescvasbgallardo.pdf http://www.uclm.es/cr/caminos/iccp/asignaturas.pdf#page=206 http://ceupromed.ucol.mx/revista/pdfart/1/27.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 17 it enabled service delivery innovation from quality perspective yao chin lin, ping heng tsai*, nhu hang ha dept. of information management, yuan-ze university 135 yuan-tung road, chung-li, taoyuan, 32003 taiwan (r.o.c.) e-mail: imyclin@saturn.yzu.edu.tw, superf2t@gmail.com*, s999212@mail.yzu.edu.tw received: 2013-11-02; accepted: 2014-01-1 abstract the objective is to understand how information and service quality to influence of it enabled service (ites), then to enhance service delivery innovation (sdi). this study bases ites to elaborate two research questions by adopting rodosek’s business reference model. consequently, we conduct a case study approach, by deriving proposals concerning the relationships among two quality perspectives, ites, and sdi in a taiwanese publishing company. in the end, we concluded three related findings. firstly, the company based on service orientated to provide basic service functions to encourage more users to enjoy their services, as well as facilitating collaborative work via the online discussion area. secondly, the company adopted advanced service functions to extend their service scopes to endusers, such as download the news into handheld devices. finally, the company utilized the supporting service functions to assist customers integrating the ideas and plans. keywords information technology enabled services; service delivery innovation; information quality; service quality mailto:imyclin@saturn.yzu.edu.tw mailto:superf2t@gmail.com mailto:s999212@mail.yzu.edu.tw multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 18 1. introduction technology enabled new service delivery approaches, also brought some problems (e.g. dependence in a service provider, risk to lose know how) (arun et al., 2004). nowadays, companies, who have based on constructed it capability, fully focused on developing new information technology (it) components. a number of studies have shown influences of firms’ a competitive advantage across it researches (lu and ramamurthy, 2011; ross et al., 1996). to be sure, lu and ramamurthy (2011) based on it capability discussing it proactive stance. he emphasised that if firms constantly keep current with new it innovations, or seek new ways to enhance the effectiveness of it use, they will create business opportunities. on the other hand, there are numerous related it theories have explained user’s behavioural intention to use behaviour by adopting it, such as, technology acceptance model (tam) (davis et al., 1989), and unified theory of acceptance and use of technology (venkatesh et al., 2003). only the issm mentioned the quality is the key factor to impact on users intention to usage, and satisfaction, such as, in the healthcare system, in the knowledge management system. however, they do not mention the impact of quality on it enabled service (ites). we consider ites as adopting services from an it platform to access, share, and modify information. however, we argue that even if it capability has provided proactive stance service, it still needs the quality concern in services and information from is to enhance and preserve the firm’s innovation capability. we firstly go back to the fundamental perspective, therefore, adopted rodosek’s (2003) business reference model to understand two relationships (provider-centric and servicecentric, and customer-centric). from the provide-centric to service-centric, gorla et al. (2010) based on issm investigating the relationship between is’s quality and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 19 organizational impact. they concluded information quality (iq) and service quality (sq) directly influence on organizational impact that contains service enhancement and information support is considered as the importance of it. he also suggests that we must consider amidst the existence of a tension associated with the notion of the technology. therefore, we focus two factors (iq and sq) to understand its influence in ites. from the service-centric and customer-centric, it enabled capability associated with high innovations within an industry (joshi et al., 2010), especially, service delivery. it assists the firm to manage knowledge sharing and transfer (lee and lim, 2005). therefore, the firm can adequately utilize ites to achieve service delivery innovation, such as tourists, who can accept new it application services, and use its functions to contact others tourists from the online tourist community instead of books and magazines. recently, many researchers have conducted studies related to ites or related to service delivery innovation (sdi) (dholakia and dholakia, 2010). however, few studies describing the relationship between ites and sdi. moreover, a number of studies aimed to discuss ites with the issues of business outsourcing and offshore service processes (davis et al., 2002). they also do not mention the impact of iq and sq on ites. in fact, in the publishing industry, iq and sq are the two main factors impacting on the quality in products, and services. therefore, we analyse the ites issue and explore its environment in is, especially analysing is development process. in more detail, we employ a case study of a real company to understand their technical framework in the design stage of their new it system and service characteristics from it services. this paper is based on it services and interpreting the relationship between iq, sq and sdi by investigating a publishing company. the objective is to investigate a professional publishing company in taiwan to understand: (1) what are the influences of iq and sq on ites in the publishing company, and (2) how to use ites to enhance sdi. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 20 for these objectives to be achieved, this study is structured as follows. the first part deals with the literature review for the development of the research. after which research framework and methodology are presented, with full details of the participants in the research, and of the instrument and procedures used. results are then presented, with a through description of the adoption process at the travel publishing company. finally, results are discussed, and conclusions are drawn. 2. theoretical foundations the theoretical background is organized into three sections. the first section describes quality perspectives in information and service. the second section explains the relationship between quality perspectives and ites. the third section concentrates on discussion ites to sdi. 2.1 information quality dimension in it iq is a term that describes the quality of the content in information systems (is). it is often pragmatically defined as “the fitness for use of the information provided”. iq focuses on the quality of is output (nelson et al., 2005). it is a sophisticated research area in the is field. in order to address the needs of information users, iq should be evaluated. previous research has used iq to measure information system success (iivari, 2005), and e-commerce success (delone and mclean, 2004). accordingly, iq is a multidimensional concept measured by dimensions such as content, format, relevance, accuracy, and completeness (nelson et al., 2005; gorla et al., 2010). nelson et al. (2005) remarks that the evaluation of different information characteristics from a user’s perspective is fairly subjective and relative to the context and task at hand. especially, the case company sees multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 21 service innovation as a critical issue in the publishing industry. therefore, this paper takes the same aspects of iq with nelson et al. (2005) and gorla et al. (2010), to classify iq into lists across three categories (information relevance, information format and information content). 2.1.1 information relevance information relevance (ir) is the extent to which information is applicable and useful for assisting decision-making. poor iq can be detrimental to system usability and hinder operational performance bringing the heighten risk exposure, and cause significant capital losses (raghunathan, 1999). cheung et al. (2008) mentioned ir is the critical element influencing information usefulness and adoption in the online communities. on the other hand, pearson et al. (2012) provided the similar point of view, through ir can pertinent to the user’s needs. for example, when a tourist intends to visit a country or create an itinerary, they will get help from travel books to plan the trip. as a travel book has vivid photos and shows the content with clear markers on a map, it helps the tourist easily understand the relations of the attractions, allowing people to highlight the attractions and place it into their itinerary. 2.1.2 information format information format (if) refers to the degree to which the information is well presented pleasing (lin et al. 2009). poor if can cause ambiguity of the information. therefore, lee et al. (2012) emphasized the importance of the if, which related to the concept of consistence, to sustain compatible information. it effects on the customer responses, such multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 22 as, style presentation and easy to understand. jeong and lambert (2001) implied that if include its design, format and links to be unambiguous, clear or readable because if related to the layout format assist customer ease of use and gathering of information on the internet (davis et al., 2002). especially on e-commerce websites, hong et al. (2004) conducted two results (displaying only one product in each row and displaying more than one product in each row) formats used to organise multiple product information on webpages. with digitalised information, these aspects of format are significantly enhanced. therefore, a publishing company who wishes to establish a tourism platform should follow the same aspects and functions to support tourists finding consistence useful information from the database and completing their itineraries. 2.1.3 information content information content (ic) is defined as the degree of valuable source of information or communicated material on a website (cudmore et al., 2011), such as using tourist website for organizing tourist purpose. ic should take responsibility for reducing the personal uncertainty, such as when organising itineraries, whilst facilitating choice and arrangement. the ic perceived quality in terms of accurate, complete, concise, useful and relevant for decision-making (nelson et al., 2005; gorla et al. 2010). from information consumer's perspective, the content of the information is one of the main characteristics of a website. the value of the website may not be assessed independently of the quality of the information content that it provides (herrera-viedma et al., 2006). however, a majority of internet users will not visit the web site again if they do not find it useful (singh, 2009). the content of the website should also meet the needs, and the presented information should also consider the credibility for encouraging people who enjoy multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 23 involving in contributing their works (cudmore et al., 2011). therefore, we take the same aspect as other research because information, data, experience or knowledge is considered a valuable asset to a business. the publishing company regarded content as their core business. 2.2 service quality dimension in it the definition of service quality is controversial and sq identified as a critical success factor for helping organisations build competitive advantages (parasuraman et al., 1985), especially high quality of services generate customer satisfaction, and the good image of the company. sq is defined as the degree of discrepancy between customers’ normative expectations for a service and their service performance perceptions (gorla et al., 2010). the concepts of it service quality are reflected via it enabled service meeting user expectations, such as satisfying users by providing services to promise. however, four dimensions are wildly discussed reliability, responsiveness, assurance, and empathy (gorla et al., 2010; kettinger et al., 2009). assurance focuses on the ability to inspire trust and confidence (yang and coates, 2010), such as professional knowledge, good manners and trustworthiness of service staff. we use assurance to understand users’ confidence in content support. for instance, a tourist publishing company utilize their professional knowledge and reputations to develop a website for delivering and sharing touchable tourist information. reliability is defined as the ability to deliver the promised service dependably and accurately, and concerns keeping promises, which includes delivery, complaint handling, etc. (brandon-jones et al., 2010). we use to evaluate the range of services in a tourist publishing company to improve the information service to their users. responsiveness describes the willingness multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 24 to help customers and provide a quick service (krepapa et al., 2003). for instance, the company adopts the responsive design ensure that visitors can easily read and use the website from any device, which is an essential feature for a tourism website. empathy measures personal attention and cares to stress the treatment of customers as individuals (liu et al., 2011). for instance, a tourist publishing company develops the tourism website based on user’s perspective to design the clear interface with basic functions to provide friendly personal empathy with customers. 2.3 it enabled service centric from quality perspectives itess are the latest buzzword in the it industry. we focus on the relationship between iq and sq and explain the basic concepts and issues involved in ites. for example, in call centers, they have a customer relationship management (crm). on the other hand, the system has provided correct customer information, and ease delivered to useful interface functions for indoor sales that can quickly response solutions to customers. therefore, it enables business by improving the quality of service is ites. several studies written on ites sectors in different countries, especially in philippines (tschang, 2005) and fiji (davis et al., 2002), with most discussing the rise of ites outsourcing or business process outsourcing (bpo) that has allowed these countries to start a new export sector focused on business services (davis et al., 2002; tschang, 2005; lahiri and kedia, 2011). however, ites consists broadly of the whole gamut of business and technical services that can be outsourced, including customer interaction services, back office operations, transcription and translation services, content development, and other services, which include engineering services and research development. however, because it has become increasingly prevalent and integrated, developing clear multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 25 boundaries around the technologies to be explored in it related research is not always been straightforward (lewis, 2008). schmidt et al. (2010) mentions that the co-creation of value in it service processes could benefit from social software, as well as considering the value of the co-creation of it services. to co-create value, interaction between the service provider and the customer has to advance along the entire value chain. it solves the interoperability problem between different concept sets by providing a specified collaboration platform, where concept designers can make agreements on the equivalence of heterogeneous concepts created in different contexts (guo et. al, 2011). in this paper, we define ites as a service model through it to achieve two things to enhance services. the first converts the integration of it and critical knowledge into digitalised data that are presented in different platforms (e.g.: digital libraries). the second uses ict to deliver content to different devices through the enabling technology to trigger the services (e.g. google). hence, content providers must facilitate collective actions to enhance it enabled capabilities in order to use them to develop content or treatments while simultaneously co-creating or collaborate with their customers and providing customised business innovations within individual customer relationships. 2.4 thinking in customer centric to service delivery a service is a method of delivering value to customers that facilitate the outcomes for customers and needs to be accomplished without costs and risks (best management practice, 2007). innovation in services can be regarded as the interplay of service concepts, service delivery practices, client interfaces, and service delivery technologies. it service delivery is what an outsourcing company does for companies in order to help them find the optimal integration between it and business. the provided it service multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 26 enables clients to manage the relationship between people, processes, technology and information to enhance new goods and new distribution mechanisms (chen et al., 2009). specialised markets are developing for low-skilled services, such as data processing and customer service delivery, as well as for critical knowledge-intensive field service activities, such as r&d and engineering design. as discussed above, the content providers must facilitate collective action to improve it-enabled process capabilities. this paper is based on sap’s sdi discussions to examine the characteristics of successful sdi, which includes four key characteristics (dawson, 2007): (1) a networked community: professional services organisations are ultimately collections of individuals, extremely specialised professionals who create their ability to create value for clients or interaction with clients. therefore, the community needs to create relationships among its participants, such as, common values, shared history, and policies (leung et al., 2010). (2) collaboration: if a company provides “black box” services without collaboration it will quickly become a commodity service provider. a professional service provider needs to provide effective support and continuously collaborate in the process with their clients to create greater value and lock in clients for the long term. however, hefetz and warner (2012) indicated that collaboration is the better solution for uncertain service delivery. (3) continuous service: sdi is an ongoing process that must be embedded in the way a professional service provider functions and develops new products and services. most importantly, the company ensure that the entire service process must continually seek better ways to deliver services to the customer (zhang and xu, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 27 2011). to this end, creativity, opinion, knowledge, experience, critique, and suggestion from users will always bring continuous service improvement. (4) enabling technology: technology is a key trigger of sdi. each of the characteristics of sdi discussed so far requires a technology platform that is modular, flexible and re-configurable. in addition, this platform embraced by the workforce to improve organisational efficiency and effectiveness (jones, 2012). moreover, it must integrate smoothly with external systems and processes to support client and supplier collaboration, and best-of-breed international resources. therefore, enabling technology facilitates the migration of services by providing solutions for improved interoperability. 3. research framework rodosek (2003) mentions a business reference model that provides three aspects that need to be addressed: service centric, provider centric and customer centric. from the service centric, which refers to elements of a service, which are independent of any provideror customer-centric issues, takes about the service functionality and quality. in the provider centric addresses the argument that services, respective of service functionality, can be provided in different ways by independent providers, and includes three key elements, (i) steps on how to provide, operate and withdraw services; (ii) the quality of services offered by providers, and (iii) policies on how to operate services. finally, customer centric refers to a facility provided by a service provider that can be offered to different customers. the framework of this paper is based on three aspects of rodosek’s model to investigate why a publishing company looks for a solution to help them use their professions to deliver new services to users. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 28 we will continue with a detailed specification of the attributes of the three identified parts of service. in the provider centric view, it service providers are currently facing a variety of challenges, which move from devices and application-oriented, to service oriented management and newly created issues, especially web-based iq, technology enabled sq or mass customisation for it services. the information might be not clear or lack accurate definitions that both cause a mismatch to happen with users. larry (2005) identifies some of the critical iq problems in collecting, preparing and presenting information for business intelligence and data mining, along with iq principles for explanation or prevention. zeithaml et al. (2002) determine that sq contains the differences between expected service and perceived service from companies. for customers, convenience refers to a generous amount of available service delivery points that are available when customers need them. some studies note that it-based service options might indirectly improve customer service because of service providers gathering customer information to improve operational efficiency and sq (randle, 1995). in the service-centric approach, managing it might be similar to managing other services that utilise equipment to provide at least part of the service product, with it service providers having to focus on defining certain components such as reusability and being standardised to optimise it service delivery. in the customer-centric approach, it is more urgent today that companies have tools to create connections with their users, tools that can directly communicate and understands users demands. nokia (2010) stress that flexible service delivery, addressing the transformation of service delivery to initiate a customer-centric approach, converged and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 29 content-based service infrastructure for voice and data services is important for their operation. 4. research methodology it is necessary for us to have an in-depth understanding of a new field. therefore, understanding the implementation of a new idea like web 2.0 into the publishing and tourism industries is essential because downstream companies (publishing companies and tourism companies) do not use a significant business reference model, whilst upstream companies (apple inc. and google inc.) are still developing new technology and creating new ideas to attract more customers. by using a case study method and based on the business reference model, this study examines the relationship between iq, sq, ites and sdi in a company. based on this information, this study will then provide generalise findings or lessons for other cases in the publishing and tourism industries. 4.1 case selection and description in taiwan, every year the ministry of economic affairs conducts an “optimizing the business of innovation and network plan” to encourage companies in different industries. they need to submit their proposal for review by a committee. these proposals are reviewed in two main different stages (application proposal and progress report). all stage four and five domain experts were invited to help investigate. in 2009, seventy companies submitted proposals. after the first review step, the committee chose sixteen potential companies to attend the last review cycle, with four representative companies multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 30 eventually getting help. the case company is selected from one of four companies that got help in the 2010 plan. our selected company is a traditional publisher, mpub (pseudonym), who intended to take advantage of it for building web 2.0 based communities up, namely ”travelpass system” to make more services and improve the way they deliver services to users. 4.2 data collection the research data collected from the report on the travelpass system implementation process, which helped us better understand how mpub to generate the new service model form the system. our data gathering concentrated on the business reference model to discuss it influences of sdi. personal interview was conducted with director, project managers and senior managers. on the other hand, the focused group meeting were conducted with the programmers, team leaders, and editors. on average, each interview lasted between one and two and a half hours, during which field notes were taken. we set two key data collection rounds, and numerous informal observations. our fieldwork was focused on mpub project teamwork and conducted by four researchers. the first two researchers conducted most of the onsite interview and investigation while the after two researchers played the role of neutral coder and experimented with an alternative analysis of the case. in the first period (from october 2009 to november 2010), we interviewed a total ten times with mpub’s directors, senior managers, project managers, and team leaders. it the first period of interview work, we discovered two key disciplinary in the case. the first disciplinary was during the investigation between the content management, and publication processes, and the second was found during the investigation of the four vital processes relating to the information systems use: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 31 submission queue, submission review, submission editing, and issue management. table 1 summaries our interview work. in the second period (december 2010-april 2012), we selected filed workers, who have it provision and know the publication process, and conducted onsite personal interview. we concentrated on mapping the relationship among quality perspectives, ites and sdi. the project team was asked to provide their in-depth stories relating to their daily works. we proceeded to analyze the it tasks and trace their contributions in the team. by integrating the stories, we restored a fuller picture of the ites and service delivery innovation from the quality perspectives. to put it simply, in the interview process, we asked team members followed list of concepts (content formulation, information delivery method, promotional functions, creation of service value or challenges from new it) answering our interview questions. after that, we used atlas.ti helping us to classify and visualize our resources. we performed the pattern-matching method, which is a useful technique for linking data to propositions, to help generalise the interview and secondary data into different categories by following the theoretical discussion. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 32 table 1 the list of interviews and participants purpose participant period 1 october 2009november 2010 period 2 december 2010-april 2012 understanding the organization and project purpose investigating the it, services processes and proper terms management layer director 3 1 senior managers 3 3 project managers* 1 5 operational layer team leaders* 3 4 web designers 2 editors 2 4.3 data analysis the analytical process involved two tasks. firstly, we analyzed the project team’s investigation process, rather than the technical infrastructure of travelling system. secondly, we asked the project team members to give the detailed descriptions of the benefit from the travelling system implementation. following, we investigated the proper terms, and focused on the team members’ perspectives in dealing with their new travelling system. the project team should continuously collate the use of new functions during the testing period of travelling system. we paid attention to the points when the team members starting to adopt the new function or allocation company’s multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 33 resources into the community. likely, uploading, listing or presenting the related travel information was investigated to construct on the travelling system. to ensure the functions and applications usability, we invested the whole design process to interact with the designers and noted in our research process. we analysed reiteratively alongside data into three stages (provider’s quality concerns, ites as a service platform, and new sdi to customers). in the first stage, we presented the provider’s quality concern basing on two quality perspectives (information and service). iq and sq are a critical issue because mpub wants to expand their publishing process, whilst, at the same time, giving accurate and precise information to customers, following the steady growth in internet infrastructure and the development of online communities. it is noteworthy that users can obtain information from the website and discuss issues with friends or bloggers who need more time to verify that the information is correct. however, this has a bearing on the publication and print media because this paper is starting by observing the iq and investigating the functions of travelpass system, and based on three elements (content, format and relevance) from theoretical aspects of evaluating iq and discussing the relationship between iq and ites. our detail analysis lesson shows in the table 2. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 34 table 2. analysis lessons in the quality perspective provide-centric related variables the quality of contents information quality relevance search (keywords, themes, itineraries) menu functions plugin external social network linkages introduce attractions download (web apps, itineraries, news) grading and collecting attractions subscribe to rss teaching guide multimedia presentation (contexts, photos, virtual reality, videos) format content service quality assurance online survey and voting plugin external social network linkages links to related blogs, travel agencies, tourism bureau attractions’ maps and street views request cooperation online registration upload personal comments email to instructors teaching guide reliability responsiveness empathy with regard to iq and sq, mpub is a professional publisher that must control the publishing process to confirm the travel content is accurate. the same for the community, mpub must create professional travel content, share their travel experience and provide the latest and newest travel information to customers. consequently, more real-time information as a backup provides accurate information that can attract more customers to use the content service offered by mpub, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 35 with their partners being more willing to pay to participate in travelpass system. on the other hand, mpub should be more responsive about not only the content service but also the service functions designed travelpass system to push more customers to choose it enabled serviceoriented functions from travelpass system. finally, based on the four aspects of service delivery by dawson, this paper thoroughly investigates the characteristics of travelpass system and discusses the relations between sdis and managing it services, and involves a number of members’ practices to ensure that it enabled services provide what has been agreed between the service provider and customers. service delivery capabilities should help better manage evolving customer requirements and execute the end-to-end execution of several new services. mpub utilises travelpass system to handle many service providers, customers and content. 5. research discussions and findings in recent years, tourism has become the most popular leisure activity in taiwan, whether domestic tourism or travelling abroad. most people like to plan free individual travel (fit) and look for related tourism information on the internet (blog and forums), instead of in newspapers and travel agencies because in the past, travellers could only get tour information from travel agencies, newspapers and international travel fairs, books and magazines. the travel agencies are focusing on selling packages and addressing other schedules for travellers at the limited time when they are in the trip. on the other hand, whether books or magazines provide attractive photos, well explained context and in-depth coverage, travellers need to pay more money to compare similar books or magazines. it is hard to engage travellers to pay for this. the most crucial issue is when the books or magazines published because the introduced attractions might have multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 36 changed and the travellers will not recognise this change. then, when they arrive at their destination, they will feel anxious and complain why the publishing company has out of date information. because published information cannot immediately renew information and time is needed for republication, the travellers will see the differences. for example, maybe one hotel has cheap accommodation in taiwan and is immensely popular, but a few years later, the hotel might have moved to another location. with the hotel still being recommended at the old address it will lead to displeasure by travellers. however, the internet technology environment has been developing. people tend to get free information from blogs and forums, therefore, travel agencies found this trend, then followed it to develop their own e-commerce platform by providing travel information and directly selling packages on the internet. both issues impacted on the publishing business. therefore, mpub must to consider the new directions to reduce the number of lost customers and their market share. mpub proposed two solutions, integrating the internal publishing process immediately to support the content in travelpass system and implementing three functions (basic, advanced and supporting service functions) to delivery new service operations in travelpass system. 5.1 information integration options enhance iq in content formulation to achieve the content services in ites mpub uses the experience of pixnet media in the online community to improve their publishing skills and develop social networking to establish a portal to provide tourism multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 37 information. to achieve this goal, mpub integrated information from their internal and external resources in the two stage processes. the first was a tagging process, which not only helped mpub’s in-house team tag articles and photos but they also collected information from their partners, communities and previous publications. on the other hand, mpub spent a lot of time digitising their paperback information and separating the contents and photos into mpub’s publishing database (mpub’s db), achieving data reutilisation. for ease of access, mpub executed the tagging process by separating tourism data into fewer labels and categories (photographs, explanations, advertisements, restructured editions and blogger interactions). data stored in mpub’s db provides information to the content management system (cms), and in order to create a consistent data structure and have efficient artwork and text editing, helped the cms provide relevant information. therefore, the data tagging process not only improve the reutilisation of materials, but also reinforced the amount of relevant information to increase its accuracy. especially, it helped the editors to quickly search the database and immediately gather information. after mpub’s db and cms was implemented, mpub utilised cms to consolidate the processed data and used the concept of patterns to apply articles and photos into the new layout format, which would be electronic and proceed using a multichannel. for example, the tagged data was connected to cms, which helped mpub optimise portable document format (pdf) files and use the characteristics of extensible markup language (xml) to export the content into different size handheld devices. second, mpub performed the modular process, which is based on structured content multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 38 delivering value-oriented travel information on the different layout formats. therefore, mpub uses a variety of information formats so that they can be presented on different devices. mpub not only try to deliver quality content and faithful information services but also redesigned the publishing process of the existing environment (books and magazines) for their customers. therefore, mpub had to extend the range of services into developing the community or the digital content to improve the publishing speed. furthermore, mpub sought to attract, maintain and develop the tourism information and service functions in travelpass system, which users or partners might depend on for accurate tourism information, to gather the latest business details and create a service continuity mechanism. the modular process provided the different sized layout formats in the multichannel, helping mpub immediately present the latest information to customers. concerning real-time information, iq is a critical issue, especially to a professional tourism publishing company to upgrade materials and proper publication content or customised content information for their customers to adopt a web 2.0 based community to increase customer satisfaction and accelerate the updating of information. hoskins (2007) suggests that an automated tagging capability ensures that all content within the source link is consistently and accurately categorised, regardless of when it is added to the portal. the modular process is both a process and product discipline offering a wide variety of advantages in the entire construction process. the modular process is applied at the product level and realised in the design development and production. the tagging and modular process helped mpub with data management, contributing to the speed of information delivery and reutilisation of materials, supplementing accurate information to attract more partners to collaborate with mpub. therefore, information relevance and information format are key issues that have helped mpub reduce the publication and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 39 response times for partners and increased the reusing of materials. one new distribution concept is content as a service (caas), whose technical foundation is referred to as data streaming to the concept of the much discussed software as a service (saas). a content provider might decide to collaborate in the delivery of such content as a service for mobile devices (this corresponds to a new type of service). the publishing company is pursuing the delivery of first-hand and corrected information to customers. therefore, they consider the advantages of the best content formulation running the website, and developed three stages (information relevance, information format and information content) to help the publishing company establish a bridge between relations and labels. mpub strengthened their experience in digital publishing to achieve the purpose of using their caas, assisting mpub to become a content-service provider and make their services easy to access and release. mpub uses the concept of caas to provide different service options and helps them become an information content provider. therefore, this study proposes the listing of three sub-findings. (1) information relevance allows the connection between database and materials to help the publishing company to reduce the time needed to search for useful information and increase information accuracy. (2) the information format focuses on a different layout to allow users to easily get in-time information from the multichannel and improve the quality of the content. (3) the information content is accessible and consistent, making the publishing company a content service provider and focused on caas to provide suitable information. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 40 5.2 designing interactive functions for the website to help users easily access service functionalities in ites. mpub initially established a website in 1999, focusing on the style of the website presentation and integrating information. mpub tried to provide a tourism channel to deliver tourism news to customers. however, during this period running the website, they faced two crucial problems leading to the website failing. first, mpub did not have any ideas about the concept of the pattern design. mpub only feeds the travel information into the system; when users visited the website, an unstructured page layout without a search function made it hard for users to find useful information. finally, the users gave up and went looking for their information on other sites. secondly, mpub aimed to deliver travel information on different experiences to readers, provide clear itineraries to customers and request travel support from agencies to complete the itineraries on the website. however, the travel agencies protested about their itineraries because the itineraries and prices were visible and included confidential business information. therefore, the website crashed. mpub used this experience of failure and adopted new community technology to design interactive functions that attract customers. mpub designed several service functions to support more internal tourism information and gathering customer experiences by posting them on the website to increase the value of the content and push partners collaborate with the services. mpub took their professional experience and integrated information from travel agencies and bloggers’ opinions through travelpass system in order to attract more users. travelpass system not only supported users collecting information but also allowed them to create their own page and save information related to their trips. mpub integrated tourism itineraries posted on multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 41 travelpass system, provided new applications supporting handheld devices and allowed customers to inspect and learn the newest tourism information from anywhere. on the other hand, mpub intended to create more enriched information in travelpass system. therefore, they provided interactive functions (online itinerary discussions, augmented reality, virtual reality, google map and google street view) and set the member functions to users. in order to integrate real-time tourism news, they provided multi-functional services, and connect to customers’ handheld devices to enable the technological experience to bring greater differentiation in the marketplace. therefore, the flexibility and digitised layout allowed mpub to push forward sdi to enable travelpass system to connect to new technological applications and achieve distinction in the marketplace to meet more users’ needs. mpub also designed free discussion areas on triapss and feacbook to attract more users, and to collect more relevant information. at the same time, mpub also shared daily tourism news, helped users forward travel news and share tourism information with more friends to help mpub look for more potential customers and prompt their services from travelpass system. in addition, mpub designs critical interactive functions integrating google map and google street view services, and their overseas project team to collect accurate information to provide precise location data. mpub utilised those functions to support users to gather information and make their own tour itineraries more precisely. mpub provided a remarkable searching function. the user could quickly select “search companion”, which includes popular items, delicacies and shopping locations to obtain related search results. on the other hand, mpub also set the vr shows the street view or the hotel room, to help user get visual information to make more appropriate decisions. mpub provides several service functions supporting customer experiences to increase multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 42 service functionality and collaborative value. therefore, this study proposes three subfindings. (1) new service function, which includes interactive functions and connective different resources helped increase the reliability of system. (2) new service function adoptions helped mpub quickly update news and attract more customers to collaborate in discussions, increasing information assurance and availability. (3) mpub designs the interactive functions and adapts to free open resources helping users not only rapidly organise their itineraries but also directly collaborate with their online friends increasing service availability. 5.3 using ites integrates backend support from iq and sq to deliver a suitable layout interface enhancement for the functions of sdi. after the observation process, travelpass system functions can be divided into three main categories. users not only want to gather tourism information from travelpass system but also intend to interact with the company, travel agencies, other users and relevant partners in different ways. therefore, the first function is the basic service function, which support reading, discussion and searching in travelpass system, or connect to other community and websites. the second function is the advanced service functions, which have a member function and personal space; travellers can leave their experience, opinions and collections on triapss. the final function concerns supporting service functions, which have the newest technology support and latest software applications in travelpass system. one supports the content, mpub developed travelpass system based on their database to improve the quality of information. travelpass system receives information from mpub’s db and complements partners to multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 43 generate supporting information, such as itineraries, tourism advertisements and recommended travel locations. moreover, mpub, based on the service-oriented function, uses travelpass system as a tourism channel to connect to the three main service functions. consequently, mpub established travelpass system based on the features of ict and the concept of web 2.0. firstly, bloggers can share information anywhere and tell a friend where are they, which attract uses to improve their mobility. secondly, mpub adopted new business reference model using advanced technology and turned their content into digital information to enhance the skill of caas. thirdly, mpub opened the discussion area to attract travellers, bloggers, experts and travel agencies to share tourism news and experiences. concerning basic service functions, travelpass system supports accurate travel information, which will attract not only customers but also partners to participate in the development of travelpass system. therefore, mpub uses travelpass system as a service bridge to collect or connect related travel information and exchange travel knowledge or experiences with other people (bloggers, editors, writers), so that users can instantly access service-oriented functions such as searching for attractions, travel themes and recommendations from travelpass system. the second is advanced service functions. travelpass system helped users consider their opinions and offer a personal service. it even provides several digital formats to extend their service to users’ handheld devices to not only provide ongoing services but also to increase information reliability, customer loyalty and satisfaction. finally, mpub established new supporting service functions, created a web 2.0-based community and addressed the functions (views in vr, ar and google map with street view or creating their own itineraries) to engage users, who spread the travel notes. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 44 mpub also encouraged greater empathy with the service building a collaborative and cocreation platform that is easier for users. 6. conclusion this paper firstly adopted rodosek’s business reference model discussing two relationships: provider-centric and service-centric, and service-centric and customercentric. we used this model to explore the relationships between iq and sq related ites and analyse how it affects sdi. this study used a case study method to provide an indepth understanding of the four constructions and descriptions that have influenced the company. we proposed the follows findings. the first, iq affects ites in content formulation, which includes three elements supporting accuracy, ease of understanding, accessibility and consistent information to customers. the following, service quality impacts on information technology enabled services in the service functionalities. therefore, the publishing company started thinking about customer demand, redesigned the interactive interfaces and added the latest technologies from free open resources to improve customer satisfaction. finally, the publishing company uses backend supplements supporting the three fundamental functions in the frontend interface. they designed suitable interfaces to achieve service delivery innovation affecting the networking community, pooled clients and partnered everyone to join to the collaborative environment delivering continuously service. a publishing company wants to speed up their business process and utilise ebusiness to enhance digital rights management and use a transactions information multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 45 platform to establish a charging mechanism. a publishing company needs to adopt the modular it approach, which aims to help companies reduce publication processes and accelerate the speed of publication, whilst the technology still sustains innovation and any kind of handheld devices that progressively become popular. therefore, this study concludes that, (1) the company developed basic service functions, based on service-oriented functions supporting certain fundamental functions and connecting to outer communities to encourage more users to enjoy their services, as well as to facilitate collaborative work via the online discussion area. (2) advanced service functions are connected to individual services, encouraging users to participate in travelpass system activities (writing travelling notes and sharing itineraries with other members) and download the news into handheld devices to extend services to end-users. (3) supporting service functions enable the new applications to provide locationbased services, which helps members have general ideas about the location and organise their itineraries. due to time and resources limits, this study has some areas that need to be improved. this research used the case study method to have an in-depth understanding about the situation. by using an interview method to collect the data, we found that there are some biases related to the respondents. respondents with individual perspectives will affect some related information. respondents without enough of a perspective and guided by others made some wrong assumptions for this study, posing a problem for future research. on the other hand, the authors probed the impact from a personal perspective, making improper judgments or conclusions. in addition, this study only focused on a single multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 46 company. that may be a problem and include some biases because in this case, these problems may have been processing for few months, but applying the findings to other cases may not be appropriate. therefore, the results do not represent whole industry. nevertheless, this exploratory study could be an extremely useful starting point for future research, which aims to study in depth what factors influence the design in the industry. therefore, future research can focus on the business process reengineering issue to discuss the impact of changing processes in a company. references arun, s., heeks, r., & morgan, s. (2004). ict initiatives, women and work in developing countries: reinforcing or changing gender inequalities in south india?. institute for development policy and management, university of manchester. best management practice. 2007. service management – itil (it infrastructure library). retrieved 29/1/2011 from: http://www.best-management-practice.com/it-servicemanagement-itil. brandon-jones, a., ramsay, j., & wagner, b. (2010). trading interactions: supplier empathy, consensus and bias. international journal of operations & production management, 30(5), 453-487. chen, j. s., tsou, h. t., & huang, a. y. h. (2009). service delivery innovation antecedents and impact on firm performance. journal of service research, 12(1), 36-55. cheung, w., hew, k., & ng, c. (2008). toward an understanding of why students contribute in asynchronous online discussions. journal of educational computing, 38, 2950. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 47 cudmore, b. a., bobrowski, p. e., & kiguradze, t. (2011). encouraging consumer searching behavior on healthcare web sites. journal of consumer marketing, 28(4), 290299. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–340. davis, c., mcmaster, j., & nowak, j. (2002). it-enabled services as development drivers in low-income countries: the case of fiji. the electronic journal of information systems in developing countries, 9. dawson, r. (2007). service delivery innovation: creating client value and enhancing profitability. retrieved from sap america inc.: http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/7038d9c0-87e7-2a1021be-998689a7dcf1?quicklink=index&overridelayout=true delone, w.h., & mclean, e.r. (2004). measuring e-commerce success: applying the delone & mclean information systems success model. international journal of electronic commerce, 9(1), 31-47. dholakia, n., & dholakia, r.r. (2010). global e-organization. in savendy g. and karwowski w., introduction to service engineering (533-543). john wiley and sons. gorla, n. (2012). information systems service quality, zone of tolerance, and user satisfaction. journal of organizational and end user computing (joeuc), 24(2), 50-73. gorla, n., somers, m. t., & wong, b. (2010). organizational impact of system quality, information quality, and service quality. the journal of strategic information systems, 19(3), 207-228. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 48 guo, b., zhang, d., & imai, m. (2011). toward a cooperative programming framework for context-aware applications. personal and ubiquitous computing, 15(3), 221-233. hefetz, a., & warner, m. e. (2012). contracting or public delivery? the importance of service, market, and management characteristics. journal of public administration research and theory, 22(2), 289-317. herrera-viedma, e., pas, g., lopez-herrera, a.g., & porce, c. (2006). evaluating the information quality of web sites: a methodology based on fuzzy computing with words. journal of the american society for information science and technology, 57(4), 538-549. hong, w., james, y.l., thong, j.y., & tam, k.y. (2004). the effects of information format and shopping task on consumers’ online shopping behavior: a cognitive fit perspective. journal of management information systems, 21(3), 149–184. hoskins, d. (2007). xml publishing with indesign cs2+. o’reilly media. huh, y.u., keller, f.r., redman, t., & watkins, a.r. (1990). data quality. information and software technology, 32(8), 559-565. iivari, j. (2005). an empirical test of the delone-mclean model of information system success. the data base of advances in information systems, 36(2), 8-27. jeong, m., & lambert, c.u. (2001). adaptation of an information quality framework to measure customers’ behavioral intentions to use lodging web sites. international journal of hospitality management, 20(2), 129-146. jones, s. (2012). egovernment document management system: a case analysis of risk and reward. international journal of information management. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 49 joshi, k.d., chi, l., datta, a. & han, s. (2010). changing the competitive landscape: continuous innovation through it-enabled knowledge capabilities. information systems research, 21(3), 472–495. kettinger, w. j., park, s. h., & smith, j. (2009). understanding the consequences of information systems service quality on is service reuse. information & management, 46(6), 335-341. krepapa, a., berthon, p., webb, d., & pitt, l. (2003). mind the gap: an analysis of service provider versus customer perceptions of market orientation and the impact on satisfaction. european journal of marketing, 37(1/2), 197-218. lahiri, s., & kedia, b. l. (2011). determining quality of business-to-business relationships: a study of indian it-enabled service providers. european management journal, 29(1), 11-24. lee, o.-k.d., & lim, k. (2005). redefining organizational information technologybased capabilities with an integrative framework for multiple level of analysis. pacific asia conference on information systems. lee, s. h., haider, a., kim, j. g., & bahador, k. m. k. (2012, november). information quality assessment in korean manufacturing organization. in acis 2012: location, location, location: proceedings of the 23rd australasian conference on information systems 2012 (pp. 1-11). acis. leung, z. c., lam, c. w., yau, t. y., & chu, w. c. (2010). re-empowering social workers through the online community: the experience of swforum in hong kong. critical social policy, 30(1), 48-73. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 50 lewis, mark o. (2008). managing the tension between standardization and customization in it-enabled service provisioning: a sense making perspective. computer information systems dissertations. retrieved from. http://digitalarchive.gsu.edu/cis_diss/24 lin, h., fan, w., & zhang, z. (2009). a qualitative study of web-based knowledge communities: examining success factors. international journal of e-collaboration (ijec), 5(3), 39-57. liu, c. t., guo, y. m., & lee, c. h. (2011). the effects of relationship quality and switching barriers on customer loyalty. international journal of information management, 31(1), 71-79. lu, y., & ramamurthy, k. (2011). understanding the link between information technology capability and organizational agility: an empirical examination. mis quarterly-management information systems, 35(4), 931. nelson, r.r., todd, p.a., & wixom, b.h. (2005). antecedents of information and system quality: an empirical examination within the context of data warehousing. journal of management system, 21(4), 199-235. parasuraman, a., zeithaml, v. a., & berry, l. l. (1985). a conceptual model of service quality and its implications for future research. the journal of marketing, 41-50. pearson, a., tadisina, s., & griffin, c. (2012). the role of e-service quality and information quality in creating perceived value: antecedents to web site loyalty. information systems management, 29(3), 201-215. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 51 raghunathan, s. (1999). impact of information quality and decision-maker quality on decision quality: a theoretical model and simulation analysis. decision support systems, 26(4), 275-286. randle, w.m. (1995). delivering the future: redefining the role of banks in a new competitive environment. bank management, 45-48. rodosek g.d. (2003) a generic model for it services and service management. in: ifip/ieee 8th international symposium on integrated network management, ieee publishing, washington, 171–184 ross, j. w., beath, c. m., & goodhue, d. l. (1996). develop long-term competitiveness through it assets. sloan management review, 38(1), 31-42. schmidt, r., dengler, f., & kieninger, a. (2010). co-creation of value in it service processes using semantic mediawiki. in business process management workshops (pp. 255-265). springer berlin heidelberg. singh, k. (2009). importance of quality content on web site. available at: http://ezinearticles.com/?importance-of-quality-content-on-web site&id=2245223 (accessed september 10, 2009), . strong, d.m., yang, w.l. & wang, r.y. (1997). data quality in context. communications of the acm, 40(5), 103-110. tschang, t. (2005). the philippines’ it-enabled services industry. available at: http://siteresources.worldbank.org/intphilippines/resources/tschang-word.pdf (accessed may 1, 2010). venkatesh, v., morris, m. g., davis, g. b. & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425–478. http://siteresources.worldbank.org/intphilippines/resources/tschang-word.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2191 social and technological sciences lin et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 17-52 | 52 yang, h., & coates, n. (2010). internal marketing: service quality in leisure services. marketing intelligence & planning, 28(6), 754-769. zeithaml, v. a., parasuraman, a., & malhotra, a. (2002). service quality delivery through web sites: a critical review of extant knowledge. journal of the academy of marketing science, 30(4), 362-375. zhang, h. y., & xu, k. (2011). study on service management innovation of service outsourcing enterprise based on knowledge value chain. in industrial engineering and engineering management (ie&em), 2011 ieee 18th international conference on (pp. 1393-1397). ieee. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 147 an assessment of information disclosures by pharmaceutical industry: evidence from india rupali khanna1 , bhupinder pal singh chahal1 1research scholar, university school of business, chandigarh university mohali, india, emails: rupaikhanna96@gmail.com; bpchahal@gmail.com received: 11 may 2018; accepted: 06 august 2019 abstract the information disclosed by the companies in their annual reports reveals much about company’s performance and prospects. investors take the information as base for decision for investment. under such circumstance, companies choose to disclose beyond what is mandatorily required. theories like agency theory, capital need theory and signaling theory support the need of voluntary disclosure. this study is about investigating the extent of voluntary disclosure in pharmaceutical sector of india which is 3rd in world in terms of volume of trade. objective: to investigate the extent of voluntary disclosure practices prevailing in pharma sector of india, for the year 2010-11 to 2017-18. significance of the study: this study aims to explore the corporate aspect of pharmaceutical sector. any growing avenue is a potential opportunity for investors looking for parking their money to get adequate returns. thus, indian pharma sector has come up in flying colors as an avenue for investors to place their money owing to its 100% fdi. investors have been looking for more and more information from this sector to ensure the safety of funds. thus, the extent of disclosures is worth studying to place a suggestion for the policymakers to introduce the changes in the present set of disclosure practices in pharmaceutical sector. https://orcid.org/0000-0002-5469-0399 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 148 research methodology: to understand the extent of voluntary disclosure, a disclosure checklist is constructed and descriptive statistics are carved out to reach the results. the checklist consists of 55 items which are not mandatory by law. the checklist is based on dichotomous scale of ‘1’ and ‘0’ representing presence and absence of the checklist item respectively. the cross-sectional analysis is carried out to investigate the year wise and company wise disclosure for eight years. findings: though the study observes an increasing trend in the disclosure scores, but the findings are alarming to state that the highest score attained by any company throughout the period of 8 years was 37 (out of 55) not even meeting 80% of the total checklist score. this shows that pharmaceutical sector is not so friendly at disclosures. the probable reasons for such startling results are discussed in the study. 1. introduction indian pharmaceutical industry has not only shown remarkable performance in the country but also established its foothold in overseas market. it is playing a vital role in promoting and sustaining development in the field of medicines and boasts of quality products in which are also approved by regulatory authorities of uk and usa. according to ficci report (2013) the pharmaceutical products have been catering to more than 95% pharmaceutical needs of the country with a population of 130crore. it has progressed significantly by shifting from traditional business practices to exploring and adapting new business strategies. indian pharma companies have come up with desired competencies not only in their manufacturing abilities but in the processing of marketing and getting huge product acceptance abroad too. international brands associated with indian pharma industries have been a significant platform for pharma companies to channelize their capabilities and convert them into opportunities to expand themselves to mark a presence on the global map. indian pharmaceutical sector has been making a spectacular growth especially since last decade. in terms of volume of business, it is placed on 3rd place and in terms of value it is at 13th position worldwide, making it one of the prominent sectors of manufacturing in the country. india owes such remarkable growth of pharmaceuticals to foreign direct investment which is 100% in green field and 75% in brown field. a highly organized sector of indian economy, pharmaceutical industry is growing at 8-9% annually. ranging from multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 149 simple headache pills to complex heart compounds, almost every type of medicine in made in india now. there has been a paradigm shift in the policies and regulations governing indian pharmaceutical industry that made, from being almost non-existent in 1970s transformed to a 6-billion usd market in world. drug control authority has done away with industrial licensing for most of the drugs. innovative scientific manpower, affordable research and development costs and equipped laboratories have given a new face to indian pharmaceutical industry which is self-reliant and technologically strong. also, the role of intellect property protection rights is adding to the strength of pharma industry. any company or individual can patent the formula which gives relief from imitation and malpractices of trade. indian pharmaceutical industry could be among pioneers to bring biogeneric i.e. generic versions of biological product in the organized form to the market. today it is world’s largest source of generic drugs, supplying almost 40 percent of total drug requirement of the us and a quarter of generic medicine demand of the uk (source: www.pharmaceuticaltechnology.com). the industry is highly fragmented with more than 20,000 registered units which are fragmented all over the country. the industry is connected to length and breadth of the country connecting kolkata in east to mumbai and pune in west extending to bengluru in south and himachal pradesh (baddi is emerging as pharmaceutical capital of india) in north. country’s large and diverse characteristic makes it a prime location for clinical trials. the industry is gradually discovering its niche of pharmaceutical formulations. apart from that, bulk drugs and drug intermediaries are gaining momentum in the industry too. over the last 35 years of its existence and growth, indian pharmaceutical market has shown its strong longevity. the significance of pharma industry can be gauged from the various parameters of its development and contributions that it makes to the country (as discussed in section 2.2 of the thesis) fierce policy reforms, over the last few decades, have pulled out the industry from the shackles of import burdens and made it self-reliant. the foreign direct investment (fdi) in pharma sector is allowed upto 100% for greenfield projects (through automatic route) and up to 74% allowed in brownfield project (through govt.approval). the fdi reforms, to welcome the foreign investment with open arms, have brought far reaching ramifications http://www.pharmaceutical-technology.com/ http://www.pharmaceutical-technology.com/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 150 and have leveraged domestic markets. the sector is expected to grow at a double digit growth rate (15% per annum approximately) between year 2015-2020. this growth rate is going to outperform the rate of global pharma industry growth which is expected to grow at the rate of 5 per cent per annum during the same period. until 1970, the recognition of product patent was not in force by law and one could only patent the process of making. this promoted the interest of researchers in pharmaceutical formulations as a result of which, india grabbed a major share in formulations and gradually became the largest producer, later exporter, of it. after the enactment of amended patent laws, the product patents came in as tool of securing and safeguarding the innovative minds. this, in fact, proved to be a boon to pharma industry. it is reflected from the fact that 20 brands of indian companies are still prevailing in market for more than 10 years from now. the resurgence of the indian pharmaceutical industry is looming large. the contribution of the pharma sector in country’s gdp is immense. being the third largest (in terms of volume) and 13th largest (in terms of value) in the world, the industry continues to be the leader of generic exports worldwide. the advantage of cost, low cost of labor, cost of inventory is much lower in the country. the multinational companies, investing in research and development in india, save up to 40% to 50% of their expenses (source:business.mapsofindia.com). it is analyzed that cost of hiring a research chemist in the us is five times higher than that of india and cost of conducting clinical trials in india is one-tenth the cost in us. india presently enjoys position in global pharma market. the country has large pool of scientists, chemists, engineers and lab technicians who have the potential to drive the industry to reach new heights. it is significant to mention that more than 80 percent of the drugs to fight aids are supplied by indian pharma companies to the world. six companies of india have the un-backed patent pool, enabling them to supply medicines to 112 countries of the world. government of india has long recognized the strategic importance of pharmaceutical sector of india. india’s policies have shown immense excellence in uplifting the industry to transform it into a global leader for manufacturing pharmaceutical and innovations in medical research. a strong correlation exists between the government’s initiative of ‘make in india’ and domestic pharmaceutical market. another move of government by enacting gst has proved a boon to the industry by safeguarding it from multiple states tax. the industry is reducing its dependence upon other states now and thus strengthening its supply multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 151 chain. department of pharmaceuticals of central government, in its report ‘pharma vision 2020’, aims to make india a major hub of new drug and development by the year 2020 the disclosure framework in india is governed by the principal act indian companies act 1956 (now 2013). the act lays down the provisions and regulations of disclosures in financial statements. it focuses on maintenance of books and presentation of annual reports. the act also lays mechanisms for issuing of standards. despite the detailed requirements of maintenance of books, the thrust of companies act is on ‘true and fair view’ of company accounts. another body that has a major role in influencing the disclosure framework is securities and exchange board of india (sebi) which governs the disclosure requirements of companies listed on stock exchanges. the reporting requirements imposed by sebi through its guidelines are in addition to those laid by the companies act which is to be followed by the companies listed on stock exchange. together, the requirements of sebi and companies act provide a legal framework for corporate reporting in india. reinforcement to the regulations of these statutory bodies, are the regulating provisions of institute of chartered accountants of india (icai) which formulates accounting standards. the icai derives its power from the chartered accountants act, 1949 (banerjee, 2002). the icai established the accounting standards board (asb) on april 21, 1977 in order to harmonize diverse accounting policies and practices of india. the main task of the asb is to formulate accounting standards, so that the council of the icai can make these standards mandatory. the asb considers the laws, customs and business environment of india while formulating its standard. icai plays pivotal role in the improvement of disclosure practices of indian companies. accounting standards are pronounced by the icai that applies to 'general purpose financial statements.' beyond these mandatory limits of disclosure requirements, there exists a set of non mandatory disclosures which are completely discretionary on the part of companies to what extent the companies want to disclose beyond what is mandatorily required. worth mentioning here that the discretionary disclosures are highlighted in a separate report called as corporate governance report. 1.2. regulatory framework of governing corporate disclosures in india 1.2.1 almost every firm in india takes a legal form as a limited liability company. financial reporting of limited liability companies is mainly regulated by companies act 2013 (the then companies act 1956). the provisions of act relating to accounting and audit are built around the concept of ‘true and fair’ disclosure which characterizes the reporting in the uk, among other countries. the act provides the financial statements in its schedules multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 152 and also lays down the penalty to be imposed both on the company and its managers responsible, for the non compliance with the provisions of the act. 1.2.2 corporate disclosures for the listed limited liability companies are also governed by the listing agreement with the stock exchange provided by the regulatory body securities and exchange board of india. clause 32 of the listing agreement with any stock exchange in india, requires a listed company to publish cash flow statements clause 41 of the listing agreement with any stock exchange in india, a listed company is required to publish unaudited half yearly results. clause 43 of the listing agreement with the stock exchange, companies are required to publish a comparison of projected gross profit, net profit and earnings per share as shown in the offer document in a public offer of shares with the actual performance, in the report of the directors in the annual report. the non-compliance with the provisions of the listing agreement results in delisting of the company’s securities from the trading on the exchange. 1.2.3 the third element of the regulatory framework that governs disclosures in india is the standards and guidance notes issued by accounting standard board (asb) of the institute of chartered accountants of india (icai). currently there are 32 accounting standards which have been issued by asb out of which 29 are mandatory and 3 are recommendatory. the noncompliance of the requirements of these standards will attract a qualification in the auditor’s report. in addition to the standards, icai also issues guidance notes and statements. guidance notes are persuasive in character. they are issued as a precursor to an accounting standard. the first and foremost standard i.e.as-1 is the harbinger of the disclosure trends. as-1 talks about disclosure of accounting policies which make it mandatory for the firms to disclose the fundamental accounting assumptions, nature of accounting policies and consideration in selecting policies. the disclosure requirements specified in the accounting standards are in addition to and not in substitution of the disclosure requirements under companies act, 2013. the corporate reporting regulations aim at providing investors with the minimum amount of information to facilitate the investment decision making (griffin & william, 1960 ) but multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 153 voluntary information is aimed at providing a clear view to stakeholders about the business’ long term sustainability (healey and palepu, 2001). however, argued that, voluntary disclosure will still remain a matter of biased information selected by managers (core, 2001). 1.3 incentives of voluntary disclosures as discussed above in the agency theory, the asymmetric may potentially bring out a breakdown in the functioning of the stock market; voluntary disclosures play a significant role to solve those information gaps. there are many factors affecting manager’s disclosures decisions. the most important factor that makes managers decide for voluntary disclosure is due to its benefits on diminishing the cost of capital. according to capital market transaction hypothesis, it is not easy for existing shareholders to make public equity when a firm has high information asymmetry level. thus, managers have incentives to disclose voluntarily in order to minimize the information asymmetry, and as a result, they can lower the firm’s cost of external borrowing when they enter into capital market transactions. however, firm may reduce voluntary disclosure due to proprietary cost associated. the studies of verrechchia (1983) and gigler (1994) show that voluntary disclosures can damage a firm’s competitive position when there does competition exist, and firms are facing a threat of entry. higher disclosure will allow an easy entry of the competitor to the industry whereas low level of disclosure will keep it protected. another factor which works as an incentive for voluntary disclosure is the litigation cost. it has two side impacts. first, firm may choose to disclose more in order to avoid litigation cost (skinner 1994). there is always a trend in the stock market that investors keep a watch on the news whether there are delays in announcement of any unfavorable news and if there is any, it is evidence that firms have not disclosed voluntarily earlier in a timely manner. they may face serious litigations due to this. so, to avoid the threat of litigation, the firm may choose to disclose more information of poor performance. on the other hand, litigation can also reduce firm’s motivation to disclose because it may increase litigation risk. the penalty imposed by the legal system may be much worse as the market players forecast the information in good faith. healy and palepu (2001) suggest that firm with high level of positive information on future earnings are followed by high level risk of litigation. thus, litigation may de-motivate firms to disclose forward looking information. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 154 thus, overall solution to reduce agency cost and information asymmetry is to make full disclosure of specific firm characteristics. in fact, it is to say that higher level of disclosure will have an impact on agents who indulge in withholding private information and hence increase transparency in the stock market. literature review in indian context, sebi has laid down statutory provisions for listed companies to ensure compliance of transparent disclosure practices. irrespective of sector, each company has to comply with those provisions. india at this time is an attractive destination of foreign investment. the investment environment in india has undergone a tremendous change. in the process of such change, knowledge of the extent of compliance is indispensable for domestic as well as foreign investors. indian pharmaceutical sector, which is 3rd largest growing sector in the world, under the aegis of companies act, 2013, icai pronouncement and sebi regulations, is not left behind in the disclosure context. literature has taken a thrust upon discussing the disclosure practices in the pharmaceutical industry in india. these studies have covered different aspects of disclosures, be it mandatory or voluntary. kamath.g (2008) talks about the disclosure of intellectual capital in reports of pharmaceutical companies of india while rajashekhar (2018) discussed corporate governance disclosure practices. dasgupta.m (2003) associates company characteristics of indian pharmaceutical industry with its financial performance whereas sachdeva et al. (2015) does an inter-sector comparison of the disclosure practices of indian listed companies including pharmaceutical sector. sharma’s (2016) brings the discussion about the practices of disclosure of valuation of intangible assets prevailing in indian pharmaceutical industry. sinha et al. (2012) investigate whether the corporate disclosure practices of indian pharmaceutical companies approach to harmonize with ifrs. in fact, after the enactment of companies (amendment) act, 2013, pharmaceutical sector has even witnessed research on corporate social responsibility disclosures in the financial reports. the disclosure framework in pharmaceutical sector has taken a shape after year 2011 (mehta & chandani; 2015). the study of mehta and chandani (2015) takes top five pharma companies listed on bse for the year 2009-10 to 2014-15 and establishes relation between their csr disclosures and financial performance. they assert that disclosure of csr and its importance on company’s strategy has significantly improved 2011 onwards. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 155 sachdeva, batra & walia (2015) investigated growth in corporate disclosure practices in selected indian companies listed on bse during the year 2005-2012 from pharmaceutical, fmcg, automobile, financial, telecom and it sector. the study shows that among all sectors, pharmaceutical sector shows the least (7%) growth in disclosure scores over the seven years as compared to fmcg sector where average increase is 26% since 2005. halder & mishra (2017) studied the factors affecting timeliness of information in indian pharmaceutical sector on a sample of top 50 pharma companies listed on bse. the authors studied the lag in number of days that companies take to disclose during the year 2010-11 to 2012-13. they found that there is an average maximum lag of 211 days in reporting information in annual reports. they asserted that age of the company, foreign shareholding and revenue from abroad have significant impact on timeliness of disclosure. the other investigation on disclosure of corporate governance practices by kalashree & rajshekhar (2018) on 53 listed indian pharmaceutical companies (mid cap and large cap) for the year 2013-14 found that among the ten disclosure segments that authors divided for disclosure score, the listed companies are liberal towards disclosing information about remuneration, and compliance and management aspects but disclose least about subsidiary companies. while studying various aspects of mandatory and voluntary disclosure in pharmaceutical sector in india, most of the previous studies ignored attributes like profitability, liquidity, volatility, market price of shares being affected by the nature and extent of disclosures. if such an association between the disclosures with the other variables is brought out, supported by the conclusive statistical results, the disclosure environment will set itself into a new pace. this study aims to contribute to the uprising pharmaceutical industry by motivating the industry for maximum extent of disclosures. as it is rightly said that, health sector, by its mere honest disclosure practices, can contribute to the healthy living of the masses. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 156 objective of the study to ascertain the extent of voluntary information disclosures in indian pharmaceutical companies listed on bombay stock exchange (bse) research methodology -sample unit: companies listed on bombay stock exchange (bse) -sample size: companies listed companies from bse (large cap having capital of more than rs. 10,000crores) -sector under study: pharmaceutical sector -period under study: 8 years from 2010-11 to 2017-18 -basis for selection of companies under study: the company is listed on the bombay stock exchange for more than three years. the basis for selection is their market capitalization the company’s ticker symbol does not suffer a halt for more than three months on stock market. the company’s voluntary data should coincide at least 50% of the check list set in by the researcher for the purpose of study. the data about the company should be available. there are 170 pharma companies listed on bse. out of these first twenty companies are large cap companies having a market capitalization of more than rs.10, 000 crore. out of the total market capitalization of rs.7, 71,998crores, the market capitalization of these twenty sums up to rs.6, 54, 792crores, which represent 84.82% of total visible pharma market on bse. rest are mid cap companies having the market capitalization between multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 157 rs.2crores to rs.10,000crores and only one small cap company having a market capitalization of less than 2crore. thus, taking into consideration the magnificent impact of big cap firms in shaping the face of pharma market, this study takes into account only large cap firms as sample under study. hence the sample is chosen among the twenty big cap companies listed on bse satisfying the criteria mentioned under ‘basis of selection of company’. the companies, whose data are unavailable for any year during the period under study (financial year 2010-11 to 201718), are wiped out. filtering such companies, the final sample size is 13 companies chosen among those total 20 big cap companies. measurement of corporate voluntary disclosure (vdcl) prior to year 1985, many studies had calculated disclosure quality but the concrete explanation about calculating the extent of voluntary disclosure was formulated by firer and meth, 1986; wallace, 1988; meek, roberts and gray, 1995. wallace et al. described disclosure as an abstract construct that does not possess its own inherent characteristics. they developed a checklist method to score the voluntary disclosure items on a dichotomous scale of assigning 1 if disclosure is there and 0 is no disclosure found and thus; calculating the total score. chau and gray (2002) have also used this checklist with some minor changes to calculate the voluntary disclosure of hong kong firms. following the same checklist method, the corporate voluntary disclosure as denoted by vdcl is arrived at, by splitting total 55 items into six categories of different items of same nature placed under one category. these 55 items are extracted from a list of 71 variables, eliminating the mandatory ones. the sub categories are vdcl1= general corporate information vdcl2= external audit committee vdcl3= financial information vdcl4=forward looking information vdcl5= employee information, social responsibility and environmental policy vdcl6= board structure disclosure multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 158 taking into consideration the mandatory disclosure framework laid under the regulations of sebi, icai and accounting standards, the author has come up with the disclosure items that are not mandatory. the author is able to identify total 55 items of different nature placed under six different categories. the author goes through the annual reports of each company for each year from 2010-11 to 2017-18 and identifies the disclosures matching with the list of disclosure items under study. 2. analysis and interpretations in the present study, in order to understand the voluntary corporate disclosure practices followed by the pharmaceuticals companies selected for the purpose of study a voluntary disclosure index has been designed based on which the calculation is done. the index designed for understanding the voluntary disclosure index considers the 55 statements related to the voluntary disclosure broadly classified into six dimensions. the response about the particular statement was considered to be binomial in nature having a zero score for a non-response item while one was assigned to a positive response item. the results of companies on a disclosure index are shown as follows: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 159 table 1: classification of information items for voluntary disclosure table 1 shows that all the statements related to voluntary disclosure in the annual reports of the pharmaceutical companies are classified into broad 6 dimensions and consist of 55 items related to the 6 dimensions under the study period (2010-11 to 2017-18). all the annual reports were considered for the study and the disclosure made in the annual reports are considered. table 2: item wise voluntary disclosure score classification of items 2018 2017 2016 2015 2014 2013 2012 2011 i. general corporate information 1. general information about the economy 92.31 100.00 100.00 100.00 92.31 100.00 92.31 92.31 2. corporate mission statements 69.23 76.92 69.23 69.23 69.23 46.15 38.46 38.46 3. brief history of the corporation (the establishment and development) 30.77 23.08 15.38 15.38 7.69 23.08 23.08 30.77 4. description of major goods/products 38.46 15.38 30.77 23.08 23.08 38.46 15.38 15.38 5. analysis of enterprises’ market share 69.23 76.92 76.92 69.23 61.54 61.54 53.85 46.15 6. business environment (economics, political…) 23.08 15.38 15.38 15.38 23.08 15.38 15.38 15.38 sl. no classification item items percentage 1 i. general corporate information 12 21.82 2 ii. external audit committee 7 12.73 3 iii. financial information 5 9.09 4 iv. forward-looking information 9 16.36 5 v. employee information, social responsibility and environmental policy 14 25.45 6 vi. board structure disclosure 8 14.55 total 55 100.00 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 160 7. statement disclosures relating to competitive position in the industry 23.08 15.38 15.38 15.38 15.38 23.08 15.38 15.38 8. description of marketing networks for finished goods/products 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 9. information of member companies 61.54 46.15 46.15 38.46 46.15 46.15 38.46 38.46 10. methods of quality control 23.08 15.38 38.46 23.08 15.38 23.08 15.38 23.08 11. company’s achieved awards 61.54 69.23 76.92 61.54 53.85 61.54 46.15 30.77 12. corporate contributions to the national economy 30.77 30.77 30.77 46.15 23.08 7.69 7.69 7.69 ii. external audit committee 1. the role and function of the audit committee 92.31 92.31 92.31 92.31 84.62 76.92 76.92 76.92 2. names and qualifications of the members of audit committee 76.92 76.92 76.92 76.92 69.23 76.92 76.92 76.92 3. number of members on audit committee 84.62 92.31 92.31 84.62 84.62 76.92 76.92 76.92 4. number of committee meetings 84.62 92.31 92.31 84.62 84.62 84.62 84.62 84.62 5. attendance at committee meetings 53.85 53.85 53.85 53.85 53.85 46.15 46.15 46.15 6. statement of independence 92.31 92.31 92.31 84.62 84.62 84.62 84.62 84.62 7. report on completed work 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 iii. financial information 1. summary of financial data for the last 3 years or over 84.62 76.92 69.23 61.54 61.54 69.23 61.54 69.23 2. share price information 92.31 92.31 100.00 92.31 84.62 92.31 92.31 84.62 3. retained profit 84.62 84.62 84.62 84.62 76.92 76.92 76.92 76.92 4. bank loan, mortgage and their use 76.92 76.92 76.92 69.23 61.54 46.15 38.46 38.46 5. advertising and publicity expenditure 15.38 15.38 23.08 15.38 0.00 0.00 0.00 0.00 iv. forward-looking information 1. factors that may affect future performance 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2. new product/service development 61.54 61.54 61.54 53.85 46.15 38.46 46.15 38.46 3. marketing plan, distribution system expanding plan 15.38 30.77 15.38 23.08 0.00 7.69 0.00 0.00 4. effect of business strategy on future performance 15.38 0.00 0.00 0.00 0.00 0.00 0.00 0.00 5. research and development expenditure 53.85 69.23 69.23 53.85 46.15 53.85 53.85 61.54 6. projection of cash flows 7.69 7.69 7.69 0.00 0.00 0.00 0.00 0.00 7. earnings per share forecast 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 8. planned capital expenditure 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 161 9. future profit forecast 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 v. employee information, social responsibility and environmental policy 1. total number of employees for the last or more years 84.62 84.62 76.92 76.92 61.54 61.54 46.15 46.15 2. category of employees by gender 53.85 53.85 38.46 23.08 23.08 23.08 7.69 7.69 3. amount of employee remuneration, remuneration policies and bonus 69.23 61.54 69.23 61.54 46.15 38.46 23.08 30.77 4. policy on employee training 38.46 38.46 30.77 15.38 7.69 7.69 0.00 7.69 5. expenses for employee training 0.00 0.00 0.00 7.69 7.69 7.69 0.00 0.00 6. reasons for change in employee number 7.69 0.00 0.00 0.00 0.00 0.00 7.69 7.69 7. qualification of the accountants 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 8. data on workplace accidents 0.00 0.00 0.00 0.00 0.00 0.00 0.00 7.69 9. disclosure of welfare policy 15.38 15.38 7.69 0.00 0.00 0.00 0.00 0.00 10. recruitment policy 30.77 23.08 15.38 7.69 0.00 0.00 0.00 0.00 11. factors of corporate culture 15.38 7.69 0.00 7.69 0.00 0.00 0.00 0.00 12. information about safety policy 84.62 84.62 46.15 38.46 23.08 23.08 7.69 30.77 13. cost of safety measures 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 14. environment protection programs 100.00 100.00 76.92 69.23 53.85 53.85 38.46 53.85 vi. board structure disclosure 1. education and professional qualification of directors 61.54 53.85 53.85 53.85 53.85 61.54 61.54 61.54 2. directors’ interests in competing businesses 30.77 30.77 38.46 23.08 23.08 30.77 23.08 23.08 3. directors’shareholding in the company and other related interests 84.62 84.62 76.92 76.92 46.15 38.46 46.15 46.15 4. number of meetings per year 92.31 92.31 92.31 92.31 84.62 84.62 84.62 92.31 5. director’s analysis of the fee and other benefits disclosure 76.92 69.23 69.23 69.23 61.54 61.54 53.85 53.85 6. role and function of the remuneration committee 84.62 76.92 84.62 76.92 69.23 61.54 61.54 61.54 7. directors’ current accounts/loans to officers 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 8. directors’interests in significant contracts 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 table 2 shows that for calculating item wise voluntary disclosure score each of the items were considered and the number of companies disclosing that particular item was divided by the total number of companies considered for the purpose of study. in this present study the total number of companies selected was 13, and hence, the maximum attainable score multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 162 is 13. the maximum score which can be achieved in a particular year for any item will be 100 if all the companies disclose item for all the years. the results are disappointing to see that only cent percent score is achieved in the first item of section i i.e. general information of the company. in rest all of the items the disclosure score is not so satisfactory. startling to see that in items like data on workplace accidents, employee trainings or director loans, the score is as low as zero in consecutive years, depicting that none of the company in those particular years have revealed any information about these items. no doubt, the good news is that the trend of disclosure is escalating year by year. table 3: company wise disclosure scores company/year 2018 2017 2016 2015 2014 2013 2012 2011 average score (201018) average % (201018) cadila 25 25 18 18 19 17 17 22 20.13 36.59 biocon 26 27 28 27 24 25 22 22 25.13 45.68 cipla 28 27 28 24 13 13 13 13 19.88 36.14 piramal 37 31 30 25 20 17 18 17 24.38 44.32 dreddy 26 28 26 23 24 26 12 12 22.13 40.23 lupin 27 24 24 25 24 24 22 21 23.88 43.41 torrent 15 14 14 10 8 7 7 7 10.25 18.64 pfizer 32 30 28 26 12 16 10 14 21.00 38.18 glaxo 15 16 15 17 15 16 15 15 15.50 28.18 divis lab 17 20 20 13 15 18 18 16 17.13 31.14 auro 32 28 30 29 28 28 25 26 28.25 51.36 ajanta 16 17 17 17 18 15 14 15 16.13 29.32 sunpharma 22 22 21 20 18 16 14 15 18.50 33.64 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 163 table 3 is about the voluntary disclosure scores obtained by the companies from the year 2010-11 to 2017-18 is calculated for the companies selected for the purpose of the study. the range of voluntary disclosure score ranges from 10.25 to 28.25 (in absolute terms) with the score range ranging from 18.64 % to 51.36 %. most of the companies could not score high in the voluntary disclosure scores. this is a disheartening picture of the disclosure practices prevailing the pharmaceutical sector. the highest score obtained is 37 (out of 55) by piramal. the highest score figure is not even equivalent to 8 percent of total score expected. where companies like auro, piramal and pfizer are among the list of companies showing good amount of disclosure, the companies like torrent, glaxo and ajanta are revealing poor disclosure scores throughout eight years of study. table 4: descriptive statistics of voluntary disclosure score 2018 2017 2016 2015 2014 2013 2012 2011 mean 24.46 23.77 23.00 21.08 18.31 18.31 15.92 16.54 standard error 1.97 1.54 1.61 1.60 1.58 1.63 1.42 1.40 median 26.00 25.00 24.00 23.00 18.00 17.00 15.00 15.00 mode 26.00 27.00 28.00 25.00 24.00 16.00 22.00 15.00 standard deviation 7.11 5.54 5.79 5.75 5.71 5.89 5.11 5.06 sample variance 50.60 30.69 33.50 33.08 32.56 34.73 26.08 25.60 kurtosis -0.93 -0.98 -1.54 -0.62 -0.56 -0.19 -0.39 0.09 skewness 0.05 -0.54 -0.27 -0.56 -0.05 0.09 0.15 0.17 the above table 4 depicts that mean score of the companies for voluntary disclosure ranges from 16.54 to 24.46 starting from the year 2011 to 2018. the mean voluntary disclosure score exhibits a steady increasing trend over the years; however during multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 164 the year 2012 the mean score marginally dipped to 15.92 from 16.54 during the year 2011. the year 2013 and 2014 showed the same mean score of 18.31 for the voluntary disclosure score. the year 2015 to 2018 showed an increasing trend starting from 21.08 (2015) to 24.46 (2018). the median of the voluntary disclosure score exhibits a steady increasing trend over the years starting from 15.00 (2011) to 26.00 (2018). the modal value of the voluntary disclosure score exhibits increasing trend over the years starting from 15.00 (2011) to 26.00 (2018), with fluctuations during the intermediate years with the score being 22.00(2012); 28.00(2016), thereby exhibiting fluctuations in the intermediate years. the sample variance of the voluntary disclosure score exhibits a steady increasing trend over the years starting from 25.60 (2011) to 50.60 (2018) thereby exhibiting higher variability in the companies reporting the voluntary disclosure scores. the sample skewness and kurtosis of the voluntary disclosure score exhibits normality of the data set over the years. graph 1: year wise mean score of voluntary disclosure 16,54 15,92 18,31 18,31 21,08 23,00 23,77 24,46 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 2017-18 year wise average voultary corporate governance disclosure scores mean score multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 165 the above graph shows mean score of the companies for voluntary disclosure ranges from 16.54 to 24.46 starting from the year 2011 to 2018. the mean voluntary disclosure score exhibits a steady increasing trend over the years; however during the year 2012 the mean score marginally dipped to 15.92 from 16.54 during the year 2011. the year 2013 and 2014 showed the same mean score of 18.31 for the voluntary disclosure score. the year 2015 to 2018 showed an increasing trend starting from 21.08 (2015) to 24.46 (2018). table 5: top five companies on the basis of voluntary disclosure score the table 5 reveals the overall average voluntary corporate governance disclosure scores of the top five companies for all eight years. the chart is topped by auro with a score of 28.25, followed by biocon (25.13), piramal (24.38), lupin (23.88) and dreddy (22.13). 3. conclusion in the present study, in order to understand the voluntary corporate disclosure practices followed by the pharmaceutical’s companies selected for the purpose of study a voluntary disclosure index has been designed based on which the calculation is done. the index designed for understanding the voluntary disclosure index considers the 55 statements related to the voluntary disclosure broadly classified into six dimensions. the response about the particular statement was considered to be binomial in nature having a zero score for a non-response item while one was assigned to a positive response item. the disclosure scores when summed up, it was found that a maximum value that a company could attain any year was 37 (piramal) out of total 55 (refer table 1). that means not even a single company discloses at least 80% of the proposed voluntary disclosure items considered under study. the companies are not disclosure friendly. though there have been strict regulations laid by the regulators of the company especially after the enactment of companies act 2013 that focus on strict disclosures, still the companies show a reluctance rank company average disclosure score 1 auro 28.25 2 biocon 25.13 3 piramal 24.38 4 lupin 23.88 5 dreddy 22.13 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 166 in disclosing the information. no doubt there seem to be increasing trend in the extent of disclosures from 2010-11 to 2017-18, yet companies like torrent pharmaceuticals are existing with a disclosure score as low as 17 out of 55 in a year. overall the increasing trends in last 8 years can be good news if the trend continues in same direction. thus, fulfilling its one objective i.e. to know that extent of disclosure, the overall score is not so satisfactory. there may be reasons inferred from the disappointing scores of the pharmaceutical sector which may be hidden in the theories explaining the need for voluntary disclosure. the theories like signaling theory state the deliberate supply of information beyond need can prove to be a reason of conflict between incentives and deterrent forces. whereas there is another theory based on agency problem stating that disclosure of information also uncovers superior information of the top management which is not supposed to be in the heads of shareholders just because they are not active players of the management. but this demand and supply in equilibrium creates information asymmetry which may lead to several other consequences. literatures have been supporting the lack of information supply as a cause to fluctuation of stock prices and diminishing market values of the companies. another aspect of information availability is linked with entering the market sources for money borrowing. the interested parties may not lend to the companies having extra disclosures, as their idea of understanding the information, which is complex to decode, may bring them into the dilemma of lending decisions (which should not happen actually). where, as per theory of cost of capital, more disclosures lead to decrease in cost of capital, the theory has proven to be contradicting itself in the studies where the disclosure surpassed beyond a minimal core. to conclude, the present disclosure system in india fails to distinguish between the very different needs of the users of the financial reports. while some users may be happy to have lengthy disclosures that may bring a positive impact on fluctuate of stocks in the market, others may be sent information that is far longer and complex to understand to make use of. the set of information useful for most users could be sort, precise and beyond a minimal core, no doubt, it has to be decided by the company to reflect its own circumstances, but the role of policy makers cannot be isolated in converging the global disclosure standards towards indian pharmaceutical sector which is the third largest income generating sector in the world. 4. references 1. accounting, f. (2014). voluntary disclosure frequency and cost of debt : an analysis in the tunisian context dorra talbi * mohamed ali omri, 6(2). https://doi.org/10.1504/ijmfa.2014.064521 https://doi.org/10.1504/ijmfa.2014.064521 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 167 2. adam, t., and v. k. goyal. "the investment opportunity set and its proxy variables." journal of financial research 31.1 (2008): 41. print. https://doi.org/10.1111/j.1475-6803.2008.00231.x 3. agarwal, s. p., monem, r. m., & ariff, m. (1996). price to book ratio as a valuation model: an empirical investigation. finance india, 10(2), 333–344. https://doi.org/10.1.1.570.3053 4. aicpa (1994), “improving business reporting-a customer focus:meeting the information needs of investors and creditors”, comprehensive report of the special committee on financial reporting (the jenkins report), 5. barry, c., and s. brown. "differential information and security market equilibrium." journal of financial and quantitative analysis 20 (1985): 407-422. https://doi.org/10.2307/2330758 6. baumann, u., and e. nier. 2003. “market discipline, disclosure, and moral hazard in banking.” in proceedings of the 2003 conference on bank structure and competition. chicago: federal reserve bank of chicago 7. bertomeu, j., beyer, a., & dye, r. a. (2011). capital structure, cost of capital, and voluntary disclosures. accounting review, 86(3), 857–886. https://doi.org/10.2308/accr.00000037 8. buzby, s.l. (1974a), "the nature of adequate disclosure", journal of accountancy, april, pp. 3 8-47. buzby, s.l. (1974b), "selected items of information and their disclosure in annual reports", accounting review, vol. xlix no. 3 july, pp. 423-43 5. buzby, s.l. (1975a), "the boundaries of adequate disclosure", the singapore accountant, vol. 10, pp. 83-91. 9. botosan, c.a. (1997). disclosure level and the cost of equity capital, accounting review, 72, pp. 323–350. botosan, c.a. (2006). disclosure and the cost of capital: what do we know? accounting and business research, international accounting policy forum, pp. 31-40. http://dx.doi.org/10.1080/00014788.2006.9730042 10. botosan, c.a. and m.a. plumlee (2002), a re-examination of disclosure level and the expected cost of equity capital, journal of accounting research, vol. 40 (1), pp. 21-40. https://doi.org/10.1080/00014788.2006.9730042 11. cooke, t. e. (1989). voluntary corporate disclosure by swedish companies. journal of international financial management & accounting, 1(2), 171–195. https://doi.org/10.1111/j.1467-646x.1989.tb00009.x https://doi.org/10.1111/j.1475-6803.2008.00231.x https://doi.org/10.1.1.570.3053 https://doi.org/10.2307/2330758 http://dx.doi.org/10.1080/00014788.2006.9730042 https://doi.org/10.1080/00014788.2006.9730042 https://doi.org/10.1111/j.1467-646x.1989.tb00009.x multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 168 12. campbell, d., shrives, p., & saager, h.b. (2001). voluntary disclosure of mission statements in corporate annual reports: signaling what and to whom?, business and society review, 106(1), pp. 65–87. https://doi.org/10.1111/0045-3609.00102 13. cheynel, e. (2013). a theory of voluntary disclosure and cost of capital. review of accounting studies, 18(4), 987–1020. https://doi.org/10.1007/s11142-0139223-1 14. choi, f.d.s. (1973). financial disclosure and entry to the european capital market, journal of accounting research, 11(2), pp. 159–175. http://dx.doi.org/10.2307/2490187 15. david, p., j. p. o'brien, and t. yoshikawa. "the implications of debt heterogeneity for r&d investment and firm performance." the academy of management journal 51.1 (2008): 165-81. print https://doi.org/10.5465/amj.2008.30772877 16. determination of the weighted average cost of capital of upc and. (2009), (october). 17. diamond, d., & verrecchia, r. (1991). disclosure, liquidity, and the cost of capital, the journal of finance, 46(4), pp. 1325–1355. http://dx.doi.org/10.2307/2328861 18. forker, j. j. (1992), "corporate governance and disclosure quality", accounting and business research, vol. 22, no. 86, spring, pp. 111-124. https://doi.org/10.1080/00014788.1992.9729426 19. francis, j., nanda, d., & olsson, p. (2008). voluntary disclosure, earnings quality, and cost of capital. journal of accounting research, 46(1), 53–99. https://doi.org/10.1111/j.1475-679x.2008.00267.x 20. gray, s. j. with mcsweeney, l. b. and shaw, j. c. (1984), information disclosure and the multinational corporation, chichester: john wiley & sons, inc. 21. gelb david.s; paul zarowin (2002) “corporate disclosure policy and the informativeness of stock prices review of accounting studies”; mar 2002; abi/inform global pg. 33 22. haggard, k. s., martin, x., & pereira, r. (2008). stock price informativeness ? financial management, 747–768. https://doi.org/10.1111/j.1755-053x.2008.00033.x 23. hail, l., c. leuz (2007), capital market effect of mandatory ifrs reporting in the eu: empirical evidence, http://www.afm.nl https://doi.org/10.2139/ssrn.1511671 https://doi.org/10.1111/0045-3609.00102 https://doi.org/10.1007/s11142-013-9223-1 https://doi.org/10.1007/s11142-013-9223-1 http://dx.doi.org/10.2307/2490187 https://doi.org/10.5465/amj.2008.30772877 http://dx.doi.org/10.2307/2328861 https://doi.org/10.1080/00014788.1992.9729426 https://doi.org/10.1111/j.1475-679x.2008.00267.x https://doi.org/10.1111/j.1755-053x.2008.00033.x http://www.afm.nl/ https://doi.org/10.2139/ssrn.1511671 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 169 24. hassan omaima a.g, peter romilly, gianluigi giorgioni, david power : the value relevance of disclosure: evidence from the emerging capital market of egypt the international journal of accounting, volume 44, issue 1, pages 79-102 25. healy, paul m. and palepu, krishna, information asymmetry, corporate disclosure and the capital markets: a review of the empirical disclosure literature (december 2000). jae rochester conference april 2000. https://doi.org/10.2139/ssrn.258514 26. ibrahim, k. (2014). firm characteristics and voluntary segments disclosure among the largest firms in nigeria. international journal of trade, economics and finance, 5(4), 327–331. https://doi.org/10.7763/ijtef.2014.v5.392 27. jensen, m.c., & meckling, w.h. (1976). theory of the firm: managerial behavior, agency costs and ownership structure, journal of financial economics, 3(4), pp. 305–360. http://dx.doi.org/10.1016/0304-405x(76)90026-x 28. kristandl, g., & bontis, n. (2007). the impact of voluntary disclosure on cost of equity capital estimates in a temporal setting. journal of intellectual capital, 8(4), 577–594. https://doi.org/10.1108/14691930710830765 29. lang, m., & lundholm, r. (1993). cross-sectional determinants of analyst ratings of corporate disclosures, journal of accounting research, 31(2), pp. 246–271. http://dx.doi.org/10.2307/2491273 30. lang, m., & lundholm, r. (1996). corporate disclosure policy and analysts behaviour, the accounting review, 71, pp. 467–492. 31. lang, h.l., r.j. lundholm (2000), voluntary disclosure and equity offerings: reducing information assymetry or hyping the stock, contemporary accounting research, vol. 17 (4), pp. 623-662. https://doi.org/10.1506/9n45-f0jx-axvw-lbwj 32. leuz, c., and verrecchia (2000) "the economic consequences of increased disclosure." journal of accounting research (2000): 91-124. https://doi.org/10.2307/2672910 33. lindblom, c.k. (1994). the implications of organizational legitimacy for corporate social performance and disclosure, critical perspectives on accounting conference. new york. 34. magness, v. (2006). strategic posture, financial performance and environmental disclosure: an empirical test of legitimacy theory, accounting, auditing and https://doi.org/10.2139/ssrn.258514 https://doi.org/10.7763/ijtef.2014.v5.392 http://dx.doi.org/10.1016/0304-405x(76)90026-x https://doi.org/10.1506/9n45-f0jx-axvw-lbwj https://doi.org/10.2307/2672910 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 170 accountability journal, 19(4), pp. 540–563. http://dx.doi.org/10.1108/09513570610679128 35. mangala, d. (2015). disclosure through annual reports: a study of indian corporate sector. international journal of research in management, science & technology, 3(2), 2321–3264. retrieved from http://www.ijrmst.org/download/vol3no2/152.pdf 36. mathew james (1999), “perception about the need for innovations in financial reporting: a survey”, the management accountant, (may), pp. 373-375. 37. meek, g. k. and gray, s. j. (1989), “globalisation of stock market and foreign listing requirement: voluntary disclosures by continental european companies listed on the london stock exchange”, journal of international business studies, vol. 20(2). https://doi.org/10.1057/palgrave.jibs.8490854 38. palepu k.g., p.m. healy and v.l. bernard (2004), business analysis and valuation: using financial statements, mason, thomson south western 39. pankaj m madhani (2008) “new models for financial reporting in the 21st century” in corporate disclosure: concepts and practices, the icfai university press 40. petrova, e., georgakopoulos, g., sotiropoulos, i., & vasileiou, k. z. (2012). relationship between cost of equity capital and voluntary corporate disclosures. international journal of economics and finance, 4(3), 83–96. https://doi.org/10.5539/ijef.v4n3p83 41. ross, s.a. (1977). the determination of financial structure: the incentive signaling approach, bell journal of economics, 8(1), pp. 23–40. https://doi.org/10.2307/3003485 42. shahriari, m., mesbahi, l., & saeedi, s. m. (2015). risk free rate in the context of indian market (vol. 1). retrieved from www.ijariie.com547 43. schutter p. and v.o connell (2006) “the trend toward voluntary corporate disclosures” management accounting quarterly, 7, no. 2 (winter) 1-9. 44. shankar tilak, "making corporate reporting practices more communicative", economic and political weekly, november 1972, pp.164-168. 45. singh, d.r. and gupta, b.n., “corporate financial disclosure in indian companies,” indian journal of accounting (june and december 1977), pp. 21-37 46. singhvi, s. s. (1972), "corporate management's inclination to disclose financial information", financial analyst's journal, july-august, pp. 66-73. solomons, d. (1989), guidelines for financial reporting standards, london: the institute of chartered accountants in england and wales. https://doi.org/10.2469/faj.v28.n4.66 http://dx.doi.org/10.1108/09513570610679128 http://www.ijrmst.org/download/vol3no2/152.pdf https://doi.org/10.1057/palgrave.jibs.8490854 https://doi.org/10.5539/ijef.v4n3p83 https://doi.org/10.2307/3003485 http://www.ijariie.com547/ https://doi.org/10.2469/faj.v28.n4.66 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 171 47. spero, l. l. (1979), the extent and causes of voluntary disclosure of financial information in three european capital markets: an exploratory study, unpublished doctoral dissertation, harvard university. 48. spence, m. (1973). job market signalling, quarterly journal of economics, 87(3), pp. 355–374 http://dx.doi.org/10.2307/1882010 49. verrecchia, r.(1990) "information quality and discretionary disclosure." journal of accounting and economics 12 (1990): 365-380. https://doi.org/10.1016/01654101(90)90021-u 50. watson, a., shrives, p., & marston, c. (2002). voluntary disclosure of accounting ratios in the uk, british accounting review, 34(4), pp. 289–313. http://dx.doi.org/10.1006/bare.2002.0213 web links: • https://www.kellogg.northwestern.edu/accounting/papers • www.mccombs.utexas.edu/~/media/.../clinch%20%20verrecchia030512.pdf • https://olin.wustl.edu/docs/faculty/dye.pdf • www.nber.org/papers/w14897 • www.emeraldinsight.com/doi/pdf/10.1108/14691930710830765 • www.rug.nl/research/portal/files/14494335/thesis.pdf • pasj.org/iijm/volume2issue7/iijm02.pdf • www.indianjournaloffinance.co.in/index.php/ijf/article/view/72089 • http://www.afm.nl • http://www.fasb.org • www.ibef.org http://dx.doi.org/10.2307/1882010 https://doi.org/10.1016/0165-4101(90)90021-u https://doi.org/10.1016/0165-4101(90)90021-u http://dx.doi.org/10.1006/bare.2002.0213 https://www.kellogg.northwestern.edu/accounting/papers https://olin.wustl.edu/docs/faculty/dye.pdf http://www.nber.org/papers/w14897 http://www.emeraldinsight.com/doi/pdf/10.1108/14691930710830765 http://www.rug.nl/research/portal/files/14494335/thesis.pdf http://www.indianjournaloffinance.co.in/index.php/ijf/article/view/72089 http://www.afm.nl/ http://www.fasb.org/ http://www.ibef.org/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 172 annexure following are the contents under each category: i. general corporate information 1. general information about the economy 2. corporate mission statements 3. brief history of the corporation (the establishment and development) 4. description of major goods/products 5. analysis of enterprises’ market share 6. business environment (economics, political…) 7. statement disclosures relating to competitive position in the industry 8. description of marketing networks for finished goods/products 9. information of member companies 10. methods of quality control 11. company’s achieved awards 12. corporate contributions to the national economy ii. external audit committee 13. the role and function of the audit committee 14. names and qualifications of the members of audit committee 15. number of members on audit committee 16. number of committee meetings 17. attendance at committee meetings 18. statement of independence 19. report on completed work iii. financial information 20. summary of financial data for the last 3 years or over multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 173 21. share price information 22. retained profit 23. bank loan, mortgage and their use 24. advertising and publicity expenditure iv. forward-looking information 25. factors that may affect future performance 26. new product/service development 27. marketing plan, distribution system expanding plan 28. effect of business strategy on future performance 29. projection of research and development expenditure 30. project of cash flows 31. earnings per share forecast 32. planned capital expenditure 33. future profit forecast v. employee information, social responsibility and environmental policy 34. total amount of employees for the last year or more years 35. category of employees by sex 36. amount of employee remuneration, remuneration policies and bonus 37. policy on employee training 38. expenses for employee training 39. reasons for change in employee number 40. qualification of the accountants 41. data on workplace accidents 42. disclosure of welfare policy 43. recruitment policy 44. factors of corporate culture 45. information about safety policy multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11531 social and technological sciences issn: 2341-2593 khanna and singh-chahal (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 147-174 | 174 46. cost of safety measures 47. environment protection programs vi. board structure disclosure 48. education and professional qualification of directors 49. directors’ interests in competing businesses 50. directors’shareholding in the company and other related interests (e.g. stock options) 51. number of meetings per year 52. director’s analysis of the fee and other benefits disclosure 53. role and function of the remuneration committee 54. directors’ current accounts/loans to officers 55. directors’ interests in significant contracts multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 14 playing, constructionism, and music in early-stage software engineering education nicolas e. golda, * , ross purvesa , evangelos himonidesa auniversity college london (ucl), uk gower street, london, wc1e 6bt *correspondence: n.gold@ucl.ac.uk received: 7 october 2021; accepted: 21 february 2022; published: april 2022 abstract understanding that design involves trade-offs, thinking at multiple levels of abstraction, and considering the cohesion and coupling between sub-components of a larger whole is an important part of software (and other) engineering. it can be challenging to convey such abstract design concepts to novice engineers, especially for materials that are themselves abstract (e.g., software). such challenges are compounded when teaching at the secondary school stage where students have limited experience of large-scale design challenges that motivate the need for abstraction at all. in this paper, we describe a method for introducing these concepts to secondary school students using lego ® and raspberry pi computers, asking them to build musical instruments as an entertaining way of motivating engagement with learning about design through play. the method has been successfully piloted in a series of three classroom sessions and key observations and experiences of using the method are presented. keywords: constructionism; construction-blocks; music; design; children; lego® to cite this article: gold, n.e., purves, r., himonides, e. (2021). playing, constructionism, and music in early-stage software engineering education. multidisciplinary journal for education, social and technological sciences, 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0002-2195-5995 https://orcid.org/0000-0003-4301-7024 https://orcid.org/0000-0002-8749-0799 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 15 1. introduction coding is an increasingly important part of school curricula with governments around the world requiring it to be taught from the earliest years (rich et al, 2019). in europe and the us, concerns have been expressed about education in informatics (barendsen & steenvoorden, 2016) and in the uk a royal society (2012) report on unsatisfactory information and communication technology (ict) teaching in schools led to curriculum changes (hm government, 2013). despite these improvements, student activities are still simple compared to the scope and scale of professional software engineering and it would be unreasonable to expect school students with limited software development experience to immediately assimilate the need for, and appreciate the problems of, large-scale software development. hislop (2008) claims that it is the scale and complexity of software systems that drives the problems and practices of software engineering, and that until students gain some understanding of this they find it hard to appreciate what software engineering tries to address. ludewig (2008) states that software engineering is not just a set of topics but is primarily a mindset like that of engineers. ali aligns software engineering with other engineering disciplines as something requiring a team effort (ali, 2006). exposing students to the kind of thinking and practice required at an early stage may therefore better prepare them for the challenges ahead. monteiro et al. also identify the desirability of signifying broader software engineering (beyond coding) at this stage of education (in their case particularly software quality) (monteiro et al., 2017). wing (2006) argues strongly for “computational thinking” (beyond programming) to be promoted to “pre-college” audiences. tsai et al. (2021) survey a range of computational thinking studies that characterize it in various ways. of particular relevance to our work, brennan and resnick (2012) defined three dimensions of computational thinking: computational concepts, computational practices, and computational perspectives. the first of these deals mainly with programming, the second with the development of design practices, and the third with designers’ views of the world and themselves. our work here is situated mainly in the second (and to some extent the third) of these dimensions. this focus also aligns with the cooperativity, creativity, critical thinking, and problem solving elements of the framework developed by doleck et al. (2017), and the abstraction and decomposition elements of selby and woollard’s (2013) definition. https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 16 in this paper, we present a pilot, exploratory, qualitative action-research study built on three sessions of activities to introduce a class of secondary-school children to computational thinking practices whilst being active, engaging and fun (alanazi (2020) identifies active recreational activities as valuable in the context of the similarly pedagogically-challenging field of mathematics). we applied educational theory (constructivist methods), used teamwork and reflection, and introduced technical concepts using non-technical approaches. our objective was to explore whether the combination of music and making could lead to effective opportunities for software engineering education and to discover the practical issues involved. the focus was therefore primarily on the exploration and feasibility of our approach to the development of computational practices and perspectives. 2. theoretical background and related work our research is underpinned by the interrelated concepts of constructivist and constructionist learning (most strongly associated with the work of piaget and vygotsky). as gogus notes, “constructivist learning claims that learners do not just absorb information. instead, learners construct information by actively trying to organize and make sense of it in unique ways” (2012, p.783) noting also that piaget asserted that individuals do this through constructing mental models and schemas, which undergo further revision through the processes of assimilation, accommodation and correction. vygotsky’s contribution was to highlight the extent to which these processes are socially and culturally influenced. thus, individuals’ constructions depend on interaction with peers, “more knowledgeable others” (often teachers, parents or workshop leaders) and prior experiences with “cognitive tools” such as language, technical, cultural and historical awareness. papert developed these concepts further to become constructionism, arguing that it is through making tangible things that we consolidate these models and schemas through physical representation (papert & harel, 1991; james, 2013). this deep and powerful connection between the concrete and the abstract often leads to descriptions of constructionism such as “thinking through our fingers” or making “objects to think with”. yet these physical representations also contribute to the social aspects of learning, since they provide a shared focus for group tasks and scaffold communication between collaborators (jensen, 2017). whilst franco and gillanders’ (2014) work is aimed at masters students https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 17 studying secondary school education, we nonetheless note their positive conclusion that where learners’ voices are heard, they take a leading role in their own learning. the desire to build tangible objects to meet given objectives, and the pride and satisfaction that can result, invokes another of papert’s theoretical principles: “hard fun”. learners, it is argued, will typically persevere in the construction of these objects, overcoming all manner of technical challenges in the process. as papert (2002) once observed, children often describe constructionist learning activities as fun because they are hard, rather than in spite of them being hard. the term ‘flow’ is most associated with csikzentmihalyi and refers to “a state of optimal experience characterized by total absorption in the task at hand; a merging of action and awareness in which the individual loses track of both time and self” (schmidt, 2010, p.605). similarly, and particularly during group-based activities, rich instances of “divergent thinking” may also occur which result in the generation of many diverse ideas of tackling the same problem. this typically involves “breaking a topic down into its various component parts in order to gain insight about the various aspects of the topic” and implies that “not only one solution may be correct” (razumnikova, 2012, p.1028). there are strong links here with another concept associated with papert and collaborator sherry turkle, but originally due to the anthropologist levi-strauss. in their analysis of computer programming style, turkle and papert (1990) famously drew distinctions between “top down planners” and “bricoleurs” who were observed to hone in on a programming solution from amongst many alternative approaches through a playful, interactive approach: “bricoleurs have goals, but set out to realize them in the spirit of a collaborative venture with the machine… bricoleur programmers prefer negotiation and rearrangement of their materials” (p.136). the term bricoleur derives from the french verb bricoler, meaning “to tinker” and, more recently, proponents of the educational maker movement have adopted the english term to describe making activities which realise learners’ personal creative goals through various “scientific and technical tools, processes, and phenomena” (bevan et al., 2015, p.99). tinkering invokes the principles of “exploration, questioning, iterative designing and testing, and problem solving” (ibid., p.99–100) to achieve creative, aesthetically satisfying and self-expressive goals. https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 18 ben-ari (1998) explored constructivism and bricolage in computer science education arguing that, whilst bricolage can help in introductory courses, it can manifest in computer science as endless debugging and ultimately fail to develop the skills of hypothesis creation and testing based on models, needed for professional practice. our aim in the work reported here was in part to lead students towards design reasoning activities that could lead to a more model-based approach. baretè, formica et al. (2017) identified music as an effective way to engage secondary-school children in learning computational thinking. they focus on coding tasks by which music is manipulated and computational thinking thereby learned. this addresses motivation but not necessarily scale and complexity. baratè, ludivico and malchiodi (2017) and ludovico, malchiodi and zecca (2017) report work in which lego ® bricks are used as a physical representation of musical elements such as scores and score segments. other authors have successfully tapped into the attraction of music performance for computing education (ruthmann et al., 2010). these works lean more towards code-level representations (either directly or in musical equivalents); while acknowledging the necessity and value of this, the work we report here was focused more on largerscale software engineering issues and computational practices (see brennan and resnick, 2012). the reported success of lego ® use in computing education suggests the potential for making-related work more broadly. rode et al. extended previous work on computational thinking to computational making, identifying several relevant concepts: creating, constructing, and material understanding (rode et al., 2015). we consider and incorporate these concepts in our work. our approach shares some characteristics of bellettini et al.’s algomotricity (bellettini et al., 2014) in the sense of having three phases and combining tangible and abstract learning about a topic at a high school level. although exploratory activities are incorporated in our work, our approach is a little more instructordirected, focuses more on explicit reflection rather than scaffolded discovery, and addresses broader concepts. lego ® has also been used in professional software (and other) engineering training and education e.g. scrum (paasivaara et al., 2014; krivitsky, 2017), team communication (schulz & geithner, 2011) and development (bulmer, 2009), and requirements engineering and dependability (kurkovsky, 2015). these works do not incorporate musical aspects in their methods. https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 19 in summary, our work is related to but distinct from that described in this section. it draws on theories of constructionist and constructivist learning, is motivated by the desire to lead students towards design reasoning at a relatively early educational stage, and draws on evidence of success in using music and making as motivational and educational strategies in coding and computational thinking, and the use of lego® in software engineering education itself. it explores broader practices and concepts than coding, uses music performance built on lego® construction activities as both motivation and a tangible material exploration, and through this aims to stimulate the development of the design skills and reflection needed for large scale software development in future. 3. pilot study 3.1. design the concepts we were interested in conveying to students are interconnected through a key principle: ‘separation of concerns’ (soc). dijkstra (1982) describes soc as focusing attention on one isolated aspect (among many) of a subject whilst being aware that this is what one is doing: not ignoring other aspects but regarding them as irrelevant from the perspective of the aspect in focus. dijkstra summarises this as “…being oneand multiple-track minded simultaneously” (p. 1). from this fundamental principle flow the concepts of abstraction, modularization, and sub-component interfacing that become increasingly relevant (indeed necessary) as the scale of systems increases, and act as the foundations of more advanced techniques such as object-orientation. these concepts require multi-level thinking and design trade-offs to consider how to arrange desired functionality among the components of a system and balance various possible solutions. abstraction (and some related concepts) also strongly feature in discussions of computational thinking (e.g. wing (2008), and selby and woollard (2013)). in musical terms, they might translate, for example, to keeping the properties and design of a guitar body in mind while focusing on the neck (and vice versa), and not losing sight of the interface that will connect them. since our aims were exploratory, we designed activities to allow observation of students’ responses without overly constraining our stimuli. we assumed they would have some familiarity with lego ® and would be ready for music-related activities consistent with their regular curricular https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 20 experiences, although did not assume specific musical knowledge. we also did not assume any explicitly-evidenced skills or knowledge of things like abstraction, design, and trade-offs. students would encounter teamwork and organisation, affordance, instruction following, creative thinking, and reflective consideration of the materials and methods produced. our programme delivery was structured into three sessions; (1) material familiarization and creative free-building of instruments, using musical performance to motivate students and help them understand physical sound creation; (2) promoting team-work aspects through a more structured instruction-led build of an mbira (a generic name for sub-saharan african instruments of the lamellophone family, consisting of a resonating chamber, sound board and a series of plucked thin metal plates called lamellae (mcneil & mitran, 2008; montague, 2011)), and introducing standardization in design; (3) introducing standardization to sound excitation through mechanism, and the separation of production from actuation by substituting physical sound production for digital synthesis. we procured many lego ® technic parts (and some from previous lego ® serious play work (purves, 2019)) to create sets for the mbiras and other activities. sets were assembled in plastic sorting boxes before transportation to school. other items included tablecloths to damp classroom noise (jensen et al, 2018), sticky notes, rechargeable speakers, and instruments (our own and classroom resources) for scaffolding and performance. instructions were created as a photo sequence (see example step in figure 1). we provided a pre-built synthesizer platform of raspberry pis with brickpi kits (dexter industries, 2019) to allow lego ® ev3 sensors to trigger sonic pi (aaron & sonic pi core team, 2021). we extended brickpi and sonic pi demo code to scan buttons, trigger notes, and select voices. a ‘modifier button’ permitted pitch shifts (allowing an octave compass over a pentatonic scale: [c, d, e]/[g, a, c]). detailed consideration was given to ethics issues. multi-stage informed consent from all relevant parties was obtained (gatekeeper consent from school senior management first, then parent/guardian consent for each child, and staff, covering photography, video, and feedback). we https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 21 sought information from the school in advance to ensure inclusivity and access, and controlled for physical health and safety risks (the fieldwork took place prior to the covid-19 pandemic). the study was carried out in accordance with the british educational research association 2018 ethical guidelines for educational research. it was approved by the ucl institute of education research ethics committee (1229 / z6364106 2019 05 174 social research). figure 1. lego® mbira designed for this activity. 3.2. experiences for our three sessions, we worked with a class of 22 uk secondary-school children aged 11– 12 and their teacher. sessions took place during weekly music lessons of about an hour. we led each session with the support of the teacher. during the sessions we took notes, photos, and video, https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 22 and (with the teacher) helped students with questions and suggestions. students sat around tables in groups of two to five. 3.2.1. session 1 affordance and performance session 1 aimed to give students understanding of the material affordance of lego ® in the context of making and playing musical instruments (reflecting recent trends in english education policy that stress the importance of musical sound as the dominant language of school music lessons (atkinson, 2018)); performance became a motivation for making. we had approximate timings in mind but adapted “in the moment” to student and session progress. as students entered the room, we played live music (improvising around a simple salsa montuno pattern), the same kind of piece within which students were asked to join at the end. we introduced the project motivation, the series of sessions, and this session in particular. students were started on the first activity: freely building a shaker out of the available bricks. we then suggested alternative instruments to build (guiro, zither, drum). once all had created instruments, we verbally taught simplified versions of standard salsa rhyt hmic patterns (e.g. son clave and tumbao) to each instrument group through imitation, then joined in as at the start. finally, we asked students to reflect on their activity, noting anonymous thoughts on sticky notes. session 1 was very successful, with all students occupied, excited, and participating fully. there was a palpable sense of ‘flow’ throughout, evidenced through very little off-task pupil behaviour and the speed at which many groups completed activities. the simplest instruments were built quickly, needing our contingency plans for additional instruments. some groups worked more collaboratively (emphatically discussing their ideas with each other, sharing or copying ideas) and others “alongside” each other rather than in collaboration, illustrating important differences between “cooperative learning” and “parallel learning” within team-based activities (kindred, 2017). the exercise led to considerable divergent thinking. shakers were produced in a variety of shapes and sizes, with some joined to each other, others including handles (initially one girl added a handle, then others on the table adopted the idea), and one a long ‘baton’-style shaker. handle strength was initially poor but, in line with constructionist theory, the designs improved following discussions between pupils and with the researchers. some shakers were subsequently repurposed as https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 23 drums. one group focused more on aesthetics and ornamenting their shakers using doors and windows. such adornments may at first seem trivial but constructionists also stress the importance of complementing the technical or scientific effectiveness of learners’ creations with personallymeaningful decoration (resnick, berg & eisenberg, 2000). in plenary reflection, one boy observed that larger, flatter and thinner ‘white wall’ bricks worked well because they “bang against things”. students also noted that using smaller pieces made more noise and rattled more, and that the ‘gate’ pieces have holes in them that release the sound more. notwithstanding the earlier point regarding decoration, they also noted that making shakers look good may not, in itself, lead to instruments that feel or sound good. reflecting on the design of handles, they noted that thicker handles were needed to avoid them breaking. when building instruments in the second stage (guiro, large shaker, hand drum) students engaged spontaneously in comparison to a real drum, or lifting the drum to extract more sound, reengaging students whose attention was starting to wane. there is evidence here of some of the concepts we were aiming to draw students towards. they were clearly considering design trade-offs between aesthetics and functionality. they also showed aspects of modular thinking in terms of developing shakers and handles separately (although it is not clear from the evidence that modularization as a concept was an explicit consideration in the minds of students). moreover, in both their descriptions of effective features of shaker design and in their ability to derive and replicate key properties of other instruments, they were demonstrating an ability to abstract particularly salient features. some of the instruments resulting from this session are shown in figure 2. 3.2.2. session 2 consolidation, reflection and direction the second session was designed to remind students of their previous work, provoke further reflection on session 1 outcomes, and prepare them for session 3. we started by recapping our aims and the salsa patterns before playing together again. we had swapped the instruments so that students were unlikely to be playing one that they created themselves, to provoke deeper reflective discussion based on inspection and playing experience of others’ designs. it was intended as a means of https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 24 consolidating and extending prior learning through exposure to novel scenarios (mintzes & wandersee, 2005). figure 2. a selection of the free-build instruments students were then directed to the mbira components and instructions and the exercise explained to them. they then started building the mbira (shown in figure 1 with two lamellae at the https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 25 same length (pitch) and one shorter — they can be adjusted to any desired length that permits vibration and thus cover three separate pitches). we asked each group to make anonymous notes about their team organization and working practice to get them thinking about their work organisation. after a period, we paused the building to present some technical points on string excitation. this “lesson closure” (ganske, 2017) was intended to highlight links between their experiences and the issues of interaction, regularity, uniformity and length of lamellae in the human interface, thus preparing them for the exercises to come in session 3. excitement was again evident on entry, with one student exclaiming “lego!”. students generally disliked playing others’ instruments (predicted by the teacher, but, in hindsight, consistent with constructionism’s emphasis on personally-meaningful object making). students grabbed shakers and started making noise, having to be calmed down. one girl remembered the rhythm to the salsa almost perfectly and the others picked their parts up once reminded. swapping the instruments was successful in terms of reflective thinking. despite their annoyance, students identified salient properties in novel designs (“i like how big it is and how it makes a loud sound”, “this one didn’t have enough space for the pieces to shake around inside so it wasn’t very loud” ), comparisons (“this one was nice but it broke so i think this one was better”) and tradeoffs (“this one sounds good but it falls apart easily”). we used these observations to introduce the role of constraint in design. some intervention was needed in this session, partly for timely completion of builds and to correct a minor error in the ordering of the mbira instructions that had confused some students. persistence is a key conative quality for engineers (adams et al., 2015) and we could model it through our resolution of this issue. in hindsight, this session did not contain sufficient parallel building activity to keep students occupied throughout, although this offered opportunities for teamwork discussions and practice. engagement was variable, with some tables working in focused sub-groups alongside other students less engaged. there were also elements of competition (“i’m ahead of you, and i haven’t even looked https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 26 at the instructions”). our (and the teacher’s) engagement with students during this period tended to refocus them. students did not always follow the instructions precisely (we failed to anticipate this), perhaps because photo sequences require careful interrogation. thus critical parts were unavailable when subsequently needed and we had to supply spares (from a limited number) or help with rebuilding to free those needed. this offered opportunities to educate students on building techniques that they had missed in the instructions. some teams exhibited resilience to these issues through teamwork, with members correcting each other as they worked. students had fewer opportunities to evidence design skills in this session: the instruction-led nature meant that problem-solving was more at the level of substituting parts, e.g., we pointed out that if there was a shortage of 13-hole lift-arms, two 5-hole and one 3-hole lift-arms could be used together to replace them. students picked up on this with one later telling another: “that’s why you use two fives and a three!”. student notes indicated a range of organizational schemes including subdivision of labour for speed and improved concentration, turn-taking, explanation to peers, mutual help and support, and looking ahead to understand instructions better. as described above, they were clearly able to exhibit thinking about trade-offs in others’ designs, engage in reflection, and consider multiple aspects of what they were doing (e.g. aesthetics vs function, physics of instruments). 3.2.3. session 3 interfacing and replacing the final session aimed to engage students with mechanical activation of the lamellae and then substituted physical sound production for digital synthesis, with the physical same interface. we completed any unfinished student instruments beforehand to level the scaffold for all groups and created one example trigger mechanism per group. we also prepared the pi-related components (figure 3) since parts arrived too late to prepare instructions. we began by briefly introducing the pi-based system, reminded students of how things were left at the end of the previous session, and explained our finishing of their instruments. we asked them to investigate the effect of lamella length on pitch, then discussed the need for a standardised https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 27 interface (e.g. by considering the viability of an mbira with the compass of an 88-key piano). students then built ‘triggers’ from our instructions (three per instrument; see figure 4). many students completed their build and we followed-up by presenting material on standardised interfaces starting by explaining piano-key mechanisms, then moving to robotic instruments to introduce the idea of separating control and sound production. students then moved to the pi systems and were supported to integrate these with their instruments and experiment (see figure 5). figure 3. pi-related components (encased pi and brickpi, frame with four sensors, portable speaker). https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 28 figure 4. lego® mbira with moveable lamellae/triggers. figure 5. pi system installed on mbira. https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 29 a final feedback sheet featuring both open and closed questions was completed by the students. students were excited at seeing the pi partial set-ups when they arrived. they found their own work from the previous week and some began to play with it while others talked. students again responded well to a reflective question about how to make the best sound: “by taking it off the bricks [base board] and holding it down”; “if you pluck them further out they are louder”. students were initially very focused when building triggers (“who wants to build the trigger?”, “have we got more of these bits?”, “i want to try to play this how it is”, “separate them into bits”, “can i borrow that yellow bit, please") although this ebbed and flowed a little after five minutes. as previously, missing pieces caused challenges owing to the diverse ways in which students had built their mbiras. specific parts were more critical to the trigger mechanisms than in the main box walls. connecting-pin colours also raised an unexpected issue. we told students that colour of bricks was not important (to free them from concerns about matching particular brick colours), however, lego ® pins vary in their frictional force and this is indicated by colour (black=high friction, light grey=low friction). it was critical to use high-friction pins for the trigger frame and low friction-pins for the rotating parts to ensure correct mechanical operation. in discussion of mechanisms, students were able to identify various things in their classroom that operated when instructed rather than by direct manipulation (computer, electronic keyboards and electric piano). there was much visible excitement when students moved to the pi systems, particularly when introduced to the ability to change the sounds (thereby evidencing an appreciation of the concept we were aiming to convey: that by abstracting sound production away from the physical properties of the object, the same interface could be used more flexibly). this session exposed students to concepts of modularization and abstraction (through the separation of actuators and sound production. students clearly understood that an external interface might be separate from the instructions that it gives to an attached operational system by their ability to identify such systems when asked. whether this translated to understanding in the context here was less explicitly evident. https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 30 4. feedback 4.1. students 70% of students reported in their session 3 feedback sheets that they preferred free-build to instruction-led activities, clearly enjoying the creative opportunities for imaginative exploration these presented. a third of answers explaining pupils’ preferred activities referenced ‘fun’ or ‘enjoyment’. a third of the reasons given referred to the independent and imaginative decisionmaking opportunities that were involved (e.g. “…really fun…helped us to use our imagination”, “…liked exploring which shakers are better…choosing the design”). only 25% of pupils indicated that they preferred instruction-based activities; of those that did, this appeared partially rooted in a belief that following the adult guidance would result in betterquality instruments (“…there was a reason for them…”; “following instructions means that you can make something cool that works…”). a further two comments suggested that following instructions was “fun” and something they liked to do. one noted elsewhere that they had “learnt how to follow instructions better and to create something that someone else has designed”. given the engagement observed, it was unsurprising that all but two students explicitly mentioned enjoyment and fun in session 1 sticky-note feedback, with the combination of lego ® and music clearly very popular. in line with papert’s concept of ‘hard fun’, a small number of notes mentioned that the session had “tested their creativity” (two different notes) and that their constructions “improved every time”. perhaps indicating a yearning to continue experimenting, a further note read, “if only there was a way to make the drums/other things sound deeper and more real”. there was also a sense of discovery: “i didn’t actually know you could make music out of lego.", “…no boring stuff.” some students focused in their session 3 feedback on the importance of methodical, detailed, persistent and collaborative approaches in design work (“…keep trying and making it stronger…”,”…take time and think…”, “…sometimes going slower makes them better…”) and two mentioned creativity and flexibility (“you can be as creative as you want”, “anything is possible with any materials even lego”, “…make music out of many different things…”). this theme also https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 31 emerged in responses to the question ‘have the sessions changed the way you think about musical instruments?’.: “…i think more about how they produce noises.” pupils were asked to consider how they might approach building another instrument if given the chance. some wished to make ‘bigger’ or ‘different’ instruments, two would have employed more colourful designs. other answers suggested that their approach to building (as opposed to the instruments themselves) would be different e.g. making sure they “have all the parts”, or “thought a bit more about the design” or “think about the sound”. two were in contrast: “follow the instructions really quickly” vs “be slower so i don’t mess up”. the ‘electronic mbira’ was rated as most exciting to play (~59%) vs the trigger version (~12%), original (~6%), or no difference (~24%). 4.2. teacher following examples of participatory research (cancian & armstead, 2001), the class teacher was central to the planning and delivery of the sessions and her feedback is integral to our understanding of what was achieved. she expected that the approach would likely succeed, with lego ® use combined with the idea of making instruments and physically creating sound being important motivations for children. the teacher anticipated that the children would learn about collaboration and working together, and as a practitioner was interested to see how this collaboration (regarded as natural in the arts) would appear in an engineering task. the activities fitted well into the curriculum and overall planning. the teacher felt that session 1 was pitched just right and achieved its aim, introducing the concepts, engaging the children, and involving considerable evaluative activity. the children learned what made effective shakers and to describe their thinking from a design viewpoint but apply it from a musical viewpoint. they were more invested in playing the music because they had created the instruments: ownership had an important role. session 2 was paced correctly but the teacher suggested improvements: split the instructions so that sub-groups produce and reflect upon sub-assemblies, and predefine team roles (e.g. searchers and builders; one pair developed a process of preparing and building) and then swap them to get https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 32 students thinking about different roles, help make design concepts more overt, and support easier reflection on team structure. the teacher also observed that, overall, the boys in the class had tended to race ahead in the instructions (perhaps causing inaccuracy in builds), whilst the girls had tended to work steadily. the final session was felt to be paced as well as possible given the time available, however, the teacher’s view was that a 4–5 week plan would have been better to fit in all the content. showing the pi at the start might have led to greater motivation and more timely completion of the builds. although students did not see the separation of concepts/concerns during building, the teacher perceived that once they were using the pis, they had recognised the concept. in terms of overall teacher feedback, the extent of evident collaboration was more than expected (she was most interested to note that there were high levels of engagement across the classroom in different students). design aspects were conveyed as well as possible given children’s desire to get going with the activities; to cover them more explicitly would need more time. learning through doing was very important and the modelling used was helpful: a scaffolded process might work well in future (with more time). although present, more explicit links to the english national curriculum in terms of learning outcomes would help engage staff. it would also be useful to provide a less specific set of builds that rely on particular pieces, but instead give examples (e.g. pictures) and let teachers drive the activity. training may be required for the computing aspects. the sequencing of activities was correct, worked effectively and had a sense of progression. the teacher felt that we had done well to get everything into three weeks, and it could even be run as a six-week exercise with additional facilitated reflection at the end, including some more explicit programming aspects to show how digital instruments could be created. despite this, the teacher felt that we had been appropriately ambitious and that this drew the children along. 5. conclusions and future work our aim was to assess whether music and making would be effective at conveying software engineering concepts in secondary schools. our evidence suggests that students engaged practically https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 33 with all of the concepts in which we were interested. they developed subsystems (e.g. handles and shakers), reflected on the interfaces between them and the quality of function of the components and their integrated whole. in plenary discussion, they evidenced awareness of the separation and abstraction of control and sound production. alongside these primary goals, they engaged and reflected on teamworking, played music, engaged in spontaneous experimentation, and importantly, had fun while doing so. overall, this pilot has shown that our approach has significant promise. it worked motivationally and engaged students practically with many of the target concepts. we learned a number of lessons that will be helpful in developing future similar interventions:  explicit presentation of key concepts is important for students to understand and apply them transferably e.g. present modularity in this context and refer back to it later from another “you’ve seen modularity in design before when you built shakers and handles in lego ® ”. this is congruent with ben-ari’s (1998) position that the ‘model’ (at an appropriate level) needs to be explicitly taught. our approach is (appropriately) bricolage-oriented in sensitizing students to the desired concepts, but this does not deny the need for later more-explicit teaching to support subsequent formal study.  the concept of abstraction needs appropriate time to be introduced fully.  following instructions likely slows down progress compared to free-building (but, as gunn (2002) argues, is a vital engineering skill).  scaffolding (e.g. staff completing partial builds) needs to be balanced with potential loss of ownership by students (teacher observation).  directed or instruction-based builds may benefit from a more explicit division of roles to improve engagement and permit parallel activity.  having all materials on-hand and a significant reservoir of spare parts is critical for success. this pilot project was conducted with school pupils aged 11-12 years. however, there are grounds to believe in the potential of similar kinds of activities to support older learners, including those in higher education. as noted, the presented activities drew heavily on constructivist learning theory. in particular, they were characterised by experiential learning, problem solving and – to a https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 34 lesser extent – ownership of the learning process by the pupils themselves. as knowles et al. (2020) note, all such aspects are consistent with core principles of andragogy – the theory and practice of teaching adult learners. moreover, research evidence in a range of disciplines suggests that practical engagement with lego ® is no less motivating for older learners (nerantzi and james, 2018). similarly, the inclusion of music-based practical activities in higher education programmes of computer science has been shown to have potential as a means of engaging learners effectively in creative, culturally-embedded and social programming tasks (bhattacharya et al, 2019). one interesting area of future work will thus be to explore this potential application to adult learners. future work with schools will be planned over a longer timescale (e.g., four to five sessions), with stronger emphasis on things like message passing and abstraction (e.g. students physically passing messages around to actuate instruments) to strengthen understanding. longer timescales will also permit the use of formalized assessment frameworks e.g., the computational thinking assessment tools of tsai et al. (2021). in addition, the breadth of topics tackled can then be increased, e.g., aspects of coding can be introduced to provide additional creative opportunities for shaping sounds in combination with physical controllers, allowing brennan and resnick’s (2012) computational concepts to incorporated in addition. other curriculum areas such as art, design, and science could also be explored. in summary, this has been a very successful pilot study, providing direct benefit to the students concerned, evidencing the potential of the approach, offering lessons to be learned for the future, and giving a strong foundation for development. acknowledgements: we are very grateful to andy purves for the idea of an mbira-style design for the primary instruments used here. we are also very grateful to the staff and students who kindly participated. lego® is a trademark of the lego group of companies which does not sponsor, authorize or endorse this work. author contributions: conceptualization, n.g., e.h., and r.p.; methodology, n.g., e.h., and r.p.; software, n.g.; formal analysis, r.p.; investigation, r.p., e.h. and n.g.; resources, n.g., e.h., and r.p.; writing original draft preparation, n.g. and r.p..; writing review and editing, n.g., r.p., and e.h.; visualization, n.g., e.h., and r.p.; project administration, n.g.; funding acquisition, n.g. and e.h. all authors have read and agreed to the published version of the manuscript. funding: this project received no external funding but was funded by the ucl department of computer science strategic research fund. https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 35 conflicts of interest: the authors declare no conflict of interest. references aaron, s. & sonic pi core team (2021). sonic pi. https://sonic-pi.net/ adams, e.a., dancz, c.l.a., landis, a.e. (2015, june 14-17). improving engineering student preparedness, persistence, and diversity through conative understanding [paper presentation – paper 12320]. 122nd american society for engineering education annual conference and exposition, seattle, wa. https://strategy.asee.org/improving-engineering-student-preparedness-persistence-and-diversitythrough-conative-understanding.pdf alanazi, h.m.n. (2020). the effects of active recreational math games on math anxiety and performance in primary school children: an experimental study. multidisciplinary journal for education, social and technological sciences 7(1), 89-112. https://doi.org/10.4995/muse.2020.12622 ali, m. r. (2006). imparting effective software engineering education. acm sigsoft software engineering notes, 31(4), 1–3. https://doi.org/10/cpxtz5 atkinson, r. (2018). the pedagogy of primary music teaching: talking about not talking. music education research, 20(3), 267-276. https://doi.org/10.1080/14613808.2017.1327946 baratè, a., formica, a., ludovico, l.a., & malchiodi. d. (2017, april 21-23). fostering computational thinking in secondary school through music an educational experience based on google blockly. proceedings of the 9th international conference on computer supported education – volume 1:csedu (pp. 117–124). porto, portugal. https://doi.org/10.5220/0006313001170124 baratè, a., ludovico, l.a., & malchiodi, d. (2017). fostering computational thinking in primary school through a lego®-based music notation. procedia computer science, 112, 1334–1344. https://doi.org/10.1016/j.procs.2017.08.018 barendsen e., & steenvoorden t. (2016). analyzing conceptual content of international informatics curricula for secondary education. in a. brodnik, & f. tort (eds.), informatics in schools: improvement of informatics knowledge and perception. issep 2016. lecture notes in computer science: vol. 9973. (pp. 14-27). springer. https://doi.org/10.1007/978-3-319-46747-4_2 bhattacharya, s., okunbor, d., sarami, c., gillespie, p. and nickolov, r. (2019). strengthening computer and mathematical sciences engagement and learning", in k.m. mack, k. winter, and m. soto. (eds.) culturally responsive strategies for reforming stem higher education, emerald publishing limited, bingley, pp. 249-258. bellettini, c., lonati, v., malchiodi, d., monga, m., morpurgo, a., torelli, m., & zecca, l. (2014). extracurricular activities for improving the perception of informatics in secondary schools. in y. gülbahar & e. karataş (eds.), informatics in schools. teaching and learning perspectives (vol. 8730, pp. 161–172). springer international publishing. https://doi.org/10.1007/978-3-319-09958-3_15 ben-ari, m. (1998). constructivism in computer science education. sigcse bulletin, 30(1), 257-261. https://doi.org/10.1145/274790.274308 bevan, b., gutwill, j. p., petrich, m., & wilkinson, k. (2015). learning through stem-rich tinkering: findings from a jointly negotiated research project taken up in practice. science education 99(1), 98–120. https://doi.org/10.1002/sce.21151 https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://sonic-pi.net/ https://strategy.asee.org/improving-engineering-student-preparedness-persistence-and-diversity-through-conative-understanding.pdf https://strategy.asee.org/improving-engineering-student-preparedness-persistence-and-diversity-through-conative-understanding.pdf https://doi.org/10.4995/muse.2020.12622 https://doi.org/10/cpxtz5 https://doi.org/10.1080/14613808.2017.1327946 https://doi.org/10.5220/0006313001170124 https://doi.org/10.1016/j.procs.2017.08.018 https://doi.org/10.1007/978-3-319-46747-4_2 https://doi.org/10.1007/978-3-319-09958-3_15 https://doi.org/10.1145/274790.274308 https://doi.org/10.1002/sce.21151 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 36 brennan, k., & resnick, m. (2012). new frameworks for studying and assessing the development of computational thinking. annual american educational research association meeting. https://damprod.media.mit.edu/x/files/%7ekbrennan/files/brennan_resnick_aera2012_ct.pdf bulmer, l. (2009). the use of lego® serious play in the engineering design classroom. proceedings of the canadian engineering education association (ceea). https://doi.org/10.24908/pceea.v0i0.3699 cancian, f.m., & armstead, c. (2001). participatory research. in e.f. borgatta, & r.j.v. montgomery (eds.), encyclopedia of sociology (2nd ed., vol. 3, pp. 2038-2044). macmillan reference usa. dexter industries (2019). brickpi. https://www.dexterindustries.com/brickpi/ dijkstra, e.w. (1982). ewd 447: on the role of scientific thought. in e.w. dijkstra, selected writings on computing: a personal perspective (pp. 60-66). springer-verlag. https://www.cs.utexas.edu/users/ewd/ewd04xx/ewd447.pdf https://doi.org/10.1007/978-1-4612-5695-3 doleck, t., bazelais, p., lemay, d. j., saxena, a., & basnet, r. b. (2017). algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: exploring the relationship between computational thinking skills and academic performance. journal of computers in education, 4(4), 355–369. https://doi.org/10/ggdpq2 franco, c., & gillanders, c. (2014). exploring innovative and creative ways of teaching. multidisciplinary journal for education, social and technological sciences 1(2), 53-69. https://doi.org/10.4995/muse.2014.2228 ganske, k. (2017). lesson closure: an important piece of the student learning puzzle. the reading teacher, 71(1), 95-100. https://doi.org/10.1002/trtr.1587 gogus, a (2012). outcomes of learning. in n. seel (ed.), encyclopedia of the sciences of learning, (pp. 2534– 2539). springer. https://doi.org/10.1007/978-1-4419-1428-6_145 gunn, c. (2002, june 16-19). following instructions [paper presentation]. american society for engineering education 2002 annual conference, montreal, canada. https://doi.org/10.18260/1-2--10912 hislop, g. w. (2008). chapter 1 software engineering education: past, present, and future. in h.j.c. ellis, s.a. demurjian, & j.f. naveda (eds.), software engineering: effective teaching and learning approaches and practices. igi global. https://doi.org/10.4018/978-1-60566-102-5 hm government (2013). national curriculum in england: computing programmes of study. https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmesof-study/national-curriculum-in-england-computing-programmes-of-study james, a.r. (2013). lego serious play: a three-dimensional approach to learning development. journal of learning development in higher education, (6). https://doi.org/10.47408/jldhe.v0i6.208 jensen, c. n. (2017). serious play approaches for creating, sharing, and mobilizing tacit knowledge in crossdisciplinary settings [unpublished doctoral dissertation]. arizona state university. http://hdl.handle.net/2286/r.a.186638 jensen, c. n., seager, t. p., & cook-davis, a. (2018), lego® serious play® in multidisciplinary student teams. international journal of management and applied research, 5(4), 264-280. https://doi.org/10.18646/2056.54.18-020 kindred, j. (2008). making cooperative learning visible without the group grade. the scholarship of teaching and learning at emu, 2(3). https://commons.emich.edu/sotl/vol2/iss1/3 https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://dam-prod.media.mit.edu/x/files/~kbrennan/files/brennan_resnick_aera2012_ct.pdf https://dam-prod.media.mit.edu/x/files/~kbrennan/files/brennan_resnick_aera2012_ct.pdf https://doi.org/10.24908/pceea.v0i0.3699 https://www.dexterindustries.com/brickpi/ https://www.cs.utexas.edu/users/ewd/ewd04xx/ewd447.pdf https://doi.org/10.1007/978-1-4612-5695-3 https://doi.org/10/ggdpq2 https://doi.org/10.4995/muse.2014.2228 https://doi.org/10.1002/trtr.1587 https://doi.org/10.1007/978-1-4419-1428-6_145 https://doi.org/10.18260/1-2--10912 https://doi.org/10.4018/978-1-60566-102-5 https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmes-of-study/national-curriculum-in-england-computing-programmes-of-study https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmes-of-study/national-curriculum-in-england-computing-programmes-of-study https://doi.org/10.47408/jldhe.v0i6.208 http://hdl.handle.net/2286/r.a.186638 https://doi.org/10.18646/2056.54.18-020 https://commons.emich.edu/sotl/vol2/iss1/3 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 37 knowles, m.s., holton, e.f., swanson, r.a, robinson, p.a. (2020). the adult learner : the definitive classic in adult education and human resource development. abingdon, oxon, routledge. krivitsky, a. (2021). lego4scrum: scrum simulation with lego. https://www.lego4scrum.com/ kurkovsky, s. (2015, july 4-8). teaching software engineering with lego serious play. proceedings of the 2015 acm conference on innovation and technology in computer science education (pp. 213–218). vilnius, lithuania. acm. https://doi.org/10.1145/2729094.2742604 ludovico, l.a., malchiodi, d., & zecca, l. (2017, november 13). a multimodal lego®-based learning activity mixing musical notation and computer programming. proceedings of the 1st acm sigchi international workshop on multimodal interaction for education (pp. 44–48). glasgow, uk. acm. https://doi.org/10.1145/3139513.3139519 ludwig, j. (2008). chapter 14 software engineering at full scale: a unique curriculum. in h.j.c. ellis, s.a. demurjian, & j.f. naveda (eds.), software engineering: effective teaching and learning approaches and practices. igi global. https://doi.org/10.4018/978-1-60566-102-5 mcneil, l. e., & mitran, s. (2008). vibrational frequencies and tuning of the african mbira. the journal of the acoustical society of america 123(2), 1169-1178. https://doi.org/10.1121/1.2828063 mintzes, j. j., & wandersee, j. h. (2005). reform and innovation in science teaching: a human constructivist view. in j.j. mintzes, j.h. wandersee & j.d. novak (eds.), teaching science for understanding (pp. 2958). academic press. https://doi.org/10.1016/b978-012498360-1/50003-9 montagu, j. (2011). mbira, kalimba, likembe. in a. latham (ed.), the oxford companion to music. oup. https://www.oxfordreference.com/view/10.1093/acref/9780199579037.001.0001/acref9780199579037-e-4306 monteiro, i. t., salgado, l. c. de castro, mota, m. p., sampaio, a. l., & de souza, c. s. (2017). signifying software engineering to computational thinking learners with agentsheets and polifacets. journal of visual languages & computing, 40, 91–112. https://doi.org/10/gbqt8h nerantzi, c. & james, a. (eds.) (2018). discovering innovative applications of lego® in learning and teaching in higher education [special issue]. international journal of management and applied research, 5(4). https://www.ijmar.org/v5n4/toc.html paasivaara, m., heikkilä, v., casper, l. & toivola, t. (2014, may 31 – june 7). teaching students scrum using lego blocks. companion proceedings of the 36th international conference on software engineering. (pp. 382–391). hyderabad, india. acm. https://doi.org/10.1145/2591062.2591169 papert, s. (2002). hard fun. http://www.papert.org/articles/hardfun.html (originally published in the bangor daily news (bangor, me)). papert, s., & harel., i. (1991). situating constructionism. in s. papert, & i. harel, constructionism (pp. 1-11). ablex publishing. purves, r. (2019, december 3). using lego to teach academic writing skills. https://www.ucl.ac.uk/teachinglearning/case-studies/2019/dec/using-lego-teach-academic-writing-skills razumnikova, o. m. (2012). divergent thinking and learning. in n. seel (ed.), encyclopedia of the sciences of learning, (pp. 1028–1031). springer. https://doi.org/10.1007/978-1-4419-1428-6_580 resnick, m., berg, r., & eisenberg, m. (2000). beyond black boxes: bringing transparency and aesthetics back to scientific investigation. the journal of the learning sciences, 9(1), 7-30. https://doi.org/10.1207/s15327809jls0901_3 https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.lego4scrum.com/ https://doi.org/10.1145/2729094.2742604 https://doi.org/10.1145/3139513.3139519 https://doi.org/10.4018/978-1-60566-102-5 https://doi.org/10.1121/1.2828063 https://doi.org/10.1016/b978-012498360-1/50003-9 https://www.oxfordreference.com/view/10.1093/acref/9780199579037.001.0001/acref-9780199579037-e-4306 https://www.oxfordreference.com/view/10.1093/acref/9780199579037.001.0001/acref-9780199579037-e-4306 https://doi.org/10/gbqt8h https://www.ijmar.org/v5n4/toc.html https://doi.org/10.1145/2591062.2591169 http://www.papert.org/articles/hardfun.html https://www.ucl.ac.uk/teaching-learning/case-studies/2019/dec/using-lego-teach-academic-writing-skills https://www.ucl.ac.uk/teaching-learning/case-studies/2019/dec/using-lego-teach-academic-writing-skills https://doi.org/10.1007/978-1-4419-1428-6_580 https://doi.org/10.1207/s15327809jls0901_3 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16453 social and technological sciences e-issn: 2341-2593 gold et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 14-38. https://doi.org/10.4995/muse.2022.16453 38 rich, p.j., browning, s.f., perkins, m., shoop, t., yoshikawa, e., belikov, o.m. (2019). coding in k-8: international trends in teaching elementary/primary computing. techtrends, 63, 311–329. https://doi.org/10.1007/s11528-018-0295-4 rode, j.a., weibert, a., marshall, a., aal, k. von rekowski, t., el mimouni, h., & booker, j. (2015, september 7-11). from computational thinking to computational making. proceedings of the 2015 acm international joint conference on pervasive and ubiquitous computing (pp. 239–250). osaka, japan. acm. https://doi.org/10.1145/2750858.2804261 ruthmann, a., heines, j.m., greher, g.r., laidler, p., & saulters. c. (2010, march 10-13). teaching computational thinking through musical live coding in scratch. proceedings of the 41st acm technical symposium on computer science education (pp. 351–355). milwaukee wisconsin, usa. acm. https://doi.org/10.1145/1734263.1734384 schmidt, j. a. (2010). flow in education. in p. peterson, e. baker, & b., mcgaw (eds.) international encyclopedia of education (pp605–611). elsevier. https://doi.org/10.1016/b978-0-08-044894-7.006084 schulz, k.-p., & geithner, s. (2011, april 12-14). the development of shared understandings and innovation through metaphorical methods such as lego serious playtm [paper presentation]. international conference on organizational learning, knowledge and capabilities (olkc), hull, uk. https://warwick.ac.uk/fac/soc/wbs/conf/olkc/archive/olkc6/papers/id_127.pdf selby, c., & woollard, j. (2013). computational thinking: the developing definition. university of southampton. https://eprints.soton.ac.uk/356481/ tsai, m.-j., liang, j.-c., & hsu, c.-y. (2021). the computational thinking scale for computer literacy education. journal of educational computing research, 59(4), 579–602. https://doi.org/10/gjhqnt turkle, s. & papert, s. (1990). epistemological pluralism: styles and voices within the computer culture. signs 16(1), 128-157. https://doi.org/10.1086/494648 the royal society (2012). shut down or restart? the way forward for computing in uk schools. https://royalsociety.org/topics-policy/projects/computing-in-schools/report/ wing, j. m. (2006). computational thinking. communications of the acm 49 (3), 33-35. https://doi.org/10.1145/1118178.1118215 wing, j. m. (2008). computational thinking and thinking about computing. philosophical transactions of the royal society a: mathematical, physical and engineering sciences, 366(1881), 3717–3725. https://doi.org/10/c24v3c https://doi.org/10.4995/muse.2022.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1007/s11528-018-0295-4 https://doi.org/10.1145/2750858.2804261 https://doi.org/10.1145/1734263.1734384 https://doi.org/10.1016/b978-0-08-044894-7.00608-4 https://doi.org/10.1016/b978-0-08-044894-7.00608-4 https://warwick.ac.uk/fac/soc/wbs/conf/olkc/archive/olkc6/papers/id_127.pdf https://eprints.soton.ac.uk/356481/ https://doi.org/10/gjhqnt https://doi.org/10.1086/494648 https://royalsociety.org/topics-policy/projects/computing-in-schools/report/ https://doi.org/10.1145/1118178.1118215 https://doi.org/10/c24v3c multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 46 internal assessment of a physical education teacher education program in the philippines using the engagement theory of program quality (etpq) stephanie g. dizona , julius ceazar g. tolentinob department of physical education, college of education, don honorio ventura state university, bacolor (2001), pampanga, philippines * correspondence: 2018002468@dhvsu.edu.ph; jcgtolentino@dhvsu.edu.ph received: 06 january 2022; accepted: 31 may 2022; published: october 2022 abstract the presence of various quality assurance measures is evidenced in the mainstream of higher education across the world and has been a well-accepted normative standard of excellence even in the philippines. however, there is a need to examine programs that strongly manifest students’ engagement in the educational system. hence, this study was sought to facilitate the conduct of an internal quality assessment of the bachelor of physical education program from a teacher education institution in a state university in pampanga, the philippines grounded on the engagement theory of program quality (etpq) (haworth & conrad, 1997). a mixed-methods sequential explanatory design (follow-up variant explanatory model) was used to examine the importance, concreteness, and effectiveness of the 16 attributes of the etpq model by the students and faculty members of the program. results of the quantitative strand revealed that the etpq attributes are generally perceived to be extremely important, highly evident, and highly to extremely effective, and yielded no significant differences between the two groups. moreover, linear regression (forward selection method) revealed ten (10) attributes of the etpq in terms of concreteness to be significant predictors of effectiveness. additionally, the qualitative strand further explored these attributes to reveal the best practices in the program. congruence between the respondents’ assessment and their narrative experiences was also confirmed. the inputs derived may pave the way for the institutionalization of quality assurance mechanisms that ventures the strong engagement of the students complementary with other stakeholders. keywords: bachelor of physical education; engagement theory of program quality; quality assurance; sequential-explanatory design; teacher education to cite this article: dizon, s. g., & tolentino, j. c. g. (2022). internal assessment of a physical education teacher education program in the philippines using the engagement theory of program quality (etpq). multidisciplinary journal for education, social and technological sciences, 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ mailto:jcgtolentino@dhvsu.edu.ph https://orcid.org/0000-0002-5777-2904 https://orcid.org/0000-0002-3728-3089 https://doi.org/10.4995/muse.2022.16959 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 47 1. introduction the world is fast-changing and education systems need to adapt from time to time. education is on its fourth industrial revolution better known as education 4.0, which focused on the advancement of technology that influenced the approach and skills in teaching and learning. consequently, 21st-century skills shall be aligned with these evolving times to prepare and produce competent and quality graduates for what is required in the future. on this account, improving the quality of teacher education programs has been a universal goal in every educational institution. the status of a program can be influenced by the changing curriculum. ensuring the quality of physical education teacher education (pete) has been challenging due to an emerging paradigm of education in the philippines. the newly implemented bachelor of physical education (bped), reshaped through the policies, standards, and guidelines (psg) of the program (commission on higher education [ched] memorandum order no. 80, series of 2017), has been designed to achieve quality physical education (qpe) in the philippines through producing highly competent and skilled physical educators. the psg was a response to the salient features of the “enhanced basic education act of 2013” (republic act no. 10533) and the 21st century philippine education framework. the pete programs in the philippines were formerly named the bachelor of physical education major in school p.e. (bpe-spe) and the bachelor of physical education major in sports and wellness management (bpe-swm) as written in article iii, section 3 of ched memorandum order no. 23, s. 2011. by the year 2017, ched officially released memorandum no. 80 stating in article iv, section 5 that “the program shall be called bachelor of physical education (bped)” (p. 4). accordingly, these metamorphoses altered the psgs which affected the sustainability of quality program goals. quality physical education (qpe) was on its critical contingency because although there were unfaltering developments, implementation of the policies in physical education was still erratic (mclennan & thompson, 2015). each program in philippine education, specifically pete, profoundly developed and sustained its quality pursuit to the mandate of article xiv, section 1 of the 1987 philippine constitution which states that “the state shall protect and promote the right of all citizens to quality education at all level…” (p. 49). similarly, the enactment of republic act no. 7722, otherwise known as the “higher education act of 1994,” declared a policy statement in section 2 stating, “the state https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 48 shall protect, foster, and promote the right of all citizens to affordable quality education at all levels...” (official gazette, n.d.) (p. 3). it further proclaimed ched to “monitor and evaluate the performance of programs and institutions of higher learning…” (section 8) (p. 7). in such cases, pete programs were strained and appeared to be impaired in yielding the rules and regulations implemented under cmo no. 15, s. 2005, entitled “institutional monitoring and evaluation for quality assurance of all higher education institutions in the philippines” (ched, 2005) because of some issues and concerns such as low quality and accessibility to education, unresponsive curriculum, and improper monitoring of implemented programs (durban & catalan, 2012). 1.1. engagement theory of program quality (etpq) the engagement theory of program quality (etpq) (haworth & conrad, 1997) was designed to evaluate and improve the quality of the programs in higher education at all degree levels. haworth and conrad (1997) aimed to classify the factors that contribute to stimulating excellence and result in students’ positive learning experiences. they interviewed 781 participants and focused on the involvement of students, faculty, and administrators in developing high-quality programs that are centered on students’ learning and development. as a result, they identified 5 clusters and 17 program attributes. the clusters (1-5) and attributes (a-q) of the proposed theory are the following: (1) diverse and engaged participants --(a) faculty, (b) students, and (c) leaders; (2) participatory cultures --(d) shared program direction, (e) community of learners, and (f) risk-taking environment; (3) interactive teaching and learning --(g) integrative learning, (h) cooperative peer learning, (i) out of class activities, (j) mentoring, and (k) critical dialogue; (4) connected program requirements --(l) tangible product, (m) planned breadth and depth, (n) professional residency; and (5) adequate resources (support) --(o) faculty, (p) students, and (q) infrastructure. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 49 figure 1. theoretical framework of the study (adopted from haworth & conrad, 1997) the etpq was used in several studies and various contexts such as educational administration (mustan, 1998), doctorate programs in educational management (caro & prado, 2014), sports and wellness programs (dimarucot, 2019; dimarucot & rosales, 2020), medical field (heat et al., 2018), master’s programs in education among christian colleges and universities (kornelis, 2004), physical education program (panganiban, 2019; orlanda, 2015), interdisciplinary studies (simmons, 2011), and guidance and counseling (warden & benshoff, 2012). mustan (1998) examined the etpq proposed in 1997 to ensure its validity and confirmed that the 17 attributes of the theory are indicators of a high-quality program. likewise, the responses of the faculty and students indicated that the suggested attributes substantially overlaid with the theory. correspondingly, the theory was examined by warden and benshoff (2012) in master’s-level graduate programs, particularly those accredited by the council for accreditation of counseling and related educational programs (cacrep). it was discovered that the etpq has the potential to assess the quality of cacrep-accredited programs in developing and maintaining their status. the validity of the etpq increased as it was employed in the context of master of education programs in the council of christian colleges and universities (kornelis, 2004). https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 50 kornelis (2004) assessed the perceptions of faculty members and students on the importance, existence, and satisfaction of the etpq attributes as quality indicators of the council of christian colleges and universities (cccu) master of education programs. faculty members and students believed that the attributes were important measures of the program. they also indicated that there was a significant difference between connected program requirements and adequate resources (clusters of etpq) based on the perceived presence of the faculty members and students. it was concluded that students’ satisfaction with the program would emerge if they perceived that connected program requirements existed. nevertheless, the quality status of master of education programs within cccu was high. the etpq was also utilized as a framework in developing a tool for the assessment of physical therapist educational program quality to define its excellence and it was concluded that there is a need for continuous engagement among the stakeholders of the program (heat et al., 2018). caro and prado (2014) formulated a causal model on the quality of doctoral programs majoring in educational management in region 10, the philippines as they examined the factors influencing its quality. all seven (7) attributes were recognized as quality indicators of the programs. these included diverse and engaged faculty, diverse and engaged students, participatory cultures, interactive teaching and learning, curriculum, adequate resources, and institutional support. meanwhile, institutional support, diverse and engaged students, interactive teaching and learning, diverse and engaged faculty, and adequate resources were imposing as significant quality indicators of the programs. it was also assessed that the quality status of doctorate programs in educational management was high. curriculum, participatory culture, interactive teaching and learning, adequate resources, and institutional support were worthy as a model for the quality of doctorate programs majoring in educational management in region 10, the philippines. the quality of the bachelor of physical education major in sports and wellness management (bpe-swm) program was assessed using the etpq from two different philippine contexts: an internal assessment at san beda university (sbu) (dimarucot, 2019) and an external assessment at the national capital region (ncr) (dimarucot & rosales, 2020). it was unveiled that the program from two different settings had almost the same assessment. dimarucot (2019) and dimarucot and rosales (2020) determined that all the attributes are important indicators of the quality of the bpeswm in sbu and ncr, respectively. however, some attributes need improvements like diverse and https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 51 engaged faculty, diverse and engaged students, risk-mentoring, and support for basic infrastructure. further, attributes such as increased student involvement, retooling of faculty on mentoring, proper orientation on professional residency and tangible products, and sufficient funding for sports facilities will be the assurance to the stakeholders in the quality of bpe-swm. on the other hand, bpe-swm in ncr also needs to reassure attributes like shared program direction, risktaking environment, integrated learning, and cooperative peer learning (dimarucot & rosales, 2020). thesis, as a tangible product, can also help in ensuring the quality of the program in sbu (dimarucot, 2019). the quality of the physical education program was assessed by the administrators, faculty and students in the philippine setting (orlanda, 2015; panganiban, 2019). implementing the service pe program is notable at batangas state university (bsu) in terms of the acquisition of objectives, curriculum, faculty teaching effectiveness, adequate facilities, and resources (orlanda, 2015). the leading indicators of the program were its objectives while resources like facilities, equipment, and supplies were placed as lagging. alternatively, panganiban (2019) assessed five (5) state universities in region iv-a offering a pe program through the administrative staff, faculty members and students. it was appraised that faculty members had a high level of awareness, and that physical literacy is the foundation of the pe program. of the five (5) quality indicators, teacher education, and supply and development were indicated as excellent and attested to have no significant difference from each other. in contrast, supply is one of the least quality indicators of the pe program at bsu (orlanda, 2015). it was also determined that assessment of teacher education, supply and development, facilities and resources, and curriculum flexibility (orlanda, 2015; panganiban, 2019) are quality indicators of the pe program while the latter is the leading predictor of all. the factors influencing students’ engagement were found to be relevant from the framework of haworth and conrad (1997) that are positively correlated to satisfaction (simmons, 2011). these include diversity-related activities, shared understanding and experiences, interaction with peers, interaction with faculty members, active and collaborative learning, integrated learning, out-of-class experiences, and academic challenge. it was described that students mostly participate in integrated learning and rarely participate in out-of-class experiences. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 52 to conclude, the etpq is a useful tool that can be used by the heis in assessing a program’s quality for its development and sustainability. although it embodies 17 attributes that are significant indicators of program quality, there are still attributes that need improvements. this had been explored in the conduct of various studies across levels of higher education such as in master’s degree programs (haworth & conrad, 1997; kornelis, 2004; warden & benshoff, 2012), doctorate programs (caro & prado, 2014), and undergraduate programs like bachelor of physical education major in sports and wellness management (bpe-swm) (dimarucot, 2019; dimarucot & rosales, 2020). this is indicative that the etpq can be utilized in bachelor’s degree programs, such as the bped program, in assessing its quality considering that the validity of the framework increased as it was used in various contexts. though studies indicated the vast use of the etpq as a parameter to measure a program’s quality by putting into premium the significant engagement of stakeholders toward the achievement of quality and excellence across programs and disciplines, there is a need to further the theory’s applicability to asian and/or philippine context. methodological gaps will also be satisfied through this research as it will be employed in a physical education teacher education program. due to the recently implemented bachelor of physical education underlying the quality of physical education, it is imperative to assess the quality of the program at the don honorio ventura state university (main campus). thus, this study was perceived to assess the quality of the bachelor of physical education program at dhvsu (main campus) using the engagement theory of program quality (etpq). this research will be significant to the administrators and policymakers, faculty, and students by assessing the quality indicators of the bped from the attributes of the etpq to further enhance and sustain the practices, policies, and system of the program. the researchers aimed to facilitate the conduct of an internal quality assessment of the bachelor of physical education program of the don honorio ventura state university, bacolor, pampanga, philippines grounded on the engagement theory of program quality (etpq). specifically, the following questions were answered: 1. how may the respondents (students and faculty members of the bped program) assess the importance of the etpq attributes? https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 53 2. how may the respondents assess the level of concreteness of the etpq attributes of the program? 3. how may the respondents assess the effectiveness of the etpq attributes of the program? 4. is there a significant difference in the assessment of importance, concreteness, and effectiveness of the attributes between the faculty members and students? 5. which of the etpq attributes assessed in terms of their concreteness are significant predictors of program effectiveness? 6. what teaching-learning experiences were confirmed by the students and faculty members as manifestations of the program’s effectiveness? 2. methodology 2.1. research design the study employed a mixed-methods sequential explanatory design with two distinct phases: quantitative, then qualitative (creswell et al., 2003). in this design, the quantitative method was followed by qualitative procedures in the second phase. the follow-up variant model was used in this sequential explanatory design because it “places greater emphasis on quantitative data, which are used by the researchers to identify significant statistical differences between groups of participants, between individuals with extreme test scores, or in the case of unexpected results” (anguera et al., 2012, p. 20). as shown in figure 2, the gathered qualitative data were analyzed to corroborate the data obtained in the quantitative phase. this variant involves an in-depth understanding of the research problems by elaborating the quantitative results through qualitative data analysis. to consolidate and condense conclusions, the results and findings from both phases were analyzed and examined (wipulanusat et al., 2020). https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 54 figure 2. the follow-up variant explanatory model (creswell & plano clark, 2007) the design was found to be suited to the conduct of an internal assessment of the bachelor of physical education program using the etpq as this provided results and findings that are not grounded within a single methodological approach. the quantitative strand determined the assessment of the entirety of the internal stakeholders of the program while the qualitative strand provided an in-depth understanding of the significant results of the assessment. 2.2. respondents and participants respondents in the quantitative strand. the respondents of the study were the faculty members and students at don honorio ventura state university, bacolor, pampanga, the philippines for the academic year 2020-2021. instructors (n = 12), assistant professors (n = 6), and associate professors (n = 2) who have handled general education, professional education, and major courses were included in a complete enumeration (census) of the faculty members of the bachelor of physical education program (n = 20). meanwhile, a complete enumeration (census) of third-year bped students (n = 141) comprised the student-respondents. third-year students are the best candidates to participate in the internal assessment of the program since they have completed 94.44 percent of the courses, or 51 of the 54 required courses. two field study courses and a teaching internship make up the three remaining courses in their fourth year (5.56 percent). furthermore, there wer e no fourth-year students https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 55 throughout the study period due to the department of education’s k to 12 transition in 2013, which includes two years of the senior high school curriculum in basic education, resulting in a two-year gap in higher education. importantly, all identified students found were regular, which implies that they take the same number of units each semester as prescribed in the curriculum as stipulated in the student handbook (don honorio ventura state university, n.d.). participants in the qualitative strand. two (2) virtual synchronous focus group (vsfg) discussions were facilitated to gather qualitative data to support the findings of the quantitative phase. the focus groups were composed of faculty members (vsfg 1) and select third-year students (vsfg 2). a purposive sampling technique was utilized to select the sample for faculty-participants who were included in the vsfgd 1. the chairperson of the program was primarily included due to their knowledge of the internal processes in the academic aspect, along with four (4) faculty members who have taught in the program since 2014. meanwhile, maximum variation sampling was used to select student-participants in the vsfgd 2. a maximum variation sampling is a method of selecting groups for the goal of optimizing diverse prospects relevant to the research problems (elmusharaf, 2016). a diverse composition of honor students, student-leaders, and regular students were considered as samples (n = 10) for the focus group. 2.3. instruments the instrument for the quantitative phase. the quantitative phase of the study assessed facultyand student-respondents’ perceptions of the importance, concreteness, and effectiveness of the 16 etpq attributes (haworth & conrad, 1997). etpq questionnaire for faculty and students. the etpq questionnaire for faculty and students (dimarucot & rosales, 2020) was adapted to assess the perceived importance, concreteness, and effectiveness of the 16 etpq attributes as applied in the bped program. it originated with 64 items and was eventually trimmed down to 49 benchmark statements following expert content validation. the benchmark statements were assessed by the faculty and students using the three (3) criteria: level of importance, level of concreteness, and level of effectiveness. using a 5-point likert scale (1 = not important to 5 = extremely important), the level of importance was used to assess the https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 56 significance of each item from the perspective of faculty and students. meanwhile, to measure the actual manifestation or existence of such items in the program, a 5-point likert scale (1 = not evident to 5 = extremely evident) was used under the same cluster subscales to assess the level of concreteness. the effectiveness of each attribute in the program was assessed using a rating scale ranging from 1 to 5 (1 = not effective to 5 = extremely effective). table 1. results of the reliability analysis of the etpq instrument for students and teachers per cluster or dimension clusters or dimensions cronbach’s alpha 1. diverse and engaged participants 0.955* 2. participatory cultures 0.912* 3. interactive teaching and learning 0.966* 4. connected program requirements 0.936* 5. adequate resources 0.944* *(cronbach’s alpha ≥ 0.70, acceptable index, fornell & larcker, 1981, nunnally & bernstein, 1994) the instrument for the qualitative phase interview protocol guide. a semi-structured interview protocol guide (ipg) was developed and was composed of open-ended confirmatory questions about the attributes of the etpq and its application to the context of the bped program. the ipg was outlined based on the following structure: (a) welcome message; (b) presentation of the topic; (c) guidelines and instruction (ground rules); (d) an opening question; (e) the open-ended questions; and (f) ending questions (krueger & casey, 2002). the ipg processes and open-ended questions were subjected to face validation by an expert in qualitative research with extensive experience in conducting fgd. moreover, the developed ipg was pilot tested to non-participating participants to ensure administrative feasibility and protocol adjustments when deemed necessary. 2.4. data-gathering procedures necessary permission from the dean of the college of education was sought to confirm the possibility of conducting the rigorous methodological procedures of the study. the purposes and detailed description were included in the letter of request. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 57 in employing the sequential explanatory strand, the following sequential procedures were employed: phase 1: procedures in the quantitative phase step 1 design and implementation of the quantitative procedures. the quantitative procedures were developed from the research questions, which focused on assessing the importance, concreteness, and effectiveness of the etpq attributes. following that, the identified groupings of completely enumerated faculty and students were determined. an electronic survey powered by google form – a free online survey software hosted by google – was used to administer the valid, reliable, and feasible instrument to them. the studentand faculty-respondents responded to the etpq e-survey. they were recruited via social media and personal contacts. an informed consent form stating who the researchers are and the link to a self-administered e-survey was included in the intended questionnaire. before their participation, a preliminary presentation and orientation of the purposes and intent of the study were initially coursed through by their instructor on the course, “the teacher and the school curriculum,” who is also one of the authors of the study to ensure that the nature and concepts of the etpq were well-comprehended by the student-participants. as a result, when communications were delivered to them, they already obtained a clear understanding and awareness of what could be expected of them in terms of participation. the survey lasted a week, and a follow-up was done. to make the statistical treatment of the data easier, a spreadsheet copy of the responses from google form was extracted. step 2 determination of the results highlights. in this step, sample selection commenced aiding in the identification of participants for the qualitative phase based on the remarkable results of the quantitative strand. overall, the focus group protocols were finalized in this process. phase 2: procedures in the qualitative phase step 3 design and implementation of the qualitative procedures. the quantitative data were used as a benchmark for the confirmatory qualitative questions. this step sought to choose participants for the two virtual focus groups (for faculty participants and the other for student participants). in this step, the qualitative data from the virtual focus groups were analyzed within the scope of thematic development and became the foundation for delivering answers to the qualitative https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 58 questions. meanwhile, the virtual fgd was guided by the guidelines proposed by dos santos marques et al. (2021), which detailed the processes of conducting virtual fgd, which is timely in researching in the context of the pandemic as shown in figure 3. figure 3. set-up of the virtual focus groups (dos santos marques et al., 2021) step 4 interpreting the results. this step highlighted the summary and interpretation of both quantitative and qualitative strands as represented with the results and findings, respectively. it ascertained the extent to which the qualitative findings explained the quantitative data. phase 3: mixing the results and findings true to its essence as a mixed-methods approach, the integration of the quantitative results and the qualitative findings in this sequential explanatory design ensured the provision of meaningful assessment of the bachelor of physical education program within the grounds of the etpq from a bi-stakeholder perspective. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 59 figure 4. process diagram of the data collection, analysis, and mixing of the results and findings 2.5. data analysis and interpretation strategies statistical analysis of data (quantitative strand). before analyses, both data sets were prepared and organized. the quantitative data were extracted from google forms into a spreadsheet, particularly on microsoft excel 365 and statistical analysis was performed using the statistical package for the social sciences (spss) version 24. descriptive statistics such as mean and standard deviation were used to analyze the responses of the facultyand student-respondents on their assessment of the importance, concreteness, and effectiveness of the etpq attributes toward the bped program. in reporting their assessment of the importance, level of concreteness, and effectiveness of the etpq attributes, the following scale and interpretation were used: data collection procedure product survey numerical data data collection procedure product virtual synchronous focus group discussion text data transcriptions analysis procedure product statistical treatment descriptive and inferential statistics analysis procedure product thematic analysis themes phase 3: mixing the data explaining the quantitative results through the qualitative findings phase 1: quantitative phase phase 2: qualitative phase https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 60 table 2. statistical ranges and their corresponding verbal interpretation rating statistical ranges verbal description importance concreteness effectiveness 5 4.21 – 5.00 extremely important extremely evident extremely effective 4 3.41 – 4.20 highly important highly evident highly effective 3 2.61 – 3.40 moderately important moderately evident moderately effective 2 1.81 – 2.60 somehow important somehow evident somehow effective 1 1.00 – 1.80 not important not evident not effective inferential statistics such as independent samples t-test was used to test the differences in the responses of the faculty members and students on their assessment of the importance, concreteness, and effectiveness of the etpq attributes in the bped program. additionally, linear regression analysis was used, employing the forward selection method to determine which of the etpq attributes, derived from the responses in the assessment of the level of concreteness, were significant predictors of program effectiveness. data analysis (qualitative strand). in analyzing and interpreting the data responses in the qualitative phase, maxqda version 2020.4.1, a software program designed for organizing qualitative and mixed methods data, was utilized for coding text segments and generating themes from the transcripts. this study followed the thematic analysis of braun and clarke (2006), viz: (1) familiarization; (2) coding the data; (3) generating initial themes; (4) reviewing the themes; (5) naming and defining the themes; and (6) writing up the report. the code-recode procedure was operated using the designed software. in formulating codes, variables indicating programs’ best practices were highlighted and created patterns. parallel variables were merged into another code to organize and easily determine their classification. the software was used to further examine the codes and to formulate relevant subthemes and major themes targeting the objectives of the study. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 61 figure 5. braun and clarke’s (2006) thematic analysis process establishing trustworthiness in data collection and organizing qualitative findings. this study evaluated the content analysis of the qualitative phase using the four (4) criteria developed by lincoln and guba (1985), namely: credibility, dependability, confirmability, and transferability. these criteria intend to establish “trustworthiness” to ensure the worth of the research findings. credibility. to establish the credibility of the research findings, researchers administered focus group discussions to have an actual interview with the participants to gather more authentic data. during the vsfgd, the facilitator established prolonged engagement by starting with casual conversations to establish rapport. the findings were shared with the participants by sending them a google drive link that consisted of transcripts with coded segments. dependability. the dependability of the research findings was established by sending the transcripts with coded segments to the participants to affirm their responses before data analysis. the formulated major themes and subthemes were objectively derived from the authentic responses of the participants in fgd and not subjected to the researchers’ perspectives to abstain from biases. raw data were appended toward the end of the paper to preserve its transparency. the formulated themes and subthemes were presented to the participants of the fgd for their affirmation. confirmability. the researchers conducted an audit trail where all records obtained from the focus group discussion were kept. the formulated major themes and subthemes were drafted in a document to validate their interdependence. the findings obtained were evaluated by an expert in https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 62 social sciences and qualitative research to verify the transcripts and examine their accuracy if the interpretations and conclusions were supported by the data. transferability. to ensure transferability, the researchers presented the research design used, the nature and characteristics of the informants, the method that was utilized, and the obtained findings from the collected data comprehensively with the belief that future researchers may replicate the study in various contexts. in the mixed methods analysis phase, extracted meta-inferences were derived through the convergence of the quantitative and qualitative outcomes of the study, which were comparatively presented in a tabular presentation. 2.6. ethical considerations the data collection procedures and treatment were governed by the international mandate for the ethical conduct of research for human participants such as the belmont report (national commission for the protection of human subjects of biomedical and behavioral research, 1979) and national provisions such as those stipulated in the national ethical guidelines for health and health-related research (philippine health research ethics board, 2018) and the philippine data privacy act of 2012 (r.a. 10173). in this study, informed consent was sought from faculty members and students in the bped program who were all of legal age and identified as having the ability to make informed decisions. there were safeguards in place to preserve confidentiality, anonymity, and privacy. details on how the data would be reported, stored, and disposed of were included, as well as the assurance that they may freely withdraw at any time for any reason without citing any consequence. transparency and ethical conduct in the collection and processing of data were rigorously adhered to in terms of the privacy and confidentiality of the obtained information, according to the data privacy act of 2012. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 63 3. results, findings, and discussions 3.1. results of the quantitative phase importance, concreteness and effectiveness of the etpq attributes. table 3 presents the assessment of the student-respondents and faculty members of the bped program regarding the etpq attributes. in terms of importance, the highest weighted means of 4.63 (sd = 0.52) and 4.85 (sd = 0.33) were noted by the studentand faculty-respondents, respectively. cooperative peer learning was viewed as “extremely important” by both respondents while the attribute diverse and engaged participants was perceived as “extremely important” as well by most of the facultyrespondents with a weighted mean of 4.85 (sd = 0.28). meanwhile, student-respondents regarded integrated learning as “highly important” based on the acquired lowest mean of 4.13 (sd = 0.63), while the faculty-respondents opined out-of-class activities as “extremely important” with the lowest mean of 4.27 (sd = 0.85). this indicates that it is vitally important that faculty members support students to be engaged in collaborative learning and peer-teaching as they deliver different instructional strategies such as real-life situated lectures and discussions, engaging and interactive hands-on activities, and conducting and participating in various out-of-class activities with the support of financial assistance from the administrators. in the level of concreteness, students and faculty members perceived cooperative peer learning as “extremely evident” in the program based on the weighted means of 4.48 (sd = 0.55) for studentrespondents and 4.58 (sd = 0.80) for faculty-respondents. meanwhile, the attribute support for basic infrastructure was assessed as “highly evident” by both groups considering a similar mean of 3.72 by the students (sd = 0.91) and faculty members (sd = 1.03), the lowest of all mean scores. these manifest that faculty members served as an encouragement to the students in engaging them to collaborate with learning and peer-teaching and provided them with different instructional strategies. the presence of administrative support in terms of funding and maintenance was also evident. the summary of the perceived effectiveness of the 16 etpq attributes disclosed that the attributes cooperative peer learning (x̄ = 4.55 [sd = 0.54] for students; x̄ = 4.58 [sd = 0.80] for faculty) and diverse and engaged participants (x̄ = 4.58 [sd = 0.59] for faculty) were assessed as “extremely effective”. however, the attributes support for basic infrastructure and out-of-class activities were translated as “highly effective” but were observed to have the lowest mean scores of https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 64 3.76 (sd = 0.89) and 3.83 (sd = 1.03), respectively. this articulates that both groups affirmed that utilizing collaborative learning, peer-teaching, and different instructional strategies are remarkably effective in the teaching-learning process. table 3. summary of the assessment of the students and faculty members on the importance, concreteness and effectiveness of the etpq attributes importance attributes students mean sd vd cooperative peer learning (highest mean) 4.63 0.52 extremely important integrated learning (lowest mean) 4.13 0.63 overall 4.41 0.63 faculty mean sd vd diverse and engaged participants (highest mean) 4.85 0.28 extremely important cooperative peer learning (highest mean) 4.85 0.33 out-of-class activities (lowest mean) 4.27 0.85 overall 4.59 0.54 concreteness attributes students mean sd vd cooperative peer learning (highest mean) 4.48 0.55 extremely evident support for basic infrastructure (lowest mean) 3.72 1.03 highly evident overall 4.10 0.72 faculty mean sd vd cooperative peer learning (highest mean) 4.58 0.80 extremely evident support for basic infrastructure (lowest mean) 3.72 1.03 highly evident overall 4.16 0.86 effectiveness attributes students mean sd vd cooperative peer learning (highest mean) 4.55 0.54 extremely effective support for basic infrastructure (lowest mean) 3.76 0.89 highly effective overall 4.16 0.66 faculty mean sd vd diverse and engaged participants (highest mean) 4.58 0.59 extremely effective cooperative peer learning (highest mean) 4.58 0.80 out-of-class activities (lowest mean) 3.83 1.03 overall 4.22 0.84 generally, all 16 attributes were assessed as extremely important (x̄ = 4.41 [sd = 0.63] for students; x̄ = 4.59 [sd = 0.54] for faculty), highly evident (x̄ = 4.10 [sd = 0.72] for students; x̄ = 4.16 [sd = 0.86] for faculty), and highly (x̄ = 4.16 [sd = 0.66] for students) to extremely effective (x̄ = https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 65 4.22 [sd = 0.84]). this implies that the program provided its students, faculty, and administrators with an assured quality education system as evident in their positive engagement and experiences. test of differences in the assessment of the importance, concreteness and effectiveness of the etpq attributes between the faculty members and students. table 4 summarizes the results of the test of differences in the assessment of the 16 etpq attributes in terms of importance, concreteness and effectiveness between students and faculty members. the following attributes were viewed differently by both respondents, viz: diverse and engaged participants (t = -2.557; p = 0.011) and diverse and engaged leaders (t = -2.684; p = 0.008) in terms of importance; diverse and engaged participants in terms of concreteness (t = -2.248; p = 0.026); and effectiveness (t = -2.048; p = 0.042). the remaining attributes have no significant differences as they do not obtain a p-value of less than 0.05. generally, faculty members and students assessed the importance, concreteness, and effectiveness of the 16 attributes with the same perspectives based on their experiences within the program. table 4. summary of the test of differences in the assessment of faculty members and students on the importance, concreteness, and effectiveness of the etpq attributes attributes criteria diverse and engaged participants diverse and engaged leaders importance t-value -2.557 -2.684 p-value 0.011 0.008 concreteness t-value -2.248 p-value 0.026 effectiveness t-value -2.048 p-value 0.042 overall: insignificant concreteness of the etpq attributes as significant predictors of effectiveness. linear regression analysis, particularly employing the forward selection method was used to determine which of the 16 etpq attributes were significant predictors of effectiveness in the bped program. table 5 features the attributes that are significant predictors of effectiveness in the program based on the perceived concreteness of the 16 etpq attributes as manifested by the student-respondents. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 66 table 5. strength of the relationship between concreteness and effectiveness model summary model r r square adjusted r square std. error of the estimate 1 .961a .924 .918 6.78468 a. predictors: (constant), var00016, var00012, var00008, var00014, var00003, var00007, var00015, var00006, var00005, var00009 table 6. one-way analysis of variance of the etpq attributes in concreteness and effectiveness anovaa model sum of squares df mean square f sig. 1 regression 73024.463 10 7302.446 158.639 residual 5984.147 130 46.032 total 79008.610 140 a. dependent variable: effectiveness b. predictors: (constant), var00016, var00012, var00008, var00014, var00003, var00007, var00015, var00006, var00005, var00009 table 7. regression analysis summary coefficientsa model unstandardized coefficients standardized coefficients t sig. b std. error beta 1 (constant) 15.776 5.235 3.014 .003 diverse and engaged leaders 1.793 .303 .204 5.924 .000 community of learners .896 .317 .109 2.822 .006 risk-taking environment 1.240 .452 .101 2.745 .007 critical dialogue 1.327 .495 .093 2.680 .008 integrated learning 1.335 .326 .143 4.096 .000 risk-mentoring .920 .315 .121 2.924 .004 planned breadth and depth of course work 2.723 .601 .143 4.534 .000 tangible products 2.240 .544 .130 4.118 .000 support for students 1.132 .393 .095 2.883 .005 support for basic infrastructure 1.572 .291 .181 5.401 .000 a. dependent variable: effectiveness results of the linear regression analysis indicated that 10 out of 16 etpq attributes were determined as significant predictors of program effectiveness, viz: https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 67 1. diverse and engaged leaders with values of β = .204, t(140) = 5.924, p<.001; 2. community of learners with values of β = .109, t(140) = 2.822, p = .006; 3. risk-taking environment with values of β = .101, t(140) = 2.745, p = .007; 4. critical dialogue with values of β = .093, t(140) = 2.680, p = .008; 5. integrated learning with values of β = .143, t(140) = 4.096, p<.001; 6. risk-mentoring with values of β = .121, t(140) = 2.924, p = .004; 7. planned breadth and depth of course work with values of β = .143, t(140) = 4.534, p<.001; 8. tangible products with values of β = .130, t(140) = 4.118, p<.001; 9. support for students with values of β = .095, t(140) = 2.883, p = .005; and 10. support for basic infrastructure with values of β = .181, t(140) = 5.401, p<.001. 3.2. findings of the qualitative phase perspectives of the student-participants on the effectiveness of the etpq attributes relevant curricular and co-curricular activities. the participants recounted their worthwhile experiences in participating in different activities relative to the program. through the strong support provided by student organizations, various activities were prepared to ensure that students’ welfare and holistic development were met while putting into quality the contents and competencies of the curriculum. one remarkable practice is the conduct of student-initiated activities that are aligned with the courses of the program as part of students’ practice in the field. some of these were local and regional conferences that provided avenues for students to be knowledgeable and competent. “for me, what is unique in the bped program is that it provides learning experiences to its students not just inside the classroom set-up but also outside.” (s-p1) https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 68 the participants also referred to sports-related activities and requirements they accomplished during their first and second year of residency as memorable and enjoyable for them. the majority of the activities were performance and outcomes-based that catered to students with practical experiences such as individual, dual, team sports, and even dances. for them, these were the curricular activities and requirements that were tiring but enjoyable as they were able to build their skills like teamwork, confidence, communication, and leadership. strong student-involvement. leadership started with simple involvement. the program involved students with various opportunities through initiatives that were conducive to learning and personality development. they involved themselves in activities that encompassed studentleadership and participation in an extended academic enrichment program such as intramurals and inter-university athletic competitions. they also had the opportunity to serve the program as they became officers, committee members, assistants, and participants in the initiated undertakings of the organizations. their involvement in these kinds of pursuits helped them to develop their leadership skills as they have been part of the governance of the program. this also served as their steppingstone to elevate their leadership abilities and successfully govern the program. “we started from being a member of the committee until i became an officer this school year...” (sp3) this implies that involvement is not merely confined within the four walls of the classroom but can also be affiliated with organizations in the program that builds strong student involvement. participants had the opportunities to develop their socialization, communication and leadership skills. consequently, they became more active in the program. cultivation of collaborative mentoring among teachers and students. strengthening positive connections with the faculty, staff, and administrators of the program established a harmonious relationship in the teaching-learning process. one key factor was effective communication. students ask for consultations and advice from their teachers who in return, provide prompt feedback for improvement. “to the dean, chairperson, or teachers, it is important to update them in every endeavor, the activities you want to implement because without them they will not be realized and related…” (sp1) https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 69 they believed that support from administrators and advisers was important for the attainment of successful activities and events. alongside communication is respect, as possessed by the participants. respect is viewed significantly to ensure a smooth transaction when communicating with the faculty members as it leads to positive implications for them. manifestations of teacher-participants on the teaching-learning experiences as indicators of satisfaction toward the program strong faculty engagement. promoting self-regulated learning maneuvered teachers to generate a strong engagement between and among students. faculty members establish a tied interface when they could administer enjoyable experiences to students which gives them the feeling of joy and satisfaction. selecting instructional activities and discussions are contextualized and based on students’ diverse needs. implementation is not exclusive to students but also the facilitators, as well. when they involve themselves in the process, they construct a strong engagement because of the bond it generates. likewise, involving students in a simple conversation and encouraging them through pep talks create a positive impression. along with the activities, incentives are one of the most notable strategies of teachers to motivate students which leads to a strong engagement. they offered grade incentives and exemption to examinations when students performed well in the class. “i am finding ways how to put myself in the level of the students based on what they need.” (f-p3) supportive administrators. the intervention of the administrators in learning resources is one of the greatest supports they have offered in the program. teaching and learning are not merely focused on the skills and knowledge of teachers and students but also the delivery of instruction through providing teaching and learning resources. without resources, the process of teaching and learning will diminish its efficiency, especially when practicing manipulative skills as part of the program’s curriculum. as a response, administrators facilitate the provision of the necessary materials, equipment, and other resources needed in the delivery of instruction. “…we provide our own instance in our classes. i bought some materials that i needed…the material that can be purchased, we will purchase it. you need to do sacrifices for the gain of the students for the program.” (f-p2) https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 70 administrators extended their support through making initiatives by offering voluntary contributions to purchase insufficient resources. they also coordinate with the university’s office of sports and development to borrow equipment when taking sports-related courses. studentparticipants confirmed that administrators fulfilled their responsibilities by providing support to them. active participation of students. students who know how to involve themselves in diversity are the students who also know how to maximize their networks. students proved that simple participation in various activities can result in a more developed individual. three characteristics were described as indicators of students’ active participation in the program, viz: enthusiasm, leadership, and social abilities. faculty members engage learners in numerous academic and extracurricular activities. accordingly, students take the opportunity to socialize and collaborate with other students as they harness their knowledge and skills. faculty members declared that students were enthusiastic about every task they were assigned regardless of the time constraints. furthermore, because of their dynamic support, student organizations were empowered as evident by the events and programs they initiated that provide other students the opportunity to take part. with that, their development as would-be-teachers grows as they learn how to plan and implement student-initiated activities which can be a stepping stone by the time they will be in the teaching field. “there are sets of learning that encourages socialization and our students were very sociable on that part...” (f-p2) “...student-organizations were empowered and these are best practices that we can be proud of…” (f-p5) this entails that the students’ enthusiasm and willingness to participate in the activities play a significant role in their growth and maturity. hence, the more eager students are to learn, the more teachers are passionate to teach. use of authentic experiences as teaching strategy. continuous inventiveness leads to innovation. intervening several modalities in teaching allows the teacher to know their students better. it encourages the students to take part in the discussion in which they can share their knowledge and perform effectively. also, involving them in the teaching process will help them https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 71 uncover their interest, which will make the experience more enjoyable. moreover, the implementation of systemic tournaments with assistance from other students allows them to be more engaged since it helps them to develop confidence, improve their sportsmanship, and have fun socializing. “we need to contextualize our discussion. we have to let them put themselves on the shoe of the teacher so that they would know.” (f-p5) “...there will be different evaluations from the different levels.” (f-p5) assessing students by merely asking simple questions is crucial as a preliminary stage in shaping their minds. they teach students to be self-sufficient and critical thinkers because of this. therefore, teachers will be able to fairly evaluate them by utilizing a rubric as their authentic assessment instrument. manifestations of student-participants on the teaching-learning experiences as indicators of satisfaction toward the program flexibility and resourcefulness in instruction. two of the good characteristics that bped students possess are flexibility and resourcefulness. in case of insufficiency, faculty members and students share mutually supportive interventions to acquire needed resources for teaching and learning ensuring a safe learning environment. “...they are looking for ways, there are available places where they can administer your activities.” (s-p4) participants confirmed that faculty members provided supervision during task performance by reminding them about safety procedures, which resulted in seamless and safe arrangements throughout class sessions. they also demonstrated their teachers’ resourcefulness in maintaining and pursuing instruction. to address issues concerning the scarcity of learning materials, equipment, facilities, and other resources, students and faculty members seek permission from the osd and municipal officials to borrow equipment and facility relative to their activities such as individual, dual, and team sports. initiatives in terms of financial aspects and extension of help are also some of the students’ interventions to make instruction flexible and resourceful. this indicates that the strong https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 72 engagement of students is evidenced in the discharge of flexible solutions to instructional challenges that complement the faculty members’ initiatives. promotion of experiential and cooperative learning toward holistic development. as a result of the program’s unique learning possibilities, it helped students achieve academic excellence in a significant and positive way. included here are written coursework such as portfolios, learning logs, reflections, research papers, and other paperwork. performance-based activities were also used to facilitate their learning experiences. these were some of the tangible results of their efforts which improved their abilities, skills, and knowledge. they created comprehensive outputs for various courses that allowed them to record what they learned in the classroom. with this, their higher-order thinking skills were strengthened. “...when you record your own learnings, in a way, you develop your creativity and your critical thinking…so, your skills in creativity and critical thinking are being enhanced which are needed to acquire as a teacher.” (s-p1) the techniques and strategies of faculty members for facilitating instruction have contributed to students’ holistic development. one technique was to instill a sense of competition in the delivery of the curriculum where students become more enthusiastic, especially when it was done in groups. with the twists, incentives, and prizes, they were tolerated and pushed to strive harder in accomplishing their tasks. physical presence was one of the most constructive and efficient ways in their field practice. reports, presentations, demonstrations, recitations, and quiz bees were all part of the program. they were also able to practice the teaching profession by writing reports which prompted them to conduct research. their teachers challenged them to be critical thinkers by giving them handouts and asking them questions. student-participants expressed their desire to increase their participation, vibrancy, and excellence in the academe because of their learning experiences. they were challenged to stretch and grow in new ways as they explored new learning opportunities, thereby, allowing them to be holistically developed. as tagare and villaluz (2021) pointed out, p. e. classes do not only develop the physical aspects of the students but also their whole selves holistically. captivating reinforcements in the teaching-learning process https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 73 driving force of faculty members to facilitate instructions. the impelling cause of facilitating effective classroom instructions is driven by students’ representations of learning. when they expressed good impressions and responses whenever teachers provide instructions, the faculty members were enthused to prompt the same energy to their students and teaching. they viewed physical education majors to be enthusiastic and vibrant as reflected in their nature of being physically oriented and well-rounded learners. the more students show willingness and positive facial and symbolic expressions, the greater the effort that teachers give to instructions. “our perspective might be ideal on how students of bped respond to an instruction.” (f-p5) guidance and support from faculty members to enrich students’ learning experiences. the persuasion of faculty members has a great influence on the teaching-learning process. words of wisdom, personality development, and provisions of actual applications were being anchored in reinforcing students’ learning experiences. students tend to show enthusiasm in participating in different activities or tasks when their teachers find ways and means to encourage and motivate them. they allow students to reflect on how important it is to take the teaching profession seriously, as they will be future educators. this is why teachers involve them in teaching and learning situations so that they may witness and practice being teachers themselves. “we supported every activity in terms of the project proposal, plans, and targets…how we maximize the learning experiences of the students into an actual participation…” (f-p2) the provision of real-life experiences to students helps them in their endeavors. faculty members lead the path for student support by developing proposals, plans, and targets as they sought for further ways to expand the bped program’s quality. these include the forming of a new organization and preparing for athletic events, where most of the competitors are commonly from the program. along with these activities is the supervision of faculty in improving students’ performance. their support stretched beyond their roles as educators because of the risks they are taking, such as investing their own money to purchase necessary materials to pursue learning. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 74 3.3. integration of results and findings results that were highlighted as remarkable in the quantitative strand of the study were captured vis-a-vis a confirmatory qualitative finding. table 8. matrix of the integrated results and findings quantitative results qualitative findings example quote diverse and engaged leaders strong studentinvolvement student-participant 5: “...being part of the committee and a pems officer has been remarkable to me.” community of learners cultivation of collaborative mentoring among teachers and students student-participant 2: “as a student, i have a better relationship with my teachers, professor, and all.” risk-taking environment guidance and support from faculty to enrich students’ learning experiences student-participant 5: “…your professors will say, “you can do it!”, “i believe in you!” these simple encouragements push me to do my best and go beyond my limits…” critical dialogue strong faculty engagement faculty-participant 5: “...encourage them to internalize what is supposed to be learned. that will help them to become critical thinkers as well because we teach our students to become independent.” integrated learning promotion of experiential and cooperative learning toward holistic development student-participant 1: “…activities are embedded in the lesson…” https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 75 quantitative results qualitative findings example quote risk-mentoring use of authentic experiences as a teaching strategy faculty-participant 5: “we only took the risk…imagine, f-p4 taught aquatics without a pool for how many semesters.” planned breadth and depth of course work manifestations of teacher-participants on the teaching-learning experiences as indicators of satisfaction toward the program faculty-participant 5: “...there was really a change in terms of the performance, the students, and i think the training you had in the training that you will be having in the future will help you and hone you to become future pe teacher...” tangible products manifestations of student-participants on the teaching-learning experiences as indicators of satisfaction toward the program student-participant 2: “...by that specific activity, reporting in every subject, i am being more prepared as a student, as future educator, and as a good speaker to deliver properly the lesson…” support for students captivating reinforcements in the teaching-learning process faculty-participant 3: “ched is giving us the curriculum but also think of ways that you can enrich, not only the activities but also the experience of the students for them to be very well-versed in their teaching career once they graduate.” support for basic infrastructure flexibility and resourcefulness in instruction faculty-participant 2: “i personally asked our president when he gave us the chart to ask questions or something to raise, so i have the guts to request for the additional covered court or facility for the progress and now is, this 2020, it was granted.” table 8 unveils the interconnection of the results and findings as the ten (10) attributes determined as significant predictors of program effectiveness were integrated into the formulated themes and subthemes. it can be manifested that the responses of the students, faculty members, and administrators of the program were coherent. this advertises that the said attributes were confirmed to be significant predictors of effectiveness as they featured the best practices shared in the program. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 76 4. conclusions relative to the results and findings, the following conclusions were derived: 1. the students and faculty members generally assessed the etpq attributes as extremely important. the results imply that the role of faculty members is imperative in the teaching and learning process. it is necessary for them to engage students in collaborative learning and peer-teaching as they deliver different instructional strategies. this is in parallel with the students’ notion that it is essential for them that their teachers relate their lectures and discussions to real-life situations. 2. both the students and faculty members generally assessed the etpq attributes as highly evident. faculty members manifested encouragement to the students in engaging them to collaborate with learning and peer-teaching. respondents also recognized the provision of fundamental infrastructures through the support of funding and maintenance. 3. while students assessed the etpq attributes to be highly effective, the faculty members perceived it to be extremely effective. the use of collaborative learning, peer-teaching, and different instructional strategies were believed to be worthwhile in the teaching and learning process. 4. there is no significant difference in the assessment of importance, concreteness and effectiveness of the attributes between the students and faculty members. this attests that faculty members and students assessed the importance, concreteness and effectiveness of the 16 attributes with the same perspectives based on their experiences within the program. 5. ten (10) attributes of the etpq in terms of concreteness were found to be significant predictors of effectiveness, viz: (a) diverse and engaged leaders; (b) community of learners; (c) risk-taking environment; (d) critical dialogue; (e) integrated learning; (f) risk-mentoring; (g) planned breadth and depth of course work; (h) tangible products; (i) support for students; and (j) support for basic infrastructure. these attributes were contributory to the implementation of the program. the best practices that were shared and recognized were fundamental and needed to improve and sustain to achieve a quality program. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 77 6. the recounted experiences of the participants revealed a dualistic perspective (students and faculty members) on the factors that contribute to program effectiveness. from the perspective of the students, a holistic dimension in the implementation of the program curriculum was made possible due to the various student-initiated activities, complemented with strong faculty support. meanwhile, the faculty members consider the stakeholders’ role in satisfying the requisites of the program for program quality because of meaningful class-based activities and actively involved students who are bound by the support of the administration. in general, the faculty members’ multidimensional pedagogical and content knowledge, along with their resourcefulness and adaptability, clearly manifests students’ satisfaction toward the program’s practices. acknowledgments: the authors would like to express their deep sense of gratitude to the college of education of the don honorio ventura state university, pampanga, the philippines for providing support and assistance toward the completion of this study. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references anguera, m. t., camerino, o., & castañer, m. (2012). mixed methods procedures and designs for research on sport, physical education, and dance. mixed methods research in the movement sciences: case studies in sport, physical education and dance, 3-27. https://doi.org/10.4324/9780203132326-9. braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa. caro, v. b., & prado, n. i. (2014). quality of the doctoral programs in educational management programs in region 10, philippines: a causal model. liceo journal of higher education research, 10(1), 253-265. http://dx.doi.org/10.7828/1jher.v.10i.672. commission on higher education. (2005). institutional monitoring and evaluation for quality assurance of all higher education institutions in the philippines. (cmo. 15, s. 2005). https://ched.gov.ph/wpcontent/uploads/2017/10/cmo-no.15-s2005.pdf. commission on higher education. (2011). policies, standards, and guidelines for bachelor of physical education major in school p.e. (bpe-spe) and major in sports and wellness management (bpe-swm) programs. (cmo. 23, s. 2011). https://ched.gov.ph/wp-content/uploads/2017/10/cmo-no.23s2011.pdf. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4324/9780203132326-9 http://dx.doi.org/10.7828/1jher.v.10i.672 https://ched.gov.ph/wp-content/uploads/2017/10/cmo-no.15-s2005.pdf https://ched.gov.ph/wp-content/uploads/2017/10/cmo-no.15-s2005.pdf https://ched.gov.ph/wp-content/uploads/2017/10/cmo-no.23-s2011.pdf https://ched.gov.ph/wp-content/uploads/2017/10/cmo-no.23-s2011.pdf multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 78 commission on higher education. (2017). policies, standards, and guidelines for bachelor of physical education (bped). (cmo. 80, s. 2017). https://ched.gov.ph/wp-content/uploads/2017/11/cmo-no.-80s.-2017.pdf. creswell, j. w., plano clark, v. l., gutmann, m. l., & hanson, w. e. (2003). an expanded typology for classifying mixed methods research into designs. a. tashakkori y c. teddlie, handbook of mixed methods in social and behavioral research, 209-240. https://us.corwin.com/sites/default/files/upmbinaries/19291_chapter_7.pdf. creswell, j., & plano clark, v. (2007). designing and conducting mixed methods research. sage. https://doi.org/10.1177/1094428108318066. data privacy act of 2012, n. p. c. §§ 8-11 (2012). https://www.privacy.gov.ph/data-privacy-act/#1. dimarucot, h. c. (2019). evaluation of bachelor of physical education major in sports and wellness management program in san beda university using the engagement theory of program quality (etpq). scientia: the international journal on the liberal arts, 8(1), 168-187. https://scientia-sanbeda.org/wpcontent/uploads/2019/04/v8n1.08-dimarucot.pdf. dimarucot, h. c., & rosales, m. j. d. (2020). educational equity through engagement in the bachelor of physical education major in sports and wellness management program in the national capital region. asia pacific journal of multidisciplinary research, 8(2). http://www.apjmr.com/wpcontent/uploads/2020/04/apjmr-2020.8.2.16.pdf. don honorio ventura state university. (n.d.). student handbook. the author. dos santos marques, i. c., theiss, l. m., johnson, c. y., mclin, e., ruf, b. a., vickers, s. m., fouad, m. n., scarinci, i. c., & chu, d. i. (2021). implementation of virtual focus groups for qualitative data collection in a global pandemic. american journal of surgery, 221(5), 918–922. https://doi.org/10.1016/j.amjsurg.2020.10.009. durban, j. m., & catalan, r. d. (2012). issues and concerns of philippine education through the years. asian journal of social sciences & humanities, 1(2), 61-69. education, science and technology, arts, culture and sports, o. g. § 1 (1987). https://www.officialgazette.gov.ph/constitutions/the-1987-constitution-of-the-republic-of-thephilippines/the-1987-constitution-of-the-republic-of-the-philippines-article-xiv/. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://ched.gov.ph/wp-content/uploads/2017/11/cmo-no.-80-s.-2017.pdf https://ched.gov.ph/wp-content/uploads/2017/11/cmo-no.-80-s.-2017.pdf https://us.corwin.com/sites/default/files/upm-binaries/19291_chapter_7.pdf https://us.corwin.com/sites/default/files/upm-binaries/19291_chapter_7.pdf https://doi.org/10.1177/1094428108318066 https://www.privacy.gov.ph/data-privacy-act/%231. https://scientia-sanbeda.org/wp-content/uploads/2019/04/v8n1.08-dimarucot.pdf https://scientia-sanbeda.org/wp-content/uploads/2019/04/v8n1.08-dimarucot.pdf http://www.apjmr.com/wp-content/uploads/2020/04/apjmr-2020.8.2.16.pdf http://www.apjmr.com/wp-content/uploads/2020/04/apjmr-2020.8.2.16.pdf https://doi.org/10.1016/j.amjsurg.2020.10.009 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 79 elmusharaf, k. (2016). qualitative sampling techniques. paper presented at training course in sexual and reproductive health research. https://www.gfmer.ch/srh-course-2016/researchmethodology/qualitative-sampling-techniques-elmusharaf-2016.htm. enhanced basic education act of 2013, o. g. § 1 (2013). https://www.officialgazette.gov.ph/downloads/2013/09sep/20130904-irr-ra-10533-bsa.pdf. fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 18(3), 382-388. https://doi.org/10.1177/002224378101800313. haworth, j. g., & conrad, c. f. (1997). emblems of quality in higher education. developing and sustaining high-quality programs. allyn & bacon. heat, a. e., altenburger, p., brechter, j. h., chleboun, g., jette, d. u., pike, g., schilling, d., topp, k., & tschoepe, b. (2018). developing a tool to assess physical therapist educational program quality with engagement theory: the american council of academic physical therapy benchmarks for excellence task force. journal of physical therapy education, 32(2), 109-117. https://doi.org/10.1097/jte.0000000000000048. higher education act of 1994, o. g. § 2 (1994). https://www.officialgazette.gov.ph/downloads/1994/05may/19940518-ra-07722-fvr.pdf. kornelis, p. c. (2004). faculty members’ and students’ perceptions of quality in master of education programs within member schools of the cccu. university of south dakota. https://www.proquest.com/openview/65f897f9361e95db506f3a757578f925/1. krueger, r. a., & casey, m. a. (2002). designing and conducting focus group interviews. https://www.eiu.edu/ihec/krueger-focusgroupinterviews.pdf. lincoln, y. s., & guba, e. g. (1985). naturalistic inquiry. sage journals. mustan, t. (1998). operationalization and preliminary testing of the engagement theory. the university of wisconsin-madison. https://www.proquest.com/openview/72e4968120bee93a2fbff8aa9285cffd/1. national commission for the protection of human subjects of biomedical and behavioral research. (1979). the belmont report: ethical principles and guidelines for the protection of human subjects of research. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.officialgazette.gov.ph/downloads/2013/09sep/20130904-irr-ra-10533-bsa.pdf https://doi.org/10.1177/002224378101800313 https://doi.org/10.1097/jte.0000000000000048 https://www.eiu.edu/ihec/krueger-focusgroupinterviews.pdf multidisciplinary journal for education https://doi.org/10.4995/muse.2022.muse social and technological sciences e-issn: 2341-2593 dizon &tolentino (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 46-80. https://doi.org/10.4995/muse.2022.16959 80 u.s. department of health and human services. https://www.hhs.gov/ohrp/sites/default/files/thebelmont-report-508c_final.pdf. nunnally, j. c., & bernstein, i. h. (1994.) the assessment of reliability. psychometric theory, 3, 248-292. https://www.scirp.org/(s(i43dyn45teexjx455qlt3d2q))/reference/referencespapers.aspx?referenceid= 1960143. orlanda, m. m. (2015). revisiting the service physical education program at the tertiary level: basis for a revitalized program. asia pacific journal of multidisciplinary research, 3(5), 29-35. http://www.apjmr.com/wp-content/uploads/2016/02/apjmr-2015-3.5.3.04.pdf. panganiban, t. d. c. (2019). quality assessment of physical education programs of state universities in the philippines. jurnal sportif: jurnal penelitian pembelajaran, 5(2), 166-174. https://doi.org/10.29407/js_unpgri.v5i2.12983. philippine health research ethics board. (2018). national ethical guidelines for health and health-related research. https://ethics.healthresearch.ph/index.php/phoca-downloads/category/4-neg. shape america. (2017). national standards for initial physical education teacher education. https://www.shapeamerica.org/accreditation/peteacherprep.aspx. simmons, j. (2011). interdisciplinary studies students’ academic and social engagement: a quantitative study. http://stars.library.ucf.edu/cgi/viewcontent.cgi?article=2713&context=etd. tagare, r., & villaluz, g. (2021). enhancing tertiary physical education by determining the activity preferences of generation z students. multidisciplinary journal for education, social and technological sciences, 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492. warden, s. p., & benshoff, j. m. (2012). testing the engagement theory of program quality in cacrep‐ accredited counselor education programs. counselor education and supervision, 51(2), 127-140. https://doi.org/10.1002/j.1556-6978.2012.00009.x. wipulanusat, w., panuwatwanich, k., stewart, r. a., & sunkpho, j. (2020). applying mixed methods sequential explanatory design to innovation management. in the 10th international conference on engineering, project, and production management (pp. 485-495). springer, singapore. https://doi.org/10.1007/978-981-15-1910-9_40. https://doi.org/10.4995/muse.2022.muse https://doi.org/10.4995/muse.2022.16959 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.29407/js_unpgri.v5i2.12983 https://doi.org/10.4995/muse.2021.15492 https://doi.org/10.1002/j.1556-6978.2012.00009.x https://doi.org/10.1007/978-981-15-1910-9_40 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 instrumentation of microscale techniques for biochemistry teaching at fes zaragoza, unam a. garcía-del-valle1, 3, m.t. corona-ortega1, m. cruz-millán1, a.g. rojasfernández1, m. aguilar-santelises2, l. aguilar-santelises1 1national autonomous university and 2national polytechnique institute, mexico. 3 corresponding author: email: aracelig567@yahoo.com.mx; faculty for higher education (fes) zaragoza, unam. batalla del 5 de mayo s/n col. ejército de oriente, iztapalapa, 09230, méxico city. phone: +52 5556 23 07 93 received: 2013-12-31; accepted: 2014-04-08 abstract biochemistry education requires laboratory sessions where theoretical knowledge may be put on test. at the same time, there is always some risk due to exposure to toxic materials, dangerous chemicals storage and waste disposal. compliance with new regulations to prevent environmental contamination may also constitute a real hindrance for biochemistry teaching as experimental science. therefore, we have designed microscale techniques, in order to reduce costs as well as the negative impact of laboratory practical sessions due to risk and environmental contamination. to develop microscale techniques does not only mean to reduce equipment size and amount of the reagents that are required for the usual experiments. microscale techniques serve particularly well as a motivating approach to experimental biochemistry teaching that produces highly motivated students at the same time that requires minor costs and decreases working time, laboratory space, amount of reagents and dangerous waste. we have demonstrated all these positive effects in biochemistry teaching and prompted the formal implementation of microscale techniques into the formal activities from the cell and tissue biochemistry laboratory i (bct-i) from the chemistry, pharmacy and biology (qfb) curricula at the national autonomous university of mexico (unam). first, we reviewed the bct-i manual, choosing all the laboratory practices that might be microscaled. then, we elaborated and validated all garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 78 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 necessary protocols to analyse linearity, accuracy and reproducibility of the determinations, demonstrating that microscale techniques allow truthful results, comparable to full scale techniques. keywords biochemistry teaching, microscale techniques, experimental biochemistry, environment protection 1. introduction since the ground-breaking discovery in 1828 demonstrating that biomolecules such as urea could be synthesised, starting with non-living elements, scientists have explored all biochemical aspects of life with increasing interest. many mysteries of life have been revealed, showing the functioning of living beings at biochemical level. however, many more questions appear after an initial question has been answered. biochemists have searched living processes with basis on a continuously increasing but never ending knowledge on biology, chemistry, physics and mathematics (mckee and mckee 2012). thanks to modern technology, we enjoy now exceptional opportunities to learn every day more and to apply our knowledge to solve problems in diverse fields, such as agriculture, anthropology, pharmacy, genetics, medicine, odontology, veterinary, forensics, toxicology and others. borders between these and many other sciences have become unclear and often arbitrary. such big overlapping is due to distinctive physical and biological properties of the elements of life. subjects constituting the study program for students following the chemistry, pharmacy and biology (qfb) curricula at the faculty for higher education (fes), zaragoza, unam gather many basic and advanced subjects on biomolecules, essential to sustain life as we know it. forming qfb professionals is also directed to prepare capable professionals who may approach the study of life from many and diverse points of view, such as public health, ecology and all others demanding an increasing knowledge on garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 79 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 functioning of life molecules and related clinical, pharmacology and technological aspects. the cell and tissue biochemistry laboratory i (bct-i) is organized for teaching of theoretical and practical knowledge, as well as to promote acquisition of experimental abilities. this kind of learning constitutes the first encounter for qfb students not only with biochemistry concepts but also with practical matters of diverse biology problems (curricula qfb 2006). to turn this situation into a successful experience, students are assisted by experienced experimenters and scientists, able to solve biological problems through experimentation. to be approved at this level, learners need not only to master theoretical concepts but also to demonstrate that they are capable of performing experimental work according to the theoretical knowledge and practical abilities that they must possess at such level. historically, science has evolved by hypothesis that must be proved with laboratory work (molina et al., 2006). new hypothesis may contrast with established knowledge and testing them require experimenters with good handling of a number of laboratory techniques. objectives at laboratory teaching usually are: a) to illustrate theoretical knowledge, b) to teach experimental techniques and c) to promote scientific interest. no doubt, having an ample knowledge of laboratory methodologies greatly facilitates the problem-solving capacity that we want to give to our students. confronting our students with the necessity to find adequate laboratory techniques to solve the problems that they face during their studies surely helps them to build up a good knowledge on experimental tools and the ability to use them correctly (lacolla 2005). additionally, during the process of learning laboratory techniques, efforts should be made to maintain a minimal risk for people as well as to reduce costs and to diminish production of chemical and biological waste. good laboratory practices (glp) are essential to guarantee the certainty of the results and to minimize or even avoid environmental contamination. garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 80 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 2. microscale instrumentation and validation 2.1 antecedents laboratory work is usually expensive and may generate dangerous waste. even big institutions have difficulties to satisfy the economic demands imposed by requirements of space, equipment, qualified personnel, laboratory organization and consumption of reagents, plastic and other materials. minor institutions with shortage of economic resources are not able to provide high quality laboratory sessions and often choose to have only demonstrative sessions. instead, we give our students the opportunity to perform the techniques by themselves, under proper supervision. here, at our national university, we also take responsibility for laboratory work within a controlled environment, reducing to a minimal level the exposure to toxic materials and diminishing the generation of waste and environmental pollution that could be caused during our work. 2.2 advantages of microscale techniques to work with microscale instead of high-consuming, waste-generating techniques is a way to work in the biochemistry laboratory, using the smallest possible volumes of reagents without decreasing the quality of the experiments performed. microscale techniques drastically reduce the amount of chemicals and laboratory materials, have lower cost and less requirement for expensive equipment. (szafran et al., 1989, singh et al., 1999, national microscale chemistry centre, 2002). among the obvious and environment-friendly advantages, we have: • minimized lab pollution. • lower release of toxic, carcinogenic or mutagenic products into the environment. • lower risk for accidents derived from fire and toxic substances (smith et al., 2008). garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 81 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 • lower costs for experiment and number of students in the laboratory. • a significant reduction (between 75 and 99 %) of chemical, biological and radioactive waste. didactic benefits and learning outcomes are considerable (pesimo 2014). techniques are easier to perform and thus, more experiments can be performed in less time with lower cost. students become curious and are forced to work carefully through all steps of the experiments. students receive full education, including awareness of the necessity to protect the environment and to work following glp (new hampshire department of environmental services 2014). at the beginning, to acquire proper equipment for microscale techniques may represent an additional cost. however, after the initial expenses, costs for material and equipment are drastically reduced. 2.3 instrumentation first, we evaluate and diagnose which techniques could be developed as microscale techniques (table 1). no practical session reason to be selected 1 standard curve basic method that can be validated and performed with minimal amount of reagents 2 extraction and identification of reserve carbohydrates from animals and plants small volumes can be used during extraction and purification of starch and glycogen 3 analysis of lipids from egg yolk small volumes can be used for extraction and purification phosphates can be quantified it may be validated 4 aminoacids small volumes can be used 5 plasma proteins small volumes can be used for extraction and purification proteins can be quantified by several methods it may be validated garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 82 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 no practical session reason to be selected 6 dna from wheat small volumes can be used for extraction and purification 7 invertase kinetics small volumes can be used to study the influence of various factors that may modify the velocity of catalytic reactions enzyme products can be quantified it may be validated table 1. techniques that may be performed in microscale. then, we modify techniques that were found susceptible of modification to microscale conditions (table 2). we tested various conditions and working volumes to find those allowing bigger saving of time and reagents while quality of work and results were as high as the original non-microscaled techniques. objective importance reagent saving 1 to design and obtain standard curves to determine albumin concentration in white egg standard curves are basic tools for biochemistry students. methods to measure proteins are discussed in relation to protein properties. 50 % 2 to extract and identify polysaccharides from various sources to learn lugol and benedict methods the biological importance of reserve carbohydrates. students extract and purify glycogen from rat liver and starch from rice. students learn methods to identify hydrolysed and non-hydrolysed lipids. 48 % 3 to extract lipids from egg yolk quantify phosphates from phosphorylated and non-phosphorylated lipids physicochemical properties of lipids and methods for lipids extraction are discussed students get acquainted with the method of kjeldahl for quantitative determination of nitrogen and methods for identification of phosphates. 69 % 4 to determine pka to verify dipolar behaviour of aminoacids students learn to identify aminoacids by pka and ionic behaviour. 50 % garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 83 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 objective importance reagent saving 5 to obtain albumin and globulin from blood to quantify proteins by the method of biuret plasma composition and protein properties are analysed. students learn the properties of plasma proteins, methods to fractionate plasma and to quantify its components 50 % 6 to extract dna from wheat germ to evaluate dna purity dna can be isolated from wheat germ. it facilitates learning of dna properties and methods for extraction, purification and quantification of dna. 50 % 7 to analyse the velocity of enzymatic reactions to analyse the ionic character of enzyme and substrate to determine km and vmáx biochemistry scholars must be acquainted with enzyme kinetics. students learn to obtain km and vmáx and analyse how ph, temperature and enzyme and substrate concentrations affect enzyme activity. this session illustrates the participation of enzyme reactions in normal metabolism. 50 % table 2. modified practical sessions 2.4 validation most laboratory practices require certification by detecting and quantifying specific products. validation is applied to demonstrate how accurate are the results obtained. validation permits to recognize linearity, precision and accuracy, and when these measurements are within certain limits, it is right to affirm that both the results and the methods used to obtain the results are correct. analytical methods describe the sequence of activities, material resources and parameters that must be fulfilled to analyse the presence of specific products. validation is a way to demonstrate that certain method fulfils its purpose with the required sensitivity and specificity (fda 2014, paquirigan and beebe 2008). garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 84 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 2.5 linearity it means that the results obtained are proportional to the concentration of the product, within certain intervals (fda 2014, paquirigan and beebe 2008). an analyst must prepare at least 5 concentrations of the reference chemical by triplicate. triplicates can be prepared separately or by diluting the highest concentrated solution. concentration in the middle must be equivalent to the analysed concentration. the interval between concentrations should be according to specifications of the method to validate. the slope of the regression line (b1), the y intercept (b0), coefficient of determination (r2) and the confidence intervals from the slope [ci (β1)] must be calculated. intervals are closely related to the purpose of the method and are therefore, expressed as a percentage of concentration of the reference chemical. concentration may be plotted in the x-axis vs the analytical response in the y-axis. acceptance criteria include: r2 ≥ 0.98 and ic (β1) ≠ 0. 2.6 precision it refers to the degree of concordance between individual analytical results, when the procedure is repeatedly applied to different aliquots of a homogeneous sample. an analyst must prepare at least 5 concentrations of the reference chemical. sextuplicates can be prepared separately or by diluting the highest concentrated solution. standard deviation (s) and coefficient of variation (cv) of the analytical response must be calculated. acceptance criteria in this case are cv ≤ 2.5% for physicochemical methods (fda 2014, paquirigan and beebe 2008). 2.7 accuracy it refers to the degree of concordance between the results obtained and the reference value. an analyst must analyse a sample with the method to validate. the same analyst garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 85 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 must prepare at least 6 samples to which the product to be analysed is added. it can be done for instance by utilizing half of the analytical sample originally required and adding the product obtained or a secondary reference until reaching 100%. added samples must be analysed by the same analyst under similar conditions and using as reference the chemical or product added to the sample. the amount of the product is determined and the percentage of recovery from the product is calculated. it is necessary to calculate the arithmetical mean, sd, cv and ci from the percentage of recovery. acceptance criteria demand 100% ci or an arithmetical mean from percentage of recovery between 97-103 % for chemical or spectrum photometric methods. the cv of the percentage of recovery must be < 3%. 2.8 validation microscaled analytical methods (1, 3, 5 and 7 from table 2) were validated and the results demonstrated linearity, precision and accuracy. as an example, tables 3, 4 and 5 show data obtained when analysing phosphate content from egg yolk. values are within accepted limits. the method complies with linearity and specifications. values in most levels are quite similar. the cv is elevated because absorbance is small, which means that the smallest variation would produce big changes. however, values observed are very similar. therefore, the method is considered precise. repetition concentration (μg) absorbance 1a 2 0.012 1b 2 0.017 1c 2 0.017 2a 6 0.055 2b 6 0.048 2c 6 0.051 garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 86 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 repetition concentration (μg) absorbance 3a 10 0.093 3b 10 0.083 3c 10 0.082 4a 20 0.166 4b 20 0.166 4c 20 0.162 5a 30 0.222 5b 30 0.252 5c 30 0.212 table 3. linearity slope (b1) 0.008 y-intercept (b0) 0.005 determination coefficient (r2) 0.985 sy/x 0.010 sb1 0.000 superior ic (β1) 0.008 inferior ic (β1) 0.007 table 4. linearity repetition concentration levels 2 μg 6 μg 10 μg 20 μg 30 μg 1 0.012 0.055 0.093 0.166 0.222 2 0.017 0.048 0.083 0.166 0.252 3 0.017 0.051 0.082 0.162 0.212 4 0.019 0.054 0.082 0.166 0.24 5 0.017 0.055 0.082 0.169 0.231 6 0.017 0.054 0.088 0.166 0.231 x� 0.016 0.053 0.085 0.166 0.231 garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 87 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 repetition concentration levels 2 μg 6 μg 10 μg 20 μg 30 μg sd 0.002 0.003 0.004 0.002 0.014 cv 14.213 5.275 5.365 1.344 6.000 table 5. precision 2.9 accuracy and protocols since results demonstrate that the method is linear and precise, it is concluded that the method is also accurate. once completed the validation procedure microscale protocols were elaborated after exhaustive literature review. 3. teacher education as means for the implementation of the microscale 3.1 antecedents scientific knowledge and technology advances are nowadays continuously increasing and evolving. speed of changes and continuous testing demands highly capacitated professionals with great curiosity and devotion. professional formation requires time and dedication as well as proper guidance and advice. to be a professional in science and education is a long life task. teachers and scientist need to cultivate their knowledge permanently. forming professionals with the right background and attitude must be considered the main objective of university teaching. garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 88 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 3.2 workshops three workshops were organized to present microscale techniques to teachers from bct1. teachers learned the techniques, appreciated the easiness of execution and the critical points that should deserve more attention in order to keep precision and accuracy of the methods. during training, there was ample discussion on advantages and disadvantages of the new techniques. at the end, the entire group agreed to adopt the new techniques for teaching biochemistry in the bct-1 program. 3.3 limitations and contributions part of this work is dedicated to develop microscale techniques and to capacitate teachers to use them and instruct their students to use them well. three workshops were offered to our teachers in order to train them properly. nearly 100 % of our academic personnel are now acquainted with the new techniques. our workshops also served to stimulate communication, teamwork, critical analysis and creativity among our colleagues. we all agree that updating and adapting our work in the teaching laboratories will contribute to improve the quality of teaching biochemistry at our university. seven new practical sessions were designed and included in the working schedule of bct-1. all microscale techniques that we added to these sessions are validated. saving time for the accomplishment of practical goals also facilitated explanation and comprehension of the subjects. new protocols allow to add additional goals such as being acquainted with additional instruments and to develop better routines for handling of materials and equipment. implementation of new protocols involves the acquisition of expensive materials and equipment and is time consuming. however, after the initial expenses, we noted an important reduction of costs. reduction of risk and generation of chemical waste is also noteworthy. an important goal is to provide our students not only with good work garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 89 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 routines and successful approaches to biological and socioeconomic problems but also with a permanent concern for the environment and toxicological risks derived from handling chemicals. in this way, we try to contribute to educate and train well-prepared professionals to produce a positive impact in our society. 4. acknowledgements this project was supported by unam (dgapa papime pe206913). the authors gratefully acknowledge the enthusiastic collaboration of araceli carrillo, sergio albarran, fabian serrano and margarita aguilar. 5. references curricula qfb. (2006) facultad de estudios superiores zaragoza. unam. available at http://www.zaragoza.unam.mx. accessed on february 5, 2014. fda (2014) analytical procedures and methods validation for drugs and biologics. available at http://www.regulations.gov. accessed on july 10, 2014. lacolla l. (2005) reflexiones acerca del trabajo practico en la enseñanza de la química. available at http://ensino.univates.br/~4iberoamericano/trabalhos/trabalho204.pdf. accessed on july 10, 2014. mckee t, mckee j. (2012) biochemistry: the molecular basis of live. 5th ed. usa: oxford university press. molina mf, farías dm, casas ja. (2006) el trabajo experimental en los cursos de química básica. 1: 51-59. available at http://portalweb.ucatolica.edu.co/easyweb2/files/44_246_v1n1molinafariascasas.pdf. accessed on july 10, 2014. national microscale chemistry centre. (2002) what is microscale chemistry? available at http://www.microscale.org/about.asp. accessed on july 10, 2014. garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 90 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2205 social and technological sciences eissn: 2341-2593 new hampshire department of environmental services. (2014) microscale chemistry. available at http://des.nh.gov/organization/commissioner/p2au/pps/ppsp/microscale.htm accessed on july 10, 2014. paquirigan al, beebe dj. (2008) microfluidics meet cell biology: bridging the gap by validation and application of microscale techniques for cell biological assays. bioassays 30:811-821. doi: 10.1002/bies.20804. pesimo ar. (2014) developing the learning outcomes of the students through microscale experiments in chemistry. open access library j 1: e479. available at http://dx.doi.org/10.4236/oalib.1100479. accessed on july 2014. singh mm, szafran z, pike rm. (1999) microscale chemistry and green chemistry: complementary pedagogies. j chem. edu. 76: 1684-1686. smith m, desha c, stasinopoulos p, hargroves k, hargroves s. (2008) lesson 11: green chemistry: reducing toxicity’ in sustainability education for high schools. the natural edge project, australia. available at http://www.naturaledgeproject.net/documents/slc/slc%20subject%20supplement%2 0-%20lesson%2011%20final.pdf. accessed on july 10, 2014. szafran z, singh mm, pike rm. (1989) the microscale inorganic laboratory: safety, economy, and versatility. j chem. edu. 66: a263-267. garcía-del-valle et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 78-91 | 91 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 60 operation of circuit breakers: data and analysis v.c. maduemea , m. j. mbunwea*, t. c. maduemea , m. ayaz ahmadb , c. v. anghel drugarinc adepartment of electrical engineering, university of nigeria nsukka, 410001, nigeria bphysics department, faculty of science, p.o.box 741, university of tabuk, 71491, saudi arabia cdepartment of electronics and informatics engineering, “eftimie murgu”, university of resita, resita, romania *corresponding author: muncho.mbunwe@unn.edu.ng & mayaz.alig@gmail.com received: 16 january 2021; accepted: 12 june 2021; published: october 2021 abstract an attempt has been made for the analysis on circuit breakers (cbs) this paper. first, the types and arcing phenomenon of oil and sf6 circuit breakers were briefly discussed. however, various cbs were analyzed in terms of certain outage frequencies and reliability indices to ascertain the most reliable cb. this was possible using data collected from the 33kv transmission company of nigeria (tcn) new haven, enugu. after the analysis, emene industrial cb had the highest value of availability of 0.9999 and the lowest tripping report while ezillo had the highest failure rate of 0.1032. keywords: circuit breaker; outage; failure rate; availability; reliability to cite this article: madueme, v.c., mbunwe, m.j., madueme, t.c., ayaz ahmad, m., anghel drugarin, c.v. (2021). operation of circuit breakers: data and analysis. multidisciplinary journal for education, social and technological sciences, 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0002-8963-0126 https://orcid.org/0000-0003-0432-4574 https://orcid.org/0000-0002-5731-5439 https://orcid.org/0000-0002-1906-0623 https://doi.org/10.4995/muse.2021.12406 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 61 1. introduction once a power system is established it is necessary to protect it from internal and external faults. so we use some protecting and sensing device like circuit breakers, relays, fuses etc (saxena, singh, ali, gandhi, 2012). power circuit breaker is one of the most important protection and control apparatus in the power system (suwanasri, hlaing and suwanasri, 2014). a circuit breaker is a switching device that interrupts the abnormal or fault current. it is a mechanical device that disturbs the flow of high magnitude (fault) current and in addition, performs the function of a switch. the circuit breaker is mainly designed for closing or opening of an electrical circuit, thus protects the electrical system from damage. circuit breakers represent one of the most critical power apparatus in the power system. they are used to change topology of the power system to accommodate various configurations in routing the load. cbs are also used to isolate faulted parts of the system as a part of the protective relaying operation (kezunovic, ren, latisko, sevcik, lucey, cook, and koch, 2005). circuit breaker essentially consists of fixed and moving contacts. these contacts are touching each other and carrying the current under normal conditions when the circuit is closed. when the circuit breaker is closed, the current carrying contacts, called the electrodes, engaged each other under the pressure of a spring. during the normal operating condition, the arms of the circuit breaker can be opened or closed for a switching and maintenance of the system. to open the circuit breaker, only a pressure is required to be applied to a trigger (circuit globe, 2017). figure 1: diagram of an oil circuit breaker (circuit globe, 2017) whenever a fault occurs on any part of the system, the trip coil of the breaker gets energized and the moving contacts are getting apart from each other by some mechanism, thus opening the circuit. according to pinnekamp (2007), several gva of power can be tamed by a circuit breaker within fractions of a second. such is the importance of this single device that tens of billions of dollars have been spent on its development over the last 100 years. https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 62 2. types of circuit breaker circuit breakers are mainly classified on the basis of rated voltages. circuit breakers below rated voltage of 1000v are known as the low voltage circuit breakers and above 1000v are called the high voltage circuit breakers. the most general way of the classification of the circuit breaker is on the basis of the medium of arc extinction. such types of circuit breakers are as follows : [1] oil circuit breaker a. bulk oil circuit breaker b. minimum oil circuit breaker [2] minimum circuit breaker [3] air blast circuit breaker [4] sulphur hexafluoride circuit breaker [5] vacuum circuit breaker [6] air break circuit breaker all high-voltage circuit breakers may be classified under two main categories i.e oil circuit breakers and oil-less circuit breaker (electrical concepts, circuit breaker and arc phenomenon, 2017). 3. arc phenomenon in circuit breaker when a short-circuit occurs, a heavy current flows through the contacts of the circuit breaker before they are opened by the protective system. at the instant when the contacts begin to separate the contact area decreases rapidly and large fault current causes increased current density and hence rise in temperature. the heat produced in the medium between contacts (usually the medium is oil or air) is sufficient to ionize the air or vaporize and ionize the oil. the ionized air or vapour, acts as conductor and an arc is struck between the contacts. the potential difference between the contacts is quite small and is just sufficient to maintain the arc. the arc provides a low resistance path and consequently the current in the circuit remains uninterrupted so long as the arc persists. during the arcing period, the current flowing between the contacts depends upon the arc resistance. the greater arc resistance will represent to the smaller the current flow between the contacts. the arc resistance depends upon the following factors: • degree of ionization the arc resistance increases with the decrease in the number of ionized particles between the contacts. • length of the arc the arc resistance increases with the length of the arc i.e. separation of contacts. https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://circuitglobe.com/arc-extinction.html http://circuitglobe.com/arc-extinction.html http://circuitglobe.com/oil-circuit-breaker.html http://circuitglobe.com/bulk-and-minimum-oil-circuit-breaker.html http://circuitglobe.com/bulk-and-minimum-oil-circuit-breaker.html http://circuitglobe.com/sf6-sulphur-hexaflouride-circuit-breaker.html http://circuitglobe.com/vacuum-circuit-breaker.html http://circuitglobe.com/air-break-circuit-breaker.html http://www.studyelectrical.com/2014/05/classification-types-of-circuit-breakers.html multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 63 • cross section of arc the arc resistance increase with the decrease in the area of cross section of the arc (electrical systems, 2017). figure 2: diagram of the sf6 circuit breaker (electrical systems, 2017) when the contacts of a circuit breaker are separated under fault conditions, an arc is struck between them. the current is thus able to continue until the discharge ceases. the production of arc not only delays the current interruption process but it also generates enormous heat which may cause damage to the system or to the breaker itself. therefore, the main problem in a circuit breaker is to extinguish the arc within the shortest possible time so that heat generated by it may not reach a dangerous value (electrical systems, 2017). 4. data and analysis the data for our analysis was collected from the 33kv transmission company of nigeria (tcn) located in new haven, enugu (tcn, tripping reports, 2016). it contained data of up to 59 feeders/cbs in enugu region for the period of three (3) months (april – june 2016). the data contained the outage (tripping) report for the feeders together with the tripping time, restoration time, type of fault, time duration before restoration. as a result of enormity of the data, we tried to group the number of outages per feeder in terms of their outage frequencies such as: i. most frequent outages: for outages greater than 100 times. ii. very frequent outages: outages between 31-99 iii. less frequent outages: between 10 -30 iv. occasional: between 3 -9 v. rare: between 1-2 https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://www.studyelectrical.com/2014/05/principles-and-methods-of-arc.html multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 64 the tables and their corresponding chart representations are given to further illustrate the frequency of outages of each feeder between april and june 2016. table 1: most frequent outages (>100) feeder outages ezillo 152 yahe 140 itigidi 133 nnewi 131 agulu 127 ehamufu 116 obosi 115 barracks rd. 104 north bank 104 umunya 102 table 2: very frequent outages (31-99) feeder outages achi 76 nnpc 76 nicuss 74 neni 73 neni 33 70 atani 70 amechi 69 isieke 67 ankpa 63 new nnpc 62 udi 62 army barracks 59 oju 47 govt house 42 wukari 40 katsina-ala 39 taraku 36 emene ind. layout 33 table 3: less frequent outages (10-30) feeder outages enugu-ukwu 27 ind.layout 26 ituku/ozalla 25 yandev 22 kingsway line2/9th mile 20 asaba 18 awada ii 18 emene 17 feeder 1 17 makurdi 16 water works 16 feeder 2 13 feeder 4 12 mobtr 10 table 4: occasional outages (3-9) feeder outages afikpo 9 golden oil 9 thinkers corner 8 aguleri 7 unn 7 feeder 3 6 kingsway line 1 5 ibagwa 4 nsukka 3 table 5: rare outages (1-2) feeder outages feeder 5 2 mobtr 2 2 mob 45 2 oji 2 agbor 1 bcc 1&ii 1 emene industrial 1 oji local 1 https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 65 (a) (c) (b) (d) (e) 0 5 10 15 20 25 30 en u g u -u kw u in d . l a yo u t it u ku /o za ll a ya n d ev a w a d a ii fe ed er 1 em en e m a ku rd i w a te r w o rk s fe ed er 2 fe ed er 4 m o bs tr less frequent outages (10-30) 0 2 4 6 8 10 occasional outages (3-9) 0 0,5 1 1,5 2 2,5 rare outages (1-2) 0 20 40 60 80 100 120 140 160 ez il lo ya h e it ig id i n n ew i a g u lu eh a m u … o bo si ba rr a … n o rt h … u m u n ya most frequent outages ( >100) 0 10 20 30 40 50 60 70 80 very frequent outages (31-99) https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 66 figure 3: charts showing the various frequency of outage of the cbs (a) most frequent outages (b) very frequent outages (c) less frequent outages (d) occasional outages (e) rare outages 5. reliability analysis according to anyaka b.o. (2012), some reliability indices were calculated from the data obtained such as: • mean time to repair (mttr) 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑂𝑂𝑂𝑂𝑇𝑇𝑇𝑇𝑂𝑂𝑂𝑂 𝐷𝐷𝑂𝑂𝐷𝐷𝑇𝑇𝑇𝑇𝐷𝐷𝑇𝑇𝐷𝐷 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑂𝑂𝑂𝑂𝑇𝑇𝑇𝑇𝑂𝑂𝑂𝑂𝑂𝑂 (1) • mean time between failures (mtbf) 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑂𝑂𝐷𝐷𝐷𝐷𝑇𝑇𝑃𝑃− 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑂𝑂𝑂𝑂𝑇𝑇𝑇𝑇𝑂𝑂𝑂𝑂 𝐷𝐷𝑂𝑂𝐷𝐷𝑇𝑇𝑇𝑇𝐷𝐷𝑇𝑇𝐷𝐷 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑂𝑂𝑂𝑂𝑇𝑇𝑇𝑇𝑂𝑂𝑂𝑂𝑂𝑂 = 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑂𝑂𝑂𝑂𝑂𝑂𝐷𝐷𝑇𝑇𝑇𝑇𝐷𝐷𝐷𝐷𝑂𝑂 𝑇𝑇𝐷𝐷𝑇𝑇𝑂𝑂 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑂𝑂𝑂𝑂𝑇𝑇𝑇𝑇𝑂𝑂𝑂𝑂𝑂𝑂 (2) • failure rate, λ λ = 1 𝑀𝑀𝑇𝑇𝑀𝑀𝑀𝑀 (3) • availability, a a = 𝑀𝑀𝑇𝑇𝑀𝑀𝑀𝑀 𝑀𝑀𝑇𝑇𝑀𝑀𝑀𝑀+𝑀𝑀𝑇𝑇𝑇𝑇𝑀𝑀 (4) it should be noted that the total period stands for the total time in consideration (i.e. 3 months = 2184 hours). after calculations, the results are shown in tables and graphs. table 6 and figure 4 shows most frequent outages reliability results. table 6. most frequent outage reliability results feeder outages duration mttr mtbf failure rate availability ezillo 152 711 4.68 9.69 0.1032 0.6743 yahe 140 544.43 3.89 11.71 0.0854 0.7506 itigidi 133 863.12 6.49 9.93 0.101 0.6048 nnewi 131 672.55 5.13 11.88 0.0842 0.6984 agulu 127 487.97 3.84 13.35 0.075 0.7766 ehamufu 116 357.3 3.08 15.75 0.0635 0.8364 obosi 115 621.29 5.4 13.59 0.0736 0.7156 barracks rd. 104 314.48 3.02 17.98 0.0556 0.8562 north bank 104 374.64 3.6 17.4 0.0575 0.8286 umunya 102 564.83 5.54 15.87 0.063 0.7412 https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 67 figure 4: availability and failure rate characteristic for most frequent outages. table 7 and figure 5 show very frequent outage reliability results. table 7: very frequent outage reliability results feeder outages duration mttr mtbf failure rate availability achi 76 534.68 7.04 21.7 0.0461 0.755 nnpc 76 170.85 2.25 26.49 0.0378 0.9217 nicuss 74 328.73 4.44 25.07 0.0399 0.8495 neni 73 408.04 5.59 24.33 0.0411 0.8132 33 70 571.7 8.17 23.03 0.0434 0.7381 atani 70 207.03 2.96 28.24 0.0354 0.9051 amechi 69 435.48 6.31 25.34 0.0395 0.8006 isieke 67 475.55 7.1 25.5 0.0392 0.7822 ankpa 63 175.12 2.78 31.89 0.0314 0.9198 new nnpc 62 89.29 1.44 33.79 0.0296 0.9591 udi 62 677.78 10.93 24.29 0.0412 0.6897 army barracks 59 194.7 3.3 33.72 0.0297 0.9109 oju 47 247.58 5.27 41.2 0.0243 0.8866 govt house 42 61.33 1.46 50.54 0.0198 0.9719 wukari 40 232.01 5.8 48.8 0.0205 0.8938 katsinaala 39 399.93 10.25 45.75 0.0219 0.817 taraku 36 254.88 7.08 53.59 0.0187 0.8833 emene ind. lay. 33 77.9 2.36 63.82 0.0157 0.9604 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 availability failure rate https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 68 figure 5: availability and failure rate characteristic for very frequent outages table 8 and figure 6 show the most frequent outages reliability results. table 8: less frequent outage reliability results feeder outages duration mttr mtbf failure rate availability enugu-ukwu 27 425.42 15.76 65.13 0.0154 0.8052 ind.layout 26 44.85 1.73 82.28 0.0122 0.9794 ituku/ozalla 25 106.53 4.26 83.1 0.012 0.9512 yandev 22 151.62 6.89 92.38 0.0108 0.9306 kingsway line2/9th mile 20 38.3 1.92 107.29 0.00932 0.9824 asaba 18 133.1 7.39 113.94 0.00878 0.9391 awada ii 18 21.17 1.18 120.16 0.00832 0.9903 emene 17 67.85 3.99 124.48 0.00803 0.9689 feeder 1 17 57.1 3.36 125.11 0.00799 0.9738 makurdi 16 44.8 2.8 133.7 0.00748 0.9795 water works 16 187.33 11.71 124.79 0.00801 0.9142 feeder 2 13 50.9 3.92 164.08 0.00609 0.9767 feeder 4 12 45.3 3.78 178.23 0.00561 0.9792 mobtr 10 6.63 0.66 217.74 0.00459 0.997 0 0,2 0,4 0,6 0,8 1 1,2 a ch i n n pc n ic u ss n en i 33 a ta n i a m ec h i is ie ke a n kp a n ew n n pc u d i a rm y ba rr a ck s o ju g o v t h o u se w u ka ri ka ts in a -a la ta ra ku em en e in d .… availability failure rate https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 69 figure 6: availability and failure rate characteristic for less frequent outages table 9 and figure 7 show the most frequent outages reliability results. table 9: occasional outage reliability results feeder outages duration mttr mtbf failure rate availability afikpo 9 193.2 21.47 221.2 0.00452 0.9115 golden oil 9 45.75 5.08 237.58 0.00421 0.9791 thinkers corner 8 18.15 2.27 270.73 0.00369 0.9917 aguleri 7 357.95 51.13 260.86 0.0038 0.8361 unn 7 56.95 8.14 303.86 0.00329 0.9739 feeder 3 6 16.68 2.78 361.22 0.00277 0.9924 kingsway line 1 5 3.83 0.766 436.03 0.00229 0.9982 ibagwa 4 26.2 6.55 539.45 0.00185 0.988 nsukka 3 91.72 30.57 697.43 0.00143 0.958 0 0,2 0,4 0,6 0,8 1 1,2 failure rate availability https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 70 figure 7: availability and failure rate characteristic for occasional outages table 10 and figure 8 show the most frequent outages reliability results. table 10: rare outage reliability results feeder outages duration mttr mtbf failure rate availability feeder 5 2 100.11 50.06 1041.95 0.00096 0.9542 mobtr 2 2 1.07 0.54 1091.47 0.000916 0.9995 mob 45 2 1.67 0.84 1091.17 0.000916 0.9992 oji 2 3.35 1.68 1090.33 0.000917 0.9985 agbor 1 4.75 4.75 2179.25 0.00046 0.9978 bcc 1&ii 1 37.73 37.73 2146.27 0.00047 0.9827 emene industrial 1 0.12 0.12 2183.88 0.000458 0.9999 oji local 1 1.93 1.93 2182.07 0.000458 0.9991 0 0,2 0,4 0,6 0,8 1 1,2 availability failure rate https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 71 figure 8: availability and failure rate characteristic for rare outages. 6. observations and conclusion from the reliability analysis carried out, the following observations are made: • in the most frequent outage result, we can observe low availabilities at itigidi and obosi feeders with corresponding high failure rates. • the very frequent outage result showed low availability values and high failure rates at achi, 33kv onitsha and udi feeders. • the feeder at enugu-ukwu has the lowest availability in the less frequent outage results. • aguleri cb has the highest failure rate in the occasional outage results • feeder 5 in asaba station has the lowest availability in the rare outage results. the availability of a system shows how reliable the system is. from our analysis, the high outages as a result of over-current and earth faults imply that the particular feeder is less reliable. by calculation, ezillo cb has the lowest availability value (0.6743) and emene industrial cb has the highest availability value (0.9999). hence, emene industrial cb has the highest reliability. however, this does not necessary mean that this feeder is the most reliable one because any cb can fail at any time due to some factors such as overloading, malfunction, weather conditions, human errors and so on. the earth-fault and over-current directional and inverse time relays should be employed in the power system to reduce the high outages due to faults on the system. 0 0,2 0,4 0,6 0,8 1 1,2 availability failure rate https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 72 compensation should also be done on areas with high loading to improve voltage profile and reactive power and hence increase transmission line load ability. acknowledgement: the authors are immensely grateful for the financial support from “african centre of excellence (ace-sped) university of nigeria, nsukka” to enable us to achieve the research (muncho j. mbunwe et al., 2019-2021). conflicts of interest: the authors declare no conflict of interest. references anyaka, b.o. (2012). reliability and maintainability of power system [lecture note]. http://engineering.unn.edu.ng circuit globe, (n.d.). circuit breaker, retrieved june 10, 2017, from open source: http://circuitglobe.com/circuit-breaker.html electrical concepts, circuit breaker and arc phenomenon, (n.d.). retrieved june 10, 2017, from open source: http://electricalbaba.com/circuit-breaker-and-arc-phenomenon/ electrical systems, (n.d.). retrieved june 10, 2017, from open source: http://skm-eleksys.com/2012/02/sf6circuit-breaker-working.html kezunovic, m. ren, z. latisko, g. sevcik, d.r. lucey, j.s. cook, w.e. and koch, e.a. (2005). automated monitoring and analysis of circuit breaker operation, ieee transactions on power delivery, 20(3), p. 1910. https://doi.org/10.1109/tpwrd.2005.848466 pinnekamp, f. (2007). the circuit breaker: a showcase of industrial product development, transmission group r&d and technology, abb, switzerland, p.31. https://new.abb.com/docs/defaultsource/technology/brosch%c3%bcre-chchrc2017final-einseitig-(002).pdf?sfvrsn=e3f37d14_2 saxena, s. singh, a. ali, m. gandhi, k. (2012). various types of circuit breakers used in power system for smooth working of the transmission line, mit international journal of electrical and instrumentation engineering, 2(2), p. 106. https://www.semanticscholar.org/paper/06918326da2b2ac6fd80620d88961b68554b72ed suwanasri, t. hlaing, m.t. and suwanasri, c. (2014). failure rate analysis of power circuit breaker in high voltage substation, gmsarn international journal, 8(1), 1-6. http://gmsarnjournal.com/home/journal-vol/journal-vol-8-no-1/ https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://engineering.unn.edu.ng/ http://circuitglobe.com/circuit-breaker.html http://electricalbaba.com/circuit-breaker-and-arc-phenomenon/ http://skm-eleksys.com/2012/02/sf6-circuit-breaker-working.html http://skm-eleksys.com/2012/02/sf6-circuit-breaker-working.html https://doi.org/10.1109/tpwrd.2005.848466 https://new.abb.com/docs/default-source/technology/brosch%c3%bcre-chchrc2017final-einseitig-(002).pdf?sfvrsn=e3f37d14_2 https://new.abb.com/docs/default-source/technology/brosch%c3%bcre-chchrc2017final-einseitig-(002).pdf?sfvrsn=e3f37d14_2 https://www.semanticscholar.org/paper/06918326da2b2ac6fd80620d88961b68554b72ed http://gmsarnjournal.com/home/journal-vol/journal-vol-8-no-1/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 madueme et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 60-73. https://doi.org/10.4995/muse.2021.12406 73 transmission company of nigeria. (2016). tripping reports april – june 2016 [unpublished raw data]. new haven, enugu, nigeria. https://www.tcn.org.ng/page_history.php muncho j. mbunwe, m. ayaz ahmad, syed khalid mustafa. (2020). an effective energy saving design strategy to maximize the use of electricity, j. math. comput. sci., 10(5), 1808-1833, https://doi.org/10.28919/jmcs/4775 mbunwe muncho j., ezema ejiofor e., ngwu anene a., c v anghel drugarin, m. rehan ajmal, m ayaz ahmad. (2021). characterization of three phase solid state var compensation scheme in three phase pulse width modulation voltage source inverter, journal of physics: conference series, vol. 1781, 012034, https://doi.org/10.1088/1742-6596/1781/1/012034 m. ayaz ahmad, irina tvoroshenko, jalal hasan baker, vyacheslav lyashenko. (2019). modeling the structure of intellectual means of decision-making using a system oriented nfo approach, international journal of emerging trends in engineering research , vol. 7(11), 460-465. https://doi.org/10.30534/ijeter/2019/107112019 mykhailo kopot, m. ayaz ahmad, vyacheslav lyashenko, syed khalid mustafa. (2020). prospects for creating sub-millimeter magnetrons, international journal of advanced trends in computer science and engineering, vol. 9(4), 6184-6188. https://doi.org/10.30534/ijatcse/2020/294942020 m. ayaz ahmad, irina tvoroshenko, jalal hasan baker, vyacheslav lyashenko. (2019). computational complexity of the accessory function setting mechanism in fuzzy intellectual systems, international journal of advanced trends in computer science and engineering, vol. 8(5), 23702377. https://doi.org/10.30534/ijatcse/2019/77852019 muncho j. mbunwe, udochukwu b. akuru, hilary u. ezea, ogbonnaya i. okoro, m. ayaz ahmad. (2020). some aspects of future energy generation in using of solar power satellites, int. j. anal. appl., 18 (1), 117-128. https://doi.org/10.28924/2291-8639-18-2020-117 https://doi.org/10.4995/muse.2021.xxx http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.tcn.org.ng/page_history.php https://doi.org/10.1088/1742-6596/1781/1/012034 https://doi.org/10.30534/ijeter/2019/107112019 https://doi.org/10.30534/ijatcse/2020/294942020 https://doi.org/10.30534/ijatcse/2019/77852019 https://doi.org/10.28924/2291-8639-18-2020-117 1. introduction 2. types of circuit breaker 3. arc phenomenon in circuit breaker evaluación de diferentes diseños de aprendizaje basado en proyectos en el entorno de un máster profesional multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 169 evaluation of different project based learning designs in an msc degree carmen garcía-berdonés, luis molina-tanco, juan pedro peña-martín, francisco garcía-lagos, gonzalo joya-caparrós, arcadio reyes-lecuona, francisco david trujillo-aguilera universidad de málaga, departamento de tecnología electrónica telf.: 952137164; fax: 952131447; berdones@uma.es received: 2013-12-15; accepted: 2014-01-28 abstract the design and implementation of different project based learning (pbl) approaches are presented in this paper. all of them were carried out in the framework of the msc degree in electronic systems for smart environments from the university of malaga. four subjects were developed using different values of the three main parameters of pbl: teamwork, self-guided learning and project complexity. during two academic years, several indicators were used to evaluate these experiences: compliance with subject time schedules, scores obtained for the students, interaction of each student in his team and satisfaction of students with the experiences. our results encourage the use of pbl in bachelor degrees but, at the same time, confirm that pbl implementation is not a trivial task when projects are complex or when a high level of autonomous learning is required from students. teamwork difficulties have also been found. so, we discuss the need of reaching a minimum level of proficiency in some key competencies before using pbl. keywords project based learning, teamwork, self-guided learning, msc degree. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 170 1. introduction convergence towards the european higher education area has led students and instructors from teacher-centered to learner-centered approaches (european communities 2009), such as problem-based learning (pbl). however if biggs’ 3p model (biggs 1993) is assumed, new teaching strategies –especially those not thoroughly experimented– should be adopted with precaution. in the 3p model, outlined in figure 1, the students’ presage variables have influence both in the learning process and in the learning product. but product and process have also impact on presage variables: what students experience in current learning processes will have an influence on how future learning is confronted. careless design or application of a teachinglearning method can compromise not only learning outcome, but also the students’ future attitude towards the method. figure 1. biggs’ 3p model multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 171 this work was motivated by the will to experiment pbl, a strategy that was new to the authors. the aforementioned precautions led them to choose a context with few, highly motivated students. specific courses within the msc degree in electronic systems for smart environments 1 (messe) from the university of malaga were chosen to put the method in practice. these are described in section 2. 1.1. parameters for pbl design and evaluation in pbl, learning is structured around a problem or a project as similar as possible to those encountered in professional settings. the students solve the problem or realize the project, which is formulated at the beginning of the teaching-learning process. aside from the guiding project, the fundamental ingredients of pbl are individual selfdirected learning and group work (chikotas 2008). successful application of pbl has been extensively described (fernández martínez 2006), although some authors have reported issues with acquisition (perrenet 2000) and evaluation (dym 2005) of specific skills, and with the necessary change of mindset required from students and from teachers (spronken 2009). this work will also look at issues that arise in the application of pbl when several design variants are considered. let us start by considering teacher intervention. in its more radical form, which we can call ‘pure’ pbl, traditional lectures are not used. students self-organize their own learning, which is driven by the need to solve the guiding problem. the teacher’s role is that of moderator and partner in the process. in a less radical approach, the teacher guides students towards the solution more or less explicitly (savery 2006). lectures might be used, for instance, but only after students have worked on the problem. 1 http://www.masterseeiuma.es/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 172 according to savery (2006), teacher intervention should be minimized if the students are to build their own learning, which is something that the ects system of the european higher education area seems to promote (european communities 2009). however, development of technical skills in the curriculum is also expected, especially those linked to regulated professions (boletín oficial del estado 2009). this in practise means that specific subjects must be covered, and a compromise must be made between pbl-style self-guided learning, and more traditional teacher-guided activities. previous work has put pbl in practice at different levels, from experiences limited to a single subject (garcía almiñana 2006) to degree-wide implementations (enemark 1994). application at wider levels allows formulation of more complex problems, which is in line with a ´pure´ pbl approach. however, this requires a high level of coordination among several teachers, which can be a challenge. finally pbl, which is rooted in social constructivism (chikotas 2008), requires from students a high degree of cooperation. thus, the challenges of teamwork are also present in pbl. an overview of these challenges and possible approaches to successful group work is given in felder (1994). the main parameters for pbl design and evaluation are depicted in figure 2. elements described in this section are shown as input parameters. the teacher must choose a specific level of each one for a specific pbl design, taking into account the technical skills of the student before the pbl activities take place. the outputs that should be observed to assess the successfulness of the process are shown too. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 173 figure 2. parameters for pbl design and evaluation 1.2. objective and phases of work we used and evaluated some of the variants of pbl described in the previous section, within the scope of the messe, in order to investigate the applicability of the methodology and, where appropriate, to extrapolate the results to the new academic degrees. it should be clarified that it was not our objective to test the hypothesis that pbl improves the quality of teaching because we are aware of the many uncontrollable variables (student profile, thematic subjects, faculty profiles, etc.) that would prevent us from drawing any conclusion about this regard. this work was carried out during two academic years under the framework of an educational innovation project at the university of malaga. moreover, this project also proposed a basic development of entrepreneurial competency in the same msc degree. specific details regarding its design and evaluation can be found in garcía-berdonés (2012), although this competency will also be mentioned here because of its use of pbl. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 174 to achieve the objectives of the project, two working phases were proposed in a cycle of design-test-redesign-test. thus, in phase i, corresponding to the 2010-11 academic course, the learning activities were designed and the results were analysed. phase ii repeated the process for the 2011-12 academic course, incorporating the redesigns advised by the results of the previous phase. the organization of this document is as follow. in the next section, there will be a brief description of the university degree and the subjects involved in this work. the following two sections describe the phases of the project. finally, the conclusions and future directions for this work are presented. 2. university degree, subjects and students involved in the study the messe is an official msc with 60 ects, offered by the telecommunication engineering school at the university of malaga. its goal is to train students in how to plan and develop electronic systems that are usually part of so-called smart environments. competences related with user-centred design and data processing techniques, both useful in this type of environments, are also developed. the messe began to be taught in the 2009-10 academic year. therefore, the work presented here was carried out during the second and third editions, affecting the teaching of 15 and 8 enrolled students, respectively, in each of the editions. all the subjects involved in this work are compulsory and they are described below. methodology for planning, management and development of projects (met), with six ects, aims to familiarize students with the work methodologies for project management, with a focus on quality. also, this subject fosters an entrepreneurial competency. pbl will be used to achieve both goals (met.quality and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 175 met.entrepreneurs). user interfaces (ui), with three ects, proposes that students are able to analyse and evaluate the usability and accessibility of a user interface. pbl is applied about usability. advanced information processing techniques (aipt), with three ects, aims to introduce students to computational intelligence techniques that are useful in the design of intelligent environments. pbl is used to help students to delimit the applicability of each technique. finally, master´s degree thesis (mt) is a twelve ects subject that intends the student to plan, design and implement an electronic prototype for a smart environment. master´s degree thesis is considered as a project. therefore, its design and planning is proposed as the guide project for the pbl associated with met.quality. 3. design, implementation and results of phase i (academic year 2010/2011) 3.1. design of learning activities and indicators for each of the learning experiences, table 1 summarizes its pbl design parameters, which are detailed below, and the main planned features for group activities, following felder (1994). regarding the project, in met.entrepreneurs students are invited to prepare a business plan for an idea related to the subject of messe. this plan has to be exposed to a guest entrepreneur by the students during the course. in met.quality, students aim to design and plan their mt, which should begin immediately after the end the course. aipt raise the solution of four different problems that mimic/simulate actual problems. students have to solve these problems using several computational intelligence techniques. ui intends to evaluate the usability of a remote control for a hotel room. the students multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 176 should design the control system before they know the usability principles that will be explained later. the mt subject was not involved in this first phase. the degree of self-direct learning that will be required to the students will depends of the scheduled time to develop each competency as well as the specific skills that students will need to complete each pbl. the contents associated with met.entrepreneurs are very new to the student, and due to limited time, students will afford the design process in a guided way, with lectures at the beginning of most of the programmed activities. although autonomous work is required to complete this activities (students need to read texts and to find information), this work is guided (preselection of texts, using templates). meanwhile, ui has a constructivist approach which takes advantage of the student preconceptions about usability, criticized in successive sessions and in a final discussion in group. again, due to time constraints, students receive lectures that guide their reviews. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 177 table 1. main characteristics of the four pbl experiences met. entrepre. met.quality aipt ui duration (ects) 1,5 4,5 3 1,5 teamwork activities 3 6 4 2 dependency among activities sequential sequential selfcontained ---- number of activities that begin with lectures 2 0 0 1 level of complexity of the project average high average average level of self-guided learning low average high low level of teamwork high high high low teamwork: number of team members 3-4 3-4 2-3 3-4 teamwork: team formation random assigned mt random random teamwork: positive interdependence high void void high teamwork: individual accountability low low low low teamwork: face to face interaction average average average average teamwork: operating reflection void void low void teamwork: appropriate use of collaborative skills documen tation 1 guided session void 1 guided session weight in the final score: teamwork activities / individual examination 20%/10% 30%/40% 70%/30% 0/100% the contents of aipt are also novel, but it is estimated that there is enough time to follow the philosophy first problem then lecture. thus we decided to provide students with little theoretical basis before facing each problem. with respect to the previous knowledge of the student about a particular problem, met.quality is between ui and aipt, and it proposes a design with the same philosophy of aipt, but with a single, longer project. teachers guide students in the realization of this project, dividing it into four blocks of linked activities. the output of a block is needed to start the next. each multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 178 block consists of three phases. in the first phase, the students must work autonomously to develop a solution to the problem, starting from a base material that is provided by teachers. in the second phase, a group discussion is performed where each student presents his or her particular solution. the ultimate aim of this phase is to agree on a joint solution. in the last phase the students present and discuss in class all the solutions developed by the groups. the teachers have previously reviewed these solutions. between phases 2 and 3, some lectures are included in order to complement the initial provided information. regarding teamwork, the teams at aipt and met worked during the whole course, whereas at ui they only worked two days: one for preparing the proposal at the beginning of the course, and other one, at the end, devoted to the concluding debate. the work posed to teams at met.entrepreneurs and ui forced a distribution of tasks among its members, while in the other two cases all teams’ members worked together on a common subject. in all cases, a classroom time was reserved to carry out the group activities, thus ensuring face-to-face interaction. pressure for completing the syllabus of course caused that no time was scheduled either to reflect on the progress of the team works or to learn new techniques to improve their performance. finally, the following indicators were used to assess the performance of each pbl: degree of compliance with their respective schedules (output time in figure 2); scores obtained by the students (output learning outcomes in figure 2), and the degree of satisfaction of students (output satisfaction in figure 2) collected through individualized surveys carried out by their assigned tutors. information on teams´ operation was also considered because of their importance for a successful pbl. this indicator was obtained by observation of team classroom activity and, in some cases, analysing team web forums. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 179 3.2. results and analysis to redesign all activities were performed in accordance with their scheduling, except at met.quality. from the second block of activities, the discussion phase was not held in half of the teams, due to their members having reached dissimilar level of learning. thus, each team performed this task at a different pace, and some of them had to do a substantial amount of work outside the classroom. this work was added to the individual tasks, necessary for the next block. the result was that students definitely quit these delayed tasks, prioritizing the final team works (which are considered for the final score). furthermore, the lectures were lengthened more than planned, which resulted in a decrease in time available for teamwork into the classroom. all these circumstances, in addition to the pressure for covering the syllabus of the course, made the previous study and discussion to be replaced by lectures in the third block (at least, that led, as a positive effect, to the synchronization of learning among the class-team). the phase of oral presentation was also removed and substituted by teacher feedback on the written deliverables. no major problems were registered in the teams' operation, except for met.quality, where two extreme problems occurred: on the one hand, a group was disengaged from the rest, delivering the task outdated and signed by only one or two of its members; on the other hand, the most active group was blocked by clashes between its leaders. both issues were successfully overcome because the rating does not solely rest on the group’s tasks (see table 1). the use of the forums was very polarized: some groups used them very intensely while the rest seldom used them. the collection of information about the degree of satisfaction with the development of subjects among students had not worked well enough due to coordination problems between tutors. however, we could detect a certain degree of demotivation with the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 180 subject met and some complaints about the timing of activities in several subjects, producing punctual, but relatively frequent, peaks of workload. regarding student scores, all students successfully overcame subjects, although a significant dispersion among scores of members of the same group was observed. 4. redesign, implementation and results of phase ii (academic year 2011/2012) regarding pbl designs, no changes were included excepting in pm.quaity, where pbl was abandoned, moving that philosophy to the master´s degree thesis subject. students are allowed to work as a team in a common part of their master thesis: the design and implementation of a basic prototype, leaving as individual work to make some improvements to that prototype. nevertheless, students could do the whole work alone if they wanted. in addition, teams did not share a common schedule anymore. a timetable was established for students to work (individually or as a team) supervised by their professor, who only advised them about the way to follow depending on their previous approaches. furthermore, some activities were included in order to increase student motivation in met (e.g. analysis of an invented story about a failure due to lack of quality management) and subject timing was changed in order to improve the coordination among them and minimise peak workload reported by students. in phase ii, the same indicators than in phase i were used, together with a web survey and a structured interview to get satisfaction of students with the subjects and teamwork. also, a questionnaire about roles within the team was also adapted and administered (gómez mujica 2003). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 181 regarding results, all the subjects were on schedule, but most of students complained about met.entrepreneurship because of lack of time to do the proposed activities. students did not blame this to the work demanded by other subjects. therefore, it can be said that, firstly, coordination among subjects has been improved thanks to structural changes carried on and, secondly, that keeping the self-learning level (not increasing autonomous activities) was a right decision, because it avoided problems with the schedule. this academic year, students also succeeded in all the subjects, although a high deviation among individual rates of component of each team was again observed. the degree of satisfaction with the subjects is high, but comparable with the rest of subjects not involved in this project. analysing results in more detail, no problems were detected with group performance in the classroom. out of it, students reported as a drawback member's availability, especially for those students who combined their studies in messe with other professional activities. nevertheless, these problems were not blocking, as students reported a high degree of satisfaction with teamwork as well. finally, student’s profiles for teamwork roles was very homogenous among both, students and roles. hence, there was no useful information there. summarizing, involved subjects obtained very similar results in both years, excepting pm.quality, where significant problems were not detected during the second year. for the master thesis subject, it was observed that students worked at very different paces and, although they worked as a team at the beginning, they gradually changed to work individually, due to problems with availability, as reported themselves. only one group kept as a team until the end and they reported in both, their master thesis dissertation and defence, that teamwork was very useful. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 182 5. conclusions, considerations and forthcoming works firstly, it is very important to notice that the implemented pbl designs have been, in general, conservative in the sense that none of them was compliant with the features of a pure pbl, as it is described in the introduction. the less conservative experience, met.quality, has been the more difficult one and, finally, professors were headed toward the withdrawal of the planned design, at the middle of the first year. nevertheless, this experience allowed us to see a possible problem in our design: the attempt to synchronize randomly composed teams during a long period. it is possible that students do not work together suitably, or even if they do it, their learning-process work-rate was different. the obtained results with the master`s degree thesis subject confirm this idea. moreover, it is important to show that there are also several experiences with less problems than the met.quality subject, the ui and aipt ones. these subjects have achieved an important simplification in their design related to the project complexity, the required self-guided learning or the demanded teamwork. in the same way, the time problems in the met.entrepreneurship subject probably could have been reduced, if we had used a more conservative pbl design with a more guided experience. the project chosen for met.quality was not sufficiently motivating. furthermore, met.quality is a low-tech subject, and thus the students’ motivation is a priori low. the motivation mechanisms developed during the second year, rather unrelated to pbl, produced better results, although it was impossible to finish them because of the students’ different profiles. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 183 finally, the obtained dispersions within the scores of member of each group show that the developed evaluation mechanism is not suitable for this matter; see garcía berdonés (2012) for more details. taking into account all of the above, it would be better to develop, in the bachelor degrees, a set of experiences with very relaxed requirements in, at least, one of the input parameters in the pbl design; or, even better, within the range of some motivating subject for the students, without taking into account the pbl as a motivating element. in addition, as it was described in the introduction, the design stage may be implemented with special emphasis. about this matter, some considerations will be shown below. problems that students have reported (difficulties to synchronize the activities of groups and failures observed in some of them) could be explained, at least in part, by the diversity of the students’ preparation about several factors: prior knowledge, self-guided learning capacity or ability to work efficiently within a group (i.e., the teamwork competency). it is convenient to remark that, from the social constructivism (chikotas 2008), pbl proposed working group learning (teaching methodology) and not learning to work in groups (competency development). therefore, although pbl is often described as a teaching methodology that develops generic skills of self-guided learning and teamwork, in our opinion, it is more accurate to say that pbl uses the level the student has in both competencies, and increases that level. returning to the biggs’ model presented in the introduction, we can venture that if the pbl design does not conform to the level that students have in both competencies, they may develop animosity towards them, rather than a higher level of domain. it would be a way to acquire the so-called accidental incompetency in walther (2007). this does not mean at all that with pbl it is not possible to develop generic competencies. rather, it means that the pbl design multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 184 requires a careful thinking about all the students’ capabilities, and not only on specific competencies, as usual. in this sense, this conclusion is fully consistent with felder (2005). figure 3 depict these reflections by extending the figure 2. figure 3. new parameters for pbl design and evaluation the experiences presented here lead us to think about some questions that go beyond pbl. if generic competencies are necessary to use pbl (or to observe law requirements or to carry on a successful professional activities), would we know how to develop them? in addition, would we be willing to invest time in it at the expense of subtracting time from specific competencies? our experience cannot answer these questions because in all our designs, the specific agenda has dominated and almost no time has been reserved for generic competencies training. in our opinion, to answer these questions is a very interesting line of future work. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 185 acknowledgments this work has been partially support by an educational innovation project (pie10-80) of university of málaga. references biggs j. (1993). from theory to practice: a cognitive systems approach. higher education research and development. 12(1), 73-86. doi: 10.1080/0729436930120107. boletín oficial del estado (2009). orden cin/352/2009. 44, 18150-18156. chikotas n. (2008). theoretical links: supporting the use of problem-based learning in the education of the nurse practitioner. nursing education perspectives. 29(6), 359-362. dym c. l., agogino a. m., eris o., frey d. d. and leifer l. j.(2005) engineering design thinking, teaching, and learning. journal on engineering education. 94(1), 103120. enemark s. and kjaersdam f. (1994). the aalborg experiment: project innovation in university education. aalborg: aalborg university press. european communities (2009), ects users’ guide http://ec.europa.eu/education/lifelong-learning-policy/doc/ects/guide_en.pdf accessed 28 december 2013 felder r. and brent r. (1994). cooperative learning in technical courses: procedures, pitfalls, and payoffs. eric document reproduction service report ed 377038. felder r. and brent r. (2005). understanding student differences. journal of engineering education. 94(1), 57-72. doi: 10.1002/j.2168-9830.2005.tb00829.x fernández martínez m., garcía sánchez j., caso fuertes a., fidalgo redondo r. and arias gundín o. (2006). el aprendizaje basado en problemas: revisión de estudios empíricos internacionales. revista de educación. 340, 397-418. http://ec.europa.eu/education/lifelong-learning-policy/doc/ects/guide_en.pdf%20accessed%2028%20december%202013 http://ec.europa.eu/education/lifelong-learning-policy/doc/ects/guide_en.pdf%20accessed%2028%20december%202013 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2201 social and technological sciences garcía-berdonés et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 169-186 | 186 garcía almiñana d. and amante garcía b. (2006), algunas experiencias de aplicación del aprendizaje cooperativo y del aprendizaje basado en proyectos. actas de jornadas de innovación educativa. zaragoza, spain. garcía berdonés c., peña martín j. p., molina tanco l., cavas toledo m., chicano garcía f. and luna valero f. (2012), aprendizaje basado en proyectos para desarrollo de la competencia emprendedora en un máster de electrónica. actas de iii jornadas sobre innovación docente y adaptación al eees en las titulaciones técnicas, granada, spain. gómez mujica, a. and acosta rodríguez, h. (2003). acerca del trabajo en grupos o equipos. acimed, 11(6) perrenet j.c., al bouhuijs p. and smits j. g. m. (2000). the suitability of problembased learning for engineering education: theory and practice. teaching in higher education. 5(3), 345-358. savery j. r. (2006). overview of problem-based learning: definitions and distinctions. interdisciplinary journal of problem-based learning. 1(1), 9-20. doi: 10.7771/15415015.1002 spronken-smith r. and harland t. (2009). learning to teach with problem-based learning. active learning in higher education. 10(2), 138-153. doi: 10.1177/1469787409104787 walther j. and radcliffe d. (2007). the competence dilemma in engineering education: moving beyond simple graduate attribute mapping. australasian journal of engineering education. 13(1), 41-51. doi: 10.1080/07294360.2012.704594. http://dx.doi.org/10.7771/1541-5015.1002 http://dx.doi.org/10.7771/1541-5015.1002 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 53 a gymkhana to discover and generate curves in a cooperative work anna vidal meló, avidal@mat.upv.es bernardino roig sala, broig@mat.upv.es vicente d. estruch fuster, vdestruc@mat.upv.es francisco j. boigues planes, fraboipl@mat.upv.es departamento de matemática aplicada, escola politècnica superior de gandia, c/ paranimf nº1, 47730-grau de gandia, valencia, 962849333, 962849309 received: 2013-12-15; accepted: 2014-01-28 abstract the student who access the university understands a curve as a function represented by an equality involving the cartesian variables x and y. but he does not know another great set of curves such as those described by parametric equations and polar coordinates. we show an experience developed in two consecutive courses in the first course of a bachelor's degree, which allows the students to discover and create many curves in parametric or polar representation, working in group. also the experience facilitate the students to achieve several generic skills. keywords mathematics, parametric curves, polar curves, general competencies. mailto:avidal@mat.upv.es mailto:broig@mat.upv.es mailto:vdestruc@mat.upv.es mailto:fraboipl@mat.upv.es http://www.upv.es/entidades/epsg/indexi.html multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 54 1. introduction in this work we describe the planning, development and results obtained from computer practices for studying parametric and polar curves. this experience has been developed during the academic courses 2011-2012 and 2012-2013 in the mathematics 2 subject corresponding to the bachelor's degree in telecommunications systems, sound and image engineering (gistsi) in the gandia campus of the universitat politècnica de valència (upv). this practices has permitted to add to the subject some general and transversal skills like teamwork, information management, use of information and communication technologies (tics) and time management (including responsibility and work planning). 1.1. the subject mathematics 2 mathematics 2 is a first year annual subject with 9 ects, distributed in such a way that 4.6 ects are devoted to theory lessons and classroom practices, 3 ects are devoted to seminary lessons and 1.4 ects are devoted to computer practices. the students can consult the teaching guide and obtain the resources of the subject in the upv poliformat platform (see figure 1). figure 1. poliformat platform multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 55 the subject program has the following units: indefinite integral and applications to the resolution of ordinary differential equations; definite integrals and improper integrals; numerical series; power series and series development; study of functions of several variables; optimization of functions of several variables and double and triple integrals. seven two hours practical computer sessions are done using matlab © (alamar 2008): 1. introduction to matlab © and mathematical applications. 2. geometry of curves: parametric and polar curves. 3. interpolation and curve fitting. 4. numeric integration. 5. numerical methods for ordinary differential equations. 6. graphical representation of functions of two independent variables, f(x,y), and its applications. 7. parametric surfaces. practices 3, 4, and 5 tackle another fundamental unit of the subject, the numerical analysis. in figure 2 interrelation between practices and theory units is shown. it can be seen how practice 2 precedes the definite integral theory unit. thus, thanks to the previous work on practice 2, the student will be able to calculate areas and volumes of solids of revolution generated from parametric or polar curves. at the end of each practice the students are required to perform a project in group (2-3 students), that they have to upload in the corresponding assignment application, established by the professor, in the poliformat platform. in the subject’s mark, 20% corresponds to the work done on the computer practices (project practices), 70% corresponds to various partial tests of theory and problems, and 10% corresponds to the portfolio containing the works performed by the student in seminaries and at home. although the realization of the practices is voluntary, students are informed about the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 56 advantage of its realization: a quarter of the students pass thanks to the practices valuation according to the analysis of the results of previous courses. figure 2. interrelation between theory and computer practices. 1.2. general skills in higher education general skills are the part of the graduate’s profile that prepare them as professionals and citizens, and have to be developed along their university studies (blanco 2009). in a study done in some universities in the united kingdom and australia (fallows 2000) is concluded that there are five of general skills which have to be developed at the university: general communication abilities; information management (search, selection, analysis and evaluation of the information coming from various sources); ability for the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 57 use of new technologies (tic utilization); teamwork, ethics and diversity recognition; personal competencies (time management, responsibility, planning and others). in section 3 of cin/352/2009 decree, which establishes the requirements for the verification of official university degree that enables to work as technical telecommunications engineer in spain, appear the competencies that the students have to acquire. we highlight here three of them: knowledge of basic matters and technologies (which enables them for the learning of new methods and technologies, as well as giving them a big versatility to adapt to new situations); problem solving skills (initiative for decision making, creativity and capability for the communication and transmission of knowledge); and finally, ability to work in a multidisciplinary group in a multilingual environment and to communicate, written and orally, knowledge, procedures, results and ideas related with telecommunication and electronics. there is no doubt that the general skills will help the university students, particularly the future technical telecoms engineers, for getting the specific skills required to practice. 1.2.1. teamwork in the computational practice described, the students have done a project in groups of 2-3 people. further each group collaborates with the others as a part of a whole for achieving a result or a final product, obtained from the aggregation of each part. the final product is a powerpoint © that includes a big number of graphics. the job of each team is a part of a global powerpoint. moreover the student needs to share learning experiences both from the partners and the professors, which requires the development of teamwork ability (blanco 2009). teamwork favors learning and helps the student in acquiring other competencies that he could not get alone. taking into account the concept of interaction, there are three types of groups: group with competitive interaction (the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 58 student only achieves the goals if his group achieves them), group with individualist interaction (the students are rewarded for the quality of their individual work) and group with cooperative interaction (the aims of the student body are interconnected, and he assumes a positive interrelation and the consecution of the team goals). the benefits of cooperation are the improvement of the students’ results, much more than those coming from individual work and higher productivity. the learning results improve when sharing objectives and working together. and this has been our bet: a cooperative interaction between teams to improve individual learning outcomes. 1.2.2. information search previously to the beginning of the academic course, in the welcome days (jornadas de acogida, jjaa), the new students have a workshop in the computer room. the workshop content includes the library, its resources and the information search. but when the course is started, the student tend to focus only in the lesson notes or in the slides that the professor have uploaded to the poliformat platform. although the teaching guide proposes a complete and varied literature, at most, only few students use the notebook of the subject. to develop skills in information search, in advance of the realization of the practice, the student has to find three equations of curves in books located in the university library. this is intended to force the student to use what he learned in the jjaa workshop localizing some texts from the teaching guide, visiting the library, manipulating texts and thus, checking that there are other books to complete the subject learning. as we indicate at the end of this paper, some students have referenced this fact in the assessment of the practice. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 59 1.2.3. use of tic in the jjaa, another workshop related with the upv intranet and the platform poliformat is performed. few weeks later students generally are familiar with poliformat and with the university e-mail but little more. however the use of tic includes also file management, document editing, surfing the internet, making presentations,… as part of the work, each team is required to provide the matlab © commands used and the powerpoint slides, in a specific format (see figure 3) available for the students in poliformat. figure 3. power point © pattern. the students have to capture the graphic representation from matlab © and adapt it to the required format, respecting the dimensions and indicating the equations and the name of the authors. later, the teacher generates a presentation with all the slides provided by the different teams, deleting the incorrect ones, classifying the rest and leaving it available in poliformat, so that every participant can see the end-product. moreover, during the practice session web resources have also been used. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 60 during the course 2012-2013 the web tool issuu has been used. it is an online service that allows displaying digitalized material as if it were a printed publication, with a format that allows visualizing two pages at the same time (like an open book or magazine) and an animated page turning (vidal 2013). 1.2.4. time management and planning the poliformat assignments, corresponding to the practice project, have a prefixed time delivery. students have two weeks from the realization of the practice for uploading the work to poliformat. only works uploaded in time are taken into account. so, each group has to plan and manage the time for completing and delivering the practice project. 2. experience development math class students of high school have studied functions f(x) and their graphs as the locus of the points (x,y) of the plane in which x belongs to the domain of definition of the function and y=f(x). they have studied straights, parabolas, conics, trigonometric functions, exponential and logarithmic functions and some others, but always through their cartesian expressions, y = f(x), and they have calculated areas determined by the graphics of the curves. however, for describing the trajectory in the plane (or in the 3dspace) of an object in many physical applications classical cartesian coordinates are not usual and parametric equations are used, where the parameter is the time t. for example, for the computation of integrals on a curve which lead to know the movement of a vector field along the curve, or for the de termination of the mass taking into account density, the parametric expression of the curve is required. on the other side there are curves that are difficult to be described by cartesian equations and the utilization of polar coordinates facilitate much their representation and study. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 61 2.1. initial planning: a mathematical gymkhana before the realization of the practice, and in order that the students search the equations of the curves to be studied in the library, it was raised to the students a kind of gymkhana (see figure 4). in the gymkhana the students had to write down the equations of three different curves, some of them in parametric equations and others in polar form, which they must find in the proposed bibliography. preparation of the gymkhana is laborious for the teacher that needs to consult and localize in the library the books where the equations appear (bradley 1998; edwards 1996; larson 2006, 2010; stewart 1999, 2003; 2005; thomas 1999). another work of the teacher is to select and organize all the curves in groups or gymkhanas of 3 curves. each course new curves are added to the collection. now 40 gymkhanas are available and the curve selection is expanded to the following web pages: http://temasmatematicos.uniandes.edu.co/trocoides http://departamento.us.es/edan/php/asig/licfis/lfipc/tema3fispc0809.pdf http://elmisantropodigital.blogspot.com.es/2012/09/mas-composiciones.html http://rectas.wordpress.com/category/pov-ray/ the gymkhana distribution among students was done in the last minutes of the previous practice. http://temasmatematicos.uniandes.edu.co/trocoides http://departamento.us.es/edan/php/asig/licfis/lfipc/tema3fispc0809.pdf http://elmisantropodigital.blogspot.com.es/2012/09/mas-composiciones.html http://rectas.wordpress.com/category/pov-ray/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 62 figure 4. sample of one of the gymkhanas. 2.2. methodology part: practice development the whole set of first year students is divided in four groups of 20 students to realize the practices. the scripts of the practices (alamar 2008) are available for the students. in the scripts, theory concepts alternate with practice called get down to work (manos a la obra¸mao). students have to work with matlab © . the professor explains the theory contents of the practice at the beginning of the session leaving some time so that the students work on the mao sections. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 63 2.2.1. script of practice: curves geometry (curves in parametric and polar equations) the structure of the script is as follows: 1. operations step by step with matlab © : operations with vectors and/or matrices which do the same computation on each of its components. 2. curves in parametric equations: the notation used is described the bezier’s curves used in shape design, automotive industry and also to identify letters’ shapes and other symbols in laser printers are cited as a curiosity. some differential geometry concepts are included: velocity vector, acceleration vector, t(t), n(t) and b(t) vectors of the movable reference system known as frênet-serret trihedral and the normal plane determined by n(t) and b(t). the last ones are essential for generating tube-shaped surfaces around a curve. finally, it is defined the arc length function, and the curve’s length. 3. some parametrizations of interest: lines, circumferences, ellipses. 4. representing graphs of curves given in parametric coordinates in matlab © : in the corresponding mao a circumference is represented, and the student is asked to modify the commands to represent a spiral, a cylindrical helix and a conical helix. 5. plane curves in polar coordinates and their representation with matlab © . 6. some curious parametric curves: , t ∈ a,b[ ]. ) ) ) z = z(t y = y(t x = x(t      multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 64 6.1. cycloids, epicycloids, hypocycloids, trochoids, epitrochoids and hypotrochoids: through the web page http://temasmatematicos.uniandes.edu.co/trocoides directed by aquiles páramo fonseca, professor in the mathematical department of the university of los andes. the student “knows what these wonderful curves are, which are their parametric equations and their most notable properties”, as there is indicated. the web page includes figures and animated programs developed with java © by the author, so that the almost inexhaustible universe of these curves can be explored. finally the student would be able to have fun with the spirograph in the same web page, a program to make attractive graphic designs (see figure 5). figure 5. spirograph for the geometric drawing of some hypothrochoids. 6.2. lissajous’ curves: appear commonly in oscilloscopes. 7. some curious polar curves: 7.1. spirals: spiral of archimedes, logarithmic spiral. 7.2. cardioid: it can show interference patterns and waves congruency which irradiate concentrically from a punctual source. thanks to this property, sensibility areas of microphones and antennas can be identified. a cardioid microphone is sensible to frontal sound and minimizes sounds coming from the rear zone. the http://es.wikipedia.org/wiki/archivo:spirog http://es.wikipedia.org/wiki/archivo:various_spirograph_des multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 65 polar pattern or directionality determines in which directions is more sensible a microphone and, therefore, where from it will take more sound. 7.3. limaçons and roses. in the last part of the practice, the representation of gymkhana’s curves starts. 3. results during the academic year 2011-2012, a total of 26 groups participated generating a total of 36 different graphs (after debugging the wrong graphics or those that do not respect the required format), 15 in polar and 21 in parametric. two of the students, voluntarily, located two more graphics that were also added. in the academic year 2012-2013, 34 teams participated, and have been generated 53 graphics, 21 in parametric curves and 32 polar curves. the complete publication is available in issuu (vidal 2013). figure 6. publication in issuu. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 66 figure 7. trochoid figures. figure 8. other parametric curves. figure 9. curves in polar coordinates. figures 7, 8 and 9 show some of the curves made by the students. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 67 4. conclusions in the academic year 2012-2013, in addition to the memory to be submitted by each group, we asked for students a review of practice. 38 students answered some interesting questions: did you find this project interesting? 81.6% answered that it has been interesting or very interesting. about the time spent making the work, 76.3% indicates that, at most they spent three hours (this time is in the provided interval). taking into account the observations and ideas given by the students in the assessment, we highlight the following ones: 1. 'it is a very interesting practice in which we can appreciate graphically much of the expressions that we use many times and we can find a big variety of possibilities in ways that normally cannot be appreciated in a theory lesson. it would be interesting to propose the construction of a roller coaster or a capricious bridge using these types of curves'. 2. 'it has been a very nice practice and not so much complicated'. 3. 'this is an entertaining practice. it is nice the way used for looking for our own equations in the library books; by one side because many of us may have discovered the library thanks to the practice and, on the other, because it is proved that you can learn math in another way, not only theory and problems'. 4. 'this practice is very good, because not only new things are learnt with matlab © , it also helps us to know the campus library and how to find the material that we need at each moment'. 5. 'in our opinion this practice has been interesting due to the curiosity of the geometric shapes produced by the equations, like we have sawn at class about the skating ramp that allows reaching the maximum velocity'. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2192 social and technological sciences vidal et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 53-68 | 68 references alamar, m., roig, b., vidal, a. (2008). fonaments matemàtics: pràctiques de laboratori, valencia: upv. blanco fernández, a. et al. (2009). desarrollo y evaluación de competencias en educación superior. madrid: narcea sa de ediciones. bradley, g.l. and smith, k.j. (1998). cálculo de varias variables (vol. 2). ed. prentice hall. edwards and penney. (1996). cálculo con geometría analítica. ed. prentice hall. fallows, s. and steven, c. (2000). integrating key skills in higher education employability, transferable skills and learning for life. london: kogan page. larson, r. and edwards, b.h. (2006). cálculo (vol. 2). mcgraw hill. larson, r. and edwards, b.h. (2010). cálculo 2 de varias variables. mcgraw hill. larson, r., hostetler, r.p., edwards, b.h. (2006). cálculo ii. pirámide stewart, j. (1999). cálculo multivariable. thomson stewart, j. (2003). cálculo de una variable. thomson. stewart, j. (2005). cálculo: conceptos y contextos. thomson thomas and finney. (1999). varias variables. addison wesley longman. vidal, a. (2013). curves of mathematical gymkhana 2012-2013 http://issuu.com/anna_vidal/docs/curves_of_mathematical_gymkhana http://issuu.com/anna_vidal/docs/curves_of_mathematical_gymkhana multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 84 use of technical computing systems in the context of engineering problems carlos llopis-albert1 , francisco rubio1 , l.m. valle-falcones2, c. grima-olmedo2 1 centro de investigación de ingeniería mecánica (ciim). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain 2 escuela técnica superior de ingenieros de minas y energía. universidad politécnica de madrid – calle ríos rosas 23, 28003 – madrid, spain corresponding author: carlos llopis-albert, e-mail address: cllopisa@upvnet.upv.es received: 09 january 2020; accepted: 12 august 2020; published: october 2020 abstract this paper presents a teaching innovation project based on applying technical computing systems as a resource to improve learning in the classroom and as a way of evaluating transversal competences (tc). by these means, students analyze complex kinematic and dynamic mechanical systems in the context of the subject dynamics of mechanical systems of the master’s degree in mechatronics engineering at universitat politècnica de valència (spain). we have observed that the use of such tools improves the students learning on the contents of the subject, allows to acquire the transversal competence related to the analysis and problem solving, and enhances the ability to understand concepts intuitively. furthermore, results clearly show a positive influence on the use of such tools for improving the professional and ethical commitment to the issues raised. keywords: transversal competencies; technical computing systems; ethical responsibility; professional responsibility; dynamics of mechanical systems. http://orcid.org/0000-0002-1349-2716 http://orcid.org/0000-0003-3465-702x https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 85 1. introduction this paper aims to assess a transversal competences (tc) through a teaching innovation project based on applying technical computing systems in the context of a mechanical engineering subject, which is taught at universitat politècnica de valència (spain). this teaching innovation project is in line with the upv objective to accredit the transversal competences to graduate students in any of the official degrees taught at the university. the upv focuses on science and technology and has more than a century of history. it is intended to advance towards training models that ensure that its students acquire the necessary skills to be able to have an adequate labor insertion. this training must be seen from a broad perspective, linked to the integral training cycle of students, which includes both undergraduate and postgraduate degrees. in this sense, the fundamental objectives of the university are (upv, 2020):  establish a strategy for the systematic evaluation of transversal competences (tc), defining where they are acquired and how they should be evaluated.  prove the acquisition of such skills. for that end, the specific objectives are focus on:  define what is a transversal competence, taking into account the different approaches collected in national and international references.  determine which are the thirteen transversal competences of the upv.  incorporate transversal competences to the traditional training of students using different strategies.  design processes for the evaluation and accreditation of transversal competences that are flexible and innovative.  implement the processes of evaluation and accreditation of tc in all the degrees taught at our university.  give visibility of the results acquired by students to society. with this strategy the university will achieve several objectives: https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 86  provide their graduates with added value that can differentiate them from other graduates and, consequently, make the studies offered more attractive compared to similar offers from other universities.  value the training of our graduates in the face of employers.  make the acquisition of competences explicit for international accreditations. 2. teaching, learning and assessment of transversal competences 2.1. transversal competences at upv the motivation for the implementation of a teaching innovation project based on applying technical computing systems lies in the need to evaluate the transversal competences (tc) as defined by the upv. tc seek to synthesize a competency profile for its graduate students, ensuring a reference framework of all degrees. here is a brief description of the 13 transversal competences defined:  tc-01. comprehension and integration. demonstrate understanding and integration of knowledge both from own specialization and in other broader contexts.  tc-02. application and practical thinking. apply theoretical knowledge and establish the process to follow the achievement of certain objectives, carry out experiments and analyze and interpret data to draw conclusions.  tc-03. analysis and problem solving. analyze and solve problems effectively, identifying and defining the significant elements that constitute them.  tc-04. innovation, creativity and entrepreneurship. innovate to respond satisfactorily and in an original way to personal, organizational and social needs and demands with an entrepreneurial attitude.  tc-05. design and project. design and evaluate an idea effectively until it is realized in a project.  tc-06. teamwork and leadership. work and lead teams effectively for the achievement of common objectives, contributing to the personal and professional development of themselves.  tc-07. ethical, environmental and professional responsibility. act with ethical responsibility, environmental and professional to oneself and others. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 87  tc-08. effective communication. communicate effectively, both orally and written, adequately using the necessary resources and adapting to the characteristics of the situation and the audience.  tc-09. critical thinking. develop critical thinking by taking an interest in foundations on which ideas, actions and judgments, both own and others, are based.  tc-10. knowledge of contemporary problems. identify and interpret problems contemporaries in their field of expertise, as well as in other fields of knowledge, paying special attention to aspects related to sustainability.  tc-11. permanent learning. use learning strategically, autonomously and flexible, throughout life, depending on the objective pursued.  tc-12. planning and time management. adequately plan the time available and program the activities necessary to achieve the objectives in academic, professional and personal terms.  tc-13. specific instruments. use techniques, skills and the necessary updated tools for the practice of the profession. transversal competences are related to a set of attitudes and values (know-how to be) and procedures (know-how to do / act). tc can be translated from one specific professional field to another. when curricula are designed, two types of competences are specified that students must develop: the specific and the transversal competences.  the specific competences are those related to a specific scientific field or academic degree and are oriented to the achievement of a specific profile of the graduate.  the transversal competences, denominated in various ways and under different perspectives (generic competencies, core competencies, key competencies, transferable competencies, soft skills, etc.), respond to those that are key and transferable in relation to a wide variety of personal, social, academic and work contexts throughout of the life. in this sense, they constitute a fundamental part of the professional profile of the degrees. these are competencies that include a set of cognitive skills and metacognitive and instrumental and attitudinal knowledge of great value for the knowledge society. the transversal competences are characterized by being: o integrative, since they favor the integral formation of students. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 88 o transferable in academic and work, personal, and social contexts. o interdependent, since when a certain tc is worked it is usual that other related transversal competences are also dealt. o multifunctional, that is, versatile and capable of responding to different types of problems. o assessable, since they must be broken down into learning outcomes to be able to accredit the level reached in them by each student. the development of each competence is staggered on a continuum that can be called curve of learning, in which partial results that provide information to students, teachers and tutors. it must consider the reference level of development that will be required at the end of the training process to accredit students’ achievement. to establish these mastery levels, it is essential to determine how to develop each one of the competences progressively. in this regard, it play a major role to take into account the experience of teachers and professionals who are working on it. the ct upv institutional project establishes three levels of mastery for each competence: 1. first level: it is developed in the 1 st and 2 nd grade courses 2. second level: takes place in the 3 rd and 4 th grade courses 3. third level: developed in the master courses. to develop and evaluate the acquisition of competences, three ways are established complementary to each other: through study plans, the bachelor's degree final project, the master’s degree final project and considering extracurricular activities. 2.2. transversal competence ct-03 in the subject dynamics of mechanical systems taught at the master’s degree in mechatronics engineering at upv the transversal competence tc-03 is worked. the transversal competence ct03 refers to the students’ capability to analyze and solve problems effectively, identifying and https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 89 defining the significant elements that constitute them. problems are new situations that require students to respond with new behaviors. solving a problem implies performing tasks that require reasoning complex processes and, in many cases, not simply a routine activity. the objective of this transversal competence is for the student to be able to apply structured procedures to solve problems, promoting thus their ability to learn, understand and apply knowledge of autonomous way. three levels of domain are established, which define what learning outcomes it would be desirable that the student had achieved at the end of the second-degree course, the fourth degree and master, respectively. for each of these three learning outcomes, a series of indicators are defined, which are the concretions of the former, so that help to focus attention on the aspects to be analyzed to detect progress.  level 1: learning outcome: analyze a problem applying the methods learned. the indicators of this level are: o define the problem by describing clearly and concisely the aspects more important. o use indicated information sources and select the correct data. o use a learned method to solve the problem proposed. o analyze the coherence of the solution/s obtained.  level 2: learning outcome: develop your own criteria to solve problems effectively and efficiently through reflection and experience. the indicators of this level are: o identify a complex problem, transform it into a several parts easier to solve and argues it. o contrast the sources of information and handle rigorous data. o use the most appropriate methodology, based on their experience, to solve the problem efficiently and justifiably. o choose an optimal solution using justified criteria.  level 3: learning outcome: solving problems individually and / or in team, in different contexts and in depth, from different approaches. the indicators of this level are: o analyze the causes and effects of problems from a global perspective long-term. o apply advanced information search criteria for the problem solving and evaluating the quality of the information. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 90 o organize in a systematic way the work for the decision-making process (individual / group). o evaluate possible solutions according to their scientific-technical feasibility and according to its difficulty of implementation. in addition to the transversal competence ct03, several specific competences are also worked in this subject. they are related to a specific scientific field or academic degree and are oriented to the achievement of a specific profile of the graduate. they are subsequently presented:  ability to model and solve dynamic problems in complex mechanical systems  ability to work in a team.  ability to apply acquired knowledge and to solve multidisciplinary problems. decisionmaking, initiative, creativity, and critical reasoning. 2.3. evaluation systems of ct the recommended evaluation proposed by the university is by means of active methodologies and instruments. they are the most compatible and coherent in a training by competencies focused on student learning. in this sense, the type of activities that are used most frequently are, among others, those that start from learning based on problems, project-oriented learning, collaborative learning (formal and informal), contextualized learning or learning through case studies. most of these methodologies rely on teamwork and collaboration but also, they seek ways of working that promote critical thinking and creativity. moreover, they are intended to assess the previous knowledge acquired. these types of initiatives are more likely to attract attention and maintain the interest of students. complex tasks that come close to the real context are meaningful tasks that allow the mobilization of knowledge, since they lead students to question, to recognize and reorganize relevant knowledge before it can be solved. the integration is the essential principle in the competency training. the activities for monitoring and integrating learning can take various forms, but they involve a formative evaluation. that is, it is about giving regularly feedback to students so that they can situate their current learning with respect to the expectations of the program and to their own https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 91 development. in this way, the process of development of students' competencies and the growth of their professional self-regulation is boosted. it is of the utmost importance to design complex activities that integrate both transversal and specific competences, which must be framed in the contents of the subjects. teachers must evaluate and collect evidence from the evaluation of both types of competences. it is worthwhile mentioning that all the subjects are part of the curriculum and that the competences are acquired throughout it. hence the dynamics of mechanical systems subject is not the only option to develop the tc-03, but what it does is contribute to the acquisition of a part of it. the evaluation of the ct-03 in this subject is performed through rubrics. in the evaluation rubrics several elements are differentiated:  three levels of mastery are defined which refer to the learning outcomes that the student must acquire at each stage.  the indicators, which are concretions of the learning results that defines that level of domain.  descriptors, which define the student's behavior with respect to the indicator that is evaluated.  an assessment scale of 4 values: d, c, b, a; which refer, respectively, to score a level of development not reached (d), in development (c), adequate (b) or excellent (a) of the competence. the result of the evaluation will be based on the weights that the teacher assigns to each one indicators and descriptors. each student will obtain a final a, b, c, d assessment as a result of the evaluation rubric. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 92 3. implementing the tc-03 in the framework of dynamics of mechanical systems the syllabus of the dynamics of mechanical systems subject taught at the master’s degree in mechatronics engineering at upv is structured in five didactic units: 1. introduction to modeling in kinematics and machine dynamics. o introduction. o objectives of computational mechanics. o applications. 2. kinematic analysis of mechanisms vector methods. o equation of the velocity/acceleration field. o kinematic relations in kinematic joints. 3. kinematic analysis of mechanisms analytical methods with numerical resolution. o coordinate systems. o constraint equations. o mobility analysis. o formulation of the position problem. o analysis of the velocity in 2d mechanisms with various degrees of freedom. o analysis of the acceleration in 2d mechanisms with various degrees of freedom. o the dynamic problem. 4. dynamic problem types – definitions. o newton's laws. o principle of virtual works. o principle of virtual powers. o lagrange equations. o force analysis. o motion analysis. 5. introduction to mechanical design: solidworks. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 93 this subject is taught by professors rubio and llopis-albert, which have an extensive experience in this field, as shown for instance in llopis-albert et al., 2015; 2018; 2019; 2019a; 2020; 2020a; rubio et al., 2015; 2016; 2019; 2019a; and valero et al., 2017; 2019; 2019a. as stated, the ct-03 is implemented using technical computing systems, while it provides a way to develop students’ engineering capabilities. the software used to work this transversal competence is wolfram mathematica, which is a technical computing system; and solidworks, which is a solid modeling computer-aided design (cad) and computer-aided engineering (cae) computer program. the evaluation will be carried out through two objective tests with multiple choices, mark of three academic works, and oral presentations in classroom of the work done during the course (table 1). these tests are focused on aspects that have been detected as critical in the evaluation of the students. the contents and the level of the tests are adapted for students of a master’s degree. regarding the academic works the students are intended to solve the kinematics and dynamics of a mechanism subject to external forces using the software wolfram mathematica, and to model it using solidworks. for instance, a certain mechanism from the series of books of artobolevski (1975) are assigned to the students (fig. 1). in solidworks they must design all pieces, provide the assembly model, the dimensioned drawings, a motion study, and a finite element analysis (fig 2). the academic work related to the resolution of the dynamics of a mechanism and the modelling using solidworks are carried out individually by each student, while the academic work related to the kinematics of a mechanism is performed in groups of two. in addition, a second-chance exam will be performed. table 1. evaluation system. description number of acts weight (%) objective tests (multiple choice) 2 40 academic work 2 30 co-evaluation 1 7,5 case 1 15 portfolio 1 7,5 https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 94 a co-evaluation procedure has also been implemented in the evaluation system, which is carried out among students based on predefined criteria. the performance and quality of the three academic works are evaluated by the classmates, as well as the level of achievement in relation to the learning objectives. the co-evaluation procedure has several advantages from both the teachers and student’s perspective. for teacher's point of view, co-evaluation can save time, while students learn to value the processes and performance of their peers. in addition, responsibility is encouraged, students understand the peculiarities of an evaluation process, collaborative work is favored, they develop skills for analysis, and it represents an opportunity to share learning strategies together. moreover, a double benefit is obtained, since on the one hand feedback is given to the partner who is being evaluated, but simultaneously the evaluated student will obtain feedback from her own work. in this sense, the professional and ethical commitment of the students is encouraged and worked in this subject. moreover, they are related to other transversal competences as defined at upv (tc-07). in short, co-evaluation leads to obtain cognitive gains and favor the achievement of bloom's taxonomy of educational objectives, which is a set of three hierarchical models employed to classify learning objectives into levels of complexity (bloom, 1956). with the oral presentations and written reports, the tc-08 is also worked, since the effective communication both orally and written is confronted in front of an audience. by means of the academic work performed in groups of two the tc-06 is dealt, which is related to teamwork and leadership. as a result, several tc are worked, although in the curricula of the master’s degree in mechatronics engineering at upv for the subject dynamics of mechanical systems only the ct-03 is mandatory to be assessed. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 95 figure 1. kinematics resolution using wolfram mathematica of the four-bar link-gear conchoidal approximate straight-line mechanism, which correspond to the 1359 artobolevski’s mechanism. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 96 figure 2. solidworks assembly of an artobolevski’s mechanism. 4. results and discussion there are 37 students enrolled in the subject of dynamics of mechanical systems belonging to the master’s degree in mechatronics engineering at upv for the 2019-2020 academic year. the results obtained have been very satisfactory given the great acceptance by the students, which have actively participated in all the activities. regarding the activities related to tc-03, all students but one have passed the objective tests. furthermore, most of them reached a high grade in the exam (with an average of 7.1 on a scale of 1 to 10), since the questions were widely addressed in classroom. although the academic works were less guided activities and student should work autonomously, they did them correctly. as aspects to improve for the next academic year, it is proposed, on the one hand, to better fit in the calendar the different installments of the academic works, taking into account the workload of the students and, on the other hand, the elaboration of improved rubrics for their evaluation. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 97 the main indicators of success of the activities carried out are the excellent pass rate and the high correlation that exists among the evaluation of the academic works, the objective tests, and the final grade for the subject. 4. conclusions this paper presents a teaching innovation project based on applying technical computing systems to evaluate the transversal competence of the upv tc-03. furthermore, the use of technical computing systems has been proven to be a valid tool to improve both the students’ performance in a mechanical engineering subject, and also, their level of satisfaction. results have shown an excellent pass rate and a high correlation among the evaluation of academic works, the objective tests, and the final grade for the subject. as future enhancements for next academic years it is expected to better fit in the calendar the different installments of the academic works and the elaboration of improved rubrics for the evaluation. references artobolevsky, i.i. 1975. mechanisms in modern engineering design: a handbook for engineers, designers and inventors. seven books. mir pubblishers, moscow. bloom, b.s. 1956. taxonomy of educational objectives: the classification of educational goals. david mckay company, p. 201-7. llopis-albert, c., rubio, f., valero, f. 2015. improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research, 68 (7), 1429-1431. https://doi.org/10.1016/j.jbusres.2015.01.027 llopis-albert, c., rubio, f., valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences, 5(1), 1-16. https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.1016/j.jbusres.2015.01.027 https://doi.org/10.4995/muse.2018.9867 https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 98 llopis-albert, c., rubio, f., valero, f. (2019). fuzzy-set qualitative comparative analysis applied to the design of a network flow of automated guided vehicles for improving business productivity. journal of business research, 101, 737-742. https://doi.org/10.1016/j.jbusres.2018.12.076 llopis-albert, c., rubio, f., valero, f., liao, h., zeng, s. 2019a. stochastic inverse finite element modeling for characterization of heterogeneous material properties. materials research express, 6(11), 115806. https://doi.org/10.1088/2053-1591/ab4c72 llopis-albert, c., valero, f., mata, v., pulloquinga, j.l., zamora-ortiz, p., escarabajal, r.j. 2020. optimal reconfiguration of a parallel robot for forward singularities avoidance in rehabilitation therapies. a comparison via different optimization methods. sustainability, 12(14), 5803. https://doi.org/10.3390/su12145803 llopis-albert, c., valero, f., mata, v., zamora-ortiz, p., escarabajal, r.j., pulloquinga, j.l. 2020a. optimal reconfiguration of a limited parallel robot for forward singularities avoidance. multidisciplinary journal for education, social and technological sciences, 7(1), 113-127. https://doi.org/10.4995/muse.2020.13352 rubio, f., llopis-albert, c., valero, f., suñer, j.l. 2015. assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering, 10 pages. article id 931048. https://doi.org/10.1155/2015/931048 rubio, f., llopis-albert, c., valero, f., & suñer, j. l. 2016. industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. robotics and autonomous systems, 86, 106–112. https://doi.org/10.1016/j.robot.2016.09.008 rubio, f., llopis-albert, c. 2019. viability of using wind turbines for electricity generation in electric vehicles. multidisciplinary journal for education, social and technological sciences, 6(1), 115-126. https://doi.org/10.4995/muse.2019.11743 rubio, f., valero, f., & llopis-albert, c. 2019a. a review of mobile robots: concepts, methods, theoretical framework, and applications. international journal of advanced robotic systems, 16(2), 172988141983959. https://doi.org/10.1177/1729881419839596 solidworks software. 2020. dassault systèmes solidworks corporation. 175 wyman street waltham, ma 02451, usa. https://www.solidworks.com/ https://doi.org/10.1016/j.jbusres.2018.12.076 https://doi.org/10.1088/2053-1591/ab4c72 https://doi.org/10.3390/su12145803 https://doi.org/10.4995/muse.2020.13352 https://doi.org/10.1155/2015/931048 https://doi.org/10.1016/j.robot.2016.09.008 https://doi.org/10.4995/muse.2019.11743 https://doi.org/10.1177/1729881419839596 https://www.solidworks.com/ https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14283 social and technological sciences issn: 2341-2593 llopis-albert el al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 84-99 | 99 upv, 2020. proyecto institucional competencias transversales. universitat politècnica de valència (upv). valencia. spain. https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf wolfram mathematica software. 2020. the wolfram centre. lower road, long hanborough. oxfordshire ox29 8fd, united kingdom. https://www.wolfram.com/mathematica/ valero, f., rubio, f., llopis-albert, c., cuadrado, j.i. (2017). influence of the friction coefficient on the trajectory performance for a car-like robot. mathematical problems in engineering, 9 pages. article id 4562647. https://doi.org/10.1155/2017/4562647 valero, f., rubio, f., llopis-albert, c. 2019. assessment of the effect of energy consumption on trajectory improvement for a car-like robot. robotica, 37(11), 1998-2009. https://doi.org/10.1017/s0263574719000407 valero, f., rubio, f., besa, a.j. 2019a. efficient trajectory of a car-like mobile robot. industrial robot: the international journal of robotics research and application, 46(2), 211–222. https://doi.org/10.1108/ir10-2018-0214 https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf https://www.wolfram.com/mathematica/ https://doi.org/10.1155/2017/4562647 https://doi.org/10.1017/s0263574719000407 https://doi.org/10.1108/ir-10-2018-0214 https://doi.org/10.1108/ir-10-2018-0214 https://creativecommons.org/licenses/by-nc-nd/4.0/ anexo 1 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 135 virtual language learning environments: the standardization of evaluation francesca romero forteza maría luisa carrió pastor universitat politècnica de valència fromero@upv.es lcarrio@upv.es tel. 963877530 fax: 963877539 received: 2013-12-15; accepted: 2014-01-28 abstract nowadays there are many approaches aimed at helping learners acquire knowledge through the internet. virtual learning environments (vle) facilitate the acquisition and practice of skills, but some of these learning platforms are not evaluated or do not follow a standard that guarantees the quality of the tasks involved. in this paper, we set out a proposal for the standardization of the evaluation of vles available on the world wide web. thus, the main objective of this study is to establish an evaluation template with which to test whether a vle is appropriate for computer-assisted language learning (call). in the methodology section, a learning platform is analysed and tested to establish the characteristics learning platforms must have. having established the design of the template for language learning environments, we concluded that a vle must be versatile enough for application with different language learning and teaching approaches. keywords standard of evaluation, virtual learning environment, language learning, world wide web. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 136 1. introduction the internet has become a very useful source of information for language learners, as they can obtain information about very specific topics or skills. the digital learning world complements conventional in-person education by providing virtual access to knowledge and other external resources that teachers can include as extra resources. in this way, learners use technology to practise a skill and they learn how to read non-static information from specific fields, searching for information. the growing role of information and communication technology (ict) on the development of pedagogical issues is worth emphasising. this is changing the role of learners, as they are faced with heterogeneous sources of information as digital information allows them to access knowledge in a variety of contexts (smartphones, laptops, tablets, etc.). this wide variety of educational models that students may encounter creates a need for goals to be prioritized, as the huge amount of information available may lead to confusion. technological advances have led to the creation of a paradigm with multiple options for the benefit of learners, but sometimes the excessive amount of information may overwhelm them (barrón 2004). conscious of the difficulties learners encounter when looking for reliable virtual language learning environments, this study sets a goal of standardising the learning platforms available on the internet. in particular, this study focuses on the study of the learning platforms that are offered to language learners. the main objective of this paper is to design an evaluation template that could help learners verify whether a virtual learning environment is appropriate for computer-assisted language learning (call). a further objective is to this template on the existing language learning platforms on the internet to check their efficacy. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 137 virtual learning is based on the so-called virtual learning environments, mentioned above, and generally known with the acronym vle. these are also called learning platforms or systems or teletraining or telematic environments (majó & marqués 2002). vles are the framework within which online instruction delivery is developed, providing many advantages in terms of the handling and provision of learning materials. guitert (2001) defined a platform as the virtual environment or specific tool that facilitates the creation of training activities on the web. it integrates several key tools in one interface so that users can carry out the necessary activities from a single environment. it can also be accessed remotely from any internet-connected device through a web browser, without the need for the installation of software on the computer; the user only needs to be connected to the server that contains the tool. content is created and distributed within the environment in html format. the platforms are open in the sense that the user can access external resources located on the internet via direct connections (ftp, urls, etc.) or navigation tools. these include individual and group communication tools such as e-mail, chat services, shared whiteboards, forums, etc. they also possess file exchange facilities, evaluation tools and also resources for the management and administration of the course by the teacher. in short, a vle includes a whole range of technological options to facilitate the teachinglearning process. for example, the first vle at the polytechnic university of valencia (upv) was merely a repository of the material made available to the student, offering a great contrast with the present version. now, this consists of a collaborative space containing a wiki in which learners can participate, and synchronous and asynchronous communication tools (the chat service, and the forum and virtual noticeboard, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 138 respectively). there are virtual spaces for file storage, plus a handful of tools for storing teaching materials or create them. these environments should not imitate the spaces and services offered in the traditional delivery mode (i.e. the classroom) (guitert 2001). both environments, digital and classroom, should be differentiated by the way learners by the mode in which learners choose to communicate and the information to be imparted. we should also take into account that vles are characterized by constant updating and adaptation due to pedagogical needs and to the evolution of technology. some researchers have stressed (babot 2004) that it is important that virtual education is well-organized and delivered effectively and, when it is, it proves to be extraordinarily effective, sometimes even more than classroom training. this author believes that online training in general resolves some problems posed by classroom education, but raises other questions such as: what kind of problems are resolved? does it benefit a real sector of the population? can we affirm that there is a demand for its development? is it really different from traditional education? all these questions are still to be answered, but perhaps some conclusions can be drawn in the near future. vle design is a relatively new and quite complex issue, as the task requires the participation of content experts and experts in computer programming. successful technology-based learning requires the application of proper teaching methods and a strategic plan to enable skills and knowledge to be acquired in an appropriate manner. the effort that the teacher would normally make in the classroom should be invested in the design of the material, the dynamic use of technology and the use of an appropriate methodology. different technologies and the different ways of using them now provide a range of options for ict-based language teaching and learning. the different educational multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 139 modes that may make use of technology should take the learning environment into account of the learning environment. an educational mode is the concept that defines the way in which the participants, the roles, the techniques, materials and technology that are part of the teaching-learning process are put together for their interaction and integration into the learning system. examples include such concepts as electronic learning, computer-based learning, online learning, web-based learning, network learning, virtual learning, community collaborative learning, blended learning, mobile learning, company training, transformative learning, interactive learning and total learning. these educational modes are described below in detail. the concepts underlying synonymous terms such as educational modes, instructional delivery or delivery mode (i.e. terms which refer to the modes in which the different tools are used) do not constitute teaching methodologies in themselves. nevertheless, we believe they should be studied in detail as they describe the different modes that can be offered to learners and it is also important to note that the choice of one educational mode or another may lead to different results. in this study, the different kinds of educational modes are important as vle are used in different educational modes and they are taken into account in the evaluation of vles. electronic learning, also known as electronic teaching or e-learning, refers to an ict-based instructional activity. this educational mode includes learning activities that are designed to be used with computers, mobile phones or tablets and can be accessed via the internet. computer-based learning refers to the instructional delivery in which every activity is carried out on a computer, whether this is guided by the learner, by the teacher or by the machine. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 140 online learning, also called online education and online training, refers to an internet-based synchronous or asynchronous learning environment that integrates all the materials, resources and communication tools necessary to produce knowledge acquisition. another educational mode is web-based learning. this involves the implementation of educational resources through a web server. it is a sub-mode of online learning given that the web is embedded in the internet. in network learning, ict is used to facilitate and foster connections among learners, between learners and tutors, and between a learning community and its learning resources (goodyear 2001). this connection can occur via the internet, an internal network or intranet. goodyear (2001) proposes this name to identify this educational mode because it does not necessarily imply interaction with online materials and with other participants. kern et al. (2008) speak of network-based language teaching with these authors claiming that the educational use of networked devices allows multidirectional communication between the agents involved in the process of teaching and learning. on the one hand, the term virtual learning is used to refer to learning activities conducted via platforms and virtual environments and, on the other hand, to denominate a resource or a service artificially created by analogy with the real world, which is represented by means of a visual system. the most well-known example is second life. the first definition is a sub-mode of web-based instructional delivery based as the materials are hosted on a web server and are accessible via a website. however, in the second definition, the virtual world can be online, contained on a cd -rom or on a hard drive, although this latter case is becoming less common since the current tendency is to update information in the cloud. this is the phenomenon known as cloud multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 141 computing, the paradigm that offers services and tools through the network without the need for any specific software or storage device. everything is on the internet, in the cloud. with the aim of not confusing the meaning of this term in this study, we refer to the first definition of virtual learning, which is the most common and widespread. another possibility for instructional delivery is community collaborative learning, which, according to casamayor (2008), is the delivery mode which has emerged from the use of web 2.0 resources. it is based on the principle that the whole is more than the sum of the different parts (casamayor 2008:197). all the information is on the web and in this sense, the users can access to the content at any time. several researchers also refer to the delivery mode known as blended learning, a hybrid form which casamayor (2008: 13) considers to be a synonym for distance learning. blended learning is defined as the combination of different forms of education that respond efficiently to the training needs of a group of people. in general, it refers to the combination of classroom sessions with the use of electronic resources. it should be noted here that, thanks to advances in ict, the differences between classroom teaching and e-learning have decreased, as many of the activities that can be performed in the classroom can also be performed online (davies et al. 2009). mobile learning can also be considered to be a delivery mode. it refers to all the teaching activities designed to be delivered on mobile devices such as wireless laptops, mobile phones, tablets or pdas. the mobile devices allow students to access the information and communication resources from anywhere at any time. another educational mode is the company training, which can be understood as customized training. it includes the instructional delivery that is designed to address a particular need at a company or public entity. this mode is mainly carried out by means of ict (davies et al. 2009). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 142 other delivery modes have been referred to in recent research. one of them is transformative learning or t–learning. it makes use of internet-based tools and it focuses on the development of the skills of the learner related to knowledge management. another recent delivery mode has received the name of interactive learning, which involves the use of interactive digital television, a hybrid between television and entertainment, also called edutainment (pindado 2010). finally, there is the concept of total learning, which expands on the idea of e-learning by taking a global view based on e-learning platforms, social networks and individual learning environments (oliva 2011). as we have seen, there are various delivery modes that can be used by language teachers or material designers, but in this study we focus on language learning. we believe that the theoretical background of the design of technology-based materials should be taken into account when evaluating a vle. here, our aim is to focus on computer assisted language learning (call), also referred to as nblt (network-based language teaching), well (web-enhanced language learning), icall (intelligent computer assisted language learning), mall (mobile assisted language learning), amongst other names. in this study, we use the acronym call to refer to the technology used in language learning, as this term is currently the most widely used by researchers. the study of the way in which language learning methodologies are delivered using computers and technology has been a matter of interest to researchers for the last five decades. more recently, the internet has ensured that the initial focus on the computer advocated by call has widened to ict technology in general. research into call includes material design, the way materials are implemented and the evaluation of the results obtained. this field of study has increased in importance in recent times, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 143 due to the new resources provided by web 2.0 and mobile devices which allow learners to decide when and how they learn. nowadays, most materials are developed for web 2.0. settings, rather than for standalone computers. furthermore, technological advances allow language learners to interact and collaborate with other learners, providing a wide range of tools that can be used for practising a language in context – a very important consideration which students find motivating. all this has contributed to a wide variety of materials and language learning courses being produced. we now have different tools that allow learners to access language learning materials and this is the reason why we consider that the name virtual language environments is more appropriate in this study, as the evaluation we propose is applicable to different electronic learning tools. vles are used by material designers to offer language learning courses and we believe it is important to evaluate them by using sound pedagogical and methodological criteria (nelson 1997; chapelle 2001). in this way, technology can provide an effective and reliable way to learn languages while using the communicative approach. vles are effective teaching tools that facilitate successful language learning and the evaluation of these environments may enable guidelines to be established which web content designers can follow (romero 2012). nevertheless, it is important to stress that this kind of material should be used jointly with different kinds of language learning, whether online or based in the classroom (kern, ware and warschauer 2008; pindado 2010). vles should be used as a tool to help language teachers and learners, but they are not more important than the language-teaching methodology itself. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 144 2. methodology in order to design a template with which to evaluate the language learning platforms found on the web, we first took into account the characteristics that an effective vle needs for use in call. the template was created after considering and assessing the opinion of the different researchers mentioned in the first section of this paper. the template we propose for this purpose includes 40 items and can be seen in table 1: table 1. template for the evaluation of virtual learning environments. items evaluation descriptors of included virtual language learning environments 1 the interface is available in the target language being learnt 2 the homepage clearly presents the sections and resources of the vle 3 the structure of the website is logical and easy to use 4 students can surf from one section to another of the website and bookmark different parts of the site 5 learners know exactly where they are at all times and can surf through the sections and resources without feeling disoriented 6 the arrangement of on-screen elements is consistent from one section to another 7 the icons and graphical metaphors are intuitive 8 the display is appropriately readable 9 instructions or a usage guide exists 10 the functions and activities of the site are self-explanatory or provide the information necessary to execute them 11 there is a frequently asked questions section (faqs) 12 the learner can save work before leaving any section or activity multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 145 13 the tool makes a record of the tasks carried out by the learner when the learning session has finished 14 there is a possibility of returning to activities by deleting the previous work session 15 the website has a search functionality 16 operative across multiple platforms (pc, mac, tablets, mobile devices, etc.) 17 the resource works with any browser (internet explorer, firefox mozilla, google chrome, etc.) 18 the resource is flexible and is able to accept and respond to feedback 19 the graphical interface is aesthetically pleasing 20 the on-screen graphic elements are well-designed 21 the icons are well-drawn 22 the screens show a balance between text and images 23 the needs of disabled people are taken into account 24 it contains tools for synchronous communication 25 it contains tools for asynchronous communication 26 it allows the implementation of various pedagogical approaches (connectivist, constructive, meaningful, collaborative learning, etc.) 27 possibility of cognitive autonomy of the student 28 all the necessary information to complete the course / subject can be found in the resource 29 all the tools and resources necessary to follow the course guidelines appropriately are included 30 the vle accepts various forms of feedback 31 the resource can record the score obtained by the student 32 the resource keeps track of student performance 33 the resource provides opportunities for listening practice multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 146 34 the resource provides opportunities for speaking practice 35 the resource provides opportunities for reading comprehension practice 36 the resource provides opportunities for writing practice 37 the resource allows for various learning styles 38 there is a study guide 39 the website includes a self-assessment system 40 the website provides a final evaluation of the student once the vle evaluation template had been created based on the experiences of previous research, we used it to evaluate different language learning environments. in this paper we present the evaluation carried out on poliforma-t, a platform used at the universitat politècnica de valència for interaction between teachers and learners. it consists of several sections: the subject programme, material design, exam design, a forum, a chat service, email, subject management, subject content, uploading of exercises, etc. this platform can be used by teachers and students to communicate and work in a virtual environment. a screenshot of the platform can be seen in figure 1: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 147 figure 1. vle of the universitat politècnica de valència. after the evaluation of this vle, the results were analysed and we considered whether the evaluation template was useful for the generation of guidelines for designers of language learning material. 3. results the results of the evaluation of the poliforma-t platform as a language learning medium are shown in table 2: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 148 table 2. evaluation of poliforma-t with the proposed template. items evaluation descriptors of included virtual language learning environments poliforma-t (upv) 1 the interface is available in the target language being learnt yes 2 the homepage clearly presents the sections and resources of the vle yes 3 the structure of the website is logical and easy to use yes 4 students can surf from one section to another of the website and bookmark different parts of the site yes 5 learners know exactly where they are at all times and can surf through the sections and resources without feeling disoriented yes 6 the arrangement of on-screen elements is consistent from one section to another yes 7 the icons and graphical metaphors are intuitive yes 8 the display is appropriately readable yes 9 instructions or a usage guide exists yes 10 the functions and activities of the site are self-explanatory or provide the information necessary to execute them yes 11 there is a frequently asked questions section (faqs) no 12 the learner can save work before leaving any section or activity yes 13 the tool makes a record of the tasks carried out by the learner when the learning session has finished yes 14 there is a possibility of returning to activities by deleting the previous work session yes 15 the website has a search functionality no 16 operative across multiple platforms (pc, mac, tablets, mobile devices, etc.) yes 17 the resource works with any browser (internet explorer, firefox mozilla, google chrome, etc.) yes 18 the resource is flexible and is able to accept and respond to feedback yes multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 149 19 the graphical interface is aesthetically pleasing yes 20 the on-screen graphic elements are well-designed yes 21 the icons are well-drawn yes 22 the screens show a balance between text and images no 23 the needs of disabled people are taken into account no 24 it contains tools for synchronous communication yes 25 it contains tools for asynchronous communication yes 26 it allows the implementation of various pedagogical approaches (connectivist, constructive, meaningful, collaborative learning, etc.) yes 27 possibility of cognitive autonomy of the student yes 28 all the necessary information to complete the course / subject can be found in the resource yes 29 all the tools and resources necessary to follow the course guidelines appropriately are included yes 30 the vle accepts various forms of feedback no 31 the resource can record the score obtained by the student yes 32 the resource keeps track of student performance yes 33 the resource provides opportunities for listening practice yes 34 the resource provides opportunities for speaking practice yes 35 the resource provides opportunities for reading comprehension practice yes 36 the resource provides opportunities for writing practice yes 37 the resource allows for various learning styles yes 38 there is a study guide yes 39 the website includes a self-assessment system yes 40 the website provides a final evaluation of the student yes multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 150 the analysis of poliforma-t as a vle for call shows that it possesses almost all of the items included in the template. it does not include a faqs section, the web has not search functionality, the screen displays more text than images and it does not accept different forms of feedback. therefore, we can say that this platform aims to facilitate the exchange of information between learners and teachers, but the designer of this platform was not interested in feedback or aspects of the design of the digital environment. the instructional delivery mode of this platform can be identified as online learning as it provides the material to learners, but it is not so interactive as to allow a flow of communication with the participants. 4. conclusions we conclude that the frequent use of technology and platforms that aim to help teachers interact with learners and implement different kinds of language learning materials means that the items included in each vle should be standardized. sometimes it seems teachers are more interested in using technology than in employing an appropriate methodology, but the latter cannot be ignored. methodology and technology should go hand in hand in order to be able to provide good materials for language learning, and this process should also be evaluated and the contents revised periodically. a vle should be a versatile tool able to support different styles of learning and teaching. in this study, poliforma-t has been evaluated with the template we have proposed and its performance has been quite satisfactory. the platform features 90% of the items evaluated, although it was created to be managed by teachers at the university and some aspects as images, faqs, etc., have not been included due to the fact that the platform was aimed at internal users. we are conscious that this is an experimental analysis and that the template proposed in this paper should be used to evaluate other multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 151 vles: then we may be able to establish whether it could be useful for the improvement of vles in general. references babot, i. (2004). perversiones del e-learning. http://www.elearningameircalatina.com/ edicion/agosto2/na_1.php. barrón, s. h. (2004) seis problemas de los sistemas universitarios de educación en línea. red. revista de eduación a distancia, 12. http://www.um.es/ead/ red/12/barron.pdf. casamayor, g. (ed.) (2008). la formación on-line. una mirada integral sobre el elearning, b-learning. barcelona: graó. chapelle, c. (2001). computer applications in second language acquisition. foundations for teaching, testing and research. cambridge: cambridge university press. davies, g. et al. (2009). introduction to computer assisted language learning (call). module 1.4. in g. davies (ed.), information and communications technology for language teachers (ict4lt). slough: thames valley university. http://www.ict4lt.org/en/en_mod1-4.htm. felix, u. (2001). a multivariate analysis of students’ experience of web-based learning. australian journal of educational technology, 17(1), 21-42, http://www.ascilite.org.au/ajet/ajet17/felix.html. http://www.elearningameircalatina.com/%20edicion/agosto2/na_1.php http://www.elearningameircalatina.com/%20edicion/agosto2/na_1.php http://www.um.es/ead/%20red/12/barron.pdf http://www.um.es/ead/%20red/12/barron.pdf http://www.ict4lt.org/en/en_mod1-4.htm http://www.ascilite.org.au/ajet/ajet17/felix.html multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2199 social and technological sciences romero and carrió (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 135-152 | 152 goodyear, p. (2001). effective networked learning in higher education: notes and guidelines. report to networked learning in higher education project (jcalt). http://csalt.lancs.ac.uk/jisc. guitert, m. (2001). los entornos de enseñanza y aprendizaje virtuales en las puertas del siglo xxi, in m. trenchs (ed.), nuevas tecnologías para el autoaprendizaje y la didáctica de lenguas, lleida: editorial milenio. kern, r., ware p. and warschauer, m. (2008). network-based language teaching. in n. van deusen-scholl and n. h. hornberger (eds.), encyclopedia of language and education, 4, second and foreign language education. spring+business media llc http://www.gse.uci.edu/person/warschauer_m/docs/network-based.pdf majó, j. and marqués, p. (2002). la revolución educativa en la era internet. barcelona: praxis. nelson, j. t. (1997). a system for the evaluation of esl web sites, http://www.wsu.edu:8080/~jtnelson/thesis/complete_thesis.html. oliva, r. (2011). el t-learning, el aprendizaje global en el siglo xxi. http://www.dimglobal.ning.com/profiles/blogs/el-t-learning-elaprendizaje-global-delsiglo-xxi. pindado, p. (2010). t-learning. el potencial educativo de la televisión digital interactive. http://www.gabinetecomunicacionyeducacion.com. romero, f. (2012). anàlisi didàctica i crítica del web per a l’autoaprenentatge del català a través del web. michigan: proquest. http://csalt.lancs.ac.uk/jisc http://www.gse.uci.edu/person/warschauer_m/docs/network-based.pdf http://www.wsu.edu:8080/~jtnelson/thesis/complete_thesis.html http://www.dimglobal.ning.com/profiles/blogs/el-t-learning-elaprendizaje-global-del-siglo-xxi http://www.dimglobal.ning.com/profiles/blogs/el-t-learning-elaprendizaje-global-del-siglo-xxi http://www.gabinetecomunicacionyeducacion.com/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 teacher identity construction and plurilingual competence: a longitudinal study about language teaching in multilingual contexts j. palou*, e. tresserras plural group, universitat de barcelona passeig de la vall d'hebron, 171, 08035 barcelona, spain * corresponding authors: email: jpalou@ub.edu; eva.tresserras@ub.edu received: 2013-12-31; accepted: 2014-08-11 abstract this article is part of a thesis dissertation focused on cognition and belief systems about prior language learning experience, teacher education and classroom practice. this document examines teacher identity construction in relation to plurilingual competence. we want to study a dynamic identity, which is constructed in the here and now. we use the european council contents for language teaching. the central purpose is analysing belief systems in relation to the teaching of languages in plurilingual contexts. we focus our analysis on teacher understandings of their new social challenges and how they respond in and to these situations. the interactional data was gathered from five focus groups of ten teachers from different educational stages. prior to this, we collected recurrent questions which had emerged in reflective narrative and individual interviews. data was analysed using a qualitative approach. activity theory is the conceptual framework to identify which tensions appear when we identify not only the theoretical discourse, but also the everyday practical classroom reality. for this reason, we distinguish between dilemmas, conflicts, critical conflicts and double binds. the impact of this proposal can be summarized in relation to the following results: urgent necessity for teacher training, sharing experiences to help developing professional identity and transferring positive experiences in schools. keywords plurilingual competence; language learning; self-confrontation; reflective training. palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 92 mailto:jpalou@ub.edu mailto:eva.tresserras@ub.edu multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 1. introduction 1.1. new terms for a new reality the elimination of physical frontiers is the removal of mental borders. learning to live with linguistic and cultural diversity would be a laudable purpose but an insufficient one if coexistence is understood as being close to each other, without bridges which cultural and linguistic differences can cross. navigating to find and contrast each other’s cultures and from contrast, enrich oneself. this is one of the greatest challenges of the world we have built. we are not referring to an epidermal issue, but to a new situation that is changing the world in which we live. challenges generate concern in what is already known. moreover, they generate new expectations which must be faced from new conceptual frameworks that give meaning to the new cultural, social and linguistic orientation. aware of that, the institutions in charge have developed documents that affect all the areas of the social activity and, in a specific way, the educational one. these documents certainly create new perspectives on what is already known, and provide when necessary, a new terminology for the emerging social situations. thus, the common european framework of reference for languages: learning, teaching, assessment (cefr) focuses on the concept of plurilingual competence and defines it as: the ability to use languages for the purposes of communication and to take part in intercultural interaction, where a person, viewed as a social agent has proficiency, of varying degrees, in several languages and experience of several cultures (cefr, 2001, p. 168). the ability to communicate and participate in an intercultural relationship that must characterize the formation of citizens, social partners, must go with a personal and collective decentralization process. the prefix "inter" of the term intercultural leads us to palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 93 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 the idea of living "inside of", "in" or "between". regardless of what is understood, the purpose is clear: the overlap is not the answer to the contact of languages and cultures; the answer is being less focused in order that each social agent understands that their culture is not the culture, that their social customs are not the social customs and that their language is not the language. the response to diversity only makes sense if we value this diversity. 1.2. the discourse of the teachers, a good mediator the formulation of new concepts cannot be understood as a static way. we are not talking about immutable principles; we are talking about the desire to give meaning to what happens through new formulations that will help to guide it. it is a dynamic process of formulation and reformulation of perceptions that can begin in the depiction of new documents by the european commission, and from here, follow a long way to the impregnation of these perceptions in social institutions, including schools. yet, this is not to stay, but to give feedback to the general intention to formulate with terms what is happening and what is preferable to happen. in order to do that, reflective training (van lier, 1996) might be a good choice. teachers, people who are living inside the complexity of the multilingual and the intercultural immediacy, are key parts in the spiral trip that helps to generate a discourse on the new social and educational contexts. as the psychologist michael cole (1998) said, the term context as “that which surrounds” is represented as a set of concentric circles that represent different levels of context. these levels, which describe interconnected systems, start with a microsystem as a main point. regarding education questions, teachers and students are the main point of it. it is often not firmly stated that teachers are the ones who work in the front line of the conflict. they are the ones who have seen how different cultures and languages met in their classrooms. the ones who have had to consider how to treat the families who are palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 94 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 unaware of the simplest institutional routines, the ones who consider what is necessary to tolerate and what is not. the ones who have seen how their role of teaching has gone back in front of a new one, more focused on mitigating the social conflict. therefore, it is necessary to approach teachers in order to know how they live their personal experiences in schools with an important linguistic and cultural diversity. this approach will only be possible if we listen to their words, the discourses generated from their experience. it is in the analysis of these talks where we can detect how they anticipate what can happen based on their impressions (lahire 2011). there are differences between what is mentioned by cefr and what the teachers think; between what they say and what they do; between the social exclusion and the political discourse; between the educational purposes and the resources that schools have. teachers’ speech filters and covers this mismatch and it is from here that it gives meaning to their performance. however, not every action is translated into speech. as noted by cicurel (2011) not everything that is real is assimilated in the verbal expression. it is possible they act without knowing the reasons, but what it is found in their talks is the intention to give a general meaning to the action. teachers’ discourse acts as a mediator since it evens out the disagreements detected between what they say and what they do, but it also acts as a mediator between the concepts formulated in a general way and its implementation in schools. a vertical perspective would avoid this evidence. the training processes, according to freeman (1996), elbaz (2001), hargreaves (1999), and woods (1996), must give priority to teachers to know how they re-conceptualize their world and how they act in it. this voice can provide feedback on the spiral process we have talked about, on the construction of a conceptual framework to be in the new multilingual and multicultural contexts, and guide them. in this field, self-confrontation defines the re-elaboration of experience, both pertinent and shared in an inter-subjective space, which allowed participants to construct knowledge in order to act in the activity. palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 95 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 1.3. learning about contradictions in a collective way without the intention to do an exhaustive literature review, there are many authors who have studied contradictions. yrjö engeström and his colleagues, within a methodological framework based on the dialectical tradition of cultural-historical activity theory, have been doing a lot of research in this field. regarding their research, contradictions cannot be observed directly, only within their manifestations. here, is an example of their thoughts: contradictions are often mentioned as significant factor behind organizational change. however, the meaning of the term contradiction is commonly left vague and ambiguous. almost any tension or aggravated problem seems to qualify as contradiction. similarly, related terms such as paradox, conflict, dilemma and double bind tend to be bundled together or used interchangeably in an ad hoc manner. such ambiguity and looseness of conceptualization are detrimental to research. there is a risk that contradiction becomes another fashionable catchword with little theoretical content and analytical power (2011, p. 368). in front of the change we have talked about, there are three options: a minimum adaptation, an escape or a transformation. according to bourdieu (1997), there is a tendency to avoid major crisis; in all likelihood this trend is associated with the first two options. but the demands of the context are such an important matter nowadays that it is impossible to hold minimizing or decreasing attitudes in a past, which is often idealized. the only thing we have is the need for a transformation that requires addressing disagreements from a holistic perspective. accordingly, our purpose is to promote this transformation. therefore, the following two issues should be considered. firstly, the specification of disagreements always goes with a certain kind of tension. secondly, transformation moves both in a vertical and in a palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 96 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 horizontal level so it affects the individual as well as their social environment and the people who are part of it. terms like conflict, tension and contradiction have been used a lot in the field of training. d. woods (1996) consolidated in his research the concept “hotspots” which was defined as: it is at points in a text where there seems to be a conflict between what is and what is believed, that beliefs stated more clearly and less consciously appear (woods, 1996, p. 71). the interest of woods’ studies is to detect conflicts which emerge in the same teachers’ discourse when they refer to what they think and what they do. other studies focus their attention on the practice because they are based on the premise that teaching practice is always performed in moments of tension. altet (2008) considered that the role of the teacher could be defined as: the management of uncertainty and the tension due to constant, mutual and interactive adjustments (altet, 2008, p. 48). taking into account the terms used to refer to all types of misalignments, engeström and sannino (2011) verified that a definition in a theoretical level was necessary. their proposal is based on this idea, on specifying how contradictions are manifested in the discourse as a prior step to categorize them. for this reason, they suggested to distinguish between “dilemma”, “conflict”, “critical conflict” and “double bind”. a dilemma is "an expression or an exchange of inconsistent evaluations, either between people or within the discourse of a single person" (engeström and sannino, 2011, p. 373). conflict involves opposition, disagreement and even obstruction between actions of different people. furthermore, a critical conflict appears if the situation leads to misunderstanding and paralysis related to the capacity to act. finally, these authors (2011, p. 374) use the term double binds to refer to the processes in which people aware of the palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 97 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 contradictions and the need to solve them ask themselves “what can we do?” instead of “what can i do?”. this is because the double bind processes involve the transition from the individual to the collective. 2. methodology 2.1. procedure and intervention our analysis is concerned with teachers’ believes in relation to language teaching in multilingual contexts, and the need for both initial education and in-service training in this field. data was gathered in two phases and it involved ten teachers from different educational levels: one of pre-school, four of primary school and four of high school. in addition, our study included a specialist in the area of didactics of linguistic diversity as all participants have been exposed to language and culture diversity in their classroom. the first phase of the process consisted of reflective narratives (bertaux, 1997). these samples were personal analyses of their own linguistic history and specific situations of their language usage. they were followed by an interview with each of the participating teachers. although the researchers led the interviews, the participants responded according to their needs and interests. through these methods, the participants could express their experiences in relation to multilingualism. their analysis allowed us to perceive conflict situations, and deal with the construction of teaching identity from a dynamic point of view. at the same time, this first set of data allowed us to detect recurring issues that guided the second phase of the project, the group sessions. five sessions in total were carried out. in the first one, the researchers pointed out issues from a more introspective approach and allowed the teachers to manage them on their own. in the other sessions, we focused on a process of self-confrontation, which consisted of the following: teachers were asked to film and record a session of class where some aspects related to both linguistic and palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 98 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 cultural diversity appeared. from this point onwards, they chose a five or ten-minute sequence to comment on with the rest. the other participants, in a very protected environment, expressed their views and comments without any intention of reaching a consensus, only deepening speculatively in the topic raised. immediately after each session, the members of the research group met to discuss its development. in these meetings, they basically considered methodological issues. however, it is necessary to note that the researchers, who were kept in the background in the joint sessions, had in that moment a space to give their opinions on the thoughts expressed throughout each session. moreover, they also expressed their views on the degree of involvement and interest that the participants had shown. text and context were mixed in these first comments characterized by immediacy and spontaneity. engeström (2001; 2010) points out that people live immersed in multiple activity systems, in multiple spheres where individual and collective activities converge. in the analysis we present, two activity systems act together: the researchers, who have pursued researching objectives, and the participants, whose objectives are their students’ learning and their own training. while the two systems consist of different purposes and activities, it is also true that they are permanently linked by the sense and meaning of the teaching practice. in this article we will focus on the session groups in which the teachers had the leading role. it is about focusing on an activity that is collective and systemic (engeström, 1999). for this reason, we take into account multivocality, which we define as multiple points of view in interaction, as not solved points that are faced through the collective debate. we will refer to the speech that the participants generated in relation to plurilingual competence. our intention is to determine how they lead the speech and what doubts are raised in order to detect structural problems that allow us to design new training devices. the results that we will present arise from the analysis of the last session, which dealt palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 99 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 with emerging issues in narrative literature, from the interviews and from the different self-confrontation exercises that were carried out. 2.2. data analysis at the beginning of this paper, we said that the conflict often appears when there is a contradiction between what teachers believe and do (or they state they do). for this reason, we explored how contradictions were manifested and which differences appeared when the experience faced the collective discourse. the starting point we considered was that the way in which participants confront and resolve contradictions offers us a new and dynamic vision of reality. this was our reasoning for taking into account both the personal and the professional history of the participants as well as the ability to put into their knowledge and experiences into play when dealing with new areas of confrontation. once the interviews and literature narratives were analysed, data showed that key concepts such as multilingualism, plurilingual competence and linguistic repertoire lacked any consistency in the discourse of teachers. they either referred to them in a confusing way or these key concepts were simply not part of their repertoire at the time they conceptualized the phenomenon. constant tensions appeared when they tried to reflect how they had put these concepts into practice in their classrooms. it was also found that the performance in the classroom responded to a wilful, but rather improvised performance. in short, the design of activities was not maintained in clear criteria and was agreed by the teaching teams. taking this information into account we oriented the collective discussion groups. the purpose was to interpret the conceptual and emotional state of the teacher working in multilingual classrooms in order to detect needs related to the processes of initial education and in-service training. the data was analysed in a qualitative approach, following the criteria established by the research group on multilingualism and language learning (plural) from, among others, the contributions by kerbrat-orecchioni (2005) on the analysis of the speech in palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 100 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 interaction. our aim is to analyse the marks that the subject leaves in what she/he says, focusing mainly on thematic and referential aspects. following engeström and sannino (2011), contradictions cannot be observed directly, but it is necessary to locate them through their manifestations. in this case, we analysed them through the speech generated among all the participants. as we have mentioned before, we will classify them into “dilemmas”, “conflicts”, “critical conflicts” and “double binds”. in this paper, the analysis of the data focused the discursive manifestation of contradictions. 3. findings we will make reference to the results focusing on the three issues already targeted that emerged from the analysis of individual data. 3.1. multilingual competence: from concept to action among the teachers who took part in the focus sessions there is a widespread agreement at the time of conceiving language as an instrumental area. furthermore, we realized that they were in favour of language learning. in this field, their beliefs and practice were always in coherence, without contradictions and thus without conflicts. as a vehicle of communication, they also agreed on taking into account the continuous involvement of all educational agents. considering the new multilingual reality of the classroom in which they work, the participants expressed the need for change to the educational and methodological approaches of the linguistic and communicative education. the dilemma arises at this point. on the one hand, they were aware of this need, but on the other hand, they also manifested that a lot of mistakes were made in the way that language is taught to beginners; as their native language does not have any presence in the classroom and thus, palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 101 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 they become invisible. at the same time, however, there were constant contradictions, manifested as double binds. they required the change as a collective way, as spiral learning where this change arises by the common conscience. for this reason, the individual data allows us to conclude that participants perceive a structural change. with this, we mean a societal change which goes beyond the current moment. as a consequence, in front of this situation, they try to give answers and find solutions as an individual way. on the other hand, in general they believe that the collective teaching is aware of the dilemma and that a lot of schools seek spaces to think about it. participants explained that this is a topic of on-going debate in teachers’ meetings and assemblies. the problem is in the gap that exists between the theory and the practice, maybe because it has never been taught to them. in their own words: there are those who talk about multilingualism but the theory is the theory and that’s all this gap between theory and practice, between what they believe and do, emphasises the lack of guidelines they have to think and design actions in the classroom. the focus does not lay on ways of doing it that go in opposite directions. the participants, above all, brought up the need to know how they can arrange specific didactics which have the new multilingual and multicultural reality that characterize their classrooms as a starting point. again we see that the dilemma appears as a lack of reflection. they perceive the lack of experience of many teachers who want to change the situation but do not know how. although they insist on the need to change many attitudes, they also feel the need to reassert themselves. here, is an example: i think schools are doing pretty well eh! but i am optimistic i think:: schools are in this way and there may be major changes. palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 102 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 this willingness to face the problem and promote transformation processes allows us to consider that its manifestations respond to a double bind. none of the participants introduced himself/herself against any of his/her colleagues, so the contradictions do not become disagreements. moreover, none of the participants expressed a feeling of paralysis or helplessness in front of the difficulties, but they always suggested new alternatives. here, are three examples: but i think but i think that it isn’t only the language but also the fact that they know that their identity is important and that we don’t have to eliminate it by the reason that they arrive in a completely different society actually there is an attempt to:: rethink|ahm| rethink isn't there? what is the current:: situation:: of immersion at schools and what’s the treatment of:: languages […] to gathe::r more stories and more experiences yes i believe that there are many schools that are on the right path:: to know how things should be done don’t they? there’s reflection and yes i mean i’m optimistic i think: that schools are on this way and there may be important changes 3.2. from action to systematization participants agreed that every society has to respect students’ cultures; origins need to be very present, especially in the learning environment and in the classroom. however, in class they talk about sporadic activities not to forget the languages. if they had to fill in a grid with the activities they carry out, they say that "we would have it quite full" at primary school and "virtually empty" at high school. we can also see a clear example of inner conflict, as a result of tension: in my specific case because they are:: such faraway cultures as |mhm| references there are few models regarding urdu or:: bengali or chinese or:: my ignorance is palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 103 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 total and i suppose that if i knew something more it would be easier to find points in common and:: but yes i admit that as a high school teacher it’s an issue i have to continue working on the lack of systematization is another important dilemma that stands out through the teachers’ manifestations. once again, this dilemma is not in a cul-de-sac, as it has a possible exit: training. when the researchers asked the participants about their experience of taking part in these sessions, their opinion had a clear consensus: they said that these sessions had helped them to realise the importance of training, they considered them extremely important if we are to make progress in a true multicultural education. once again, it becomes clear that we should not look for contradictions in the difference of opinions, but in the lack of information. the fragment that follows exemplifies this perception: a lot of teachers that sometimes don’t start with it no because they don’t want don’t want or have a bad attitude but because sometimes they don’t know how start with this topic you know? […] for the experience that i have in schools i don’t see bad attitudes among the teachers right? they say that after these sessions they have become aware of the large amount of resources and strategies that can be used: realize:: that there are other ways that there are many:: ways to focus on my job making use of the students’ linguistic repertoires, which are very complex and often unknown, is an ideal way of immersion in other cultures, they say. resources like these, which deal with language in its maximum functionality, would reduce the situations of hatred, rejection, the stereotypes and the negative representations that concern the participants. the sessions they have shared with other teachers have helped them to understand that we always need to consider the language and the communicative situations when we use the language, especially in multilingual environments. palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 104 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 the fact of having shared successful experiences is the best way to move forward. getting to know colleague situations and daily contextualised practices promote reflection and, as a consequence, the construction of knowledge and new forms of actuation in this field: from a doubt we ask and learn don’t we? the participants also insist on the importance of these experiences for the students. making them part of learning, through practice and reflection, is the best way to develop communication and multiculturalism. never before, had anybody made them think on their own languages. a participant explains: i have realised that it has worked with them. talking about their own languages has been a spectacular breakthrough". 3.3. contradiction and transformation: need of change there is always a reflection between what happens in the classroom and what happens later. we have realized that this reflection covers the previous experience and the daily practice, what is individual and what is social. we have already commented that conflict appears when there is contradiction and this is the beginning of change, of transformation. through this study, in which we can see how horizontal learning opens new ways of change, some tensions disappear as a consequence of sharing experiences and thoughts. the collective overtakes the individual when it comes to share dialogue, to reflect and build common knowledge. in this regard, the need of training appears in all the participants, not only regarding plurilingual competence but also in practical training. we would need to give something such as a handbook or a series of ideas of to formulate and to act and to put them into practice and then |ehm| see how how it works right? i mean going beyond p-put like:: like a series of experiences that:: after all we all agree on the same thing don’t we? palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 105 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 i believe that sharing experiences is good you know! successful because in this way with a few clear examples teachers see it and then it is as if they became confident to say let’s start? let’s try it! 4. conclusions at the beginning of this paper we talked about the didactics of languages and cultures and cultural and intercultural teacher awareness as ways to understand new situations of inclusion and learning in multilingual environments. in order to develop the conceptual framework which guides this thesis dissertation, we asked ourselves which contradictions are appeared and in which four kinds of discursive manifestations are they expressed when the participants share their previous experience and daily practice in multilingual environments? although the analysis of the whole data is not finished yet, throughout this part of the corpus we point out the need for teachers to make sense of the social changes they live in first person; participants can only do it in contexts where speech is controlled by themselves. this, regarding discursive manifestations, implies some considerations. one the one hand, tensions become dilemmas in personal situations, in ways where the individual action is translated into speech. their daily practice involves dilemmas, as well as the self-confrontation between what they believe and do. on the other hand, we identify double binds when participants try to give a general meaning to the action, which often becomes an unbalanced situation. the current agreement, the collective awareness and the need of change is the first step, as a dynamic field, to make the necessity of creating new formative devices explicit. we also highlight that conflicts and critical conflicts are indeed less frequent than dilemmas or double binds, maybe because participants do not manifest disagreements or resistances in their thoughts and actions or maybe because they understand other participants’ experiences, a key signal of current need for change. for all these reasons mentioned, more data has to be analyzed in order palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 106 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 to investigate around these hypothetical agreements deeply. however, we are considering a possible contribution in this field. the responses to these situations and the construction of new interpretative frameworks are never individual. for this reason, it is more likely that there are more double binds than conflicts. dialectics appears when we contradict what we are saying and, in that moment, we evolve when we deny what we previously said. finally, both in this particular paper and in all the research the asymmetrical relation between researchers and teachers world, who are in real contact with multilingual diversity is often perceived. in this field, reflective training seems to be a good alternative to decrease these differences. participants need to share their individual experiences, which are only expressed in environments where they feel completely comfortable. teachers need to be asked to wonder their daily life and confront their own believes with beliefs of the others who are in the same situation. overall, the transformation processes, always motivated by remarkable contradictions, should be seen as horizontal learning, emphasized when heterogeneous groups are created and their voices are collected. 5. references altet m. (2008). tensions, régulations et ajustements dans les pratiques enseignantes: analyse de la dynamique des interactions maître-élèves observées dans un débat scientifique. in i. vinatier y m. altet (dir.) analyser et comprendre la pratique enseignante, (pp47-56). rennes: presses universitaires de rennes. bertaux d. (1997). les récits de vie. parís: nathan. bordieu p. (1997). razones prácticas. sobre la teoría de la acción. barcelona: anagrama. cicurel f. (2011). les interactions dans l'enseignement des langues agir professoral et pratiques de classe. paris: didier. palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 107 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 cole m. (1996). cultural psychology: a once and future discipline. usa: harvard university press. council of europe. (2001). common european framework of reference for languages: learning, teaching, assessment. council of europe: cambridge university press. elbaz f. (2001). research on teacher’s knowledge: the evolution of a discourse. journal of curriculum studies, 23 (1), 1-19. engeström y. (1999). activity theory and individual and social transformation. in engeström y.; miettinen r. and punamaki r.l. (ed.). perspectives on activity theory (pp.19-38).cambridge: cambridge university press. (2001). expansive learning at work: toward an activity theoretical reconceptualization. journal of education and work, 14 (1), 133-156. engeström y. i sannino a. (2010). studies of expansive learning: foundations, findings and future challenges. educational research review, 10, 1-24. (2011). discursive manifestations of contradictions in organizational change efforts. a methodological framework. journal of organizational change management, 24, 3. freeman d. (1996). the unstudied problem: research on teaching learning in language teaching. in freeman d.; richards j.c. (ed.). teacher learning in language teaching (pp. 351-378). new york: cambridge university press. hargreaves a. (1999). sentir-se mestre: les emocions d’ensenyar i el canvi educatiu. temps d’educació, 22, 271-291, 33-59. kerbrat-orecchioni c. (2005). le discours en interaction. parís: armand colin. lahire b. (2011). l’acteur entre dispositions et contextes d’action. in bigot v. and cadet l. (dir.). discours d’enseignants sur leur action en classe. enjeux théoriques et enjeux de palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 108 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2225 social and technological sciences eissn: 2341-2593 formation. paris: riveneuve éditions. tusón a. (1995). el análisis de la conversación. barcelona: ariel. van lier l. (1996). interaction in the language curriculum. awareness, autonomy & authenticity. london: longman group limited. woods d. (1996). teacher cognition in language teaching: beliefs, decision-making and classroom practice. cambridge: cambridge university press. palou and tresserras (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 92-109 | 109 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 45 adolescents and values portrayed in music nowadays maría gemma sánchez gonzález1 university of extremadura, muñoz chaves, 3 10003 cáceres, spain. e-mail address: sanchez_gemma@hotmail.com received: 11 july 2020; accepted: 06 september 2020; published: october 2020 abstract nowadays, the close relationship between adolescents and music is a fact reinforced by new technology. music, in general, reflects values and attitudes in society and serves different purposes in life, however the crisis of values is a constant threat that may be eased by using music as a tool to engage with young people and reinforced ethical values. this work aimed at assessing perceptions of explicit content in pop music by 16-year-old adolescents by means of a questionnaire with a series of short-open questions. the results confirmed a relaxed attitude towards current social values but yet a robust self-claimed personality in young people keywords: adolescents; explicit content; music; ethical values https://creativecommons.org/licenses/by-nc-nd/4.0/ https://orcid.org/0000-0003-1254-9933 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 46 1.introduction nowadays it can be observed that younger generations are involved in a culture of free music and therefore new business opportunities have consequently emerged over the last few years (fouce, 2010). the music industry is constantly looking into different ways to call the attention of young potential listeners by portraying innovative sex standards, flexibility of profanity, among others in order to express personal experiences and making it explicit is the key to make their way to the music charts (knobloch-westerwick et al, 2006). further research conducted linguistic analysis of popular songs 1980–2007 to demonstrate that changes in the language "tune in to psychological change" (dewall, pond, campbell & twenge, 2011). it also showed that over time, the use of words related to self-focus and antisocial behavior increased, whereas words related to other-focus, social interactions, and positive emotion decreased. song lyrics are a clear example of social changes occurring at a rapid pace. in line with the above, highly sexualised lyrics have been analysed to show that the relationship between the body and sex is unambiguously portrayed in contemporary media. far too often lyrics show extreme explicit sex acts and body parts that have taken over personality or any other qualities of the person/s involved. sex has become one of the main basis of relationships, even in so-called typical love songs where the listener, back in time, would have never expected that type of graphic vocabulary. (aubrey, 2006) relaxation of ethical values in the name of globalisation has affected all paths of life, from language etiquette to social manners resulting in a general crisis and change of values (amuthavalli; sridevi, 2014; kumar, 2015; pajaziti, 2014) that is affecting western society and is being experienced, firsthand, by those who are most vulnerable, children and young adults. besides, variety, forms of expression and specially globalisation (bauman, 1998) has caused a change in song lyrics. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 47 as claimed by saarikallio & erkkilä, (2007) music functions as a mood regulation device; this statement is reinforced by sloboda & juslin (2001) who claim that music can regulate and satisfy personal mood-related goals. music can be used for several purposes, among which, the most used are "mood enhancement, coping with distress, identity construction and social identity formation" (bogt, mulder, raaijmakers & gabhainn, 2011). "music’s consoling effects were reported as resulting particularly from the sound and texture of the music itself, from attribution of personal meaning to music’s lyrics, and, to a lesser extent, from perceptions of closeness to artists and other listeners" (bogt, mulder, raaijmakers & gabhainn, 2011). the fact that young people are easily influenced by the music they listen to would cause them to follow the social trends those songs evoke (babtrakinova, voloshko & snistaryova, 2017). music, being one of the most beneficial and influential methods to engage with people (papinczak, z. & dingle; genevieve & stoyanov, s.; hides, l. & zelenko, o., 2015), is actually depicting a changing society that lacks an established set of values which could correlate with music preferences (rentfrow, p., & gosling, s., 2003) and social standards, therefore affecting life as a whole, therefore, can explicit music content influence the perception of adolescents towards trendy social standards and own ethical aspects of their personality? 1.1 music as a resource music is one of the main resources of intellectual, cultural, artistic and, why not, technological and economical depth. young people spent large amounts of money and time listening to music and the music industry is aware of that and they generate a great deal of music through different communication channels (roberts, henriksen, & foehr, 2009). "music is their soundtrack during this intense developmental period". on average, adolescents listen to music for up to three hours daily and accumulate more than 10,000 hours of active music listening throughout adolescence (roberts et al., 2009; tarrant, north, & hargreaves, 2000; zillmann & gan, 1997). due to the fact that they are constantly listening to music they become young music experts. https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.tandfonline.com/doi/full/10.1080/02673843.2011.650182?src=recsys https://www.tandfonline.com/doi/full/10.1080/02673843.2011.650182?src=recsys https://www.tandfonline.com/doi/full/10.1080/02673843.2011.650182?src=recsys https://www.tandfonline.com/doi/full/10.1080/02673843.2011.650182?src=recsys multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 48 starting by the close relationship between music and teenagers, it is stated that when a teenager is going through emotional distress or low stability, it is music that consoles them and regulate moods and emotions (taruffi & koelsch, 2014). teenagers, no doubt, engage with music and if that piece of music contains strong lyrics, they might engage with them and make them part of their lives. besides, music serves as therapy when working through conflictive situations and lets feelings arouse (laiho, 2002; sloboda, 1992; ziv, 2004). 1.2 music as a value family passes values from generation to generation (ilisko, d. & kravale-paulina, m., 2015), they are linked to ethics and morals inherited from our ancestors. self-exploration, selfimprovement and self-recognition are present in the human value system and guide our patterns of activity, which in turn are intrinsic to the behaviour pattern itself (wilson & dufrene, 2009). our values guide our actions, emotions and behaviour, which is significant for different fields of study such as sociology, philosophy, education and psychology. values prepare us to choose actions according to consequences involved in those actions. music is universal and so are values; music can change perceptions and so can values. according to abbott (2002:10), songs express cultural aspects such as feelings or ideas, which can be received through the listening of them. all in all, music is a multi-purpose tool for young people, as it comes with traditional cultural baggage as well as new social trends. 1.3 music nowadays nowadays, popular songs overuse sexual vocabulary and insults as demonstrated by various studies by hall et al. (2012) about sexualisation in the billboard top music lyrics 1959-2009, also reported by martino et al (2006) regarding consumption of music with degrading sexual references. songwriting experts claim that explicit content in music occurs so often that the only way to call https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 49 the attention of listeners is to add even stronger language. all in all, patterns of behaviour are portrayed by music. music is said to a reflection of social beliefs and attitudes therefore, taking into account the global crisis of values young people are going through sortheix, f. & parker, p.; lechner, c. & schwartz, s., 2017) and today´s music lyrics reflecting general moods and views young people have, music could be the savior, a tool to engage with young people from an early age and influence their ethical values, which then will turn into positive attitudes. 1.4 music as an engaging tool to choose the appropriate type of music is essential to young people use it as a tool (fonsecamora, m.c.; toscano-fuentes, c.,& wermke, k. (2011) for different needs, such as a calming environment, a family atmosphere, a friendly exciting environment, mood regulation (saarikallio, s., & erkkilä, j. (2007), as the more understanding of the lyrics of a song, the more aware an individual would become. themes and references in popular music lyrics do matter since it focuses on the fact that popular music reflects what is on the minds of the artist as well as of the individual who consumes that type of music; after all, lyrics mirror social trends related to worries, concerns, aspirations (christenson, p. g., de haan-rietdijk, s., roberts, d. f., & bogt, t., 2019). the emotional part of this experiences is valuable as it moves and changes perceptions of people and create the "domino" effect, causing human nature value reconsideration and attitude change (rokeach, 1973). 2.methods and instruments moved by the research question: can explicit music content influence the perception of adolescents towards trendy social standards and own ethical aspects of their personality? the https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 50 approach of the study was fundamentally psychological in the sense that the aim was to discover internal explanations for human behaviour. the first stage involved searching for related past literature on how lyrical content may affect attitudinal change on young people, followed by a collection of top singles from official international music site covering usa and british music (billboards.com) based on popularity and sales (2014-2019) with the aim to confirm the relaxed social and ethical attitude portrayed in songs, mainly by an overuse of profanity. the songs chosen were love/melodic as easy-listening would not generally disturb anyone and be generally enjoyed. the second phase of the research, once permission from school managers was granted, implied the quantitative method (creswell & clark, 2017) as these proved to be easy, fast and costeffective taking into account the number of participants. internet search tools, paper surveys, recording systems and statistics applications were used for both methods in order to reach utmost reliability, validity and objectivity. a total of 76 secondary students (36 males and 40 females in total) divided into three groups of 20 students and one group of 16 students (in this case, these subjects belonged to the bilingual branch offered by that particular school and were studying music at the local conservatory) aged 16 from four different local schools, were addressed based on purposive sampling (saunders et al., 2003: 175) as they most suited this research project mainly due to their age (still on a developmental stage) and experience with music, which would reveal awareness of the matter and repercussion on their daily routines. the students were encouraged to take part in the survey by providing research credit. besides, in order to interpret in-depth information about the awareness of the matter and views of adolescents on profanity and explicit content affecting their personality, two short-open questions were posed during the ethics class, as it was considered to be a topic for debate regarding critical pedagogy. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 51 as the atmosphere was confidential and at the same time it provided an entertaining environment for the students, that allowed the participants to feel free to respond to the posed questions and even discussion afterwards in order to challenge views (saunders et al., 2003: 270). this method allowed freedom of speech among the participants and a place for debate to compare different views and obtain a general overview of the beliefs and provide new insights based on experiences and meanings provided by lyrics. the methods used may be self-conducted and predict similarities in future research for similar groups of participants. 3. results starting from the point of knowledge provided by previous literature related to the influence of music lyrics on adolescents, the hypothesis of this work was: exposure to explicit music content would lead young people to follow social unethical trends evoked in it. in order to analyse results, the inferential statistical method was used (glass & stanley, 1980), generally applied to social studies. firstly, from a lexicon point of view regarding the use of profanity, after the analysis of 65 love/melodic songs in search for the most common used profane words, eleven different strong words were found (ass, bitch, damn, dick, fuck, hoe, nigga, piss off, pussy, shit, whore), however, the highest occurrence of profanity was represented by the f-word and the s-word followed by the b-word and n-word. secondly, the participants were asked to chose between explicit or non-explicit music content and give reasons behind their choices. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 52 out of 72 participants, results reveal that 80.6% preferred the explicit version as they expected real feelings from the artist and they considered this was a better and more emphatic way to provoke reactions in the listener. on the other hand, 16.7% of participants preferred the nonexplicit version as they believed for love songs it would be preferable to have softer, nicer, kinder lyrics. finally, 2.8% showed no preference between the two versions as they considered them the same. 2.when the participants were asked if they thought explicit music content would actually influence their personal ethical values and vision of life, the responses were: "with good foundations and manners behind my personality should stay firm and not change", "nobody and nothing can change my personality or the way i envisage life", "strong language is something normal nowadays and nobody finds it disturbing", "lyrics are just words, not actions", "rhythm is what really matters, so in that sense, i do not pay attention to music lyrics", "it only affects you if you live in similar environments of those shown in the song and if you are easily impressionable", "it may influence your personality as it is related to everyday situations and if you are going through the same", "only if you are really young, it may influence you"-"personality does not change but language may be affected" (various subjects, 2019). out of 72 participants, 70 claimed that explicit content in songs would generally not change or affect their personality if well-established, as lyrics are just words and, after all, they mostly paid attention to the rhythm of the song. they also stated that profanity in music is normalised in everyday situations and did not call the attention of people as much. on the other hand, only 2 participants were concerned about young people being influenced in terms of language if they were continuously exposed to strong lyrics. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 53 4. discussion music, in all its representations, has no doubt been affected, showing an increase of profanity (hall et al, 2012), probably due to the influence of other music genres and the effect of globalization (bauman, 1998). relaxation of social standards and values is a fact and that is reflected in song lyrics, considered as just words by young people, who in turn prefer direct expressions rather than epithets. as nowadays profanity has flooded all top music charts, romantic music material has also experienced it, therefore impacting on adolescents´ perceptions of what is evoked in them, causing them to actually prefer explicit material as they consider it real therefore there is general acceptance of profanity (coyne et al., 2011) however, at the stage of this work no relevant impact on their physical and physiological development as they claimed to have a well-established personality. as the aim of the research was to assess the influence of explicit content on their personal values, statistics showed that most participants would not consider explicit content as an influential factor in their lives, however as it is a long-term effect and the participants were still going through developmental stages, more time would be needed to confirm this hypothesis. this statement provided an overview of young people claiming to have established personality pillars to guide their actions and although music lovers, they preferred things to be called by their names, in the sense that they were used to profanity in their lives and considered that type of language as standard, a type of language that was direct and honest. all in all, it can be stated that their attitude towards profanity was relaxed, therefore not much of a negative impact on their values could be perceived at that stage; more time would be needed for an adolescent to complete their developmental stage and be able to observe any relevant behavioural impact. this aspect was of importance to realise that adolescents considered they has a https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 54 solid personality able to face unethical circumstances, which could also be used for ethical academic purposes by combining different resources such as command of a language and music can provide a different perspective of other cultures (abbott, 2002). most participants claimed that explicit language in songs would not change or affect their personality if well-established, as lyrics are just words and, after all, they mostly paid attention to the rhythm of the song. they also stated that profanity in music was normalised in everyday situations and did not call the attention of people as much. at this stage, h1 could not be confirmed. 4.1 findings the assessment of the correlation between profanity reflected by social trends in music and acceptance of it (coyne et al., 2011) by adolescents as a consequence of the relaxation of social values provides scientific contributions that can be applied to support, namely, social, cultural and academic fields of study. the findings provided guidance to ethics professionals, who can use music to create critical thinking about the different cultures evoked in the lyrics depending on the language. this would, in turn, to improve the understanding of values among all communities and the importance of social responsibility. the attractiveness of music for adolescents led to active participation and revealed sets of selfclaimed ethical values shared amongst them, resulting in the majority of participants providing similar statements. the analysis of data obtained provided social as well as academic relevance as this research backs up studies carried out in the past to corroborate similar results, especially concerning the https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 55 close relationship between music and adolescents (taruffi & koelsch, 2014) and impact on social behaviour (wilson & dufrene, 2009), this could would broaden their critical skills and be used and applied for academic purposes (fonseca-mora, m.c.; toscano-fuentes, c.,& wermke, k., 2011). some profane words seem to be overused, which shows an attitude of relaxation towards them in contrast with most powered-words which are less commonly used probably due to the rank of power-charged words that society has established. the results would benefit all parties involved in the study as it raise general awareness about a topic seen from a parental, academic and social point of view. on the other hand, the strengths of the study included available fast, low-cost data gathering which enhanced the accuracy and credibility of the research and easiness for replication. 4.2 limitations and recommendations the age of the groups only showed perspectives from specific age group, therefore generalization may not be applied to other age groups. as recommendations, future research on how explicit songs may also affect l2 learners is advised as it could broaden the literature of how foreign language are perceived by listening to music and repercussion on their social or language skills as well as ethical standards. 5. concluding comments music provides the appropriate scenario to engage with people ((papinczak, z. & dingle; genevieve & stoyanov, s.; hides, l. & zelenko, o., 2015) regarding universal ethical values which, no doubt, will provide young people with beneficial outcomes such as the improvement and appreciation of relationships. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 56 adolescents were proud to claim their personality could be not be affected by explicit content evoked today in music, which revealed maturity at that age when dealing with constructs difficult to measure in a short period of time. as young people are constantly being influenced by different aspects of life, explicit content in music may not, at this stage, be as relevant as previously expected. on the other hand, the results proved to be efficient to provide a framework to change perceptions and help enhance values using something as revealing as music. as music is an enjoyable activity that expresses feelings (laiho, 2002; sloboda, 1992; ziv, 2004) and values of society, people can benefit from it, especially during academic periods, which in turn will bring discussion in the ethics class and could collaborate to palliate the crisis of value and create a more critical perspective in students and improve quality of life. references abbott, m (2002). using music to promote l2 learning among adult learners. tesol journal 11,1 (2002): 10-17. amuthavalli, t., & sridevi, m. (2014). a study of the value orientation of degree college students. mier journal of educational studies, trends & practices, jammu: model institute of education & research, v. 4, n. 2, p. 262-271. aubrey, j. s. (2006). effects of sexually objectifying media on self-objectification and body surveillance in undergraduates: results of a two-year panel study. journal of communication,56,pp.366-386. https://doi.org/10.1111/j.14602466.2006.00024.x babtrakinova o.i., voloshko a.a., & snistaryova p.a. (2017). influence of modern music on young generation. human and society, 2(3), 4-6. https://doi.org/10.21661/r-462119 bauman, z. (1998). globalization, the human consequences. columbia university press; https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1111/j.1460-2466.2006.00024.x https://doi.org/10.21661/r-462119 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 57 bogt, t., mulder, j., raaijmakers, q., & gabhainn, s. (2010). moved by music: a typology of music listeners. psychology of music. https://doi: https://doi.org/10.1177/0305735610370223 coyne, s., stockdale, l., nelson, d., & fraser, a. (2011). profanity in media associated with attitudes and behavior regarding profanity use and aggression. pediatrics. https://doi.org/10.1542/peds.2011-1062 creswell, j. w., & clark, v. l. p. (2017). designing and conducting mixed methods research. london: sage publications christenson, p. g., de haan-rietdijk, s., roberts, d. f., & ter bogt, t. f. m. (2019). what has america been singing about? trends in themes in the u.s. top-40 songs: 1960–2010. psychology of music, 47(2), 194–212. https://doi.org/10.1177/0305735617748205 dewall, c. n., pond, r. s., jr., campbell, w. k., & twenge, j. m. (2011). tuning in to psychological change: linguistic markers of psychological traits and emotions over time in popular u.s. song lyrics. psychology of aesthetics, creativity, and the arts, 5(3),200–207. https://doi.org/10.1037/a0023195 fonseca-mora, m.c.; toscano-fuentes, c.,& wermke, k. (2011). melodies that help: the relation between language aptitude and musical intelligence. anglistik international journal of english studies. 22(1):101-118. fouce, h. (2010) de la crisis del mercado discográfico a las nuevas prácticas de escucha. comunicar (34). pp. 65-72. issn 1134-3478. glass, g.v., & stanley, j.c. (1980). métodos estadísticos aplicados a las ciencias sociales. barcelona:editorial prentice/hall internacional hall, p., west, j. h. c., & hill, s. (2012). sexualization in lyrics of popular music from 1959 to 2009: implications for sexuality educators. sexuality & culture: an interdisciplinary quarterly,16, pp.103-117. https://doi.org/10.1007/s12119-011-9103-4 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1177/0305735610370223 https://doi.org/10.1542/peds.2011-1062 https://doi.org/10.1177/0305735617748205 https://psycnet.apa.org/doi/10.1037/a0023195 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 58 ilisko, d. & kravale-paulina, m. (2015). sustainability of family values through the generation as viewed by the bachelor program students. society, integration, education. proceedings of the international scientific conference. https://doi.org/10.17770/sie2013vol1.1541. juslin, p. n., liljeström, s., västfjäll, d., & lundqvist, l.-o. (2010). how does music evoke emotions? exploring the underlying mechanisms. in p. n. juslin & j. a. sloboda(eds.), series in affective science. handbook of musi.c and emotion: theory, research, applications (p. 605–642). oxford university press knobloch-westerwick, s., musto, p., & shaw, k. (2006). rebellion in the top music charts: defiant messages in rap/hip hop and rock music— 1993-2003. presented at: the international communication association conference; dresden, germany. kumar, v. (2015). essential principles for quality education. international journal of multidisciplinary educational research, visakhapatnam: sucharitha publications, v. 4, n. 5, p. 19-29. laiho, s. (2002). musiikki nuorten kehityksen ja psyykkisen itsesäätelyn tukena, ma thesis. university of jyväskylä, finland. martino, s., collins, r.; elliott, m.; strachman, a., kanouse, d., & berry, s. (2006). exposure to degrading versus nondegrading music lyrics and sexual behavior among youth. pediatrics. https://doi.org/10.1542/peds.2006-0131 pajaziti, a. (2014). values and academia: a sociological-empirical approach. revue des sciences politiques, craiova: editora universitaria craiova, v. 43, p. 125-140. papinczak, z. & dingle; genevieve & stoyanov, s.; hides, l. & zelenko, o. (2015). young people's uses of music for wellbeing. journal of youth studies. 18. https://doi.org/10.1080/13676261.2015.1020935. https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.researchgate.net/deref/http%3a%2f%2fdx.doi.org%2f10.17770%2fsie2013vol1.154 https://doi.org/10.1080/13676261.2015.1020935 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 59 rentfrow, p., & gosling, s. (2003). the do re mi's of everyday life: the structure and personality correlates of music preferences. journal of personality and social psychology. https://doi.org/10.1037/0022-3514.84.6.1236 roberts, d. f., henriksen, l., & foehr, u. g. (2009). adolescence, adolescents, and media. in r. m. lerner & l. steinberg (eds.), handbook of adolescent psychology: contextual influences on adolescent development (p. 314–344). john wiley & sons inc. rokeach, m. (1973). the nature of human values. new york: free press. saarikallio, s., & erkkilä, j. (2007). the role of music in adolescents’ mood regulation.psychology of music, 35, pp.88-109. saunders, b., sim, j., kingstone, t., baker, s., waterfield, j., bartlam, b., & jinks, c. (2017). saturation in qualitative research: exploring its conceptualization and operationalization. quality and quantity, 1–15. https://doi.org/10.1007/s11135-017-0574-8 sloboda, j.a. (1992). empirical studies of emotional response to music, in m.r. jones and s. holleran (eds) cognitive bases of musical communication, pp. 33–45. washington, dc: american psychological association71–104. new york: oxford university press. sortheix, f. & parker, p,; lechner, c. & schwartz, s. (2017). changes in young europeans' values during the global financial crisis. social psychological and personality science. https://doi.org/10.1177/1948550617732610. tarrant, m., north, a.c., & hargreaves, d.j. (2000). english and american adolescents’ reasons for listening to music. psychology of music, 28, 166–173. taruffi, l., & koelsch, s. (2014) the paradox of music-evoked sadness: an online survey. plos one 9(10): e110490. https://doi.org/10.1371/journal.pone.0110490 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.researchgate.net/deref/http%3a%2f%2fdx.doi.org%2f10.1037%2f0022-3514.84.6.1236 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0145 https://doi.org/10.1177/1948550617732610 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.14001 social and technological sciences issn: 2341-2593 sánchez-gonzález (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 45-60 | 60 zillmann, d., & gan, s.-l. (1997). musical taste in adolescence. in d. j. hargreaves & a. c. north (eds.), the social psychology of music (p. 161–187). oxford university press. ziv, n. (2004) the relationship between social circumstances and recounted musical experiences, paper presented at the 8th international conference on music perception and cognition, evanston, il, august. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 92 activity preferences of generation z students for tertiary physical education: implications for curriculum enhancement ruben l. tagare, jr.a , geraldine d. villaluzb a instructor, department of physical education, institute of sports, physical education, and recreation, university of southern mindanao, kabacan, 9407, philippines b parttime faculty, school of education, university of san carlos, cebu city 6000, philippines * correspondence: tagareruben@usm.edu.ph received: 16 january 2020; accepted: 26 february 2021; published: 1 march 2021 abstract this research aimed to look into generation z students' preferred activities to enhance the curriculum content of tertiary physical education in one of the leading universities in the philippines using the phenomenological research design due to the fact that a great number of students in physical education have dropped out. a total of 110 research participants were involved in focus group discussions, in-depth interviews, and extensive field observation. results from focus group discussions, interviews, and extensive observations resulted to five concerns: less lectures, practical activities needed, minimal competition programs, and group activities. findings from this study served as basis for a contextualized physical education program for generation z students in southern philippines. keywords: activity preference in physical education; generation z students; enhancing tertiary physical education; phenomenology research to cite this article: tagare, r., & villaluz, g. (2021). enhancing tertiary physical education by determining the activity preferences of generation z students. multidisciplinary journal for education, social and technological sciences, 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0003-1628-6229 https://orcid.org/0000-0002-3929-6502 https://doi.org/10.4995/muse.2021.15492 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 93 1. introduction generation z students have gained a lot of interest because of their distinct characteristics and behaviors. one of their generation's significant distinctions is that they were born in a world where technology is pervasive. the biographies of these students consistently suggest that mentors in the academe must consider these new students’ characteristics to cultivate them efficiently. generation z was born between 1995 and 2010 (rothman, 2014), accounting for one-third of the philippine population. their age is thought to be remarkable in many respects since they did not see computers, the internet, and technology as merely weapons but as an essential part of their lives (philippine statistics authority, 2019). generation z students were seen as digital natives in the global culture, educated in a technologically advanced society, nurtured in various environments and terrorism issues, and linked globally in the virtual home. despite being more innovative, more adaptable, and inclusive of different cultures, generation z students face challenges and issues that will be significant concerns for many industries in the coming years. they are egocentric mammals who imagine themselves as ceos and owners with their own companies rather than workers after college graduation. generation z students are ready to get what they want right away, prematurely old, pampered, and risk-averse (singh, 2014). researchers discovered that in the united states, most educators in the academe made appropriate changes to their instructional strategy to make it more immersive and experiential-based, considering the characteristics of generation z students. they also discovered that generation z students' distinct and unusual characteristics pose a problem in conventional classroom structures. educators now recognize that the traditional teaching–learning approach is no longer reliable and applicable for these students (skiba and barton, 2006). since 2018, philippine higher education institutions have been welcoming pioneer senior high school graduates, and research has shown that there is a need for new approaches in the teaching-learning phase, which would be a step toward permanently changing the landscape of philippine education in irreversible ways. generation z students were extremely busy, creative, multi-taskers, digitally literate, and ethnically diverse. scholars have identified this generation as https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 94 self-assured, unique, under strain, team-oriented, and effective. since they depend on good results to gain high scores and marks, academic competitions are intense and occur regularly to this generation (debard, 2004). apart from non-participative students during class activities, the growing number of dropouts in physical education classes in one of the leading universities in the philippines is a significant concern. table 1 graph of the total number of dropouts in physical education classes in one of the leading state universities in the philippines (information and communication technology office, 2019). in the year 2020, an inquiry was undertaken to find a clear solution to this issue by describing the flaws in the implementation of tertiary physical education in the abovementioned school from the perspective of generation z students. the investigation revealed five (5) significant concerns that transpired during the triangulation of the results. these were: duplication of activities from senior high school physical education, stress due to very high physical fitness test standards, very timeconsuming take-home tasks in physical education that affect their time for other matters, very high teachers’ expectations, and teachers’ absenteeism. apart from the students' problems, the program quality of the new tertiary physical education of the university was one of the leading factors in the growing number of dropout students. as a result of this analysis, it is proposed that the existing 30 32 43 56 0 10 20 30 40 50 60 1st semester, ay 20172018 (30) 2nd semester, ay 20172018 (32) 1st semester, ay 20182019 (43) 2nd semester, ay 20182019 (56) https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 95 curriculum be revisited and updated and suggest contextualizing the course material by adding the local notions and preferred activities of generation z students (tagare and villaluz, 2019). with the current problem found, this study aimed to look into the activity preferences of generation z students in their tertiary physical education classes as a way to strengthen the course offering. this research shows how educational content can be enriched by integrating local knowledge and how learning environments can be strengthened by grasping native principles as a platform for enhancing a curriculum that is tailored to students' generation patterns, preferences, and expectations. this phenomenology investigation has created a venue for generation z students to express themselves by airing out their opinions and suggestions, which acted as the base principles for understanding their behavior interests in tertiary physical education. critical themes were generated and considered to improve the physical education curriculum content and offerings. this analysis should be used as a guide for program authors, instructors, and scholars to show that instructional improvement can be achieved by looking into students' behavior to tailor learning environments to their preferences and interests. considering the design of this study, a research question was formulated: 1. what are the preferred activities of generation z learners for tertiary physical education? 2. methods and materials 2.1 research design this research employed a qualitative research design, specifically applying the steps of the phenomenology approach. the experiences of generation z students in their physical education classes were analyzed through multiple data collection procedures to create a realistic portrait of how they behave, think, and talk to realize the outcomes of this study. phenomenology is an analytical description of a scenario, traditionally written in the third person and empirically based on data gathered from researchers at a field site (creswell, 2012). in this analysis, the researcher s https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 96 invested enough time in the actual locations of the research participants for focus group discussions, in-depth interviews, non-obtrusive participant observations, journal writing, and field notes to accurately gather the correct evidence needed to closely address generation z’s preferred meaningful activities for physical education. 2.2 research procedures to perform a qualitative phenomenology study, it was necessary first to seek approval from the head of the schools and acquire an informed consent contract from the research participants. following the approval of the required information, the researcher performed a series of orientations to clarify the study's goals and the costs and advantages of the participants' participation. the qualitative analysis relied heavily on fieldwork. following the preliminaries, immersion in the actual study setting started to provide a more detailed and accurate image of generation z students' experiences in physical education. in this phenomenological approach, indepth interviews were conducted in sites where generation z students gather and spend their leisure hours. primary sources were obtained through unobtrusive observations, journal writing, memoing, and focus group discussions. to complement the primary evidence, nonobtrusive participant observation and journal writing were used. speech recorders, video camera recorders, and journal writing were used to collect the participants' responses. the best method for authentically capturing the attitudes and emotions of the study participants was to observe them in their natural environment, where they could easily share their feelings. 2.3 research participants since there are so many generation zs in the research setting, the sample participants were selected using a purposive sampling process. the key participants for this research were 110 generation z students who are born between 1995 2010. students from the university laboratory school, kabacan national high school, and tertiary students from the university of southern mindanao served as primary participants for this research. https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 97 2.4 research instrument in this study, casual and semi-structured dialogues were used to perform the interviews. open-ended guide questions were used to help capture the participants' responses. the guide questions were subjected to an ethics review to ensure that no derogatory contents could affect the research participants' emotional well-being. these approaches seem to be informal discussions, but they have a clear and implicit goal: to understand how generation z students conceptualize and arrange their interactions into concrete categories. 2.5 data analysis and interpretation word analysis, coding, and thematization of concepts were used to analyze and interpret the data. open codes were generated to classify the most important and recurring words and ideas mentioned by the research participants. after the open coding, axial coding was performed to build the relationship between concepts and exclude outlier ideas and phrases. after the coding, essential themes have been established. validation and triangulation were also used to ensure the trustworthiness and reliability of gathered data. 3. results and discussions research question 1. what are the preferred activities of generation z learners for tertiary physical education? as can be gleaned from the result, generation z students unveiled “inclusion of larong pinoy (filipino recreational activities),” “holistic activities for physical education,” “lessen the lectures, focus on practical activities,” “minimize the competition in physical education classes,” and “inclusion of group activities” as their preferred activities that they wish to be integrated into their tertiary physical education classes. these ideas are noticeable in the following excerpts of their responses below: 1. inclusion of larong pinoy (filipino recreational activities). the “larong pinoy,” or filipino recreational games, was one of the top responses from generation z students as an activity that they would like to be included in their physical https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 98 education classes. apart from its nostalgic importance as a representation of filipino culture, research participants clarified that larong pinoy provides filipino youths with entertainment, relaxation, and the growth of motor skills, strategic logic, social maturity, and problemsolving skills. they shared concern about preserving filipino traditional games, as these traditions are increasingly losing their importance due to the threat of digital era. they agree that having larong pinoy in physical education classes encourages nationalism and love for traditional filipino games and pastimes. they still like conventional games because they experience less strain and tension while playing them, as reflected in the following responses: “i would prefer including the larong pinoy (filipino recreational games) to retain it, and so that we could have more physical activities rather than just using gadgets…” (participant 70) “…it would be better if the larong pinoy (filipino recreational games) including luksong baka and other games will be rekindled and played by the youth today as these games are entertaining and are boredom-busters.” (participant 71) existing studies have established a connection between participation in recreational activities and academic achievement. the researchers discovered that time spent outdoors and participating in recreational activities was a good predictor of academic grades, even when achievement test scores were taken into account in a sample of middle school and college students. they discovered that participation in recreational activities helped predict academic achievement even after gender, grade level, ethnic origin, free lunch eligibility, and level of adult supervision after school were statistically controlled (barnett & weber, 2008). further, biddle & asare (2011) and eime et al. (2013), as cited by alanazi (2020), demonstrated that those who participate actively in recreational activities experience less symptoms of anxiety than those who do not. previous research findings indicate that active https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 99 involvement in recreation is a critical predictor of mental health (biddle & asare, 2011; eime etal., 2013; rafferty et al., 2016; johnson, connolly, & tully, 2017). additionally, the literature demonstrates that involvement in leisure activities increases self-esteem, self-efficacy, self-concept, chance for social interaction, and decreases alienation and loneliness. recreation has been shown to be beneficial in the treatment of different types of stress and anxiety (goldstein, 2012; paggi et al., 2016; whitebread et al., 2017). 2. holistic activities for physical education. additionally, the research participants favored holistic exercises and emphasized that their physical education classes do not solely concentrate on physical development. for them, learning is more enjoyable and meaningful if they can participate in various activities that help them grow their whole selves. additionally, it will boost their confidence and involvement, ultimately resulting in increased participation in physical education classes. physical education is an essential part of the educational process since it helps develop and balance a growing individual's physical, academic, social, and emotional dimensions by professionally chosen physical activities. physical education subjects must take a comprehensive approach to cater to the development of each learner's well-being by providing diverse but appropriate learning experiences. the following excerpts from the research participants' answers indicate that they still tend to involve holistic activities: “…i hope for the inclusion of mental, emotional, and social activities to enjoy and to learn for greater participation. it should not only focus in physical trainings.” (participant 35) “…it should not only focus on physical activities, i want to have an activity that is beyond physical development because there are also students who are not kinesthetically inclined, so even if they try harder, but they will also take ample time to learn a specific skill. with that, i want to suggest for us to experience holistic https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 100 activities that could bring development in physical, mental, and social…” (participant p31) numerous sources suggest that in the twenty-first century of teaching and learning, holistic education should be applied to new waves of students to foster a more realistic worldview, encourage more holistic development and nurture the relationship between the different facets of the personality (intellectual, physical, moral, mental, social, and aesthetic) (mahmoudi et al., 2012). bhardwaj (2016) reinforced this notion further in his inquiry into the value of holistic education in human life, revealing that education is not limited to acquiring knowledge. fundamentally, it should be about instilling holistic ideals in all students to develop into responsible people. additionally, his research indicates that in the educational program, it should be observed that values and holistic activities are integrated in an engaging manner such that students master them at any stage of their education. on the other side, the value must be emphasized equally since it is critical to students' holistic growth. 3. lessen the lectures – focus on practical activities generation z students desire to reduce lecture time in physical education classes, stating that they spend most of their day exposed to other topics taught through the lecture process. the research participants stated that they were dissatisfied with physical education due to the excessive lectures. additionally, the generation z students recommend that teachers incorporate stress-relieving activities into their lessons rather than relying solely on lectures. additionally, generation z students chose practical exercises in physical education over sitting and listening to their teachers' lecture to alleviate academic tension associated with applied and advanced courses. to help them appreciate physical education classes more, they should find enjoyment and enthusiasm in their learning activities to provide a brief respite from their stresses. as a result, their willingness to reduce the lectures is expressed in the following responses: https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 101 “…for it not to be boring, lectures should be lessened because other subjects are already doing lots of lectures. physical education should focus more on giving practical activities rather than lecturing the whole period” (participant 81) “…the teacher shouldn’t focus merely on lectures; physical education should focus on stress-relieving activities…” (participant 49) previous research findings reveal that schools can develop 21st-century knowledge by incorporating practical activities into the curriculum (morris, 2019). several scholars demonstrated that one effective strategy for fostering behavioral improvement is to provide students with dynamic and realistic experiences that train them for real-world jobs. as a result, multicomponent school-based interventions that typically provide enhanced practical activities are effective in students' physical activity (voogt, 2013). additionally, generation z students demonstrate an eagerness to participate effectively in the teaching–learning phase since they have a low attention period of about 20 minutes (proctor & prevatt 2009). recent research findings indicate that generation z students cannot sustain an excessive amount of lecture approaches because they get bored with the teacher-centered strategy, frustrated with the lack of opportunities for student activity, and disinterested in their passive involvement in the learning process (marshall, 2016). 4. minimize the competition in physical education classes prompted by the pressures of their applied and advanced courses, generation z students chose to minimize the intensely competitive learning activities in their physical education classes to alleviate the strain. additionally, they emphasized that making competitive group assignments is unjust because they do not share the same competencies; https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 102 thus, some will receive a high grade due to their talent while others will not because it is not their field. further, providing highly competitive activities in physical education make them more grade-oriented, which contradicts the central aim of physical education in the school curriculum which is to provide students with enjoyable and meaningful interactions that will help them realize the conviction to engage in lifelong physical activity to promote fitness. the following are excerpts from respondents' answers about why they choose to avoid competitions in their physical education classes: “…competition should be minimized. like groupings in dancing where you will be given grade as one group. i think it is not fair enough to depend the grade on it since the skills of each one differs from one another…” (participant 57) “for our physical education class, we prefer to lessen the highly competitive course works for us to have fun and enjoyment while learning. because if thorough competitions are still integrated into this subject, we tend to focus on beating other groups to earn better grades, which adds too much pressure. allocating so much time to prepare for competitions beyond class hours can ruin our schedules and add more stress to us…” (participant 69) scholars have shown that classroom competitions result in stress and anxiety, especially when they encourage academic competitiveness among individual students. this tension can cause students to prioritize academics over other interests and extracurricular activities, resulting in an unbalanced school experience. a less-than-perfect outcome will affect the child's feelings since they may be upset if they do not succeed, which will erode their trust and self-confidence (bao, 2009). additionally, if a classroom competition does not have an attainable target for all students, the game becomes unbalanced, with certain students leaping forward and others falling behind. this is the kind of rivalry that can result in distress and a decreased motivation https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 103 to engage in similar activities. an unhealthy rivalry could be solely based on competing rather than honing a new talent or functioning cooperatively (wilkins, 2012). 5. inclusion of group activities. the last activity that generation z students desire to incorporate into their tertiary physical education courses is group activities. they prefer group collaborations over individual tasks because they believe that it foster a more harmonious relationship with their peers and boost their morale while completing an academic mission. it is much more beneficial to include community activities in tertiary physical education to alleviate their shyness and produce high-quality outputs through their collective efforts. additionally, they prefer group practices in tertiary physical education because they can share thoughts and inputs with their classmates. their ability to incorporate community activities into their physical education class is shown in the following responses: “…if you have groupmates then you tend to blend with the group where you don’t get shy compared to that of being alone where you get shy most of the time” (participant 52) “…it’s better if it is with groups because you have each other in doing the activities…” (participant 12) this viewpoint is consistent with lafave's (2019) observations that group activities are recommended in the twenty-first-century teaching-learning phase because it fosters collaboration, strategic thought, enhances partnerships, broadens logical viewpoints, shakes things up, and increases self-confidence, both of which generation z students need more in preparation for the world of work. choi & yong (2015) elaborated on the benefits of group activities for students, stating that they enable them to tackle more complex problems than they may on an individual assignment, practice delegating roles and responsibilities, share diverse https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 104 perspectives, provide a pool of knowledge and skills, receive social support and encouragement to take risks, and develop new aptitudes. 4. conclusions based on the findings, this research concludes that generation z students are under stress and pressure due to their academic pursuits, as shown by their responses during interviews. this involves academic course works and demands from their teachers and society, which urges their desire to eliminate intensely competitive academic activities in physical education to de-stress. they prefer to incorporate recreational activities that allow students to have fun and enjoy themselves while they learn. generation z students also value friendship, as shown by their preference for social activities and the reduction of highly competitive academic tasks in physical education. additionally, this study concludes that generation z students retain a sense of patriotism, as shown by their preference to include larong pinoy (filipino recreational activities), which they fear would soon become extinct due to technological advancement. additionally, this study concludes that students preferred to engage in various activities that focused on their holistic development. finally, this study concludes that the research participants are committed students, as shown by their enthusiasm for increased participation in their physical education classes. generation z students prefer fewer lecture lessons and more time for practical and valuable tasks over sitting and listening to their teachers. acknowledgements: the researchers would like to thank all individuals who have contributed to the success of this study. to the examining panel, dr. maria rosita a. hernani, dr. antonio batomalaque, and dr. rita may p. tagalog for the substantial information and constructive criticisms that they have provided for the betterment of this study. to the commission on higher education, university of san carlos, and university of southern mindanao for supporting the researchers and providing necessary favors throughout the study. to the research participants, for their time allotted to participate in the data gathering and validation. above all, to almighty god for the strength, guidance, and protection. funding: this research was funded by the philippine commission on higher education. conflicts of interest: the authors declare no conflict of interest. the funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 105 references alanazi, h. (2020). the effects of active recreational maths games on maths anxiety and performance in primary school children: an experimental study. multidisciplinary journal for education, social and technological sciences (pp.89-112). https://doi.org/10.4995/muse.2020.12622 bao, c. (2009). within the classroom walls: critical classroom processes, students' and teachers' sense of agency, and the making of racial advantages and disadvantages. [published dissertation, boston college, proquest dissertations publishing]. barnett, l. a., & weber, j. j. (2008). perceived benefits to children from participating in different types of recreational activities. journal of park & recreation administration, 26(3). biddle, s. j. h., & asare, m. (2011). physical activity and mental health in children and adolescents: a review of reviews. british journal of sports medicine, 45, 886-895. http://dx.doi.org/10.1136/bjsports2011-090185. bhardwaj, a. (2016). importance of education in human life: a holistic approach. international journal of science and consciousness. vol. 2 | no. 2, pages 23-28. http://www.ijsc.net/docs/issue4/importance-of-education-in-human-life.pdf choi, j., & yong, h. (2015). conceptual group activity recognition model for classroom environments. piscataway: the institute of electrical and electronics engineers, inc. (ieee). creswell, j. w. (2002). educational research: planning, conducting, and evaluating quantitative (pp. 146166). upper saddle river, nj: prentice hall. debard, r. (2004). serving the millennial generation, millennials coming to college (pp. 33-45). new directions for student services http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.472.8236&rep=rep1&type=pdf eime, r. m., young, j. a., harvey, j. t., charity, m. j., & payne, w. r. (2013). a systematic review of the psychological and social benefits of participation in sport for children and adolescents: informing development of a conceptual model of health through sport. international journal of behavioral nutrition & physical activity, 10, 98-118. https://doi.org/10.1186/1479-5868-10-98. goldstein, j. (2012). play in children’s development, health and well-being. toy industries of europe. johnson, a., connolly, p., & tully, m. a. (2017). school-based physical activity interventions and wellbeing in children: a systematic review and intervention complexity assessment. https://campbellcollaboration.org/media/k2/attachments/ecg_johnson _title.pdf. lafave, s. e., granbom, m., cudjoe, t. k. m., gottsch, a., shorb, g., & szanton, s. l. (2019). attention control group activities and perceived benefit in a trial of a behavioral intervention for older adults. research in nursing & health, 42(6), 476-482. https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4995/muse.2020.12622 http://dx.doi.org/10.1136/bjsports-2011-090185 http://dx.doi.org/10.1136/bjsports-2011-090185 http://www.ijsc.net/docs/issue4/importance-of-education-in-human-life.pdf https://doi.org/10.1186/1479-5868-10-98 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 tagare and villaluz (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 92-106. https://doi.org/10.4995/muse.2021.15492 106 mahmoudi, s., jafari, e., nasrabadi, h., & liaghatdar, m. (2012). holistic education: an approach for 21 century. international education studies, v5 n2 p178-186. https://eric.ed.gov/?id=ej1066819 marshall, d. (2016). diagnostic validity of attention-deficit/hyperactivity disorder in college students: a comparison of dsm-iv-tr and dsm-5 [published dissertation, the florida state university, proquest dissertations publishing]. morris, n. (2019). to the 21st century, and beyond! investigating the practical ways that secondary school teachers can develop the “21st century competencies” in their students [published dissertation, university of toronto (canada), proquest dissertations publishing]. paggi, m., jopp, d., & hertzog, c. (2016). the importance of leisure activities in the relationship between physical health and well-being in a life span sample. gerontology, 62, 450-458. https:/doi.org/10.1159/000444415. philippine statistics authority. (2019, june 14). youth: psa. philippine statistics authority. https://psa.gov.ph/content/youth proctor, b. e., & prevatt, f. (2009). confirming the factor structure of attention-deficit/hyperactivity disorder symptoms in college students using student and parent data. journal of learning disabilities, 42(3), 250-9. rafferty, r., breslin, g., brennan, d., & hassan, d. (2016). a systematic review of school-based physical activity interventions on children’s wellbeing. international review of sport and exercise psychology, 9(1),1-16. https://doi.org/10.1080/1750984x.2016.1164228 rothman, d. (2016). a tsunami of students called generation z. international journal of social sciences. http://www. mdle. net/joumafa_tsunami_of_students_called_generation_z. pdf. singh, a. (2014). challenges and issues of generation z. iosr journal of business and management, 16, 5963. skiba, d. j., & barton, a. j. (2006). adapting your teaching to accommodate the new generation of students. online journal of issues in nursing, 11(2), 5. https://www.ncbi.nlm.nih.gov/pubmed/17201579 tagare, r. & villaluz, g. (2019). drawbacks in the delivery of tertiary physical education among the generation z students [unpublished masters’ thesis]. the university of san carlos. voogt, j., erstad, o., dede, c., & mishra, p. (2013). challenges to learning and schooling in the digital networked world of the 21st century. journal of computer assisted learning, 29(5), 403-413. wilkins, a. (2012). push and pull in the classroom: competition, gender and the neoliberal subject. gender and education, 24(7), 765. http://ezproxy.usc.edu.ph:2219/10.1080/09540253.2011.606207 whitebread, d., neale, d., jensen, h., liu, c., solis, s. l., hopkins, e., hirsh-pasek, k. zosh, j. m. (2017). the role of play in children’s development: a review of the evidence. the lego foundation, dk. https://doi.org/10.4995/muse.2021.15492 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://eric.ed.gov/?id=ej1066819 https://doi.org/10.1159/000444415 https://psa.gov.ph/content/youth https://doi.org/10.1080/1750984x.2016.1164228 https://www.ncbi.nlm.nih.gov/pubmed/17201579 http://ezproxy.usc.edu.ph:2219/10.1080/09540253.2011.606207 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 113 optimal reconfiguration of a limited parallel robot for forward singularities avoidance carlos llopis-albert1 , francisco valero1 , vicente mata1 , rafael j. escarabajal1 , pau zamora-ortiz1 , josé l. pulloquinga1 1centro de investigación de ingeniería mecánica (ciim). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding author: carlos llopis-albert, e-mail address: cllopisa@upvnet.upv.es received: 18 november 2019; accepted: 21 march 2020; published: april 2020 abstract the positioning of the anchoring points of a parallel kinematic manipulator has an important impact on its later performance. this paper presents an optimization problem to deal with the reconfiguration of a parallel kinematic manipulator with four degrees of freedom and the corresponding algorithms to address such problem, with the subsequent test on an actual robot. the cost function minimizes the forces applied by the actuators along the trajectory and considers singular positions and the feasibility of the active generalized coordinates. results are compared among different algorithms, including evolutionary, heuristics, multi-strategy and gradient-based optimizers. keywords: parallel robot; non-linear optimization; rehabilitation; trajectory; singularity 1. introduction currently, there is a growing interest in robot trajectory planning (dash, chen, yeo, & yang, 2005; rubio, llopis-albert, valero, & suñer, 2016; valero, rubio, & llopis-albert, 2019). different optimization approaches are being proposed for this kind of problems (llopis-albert, rubio, & https://doi.org/10.4995/muse.2020.13352 http://orcid.org/0000-0002-1349-2716 http://orcid.org/0000-0003-2295-4035 https://orcid.org/0000-0003-2255-0567 https://orcid.org/0000-0001-6535-9098 https://orcid.org/0000-0002-7284-2181 https://orcid.org/0000-0003-0403-4593 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 114 valero, 2018) including the notions, methods, and operations of mobile robots (rubio, valero, & llopis-albert, 2019). particularly, parallel kinematic manipulators (pkms) has drawn special attention. compared with serial robots, pkms can manage higher velocity, accuracy and load capability. however, they exhibit more limited workspace and forward kinematics singularities (arakelian, briot, & glazunov, 2008; gosselin & angeles, 1990; xianwen kong & gosselin, 2002), which entail a set of characteristics: a) at least one degree of freedom (dof) turns uncontrollable; b) they cannot resist some exerted wrenches; c) they are not able to leave such singularity without external help; d) the forces in its joints tend to infinity; and e) it is likely that the manipulator adopts another assembly configuration. this problem can be tackled by a rigorous trajectory planning of the robot’s end-effector, which must consider the avoidance of singularities within the workspace and actuation demands. the reconfiguration of the pkm can help with this task (patel & george, 2012). this paper addresses the geometrical redesign of a reconfigurable pkm (rpkm) meant for knee rehabilitation. the trajectories of the mobile platform of the rpkm depend on the patient’s rehabilitation procedure and cannot be easily adapted for singularity avoidance (araujo-gómez, díaz-rodríguez, mata, & gonzález-estrada, 2019; araujo-gómez, mata, díaz-rodríguez, valera, & page, 2017; vallés et al., 2018). the reconfiguration is treated as a non-linear optimization problem where the design variables are the positions of the four limbs linked to the fixed and mobile platforms, whereas the objective function comprises the total active force needed to follow a defined trajectory subject to several constraints on the value of the determinant of the forward jacobian and on the limit values allowed for the active generalized coordinates. this optimization problem is solved by means of various approaches, including evolutionary algorithms, heuristics optimizers, multi-strategy algorithms and gradient-based optimizers (yang, 2017). finally, the results can be compared despite the complexity that the assessment of these optimization algorithms imply (beiranvand, hare, & lucet, 2017). this paper is organized as follows: section 2 explains the kinematic and dynamic modeling of the 3ups+rpu pkm, including the intrinsic forward singularities and the optimization approach. https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 115 section 3 shows the application of the methodology to different cases, and section 4 states the conclusions. 2. methodology 2.1. kinematic model and forward singularities this paper deals with the optimization of a pkm reconfiguration in order to avoid forward singularities when moving around its workspace. the analyzed pkm is a reconfigurable robot with four dof (two translations and two rotations) for knee diagnosis and rehabilitation (vallés et al., 2018). this pkm is named 3ups+rpu by its architecture, where the underlined letter is the actuated joint. the universal, prismatic, revolute and spherical joints are represented by u, p, r, and s respectively. in fig. 1 is presented the kinematic modeling implemented for the 3ups-rpu pkm with 3 identical external limbs and a central limb. in this pkm the actuated joints are the prismatic ones. the fixed reference system is denoted by �𝑂𝑂𝑓𝑓 − 𝑋𝑋𝑓𝑓𝑌𝑌𝑓𝑓𝑍𝑍𝑓𝑓�, while the reference system attached to the mobile platform is given by {𝑂𝑂𝑚𝑚 − 𝑋𝑋𝑚𝑚𝑌𝑌𝑚𝑚𝑍𝑍𝑚𝑚}. figure 1. kinematic modeling for 3ups-rpu pkm. https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 116 the coordinates of the origin of the mobile reference system attached to the mobile platform are xm and zm. the angles rotated by the mobile platform regarding ym and zm are represented by θ and ψ, respectively. note that ym and the angle rotated regarding xm (𝜙𝜙) are always zero because of the pkm topology. the location of the connection points to the fixed platform is defined by the radius r, the angles βfd, βfi and the distance ds along the xf. regarding the mobile platform, the location of the vertices depend on rm, βmd and βmi. eventually, the geometric reconfiguration of the 3upsrpu pkm to be minimized is based on 7 geometrical parameters (r, βfd, βfi, rm, βmd, βmi and ds). this study uses these 7 geometrical parameters as the design variables. the modeling of the manipulator using denavit-hartenberg notation is developed by a set of 22 generalized coordinates 𝑞𝑞𝑖𝑖𝑖𝑖 (table 1). the subscript 𝑖𝑖 denotes the number of the limb and 𝑗𝑗 the coordinate within the limb, see fig. 1. table 1. general coordinates in denavit-hartenberg notation. joint i j αi ai di θi universal 1,2,3 1 -π/2 0 0 qij 1,2,3 2 π/2 0 0 qij prismatic 1,2,3 3 0 0 qij 0 spherical 1,2,3 4 π/2 0 0 qij 1,2,3 5 π/2 0 0 qij 1,2,3 6 π/2 0 0 qij revolute 4 1 -π/2 0 0 qij prismatic 4 2 -π/2 0 qij π universal 4 3 -π/2 0 0 qij 4 4 0 0 0 qij the inverse kinematic problem can be posed as a set of explicit expressions in function of the actuated generalized coordinates q13, q23, q13, q42 and the design variables: https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 117 (q13)2 = r2 + �2 ∙ xm − 2 ∙ cθ ∙ cψ ∙ rm� ∙ r + rm2 + +�2 ∙ zm ∙ sθ − 2 ∙ cθ ∙ cψ ∙ xm� ∙ rm + xm2 + zm2 (q23)2 = r2 − 2 ∙ r ∙ �� cθ ∙ cψ ∙ cfd ∙ cmd − sψ ∙ cfd ∙ smd + +cψ ∙ sfd ∙ smd + cθ ∙ sψ ∙ sfd ∙ cmd � ∙ rm + cfd ∙ xm� + +rm2 − 2 ∙ rm ∙ �sθ ∙ cmd ∙ zm + sψ ∙ smd ∙ xm − cθ ∙ cψ ∙ cmd ∙ xm� + +xm2 + zm2 (q33)2 = r2 − 2 ∙ r ∙ �� cθ ∙ cψ ∙ cfi ∙ cmi − sψ ∙ cfi ∙ smi + +cψ ∙ sfi ∙ smi + cθ ∙ sψ ∙ sfi ∙ cmi � ∙ rm + cfi ∙ xm� + +rm2 − 2 ∙ rm ∙ �sθ ∙ cmi ∙ zm + sψ ∙ smi ∙ xm − cθ ∙ cψ ∙ cmi ∙ xm� + +xm2 + zm2 (q42)2 = ds2 − 2 ∙ ds ∙ xm + xm2 + zm2 ⎭ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎫ (1) where cθ, sθ, cfd, sfd denote cos(θ), sin(θ), cos(βfd), sin(βfd), respectively. the relation between actuated generalized velocities and the velocities of the mobile platform is determinate by time derivate of the equations (1). the velocity relations through a matrix expression is: φ𝑎𝑎 ∙ � �̇�𝑞13 �̇�𝑞23 �̇�𝑞33 �̇�𝑞42 � = φ𝑥𝑥 ∙ ⎣ ⎢ ⎢ ⎡ �̇�𝑥𝑚𝑚 �̇�𝑧𝑚𝑚 �̇�𝜃 �̇�𝜓 ⎦ ⎥ ⎥ ⎤ (2) where φ𝑎𝑎 is the inverse jacobian and φ𝑥𝑥 the forward jacobian. an inverse singularity is presented when the determinant of φ𝑎𝑎 becomes zero, and a forward singularity occurs with determinant of φ𝑥𝑥 is equal to zero. for the pkm under study, the φ𝑎𝑎 is equal to the identity matrix, which prevent the occurrence of inverse singularities. on the other hand, the φ𝑥𝑥 is a function of the four dof of the mobile platform (xm, zm, θ, ψ). in that case, the 3ups+rpu pkm will undergo a forward singularity. https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 118 2.2. dynamic model the dynamic model of the parallel manipulator can be obtained by applying the d’alembert’s principle and the principle of virtual power (tsai, 1999): −𝑄𝑄�⃗ 𝑖𝑖𝑖𝑖 + φ𝑞𝑞 𝑇𝑇 ∙ 𝜆𝜆 = 𝑄𝑄�⃗ 𝑔𝑔𝑔𝑔𝑎𝑎𝑔𝑔 + 𝑄𝑄�⃗ 𝑒𝑒𝑥𝑥 + 𝑄𝑄�⃗ 𝑓𝑓𝑔𝑔𝑖𝑖𝑓𝑓 + 𝑄𝑄�⃗ 𝑎𝑎𝑓𝑓𝑎𝑎 φ𝑞𝑞 ∙ �⃗̈�𝑞 = 𝑏𝑏�⃗ (3) where: 𝑄𝑄�⃗ 𝑔𝑔𝑔𝑔𝑎𝑎𝑔𝑔 the gravitational generalized forces 𝑄𝑄�⃗ 𝑖𝑖𝑖𝑖 the inertial generalized forces 𝑄𝑄�⃗ 𝑓𝑓𝑔𝑔𝑖𝑖𝑓𝑓 the friction generalized forces 𝑄𝑄�⃗ 𝑒𝑒𝑥𝑥, the external generalized forces applied to the mobile platform 𝑄𝑄�⃗ 𝑎𝑎𝑓𝑓𝑎𝑎 the active generalized forces exerted by the actuators φ𝑞𝑞 the restriction jacobian matrix 𝜆𝜆 the vector of lagrange multipliers �⃗̈�𝑞 the generalized accelerations and 𝑏𝑏�⃗ the vector comprising the acceleration terms quadratic in velocities. the �⃗�𝑞 is a set of generalized coordinates from the active (independent) and passive joints (secondary), organized as: �⃗�𝑞 = � 𝑞𝑞11, 𝑞𝑞12, 𝑞𝑞21, 𝑞𝑞22, 𝑞𝑞31, 𝑞𝑞32, 𝑞𝑞41, 𝑥𝑥𝑚𝑚, 𝑧𝑧𝑚𝑚, 𝜃𝜃, 𝜓𝜓��������������������������� 𝑞𝑞13, 𝑞𝑞23, 𝑞𝑞33, 𝑞𝑞42����������� 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 (𝑞𝑞𝑠𝑠) 𝐼𝐼𝑆𝑆𝑆𝑆𝑆𝑆𝐼𝐼𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝐼𝐼 (𝑞𝑞𝑖𝑖) � 𝑇𝑇 (4) for the 3ups+rpu pkm, the φ𝑞𝑞 matrix is defined by deriving respect to �⃗�𝑞 the subsequent 11 constraint equations: https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 119 ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ 𝐶𝐶11 ∙ 𝑆𝑆12 ∙ 𝑞𝑞13 − 𝑅𝑅 − 𝑥𝑥𝑚𝑚 + 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝜃𝜃 ∙ 𝐶𝐶𝜓𝜓 −𝐶𝐶12 ∙ 𝑞𝑞13 + 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝜃𝜃 ∙ 𝐶𝐶𝜓𝜓 𝑆𝑆11 ∙ 𝑆𝑆12 ∙ 𝑞𝑞13 − 𝑧𝑧𝑚𝑚 − 𝑅𝑅𝑚𝑚 ∙ 𝑆𝑆𝜃𝜃 𝐶𝐶21 ∙ 𝑆𝑆22 ∙ 𝑞𝑞23 + 𝑅𝑅 ∙ 𝐶𝐶𝐹𝐹𝐹𝐹 − 𝑥𝑥𝑚𝑚 − 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝑀𝑀𝐹𝐹 ∙ 𝐶𝐶𝜃𝜃 ∙ 𝐶𝐶𝜓𝜓 + 𝑅𝑅𝑚𝑚 ∙ 𝑆𝑆𝑀𝑀𝐹𝐹 ∙ 𝑆𝑆𝜓𝜓 −𝐶𝐶22 ∙ 𝑞𝑞23 + 𝑅𝑅 ∙ 𝑆𝑆𝐹𝐹𝐹𝐹 − 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝑀𝑀𝐹𝐹 ∙ 𝐶𝐶𝜃𝜃 ∙ 𝑆𝑆𝜓𝜓 − 𝑅𝑅𝑚𝑚 ∙ 𝑆𝑆𝑀𝑀𝐹𝐹 ∙ 𝐶𝐶𝜓𝜓 𝑆𝑆21 ∙ 𝑆𝑆22 ∙ 𝑞𝑞23 − 𝑧𝑧𝑚𝑚 + 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝑀𝑀𝐹𝐹 ∙ 𝑆𝑆𝜃𝜃 𝐶𝐶31 ∙ 𝑆𝑆32 ∙ 𝑞𝑞33 + 𝑅𝑅 ∙ 𝐶𝐶𝐹𝐹𝐹𝐹 − 𝑥𝑥𝑚𝑚 − 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝑀𝑀𝐹𝐹 ∙ 𝐶𝐶𝜃𝜃 ∙ 𝐶𝐶𝜓𝜓 − 𝑅𝑅𝑚𝑚 ∙ 𝑆𝑆𝑀𝑀𝐹𝐹 ∙ 𝑆𝑆𝜓𝜓 −𝐶𝐶32 ∙ 𝑞𝑞33 − 𝑅𝑅 ∙ 𝑆𝑆𝐹𝐹𝐹𝐹 − 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝑀𝑀𝐹𝐹 ∙ 𝐶𝐶𝜃𝜃 ∙ 𝑆𝑆𝜓𝜓 + 𝑅𝑅𝑚𝑚 ∙ 𝑆𝑆𝑀𝑀𝐹𝐹 ∙ 𝐶𝐶𝜓𝜓 𝑆𝑆31 ∙ 𝑆𝑆32 ∙ 𝑞𝑞33 − 𝑧𝑧𝑚𝑚 + 𝑅𝑅𝑚𝑚 ∙ 𝐶𝐶𝑀𝑀𝐹𝐹 ∙ 𝑆𝑆𝜃𝜃 −𝑆𝑆41 ∙ 𝑞𝑞42 − 𝑥𝑥𝑚𝑚 + 𝑆𝑆𝑑𝑑 𝐶𝐶41 ∙ 𝑞𝑞42 − 𝑧𝑧𝑚𝑚 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ = 0�⃗ 11𝑥𝑥1 (5) grouping �⃗̈�𝑞 and 𝜆𝜆 the eq. (3) can be rewritten in matrix form, it can be expressed as follows: � 𝑀𝑀 �φ𝑞𝑞� 𝑇𝑇 φ𝑞𝑞 0 � ∙ ��⃗̈�𝑞 𝜆𝜆 � = � 𝑄𝑄�⃗ 𝑓𝑓𝑓𝑓 + 𝑄𝑄�⃗ 𝑔𝑔𝑔𝑔𝑎𝑎𝑔𝑔 + 𝑄𝑄�⃗ 𝑒𝑒𝑥𝑥 + 𝑄𝑄�⃗ 𝑓𝑓𝑔𝑔𝑖𝑖𝑓𝑓 + 𝑄𝑄�⃗ 𝑎𝑎𝑓𝑓𝑎𝑎 𝑏𝑏�⃗ � (6) where 𝑄𝑄�⃗ 𝑖𝑖𝑖𝑖 is divides in the mechanical system mass matrix (m), and the generalized forces related to coriolis and centrifugal accelerations (𝑄𝑄�⃗ 𝑓𝑓𝑓𝑓). in this case the 0 is an 11x11 null matrix. the velocity of the general coordinates (�⃗̇�𝑞), using coordinate partitioning method (wehage, wehage, & ravani, 2015), can be express in function of the independent coordinates as: �⃗̇�𝑞 = � �̇�𝑞��⃗ 𝑑𝑑 �̇�𝑞��⃗ 𝑖𝑖� = � −�φ𝑞𝑞 𝑑𝑑� −1 ∙ φ𝑞𝑞 𝑖𝑖 1 � ∙ ��̇�𝑞��⃗ 𝑖𝑖 � = 𝑅𝑅∗ ∙ ��̇�𝑞��⃗ 𝑖𝑖 � (7) in this case, φ𝑞𝑞𝑖𝑖 and φ𝑞𝑞𝑠𝑠 are parts of the restriction jacobian matrix φ𝑞𝑞 related to the independent and secondary generalized coordinates, respectively; 1 is a 4x4 identity matrix. multiplying both sides of eq. (6) by 𝑅𝑅∗ the equation of motion can be compactly written follows: (𝑅𝑅∗)𝐹𝐹×𝑁𝑁𝑇𝑇 ∙ �𝑀𝑀𝑁𝑁×𝑁𝑁 ∙ �⃗̈�𝑞𝑁𝑁×1 − 𝑄𝑄�⃗ 𝑓𝑓𝑓𝑓𝑁𝑁×1 − 𝑄𝑄�⃗ 𝑔𝑔𝑔𝑔𝑎𝑎𝑔𝑔𝑁𝑁×1 − 𝑄𝑄 �⃗ 𝑓𝑓𝑔𝑔𝑖𝑖𝑓𝑓𝑁𝑁×1 − 𝑄𝑄 �⃗ 𝑒𝑒𝑥𝑥𝑁𝑁×1� = = (𝑅𝑅∗)𝐹𝐹×𝑁𝑁𝑇𝑇 ∙ 𝑄𝑄�⃗ 𝑎𝑎𝑓𝑓𝑎𝑎𝑁𝑁𝑥𝑥1 = (𝑅𝑅 ∗)𝐹𝐹×𝑁𝑁𝑇𝑇 ∙ (𝑄𝑄𝑎𝑎𝑓𝑓𝑁𝑁×𝐹𝐹 ∙ �⃗�𝐹𝑎𝑎𝑓𝑓𝑎𝑎𝐹𝐹×1) (8) where 𝑁𝑁 and 𝐹𝐹 are the number of generalized coordinates and the independent coordinates respectively. for this study, 𝑁𝑁 = 15 and 𝐹𝐹 = 4. �⃗�𝐹𝑎𝑎𝑓𝑓𝑎𝑎𝐹𝐹×1 are the forces belonging to the actuators on https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 120 the pkm. it is worth mentioning that the right-side term (𝑅𝑅∗)𝐹𝐹×𝑁𝑁𝑇𝑇 ∙ 𝑄𝑄�⃗ 𝑎𝑎𝑓𝑓𝑁𝑁×𝐹𝐹 of eq. (8) is the identity matrix. the equation of motion can be further developed by considering friction force only in the prismatic actuators, thus only affecting the active generalized coordinates, hence: (𝑅𝑅∗)𝐹𝐹×𝑁𝑁𝑇𝑇 ∙ �𝑀𝑀𝑁𝑁×𝑁𝑁 ∙ �⃗̈�𝑞𝑁𝑁×1 + 𝑄𝑄�⃗ 𝑓𝑓𝑓𝑓𝑁𝑁×1 + 𝑄𝑄�⃗ 𝑔𝑔𝑔𝑔𝑎𝑎𝑔𝑔𝑁𝑁×1 + 𝑄𝑄 �⃗ 𝑒𝑒𝑖𝑖𝑎𝑎𝑁𝑁×1� + �⃗�𝐹𝑓𝑓𝑔𝑔𝑖𝑖𝑓𝑓𝐹𝐹×1 = �⃗�𝐹𝑎𝑎𝑓𝑓𝑎𝑎𝐹𝐹×1 (9) in which the friction force assigned to the generalized active coordinates is represented as: �⃗�𝐹𝑓𝑓𝑔𝑔𝑖𝑖𝑓𝑓 = ⎣ ⎢ ⎢ ⎡ −𝑑𝑑𝑖𝑖𝑠𝑠𝑆𝑆(�̇�𝑞13) ∙ (𝜇𝜇𝑓𝑓 + 𝜇𝜇𝑔𝑔 ∙ |�̇�𝑞13|) −𝑑𝑑𝑖𝑖𝑠𝑠𝑆𝑆(�̇�𝑞23) ∙ (𝜇𝜇𝑓𝑓 + 𝜇𝜇𝑔𝑔 ∙ |�̇�𝑞23|) −𝑑𝑑𝑖𝑖𝑠𝑠𝑆𝑆(�̇�𝑞33) ∙ (𝜇𝜇𝑓𝑓 + 𝜇𝜇𝑔𝑔 ∙ |�̇�𝑞33|) −𝑑𝑑𝑖𝑖𝑠𝑠𝑆𝑆(�̇�𝑞42) ∙ (𝜇𝜇𝑓𝑓 + 𝜇𝜇𝑔𝑔 ∙ |�̇�𝑞42|)⎦ ⎥ ⎥ ⎤ (10) where µv and µc are the viscous and coulomb coefficients, respectively. 2.3. objective function and optimization constraints the reconfigurations process, based on previous works (araujo-gómez et al., 2019; vallés et al., 2018), looks for the optimal set of geometric parameters of the pkm for a specific mobile platform trajectory. the reconfiguration of the 3ups+rpu (i) prevents forward singularities inside the workspace (determinant of the φ𝑥𝑥 different from zero), and (ii) avoids large control actions in the vicinity of the singular configurations. the physical bounds of the seven design variables of the pkm (r, βfd, βfi, rm, βmd, βmi and ds) showed in fig. 1 are: ⎩ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎧ 0.30 𝑚𝑚 ≤ 𝑅𝑅 ≤ 0.50 𝑚𝑚 0.10 𝑚𝑚 ≤ 𝑅𝑅𝑚𝑚 ≤ 0.30 𝑚𝑚 −0.15 𝑚𝑚 ≤ 𝑆𝑆𝑑𝑑 ≤ 0.15 𝑚𝑚 0.10 𝑆𝑆𝑆𝑆𝑆𝑆 ≤ 𝛽𝛽𝐹𝐹𝐹𝐹 ≤ 𝜋𝜋 2 𝑆𝑆𝑆𝑆𝑆𝑆 0.10 𝑆𝑆𝑆𝑆𝑆𝑆 ≤ 𝛽𝛽𝐹𝐹𝐹𝐹 ≤ 𝜋𝜋 2 𝑆𝑆𝑆𝑆𝑆𝑆 0.10 𝑆𝑆𝑆𝑆𝑆𝑆 ≤ 𝛽𝛽𝑀𝑀𝐹𝐹 ≤ 𝜋𝜋 2 𝑆𝑆𝑆𝑆𝑆𝑆 0.10 𝑆𝑆𝑆𝑆𝑆𝑆 ≤ 𝛽𝛽𝑀𝑀𝐹𝐹 ≤ 𝜋𝜋 2 𝑆𝑆𝑆𝑆𝑆𝑆 (11) https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 121 the set of rehabilitation trajectories are discretized into a n number of passing through points. at these points we solve the inverse dynamics of the 3ups+rpu pkm, then we define the objective function as the sum of the square of the active generalized forces (�⃗�𝐹𝑎𝑎𝑓𝑓𝑎𝑎): 𝑓𝑓(𝑅𝑅, 𝑅𝑅𝑚𝑚, 𝑆𝑆𝑑𝑑, 𝛽𝛽𝐹𝐹𝐹𝐹 , 𝛽𝛽𝐹𝐹𝐹𝐹, 𝛽𝛽𝑀𝑀𝐹𝐹, 𝛽𝛽𝑀𝑀𝐹𝐹) = ∑ ∑ �𝐹𝐹𝑖𝑖𝑖𝑖� 24 𝑖𝑖=1 𝑖𝑖 𝑖𝑖=1 (12) to ensure that the ‖φ𝑥𝑥‖ ≠ 0 for all configurations part of the rehabilitation trajectory, the next constraints must be met: �‖φ𝑥𝑥‖𝑆𝑆𝑆𝑆𝑓𝑓 − ‖φ𝑥𝑥‖𝑖𝑖� < �‖φ𝑥𝑥‖𝑆𝑆𝑆𝑆𝑓𝑓�; 𝑖𝑖 = 1,2 … , 𝑆𝑆 (13) with: ‖φ𝑥𝑥‖𝑔𝑔𝑒𝑒𝑓𝑓 = 𝑚𝑚𝑆𝑆𝑥𝑥(‖φ𝑥𝑥‖𝑖𝑖); 𝑖𝑖 = 1,2 … , 𝑆𝑆 (14) if both sides of the constraint (13) are squared, it can be rewritten as: 2 ∙ ‖φ𝑥𝑥‖𝑆𝑆𝑆𝑆𝑓𝑓 ∙ ‖φ𝑥𝑥‖𝑖𝑖 − ‖φ𝑥𝑥‖2𝑖𝑖 > 0; 𝑖𝑖 = 1,2 … , 𝑆𝑆 (15) the final optimization constraint is referring to the length of each actuator. the length of the actuated joints must be between the minimum (𝑙𝑙𝑚𝑚𝑖𝑖𝑖𝑖) and maximum (𝑙𝑙𝑚𝑚𝑎𝑎𝑥𝑥) length of each limb. 𝑙𝑙𝑚𝑚𝑖𝑖𝑖𝑖 ≤ 𝑞𝑞𝑖𝑖3 ≤ 𝑙𝑙𝑚𝑚𝑎𝑎𝑥𝑥; 𝑖𝑖 = 1,2,3 𝑙𝑙𝑚𝑚𝑖𝑖𝑖𝑖 ≤ 𝑞𝑞𝑖𝑖2 ≤ 𝑙𝑙𝑚𝑚𝑎𝑎𝑥𝑥; 𝑖𝑖 = 4 (16) the minimization of the penalty function (12) subjected to non-linear constraints (15) and (16) represents a non-linear optimization problem. in this study, the optimization problem is solved by several approaches, which covers evolutionary algorithms, heuristics optimizers, multi-strategy algorithms and gradient-based optimizers. 2.4. optimization approaches comparison optimization techniques can be classified as either local (commonly gradient-based) or global (commonly non-gradient based or evolutionary) algorithms. however, it is worth mentioning the difficulties in comparing the performance of several optimization algorithms (beiranvand et al., https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 122 2017). therefore, we have carried out the optimization algorithm comparison following the recommendations of those authors. our research team used these optimization algorithms: a) evolutionary algorithms (ea), which use mechanisms inspired by biological evolution. b) heuristic methods use a heuristic function to solve the problem. c) multi-strategy algorithms combine the strengths of different approaches. d) gradient-based are iterative methods using the gradient information. using the model frontier framework (www.esteco.com) all these optimization approaches are compared. there is an exhaustive explanation about this optimization algorithm in (yang, 2017). 3. case studies a set of 8 trajectories have been tested for knee rehabilitation. all of them are non-feasible in terms of forward singularities and actuators out of range, so they require a reconfiguration. in table 2, the characteristics of those trajectories are featured, regarding the motion of the mobile platform as well as the difficulties found during the execution. table 2. test trajectories. (1) forward singularities, (2) actuators out of range. trajectory horizontal vertical inclined straight line ellipse constant orientation tr1 (1) tr3 (2) tr5 (2) tr7 (1) and (2) variable orientation tr2 (1) tr4 (1) tr6 (1) and (2) tr8 (1) and (2) the reconfiguration involves 7 design variables, but only 4 are optimized (𝑅𝑅, 𝑆𝑆𝑑𝑑, 𝛽𝛽𝐹𝐹𝐹𝐹, 𝛽𝛽𝐹𝐹𝐹𝐹), while the other 3 are kept constant (𝑅𝑅𝑚𝑚, 𝛽𝛽𝑀𝑀𝐹𝐹, 𝛽𝛽𝑀𝑀𝐹𝐹). the initial parameters of the manipulator are defined in eq. (17) and are intended to avoid a trivial singular configuration. the physical bounds of the optimized design variables are those presented in eq. (11). moreover, the actuator angles must be less than 0.7854 rad and their lengths must lie between 0.575 and 0.775 m. https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 123 ⎩ ⎪⎪ ⎨ ⎪⎪ ⎧ 𝑅𝑅 = 43 𝑆𝑆𝑚𝑚 𝑅𝑅𝑚𝑚 = 23 𝑆𝑆𝑚𝑚 𝑆𝑆𝑑𝑑 = 5 𝑆𝑆𝑚𝑚 𝛽𝛽𝐹𝐹𝐹𝐹 = 45𝑜𝑜 𝛽𝛽𝐹𝐹𝐹𝐹 = 48𝑜𝑜 𝛽𝛽𝑀𝑀𝐹𝐹 = 90𝑜𝑜 𝛽𝛽𝑀𝑀𝐹𝐹 = 100𝑜𝑜 (17) table 3 summarizes the results obtained when applying the different optimization strategies for trajectory 2. the optimized design variables avoid forward singularities, which is shown by the fact that the minimum value of eq. (15) is greater than zero (table 3). the pilopt algorithm presents the best performance. however, results greatly depend on the tuning of the specific parameters of each algorithm, e.g., the stopping conditions, population size or step sizes (beiranvand et al., 2017). in fact, the main reason why the pilopt algorithm outperforms the rest is that it only requires one parameter, which is the number of design evaluations determining when the algorithm stops, occurring when no improvement in the pareto efficiency is observed. table 3. optimized design variables for trajectory 2. algorithm 𝜷𝜷𝑭𝑭𝑭𝑭 (º) 𝜷𝜷𝑭𝑭𝑭𝑭 (º) r (cm) ds (cm) objective function (n2) minimum value of eq. (15) evolutionary algorithms nsga-ii 180 48 40 15 154,547.26 1.64·10-4 moga-ii 174 60 42 15 149,220.00 2.22·10-4 armoga 84 168 32 9 143,290.00 1.82·10-4 evolution strategies 180 48 32 -1 129,375.02 2.18·10-4 heuristics optimizers mosa 84 156 32 13 147,530.00 9.39·10-5 mopso 67 21 22 -15 106,449.01 3.21·10-4 multi-strategy algorithms hybrid 174 60 42 13 150,060.00 2.11·10-4 pilopt 66 18 22 -15 104,010.00 1.99·10-4 fast 177 173 40 15 193,527.55 1.32·10-4 mego 63 21 20 -15 110,761.44 1.55·10-4 gradient-based optimizers mipsqp 132 138 30 15 212,920.00 3.20·10-5 after solving the optimization problem using the pilopt algorithm, several results are presented. fig. 2 shows the geometrical robot reconfiguration from the original robot design for the second trajectory and for both the fixed base (left) and the mobile platform (right). https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 124 figure 2. geometrical robot reconfiguration from the original robot design for the second trajectory and for both the fixed base (left) and the mobile platform (right). the yellow dots correspond to the original configuration of the pkm, while the red lines lead to the final configuration. bearing in mind that the pilopt algorithm leads to the best results, the 8 non-feasible trajectories are solved using this optimization technique. table 4 illustrates the optimal reconfiguration design variables of the robot using the pilopt algorithm. the robot reconfiguration prevents high values of generalized forces and the problem of a direct singularity. table 4. optimized design variables for the 8 non-feasible trajectories. trajectory 𝜷𝜷𝑭𝑭𝑭𝑭 (º) 𝜷𝜷𝑭𝑭𝑭𝑭 (º) r (cm) ds (cm) objective function (n2) 1 18 30 28 -7 101,130.00 2 66 18 22 -15 104,010.00 3 60 12 30 9 79,627.00 4 60 18 36 15 58,168.00 5 72 30 26 -9 109,410.00 6 48 6 30 1 65,622.00 7 90 25 35 15 83,526.00 8 48 114 32 15 248,780.00 results show that there is not the best optimization for all types of optimization problems, because each algorithm has its advantages and disadvantages. in general, local algorithms are better when design variables are greater than 50, with high computational cost, with a little significance numerical noise, when local minima are not a problem and when gradients are easily available. inversely, global algorithms are recommended with less than 50 design variables, with significance https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 125 numerical noise, where gradients do not exist, when global optimum is needed and when there are discontinuous objective or constraint functions. eventually, global methods should be used only in cases where efficient local search is not feasible. 4. conclusions in order to apply the required movements for diagnosis and rehabilitation tasks of anterior cruciate ligament of human knee, a pkm robot with 4 dof comprised of 3ups-rpu was designed, and the kinematics and dynamics modeling has been presented. during the execution of certain rehabilitation trajectories, the forward jacobian becomes singular, so in order to prevent control problems a geometrical and kinematical reconfiguration of the manipulator has been considered. this leads to the achievement of the generalized coordinates that were initially outside range of prismatic actuators. as it is not possible to modify the rehabilitation trajectories because they are prescribed by the physical therapist, the robot reconfiguration raises as the only solution of such problem. thus, it is needed to modify the points of insertion of the limbs on both the mobile and fixed robot platforms. a non-linear optimization solver has been proposed to approach the reconfiguration problem. the penalty function to be minimized sums the square of the active generalized forced. the constraints include the imposition of the robot actuated joints to lie within an admissible range, and the non-singularity of the forward jacobian. using d'alembert's dynamics inverse model of the pkm and the principle of virtual power the optimal redesign problem of the robot has been tackled. we have used different optimization strategies to solve it. the rehabilitation therapies cover a set of 8 non-feasible trajectories. the second non-feasible trajectory was optimized by using different optimization techniques to find the best one. results clearly show that the pilopt algorithm outperforms the other algorithms for the problem in hand. https://doi.org/10.4995/muse.2020.13352 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 126 the rest of non-feasible trajectories were optimized using pilopt and the results show that the forces required to carry out these trajectories are much lower than those of the initial configuration of the robot and that the active generalized coordinates fall within the physical ranges of the actuators. acknowledgements this work was supported by the spanish ministry of education, culture and sports through the project for research and technological development with ref. dpi2017-84201-r. references arakelian, v., briot, s., & glazunov, v. (2008). increase of singularity-free zones in the workspace of parallel manipulators using mechanisms of variable structure. mechanism and machine theory, 43(9), 1129– 1140. https://doi.org/10.1016/j.mechmachtheory.2007.09.005 araujo-gómez, p., díaz-rodríguez, m., mata, v., & gonzález-estrada, o. a. (2019). kinematic analysis and dimensional optimization of a 2r2t parallel manipulator. journal of the brazilian society of mechanical sciences and engineering, 41(10), 425. https://doi.org/10.1007/s40430-019-1934-1 araujo-gómez, p., mata, v., díaz-rodríguez, m., valera, a., & page, a. (2017). design and kinematic analysis of a novel 3ups/rpu parallel kinematic mechanism with 2t2r motion for knee diagnosis and rehabilitation tasks. journal of mechanisms and robotics, 9(6), 061004. https://doi.org/10.1115/1.4037800 beiranvand, v., hare, w., & lucet, y. (2017). best practices for comparing optimization algorithms. optimization and engineering, 18(4), 815–848. https://doi.org/10.1007/s11081-017-9366-1 dash, a. k., chen, i. m., yeo, s. h., & yang, g. (2005). workspace generation and planning singularity-free path for parallel manipulators. mechanism and machine theory, 40(7), 776–805. https://doi.org/10.1016/j.mechmachtheory.2005.01.001 gosselin, c., & angeles, j. (1990). singularity analysis of closed-loop kinematic chains. ieee transactions on robotics and automation, 6(3), 281–290. https://doi.org/10.1109/70.56660 https://doi.org/10.4995/muse.2020.13352 https://doi.org/10.1016/j.mechmachtheory.2007.09.005 https://doi.org/10.1007/s40430-019-1934-1 https://doi.org/10.1115/1.4037800 https://doi.org/10.1007/s11081-017-9366-1 https://doi.org/10.1016/j.mechmachtheory.2005.01.001 https://doi.org/10.1109/70.56660 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.13352 social and technological sciences issn: 2341-2593 llopis-albert et al. (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 2 (2020): 113-127 | 127 llopis-albert, c., rubio, f., & valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences, 5(1), 1. https://doi.org/10.4995/muse.2018.9867 patel, y. d., & george, p. m. (2012). parallel manipulators applications—a survey. modern mechanical engineering, 02(03), 57–64. https://doi.org/10.4236/mme.2012.23008 rubio, f., llopis-albert, c., valero, f., & suñer, j. l. (2016). industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. robotics and autonomous systems, 86, 106–112. https://doi.org/10.1016/j.robot.2016.09.008 rubio, f., valero, f., & llopis-albert, c. (2019). a review of mobile robots: concepts, methods, theoretical framework, and applications. international journal of advanced robotic systems, 16(2), 172988141983959. https://doi.org/10.1177/1729881419839596 tsai, l.-w. (1999). robot analysis and design. john wiley & sons, inc. new york, ny, usa ©1999. valero, f., rubio, f., & llopis-albert, c. (2019). assessment of the effect of energy consumption on trajectory improvement for a car-like robot. robotica, 37(11), 1998–2009. https://doi.org/10.1017/s0263574719000407 vallés, m., araujo-gómez, p., mata, v., valera, a., díaz-rodríguez, m., page, á., & farhat, n. m. (2018). mechatronic design, experimental setup, and control architecture design of a novel 4 dof parallel manipulator. mechanics based design of structures and machines, 46(4), 425–439. https://doi.org/10.1080/15397734.2017.1355249 wehage, k. t., wehage, r. a., & ravani, b. (2015). generalized coordinate partitioning for complex mechanisms based on kinematic substructuring. mechanism and machine theory, 92, 464–483. https://doi.org/10.1016/j.mechmachtheory.2015.06.006 www.esteco.com. (n.d.). retrieved june 10, 2019, from https://www.esteco.com/ xianwen kong, b., & gosselin, c. m. (2002). kinematics and singularity analysis of a novel type of 3-crr 3-dof translational parallel manipulator. international journal of robotics research, 21(9), 791–798. https://doi.org/10.1177/02783649020210090501 yang, x. (2017). optimization algorithms optimization and metaheuristic algorithms in engineering. (march). https://doi.org/10.1007/978-3-642-20859-1 https://doi.org/10.4995/muse.2020.13352 https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.4236/mme.2012.23008 https://doi.org/10.1016/j.robot.2016.09.008 https://doi.org/10.1177/1729881419839596 https://doi.org/10.1017/s0263574719000407 https://doi.org/10.1080/15397734.2017.1355249 https://doi.org/10.1016/j.mechmachtheory.2015.06.006 https://www.esteco.com/ https://doi.org/10.1177/02783649020210090501 https://doi.org/10.1007/978-3-642-20859-1 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 13 effects of al-miap-based learning management to promote digital intelligence for undergraduate students phunaploy sathiyaa , nilsook prachyanunb , nookhong jarumona,* asuan sunandha rajabhat university (ssru), bangkok , thailand. email: satiya.ph@ssru.ac.th bking mongkut's university of technology north, bangkok, thailand. email: prachyanun.n@kmutnb.ac.th; *corresponding author: jarumon.no@ssru.ac.th received: 19 july 2020; accepted: 08 march 2021; published: april 2021 abstract this paper has two main aims: the first is to study preand postleaning achievement utilizing al miap to promote copyright digital intelligence for suan suan sunandha rajabhat university undergraduate students; and the three is to investigate the appropriateness of leaning management adopting an al miap leaning model to promote the students’ digital intelligence. an al miap learning model was utilized in the processes to promote five digital intelligence skills, i.e., respecting copyright, prevention, checking before sharing, threat awareness and using safely. fifty-eight suan sunandha rajabhat university undergraduate students in the three-year enrolled in the introduction to digital economy course and were selected in a sample group. the tools were: kahoot!, google forms, content network chart, preand post-test, digital media copyright test and the al miap learning model. percentage, mean, s.d, and t-test were applied in the research. the study revealed that the post-learning achievement was better that the prelearning achievement with statistical significance at 0.01; the students gained total scores of 17.06, which was 7.16 points higher than the pre-learning scores, signifying that their digital intelligence skills were better. as for al miap leaning model, it revealed that this model was appropriate at the highest level. keywords: learning management; active learning management; al miap learning model; digital intelligence and copyright literacy to cite this article: sathiya, p., prachyanun, n., jarumon, n. (2021). effects of al-miap-based learning management to promote digital intelligence for undergraduate students. multidisciplinary journal for education, social and technological sciences, 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048. https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4995/muse.2021.14048 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 14 1. introduction investment in human resources and knowledge must be emphasized in order to aid longterm competitive capability development in thailand; these two aspects are the supporting factors that help thailand advance into the future. they also shape the ways and development path in alignment with shifting trends in technology, demography and environmental, geopolitics and geopolitical power shifting, etc. the investment must emphasize a foundation for long-term and further development, rather than a short-term one. it must also emphasize human development to foster knowledge and skills in conformity with ways of working and life in the 21st century. constructivism based on fields of research that thailand has the potential and development of large infrastructure to support research and innovation in order to allow self-development of technology and innovation in thailand. this will pave the way to becoming a leader in fields of research such as reducing foreign resource dependency, lowering technology import, which causes economic deficit (budget deficit), building up long-term competitive capability with other nations, and for raising thailand to stand with developed countries (national council for higher education, science, 2019) according to educational management, instructional methodologies or models that are in alignment with the national scheme of education should be developed by applying advanced technology to current circumstances to encourage copyright digital intelligence. this development is in accordance with the rapidly changing learning management environment of the 21st century. as (chuangprakhon, s., santaveesuk, p., & nilsook, 2018) put it, “a successful person or a person that can sustainably live their live in the modern world must possess a strong will, strength and wide creative ideas and he/she must have good wit, be well-versed, capable to think on their feet and must be constructivist throughout their life”. learners must be capable of adapting their methods of applying technology to support learning in conformity with a study conducted by toda, valle, & isotani, (2018), which emphasized that a person that aims to apply technology needs to constantly improve their knowledge to improve learning processes. nevertheless, digital knowledge is not sufficient to dictate digital citizenship . according to (gómez-ejerique & lópez-cantos, 2019), critical thinking is essential for the digital society as leaners need the knowledge to improve https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 15 discretionary usage and thinking; therefore, information technology and supporting skills are necessary to academic literature related to knowledge concerning privacy and internet safety, creativity, ethics and responsibility (kaeophanuek, na-songkhla, & nilsook, 2019). digital intelligence quotient is “the sum of social, emotional, and cognitive abilities that enable individuals to face the challenges and adapt to the demands of life in the digital world ”. in other words, it includes knowledge, skills, attitude, and values necessary to a life in the digital world or communication skills and social online access. the quotient is a study and development by the dq institute, which is an agency established through collaboration between public and private sectors around the world and the world economic forum. it aims to allow all children in every country to receive education, to have fine digital citizen skills and to live their digital life safely. with rapid, free and borderless communication, children and juveniles born in the it era are surrounded with digital devices and the internet; therefore, by the nature of such communication with seeming anonymity (no face-to-face contact), this makes their perception and way of life different from previous generations. even though the digital devices and internet offer convenience, there are some dangers such as health, technological addiction (if excessively used) or online crime, cyber threats, and cyberbullying. as such, citizens-to-be must be able to keep up with the media and information, and possess digital intelligence skills to be able to live in an online society and real life without harming them or others. their family, school, public sector, and relevant agencies must uplift the juvenile to be a digital citizen, possessing knowledge and understanding of internet usage. this consists of eight elements: (1) personal data protection, (2) good critical thinking skills, (3) personal history data management, (4) ethical technology usage, (5) personal data safety, (6) cyber threats countermeasures, (7) time management, and (8) good identity protection (media., 2018) in light of this, we conducted a study on learning management by adopting an al miap model to identify results in order to promote copyright digital intelligence in suan sunandha rajabhat university undergraduate students. an al miap model was mainly utilized, and it consisted of four processes: motivation, information, application and progress (jabtien, 2015). https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 16 active learning was also applied; this consisted of six main focuses: (1) brain development such as thinking, problem solving, knowledge application, (2) participatory learning, (3) knowledge construction and learning management, (4) instructional participation, knowledge construction, interaction and cooperation in collaborative rather than competitive ways, (5) co-responsibility and discipline, and task assignment, and (6) simulation for reading, speaking, listening and thinking (phutthikun, 2015), which were also utilized in learning activities. in addition, two sets of digital intelligence skills were applied in the instructional design: (1) good identity protection, and (2) personal data protection to promote digital intelligence in learners, making them able to keep up with digital media copyright and use such media safely. 2. miap’s principles and theories a miap teaching technique consisted of four processes: step 1: motivation: motivating the interest of learners, i.e. clear introduction to the title and objectives with instructional materials for motivating their interest, spending the time concisely, encouraging learners in the class to collaborate and a conclusion to guide learners into the next step, suitably and relevantly. step 2: information, i.e. learners obtained knowledge from textbooks, self-learning, meta skills by obtaining information from several resources, teaching aids and planning a step from providing less and easier knowledge to more and harder ones. step 3: application, i.e. applying the obtained knowledge, allowing an experiment/simulation after obtaining the content in order to measure how well leaners retain such knowledge (obtaining less or more), memory review to prevent data loss, aiding learning development by urging the use of intellectual power and problem solving in learning and promoting learning transfer. step 4: progress: evaluating the learning achievement, i. e. evaluating learning outcome or progress, measuring and assessing outcomes – skills, behaviors, knowledge and attitude https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 17 3. principles and theories of active learning active learning processes: 1) brain power development, i.e. thinking, problem solving and knowledge application; 2) participatory learning; 3) knowledge construction and learning management system created by learners; 4) classroom participation, knowledge construction, interaction, cooperation in collaborative rather than competitive ways; 5) co-responsibility, discipline and task assignment; 6) simulation for reading, speaking, listening and thinking; 7) higher order thinking skills; 8 ) an opportunity for integration of data, news and information and principles to create concepts; 9) learning management facilitated by teacher in order to allow self-learning by learners; 10) experiencebased knowledge, knowledge construction, conclusion and review by learners. 4. digital intelligence quotient (dq) the digital intelligence quotient is the sum of the social, emotional and cognitive abilities that enable individuals to face the challenges and adapt to the demands of life in the digital world. the scope of digital intelligence covers necessary knowledge, skills, attitude and values for a digital life – in other words a communication skill and online socialization. two digital intelligence skills were applied in the instruction : https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 18 1) digital citizen identity protection: this is a skill of good online and real-world identity creation and management. good identity means the capability of a digital media user to build a positive identity, i.e. thought, feeling and action with discretion about data communication and commentary, to sympathize with social media users, to take responsibility for their actions and to act in accordance with laws and online ethics, e.g., respecting copyright, no cyberbullying or hate speech. 2) privacy management: this is a skill regarding discretion about personal data management, online personal data protection, especially online sharing to prevent privacy breaches, internet literacy – threats, i.e., malware, viruses, and internet fraud. https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 19 figure 1. al miap learning model 5. processes four processes were divided as follows: step 1: motivation: 15 minutes for introduction, informing about the course and lesson objectives before proceeding with the lecture, pretesting before learning with kahoot! to create a participatory atmosphere. step 2: information: 1: 30 hours for course introduction, discussion and presentation by powerpoint; q&a in certain parts to create interaction between the class. step 3: application: 30 minutes for brainstorming by a group, each of which consists of 57 learners, google slides utilized with creative commons (cc) license required in order to encourage teamwork, discussion and exchange of ideas, and to determine the rights which usages are allowed for recipients as follows: 1. digital law: respecting copyright, identity or property or works created by third party to distribute in digital format. 2. digital protect: personal data protection, preventing a person with malicious intent from the digital world. 3. digital check: checking before sharing, and sharing personal data with caution. 4. digital threat: learning internet threats. 5. digital security: using safely, and knowing how to secure data and digital devices. step 4: progress: five minutes for class presentation, q&a by classmates and teachers, checking the completed work, assessing digital intelligence in the presenting group to encourage discussion, explicit self-expression and doing digital intelligence questionnaire via google forms. 6. objectives https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 20 1) to study preand postleaning achievement by utilizing al miap to promote copyright digital intelligence for ssru’s undergraduate students; 2) to investigate the appropriateness of leaning management adopting an al miap leaning model to promote the students’ digital literacy. . 7. delimitation 1) population and sample group population (n): undergraduate students, information science, suan suan sunandha rajabhat university sample group = 58 undergraduate students (3rd year) enrolled in the introduction to digital economy course, suan suan sunandha rajabhat university 2) study variables independent variable: al miap-based learning management dependent variable: digital intelligence promotion 8. tool design tools consisted of the following: 1) al miap learning model 2) preand post-testing: digital intelligence – digital copyright test (20 questions) 3) al miap evaluation forms of the introduction to digital economy course to promote digital intelligence a 5-item likert scale was used to evaluate all four processes (motivation, information, application and progress) was applied. the scale was as follows: https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 21 scale comment/appropriateness 5 highest 4 high 3 moderate 2 low 1 poor interpretation was interval as follows: average comment/appropriateness 4.50-5.00 highest 3.50-4.49 high 2.50-3.49 moderate 1.50-2.49 low 1.00-1.49 poor research processes of al miap learning model step 1: motivation: 15 minutes for introduction, informing the course and lesson objectives before proceeding with lecture, pre-testing before learning with kahoot to create participatory atmosphere step 2: information: 1:30 hours for course introduction, discussion and presentation by powerpoint, q&a in certain parts to create interaction in the class. step 3: application: 30 minutes for brainstorming by a group each of which consists of 5-7 learners, google slides utilized with cc (creative commons) license required in order to https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 22 encourage teamwork, discussion and exchange of ideas and to determine the rights which usages are allowed for recipients; there were 5 topics as follows: 1. digital low: respecting copyright, identity or property or works created by third party to distribute in digital format 2. digital protect: personal data protection, preventing a person with malicious intent from the digital world 3. digital check: checking before sharing and sharing personal data with caution 4. digital threat: learning internet threats 5. digital security: using safely and know how to secure data and digital devices step 4: progress: 5 minutes for class presentation, q&a by classmates and teachers, checking finished works, assessing digital intelligence in the presenting group to encourage discussion, explicit self-expression and doing digital intelligence questionnaire via google forms 4) kahoot! and google forms kahoot! is a game-based learning platform that allows learners to do quizzes; learners must answer the questions quickly and correctly. the created game is accessed by a smart-phone or computer connected to the internet; it is played individually and in competition with the whole class. the total score is displayed after finishing the game (leaners know how well they fared instantly); then, all results (as completed by all learners) are finalized to identify the highest and lowest scores. this game allows the learners to compete with themselves and their classmates, to have a good time and to participate in activities in class. google forms is a survey administration application for creating a test with questions and answers. this application can check what choices learners have chosen or what answer they submit; at the same time, they can also see what the correct choices/answers are. total scores are displayed and can be set to display the true answer; moreover, the application provides a comparative data graph to compare each question and overall test. this enables the teacher to check whether the test is too hard or easy. https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 23 9. results the preand post-learning achievement of the 58 undergraduate students (2nd year) by applying al miap is shown in table 1: table 1. comparison of learning achievement between pre-learning achievement and post-learning achievement from 58 undergraduate students (2nd year) with al miap learning model. score full score x̅ s.d. t-test sig. pre-learning 20 9.90 1.49 21.15 .01 post-learning 20 17.06 0.94 **p< .01 according to table 1, the learning achievement of the 58 undergraduate students (3nd year) with the al miap learning model to promote digital intelligence is better in the post -learning period than the pre-learning period, with statistical significance at 0.01; the total score is 17.06, signifying that the digital intelligence of learners increased by 7.16 points. in conclusion, using an al miap learning management to promote digital intelligence can increase the score. table 2. al miap al mial learning model evaluation descriptions x̅ s.d interpretation step 1 : motivation inform learning goals pre-test by kahoot 5.00 0.00 highest https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 24 descriptions x̅ s.d interpretation conclude before introduction into the lessons step 2 : information explain course details establish topic for discussion related to digital intelligence q&a 4.75 0.50 highest step 3 : application identify problems and give an assignment explain how to use google slides describe how to create cc assign group tasks, create works by using google slides and apply cc license 4.25 0.50 high step 4: progress presentation check and evaluate completed works inquire and exchange ideas based on presented works post-test to evaluate at the end of the class 4.50 0.58 highest total 4.63 0.39 highest based on table 2, the appropriateness of an al miap learning model applied to promote digital intelligence of the students is highest (x̅ = 4.63, s.d. = 0.39). https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 25 10. conclusion based on the objectives, it was concluded that: 1) in regard to preand post-learning management with an al miap model to promote digital intelligence of ssru’s undergraduate students, based on the preand post-test (20 points), post-learning achievement was higher than the pre-learning period, with statistical significance at 0.01 (total score: 17.06); this signifies better digital intelligence (average score 7.16 increased). this shows that an al miap learning model is able to promote a better score (digital intelligence). 2) based on an evaluation on the four processes (al miap) by four experts, the appropriateness of learning management with an al miap model to promote digital intelligence of ssru’s undergraduate students was of the highest level (x̅ = 4.63, s.d. = 0.39); the miap (motivation, information, application, progress) was assessed as follows: 2.1) mean of appropriateness from step 1 (motivation) was of the highest level (x̅ = 5.00, s.d. = 0.00); 2.2) mean of appropriateness from step 2 (information) was of the highest level (x̅ = 4.75, s.d. = 0.50); 2.3) mean of appropriateness from step 3 (application) was of the highest level (x̅ = 4.25, s.d. = 0.50); 2.4) mean of appropriateness from step 4 (progress) was of the highest level (x̅ = 4.75, s.d. = 0.50). 11. discussion according to the study, post -learning achievement was better than pre -learning achievement, with statistical significance at 0.01 (total score: 17.06), thus signifying better digital intelligence (average score increased to 7 .16); this is in line with a study conducted by lerdrungporn, p., wattananarong, k ., & wiriyanon, (2017), which compared the learning https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 26 achievement (average score) of students’ learning with instructional management for touch-screen computers and students’ learning with conventional methods using independent t-tests (two sample groups); the comparison showed statistically significant difference at 0.05; (alanazi, 2020) thus, the t-test was feasible in the study. second, according to the study, appropriateness was highest (x̅ = 4.63, s.d. = 0.39); this is in line with a study on an instructional model of map on cloud computing technology of the unde rgr a dua t e s t ude nt s i n or de r t o pr omot e 21 s t c e nt ur y l e a r ni ng s ki l l s c onduc t e d by thanachawengsakul, n., & jeerungsuwan, (2019); they revealed that the appropriateness was of the highest level (x̅ = 4.90, s.d. = 0.32). further, a study conducted by leela, s, chookkaew, s. & nilsuk, (2019) revealed that the appropriateness of micro learning management development by utilizing living books to promote computational thinking was of the highest level (x̅ = 4.60, s.d. = 0.51) by applying multimedia, image, video, freeware and gameplay as the channels for instructional management; this also emphasized the that the activities conformed to the learners’ skills to encourage copyright digital intelligence and class participation. this kind of instructional management was different from original miap, as it focused on activities and learnin g in a class only due to limitations of the premise causing learners to come to learn in the university. 12. suggestions the suggestions are as follows: 1) it is important to raise the student’s awareness of cc license to use image or media in the assigned works created with google slides. 2) in application process, simulation related to lessons may be feasible to allow the students to learn in practical way i .e., assigning tasks for student to design a business model with a simulation of copyright infringement and solution, etc. 3) rubric score should be applied for checking the completed works 4) video with current interesting topics should be added for encouraging an analytical cooperation, class discussion between students and for allowing them to get a bigger picture. https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 27 acknowledgements: we would like to express our gratitude to the office of general education and innovative electronic learning, information science program, humanities and social science, ssru and vocational and technical education, science and technology research institute, king mongkut's university of technology north bangkok to extent their support for this study. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references alanazi, h. m. n. (2020). the effects of active recreational math games on math anxiety and performance in primary school children: an experimental study. multidisciplinary journal for education, social and technological sciences, 7(1), 89. https://doi.org/10.4995/muse.2020.12622 aparicio, f., morales-botello, m. l., rubio, m., hernando, a., muñoz, r., lópez-fernández, h., … buenaga, m. de. (2018). perceptions of the use of intelligent information access systems in university level active learning activities among teachers of biomedical subjects. international journal of medical informatics, 112(may 2017), 21–33. https://doi.org/10.1016/j.ijmedinf.2017.12.016 berlinski, s., & busso, m. (2017). challenges in educational reform: an experiment on active learning in mathematics. economics letters, 156, 172–175. https://doi.org/10.1016/j.econlet.2017.05.007 brian chen, c. c., kathy huang, c. c., gribbins, m., & swan, k. (2018). gamify online courses with tools built into your learning management system (lms) to enhance self-determined and active learning. online learning journal, 22(3), 41–54. https://doi.org/10.24059/olj.v22i3.1466 budoya, c. m., kissake, m., & mtebe, j. (2019). instructional design enabled agile method using addie model and feature driven development method. international journal of education and development using information and communication technology, 15(1), 35. chuangprakhon, s., santaveesuk, p., & nilsook, p. (2018). a model of distance learning for music art of higher education in thailand. technical education journal king mongkut’s universily of technology north bangkok., 19(3), 39–46. costa, c. j., aparicio, m., & raposo, j. (2020). determinants of the management learning performance in erp context. heliyon, 6(4), e03689. https://doi.org/10.1016/j.heliyon.2020.e03689 dostál, j., wang, x., steingartner, w., & nuangchalerm, p. (2017). digital intelligence new concept in context of future of school education. iceri2017 proceedings, 1(november), 3706–3712. https://doi.org/10.21125/iceri.2017.0997 duin, a. h., & tham, j. (2020). the current state of analytics: implications for learning management system (lms) use in writing pedagogy. computers and composition, 55, 102544. https://doi.org/10.1016/j.compcom.2020.102544 https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.4995/muse.2020.12622 https://doi.org/10.1016/j.ijmedinf.2017.12.016 https://doi.org/10.1016/j.econlet.2017.05.007 https://doi.org/10.24059/olj.v22i3.1466 https://doi.org/10.1016/j.heliyon.2020.e03689 https://doi.org/10.21125/iceri.2017.0997 https://doi.org/10.1016/j.compcom.2020.102544 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 28 elfeky, a. i. m., masadeh, t. s. y., & elbyaly, m. y. h. (2020). advance organizers in flipped classroom via elearning management system and the promotion of integrated science process skills. thinking skills and creativity, 35(november 2019), 100622. https://doi.org/10.1016/j.tsc.2019.100622 elia, g., margherita, a., & passiante, g. (2020). digital entrepreneurship ecosystem: how digital technologies and collective intelligence are reshaping the entrepreneurial process. technological forecasting and social change, 150(september 2019), 119791. https://doi.org/10.1016/j.techfore.2019.119791 feldman, r., ewing, t., & jeruss, s. (2013). ucla journal of law & technology. ucla journal of law & technology, 21(1), 1–82. frost, r. d., matta, v., & macivor, e. (2015). assessing the efficacy of incorporating game dynamics in a learning management system. journal of information systems education, 26(1), 59–70. ge, s. s., guerra, t. m., lewis, f. l., principe, j. c., & colnarič, m. (2014). computational intelligence in control antónio e ruano 1. ifac proceedings volumes, 47(3), 8867–8878. https://doi.org/10.3182/20140824-6-za1003.01164 gómez-ejerique, c., & lópez-cantos, f. (2019). application of innovative teaching-learning methodologies in the classroom. coaching, flipped-classroom and gamification. a case study of success. multidisciplinary journal for education, social and technological sciences, 6(1), 46. https://doi.org/10.4995/muse.2019.9959 kaeophanuek, s., na-songkhla, j., & nilsook, p. (2019). a learning process model to enhance digital literacy using critical inquiry through digital storytelling. ijet international journal emerging technologies in learning, 14(3), 22–37. https://doi.org/10.3991/ijet.v14i03.8326 khotmanee, w., kaeotasaeng, w., & akatimagool, s. (n.d.). asia pacific journal of science and technology model ; a case study of basic antenna design. 1–8. leela, s, chookkaew, s. & nilsuk, p. (2019). development of a model for managing micro-learning with live books that promote computational thinkin. national academic conference. academic information science, 8. lerdrungporn, p., wattananarong, k., & wiriyanon, t. (2017). the development of learning management system for tablets. technical education journal king mongkut’s universily of technology north bangkok., 3(1), 78– 86. llantos, o. e., & estuar, m. r. j. e. (2019). characterizing instructional leader interactions in a social learning management system using social network analysis. procedia computer science, 160(2018), 149–156. https://doi.org/10.1016/j.procs.2019.09.455 media., o. of h. p. f. a. the c. and y. m. p. f. i. of j. (2018). digital intelligence. retrieved from http://cclickthailand.com/contents/general/dq3.pdf moreira, f., ferreira, m. j., & seruca, i. (2018). enterprise 4.0 the emerging digital transformed enterprise? procedia computer science, 138, 525–532. https://doi.org/10.1016/j.procs.2018.10.072 national council for higher education, science, r. and i. p. c. a. the o. of the s. p. b. r. and innovation. (2019). policy and strategy for higher education, science, research and innovation 2020 2027 and science plans research and innovation. retrieved from a learning process model to enhance digital literacy using critical inquiry through digital storytelling perry, m. (2016). global governance of intellectual property in the 21st century: reflecting policy through change. https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.tsc.2019.100622 https://doi.org/10.1016/j.techfore.2019.119791 https://doi.org/10.3182/20140824-6-za-1003.01164 https://doi.org/10.3182/20140824-6-za-1003.01164 https://doi.org/10.4995/muse.2019.9959 https://doi.org/10.3991/ijet.v14i03.8326 https://doi.org/10.1016/j.procs.2019.09.455 https://doi.org/10.1016/j.procs.2018.10.072 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sathiya et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 13-29. https://doi.org/10.4995/muse.2021.14048 29 in global governance of intellectual property in the 21st century: reflecting policy through change. https://doi.org/10.1007/978-3-319-31177-7 phutthikun, s. (2015). quality of students derived from active learning process. journal of educational services. burapha university, 6(2), 1–13. samuelson, p. (2017). strategies for discerning the boundaries of copyright & patent protections. notre dame law review, 92(4), 1493–1538. stiakakis, e., liapis, y., & vlachopoulou, m. (2019). developing an understanding of digital intelligence as a prerequisite of digital competence. the 13th mediterranean conference on information systems (mcis), 1– 14. thanachawengsakul, n., & jeerungsuwan, n. (2019). instructional model of miap on cloud computing technology of the undergraduate students in order to promote 21st century learning skills. journal of education naresuan university, 20(4), 58–69. toda, a. m., valle, p. h. d., & isotani, s. 1007_97.-3-319-97934-2_9. bibij. (2018). higher education for all. from challenges to novel technology-enhanced solutions. hefa 2017: higher education for all. from challenges to novel technology-enhanced solutions, communicat(august), 143–156. https://doi.org/10.1007/978-3-31997934-2 https://doi.org/10.4995/muse.2021.14048 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1007/978-3-319-31177-7 https://doi.org/10.1007/978-3-319-97934-2 https://doi.org/10.1007/978-3-319-97934-2 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 1 teaching methodologies for combustion science within the european higher education area s. molina * , j. javier lópez, j.m. garcía-oliver, a. garcía dpto. de máquinas y motores térmicos. universitat politècnica de valència, camino de vera s/n, 46022 valencia, spain. * corresponding author: email: samolina@mot.upv.es; phone: + 34 963879651 received: 2013-12-15; accepted: 2014-01-28 abstract the main objective of the present work is the selection and integration of different methodologies among those applicable within the framework of the european higher education area to combine teaching methods with high implication from both lecturers and students. the aim is to enhance the students’ motivation, by means of which they should learn both combustion fundamentals as well as particular technological applications. thus, the selected methods are master classes, guided work by means of learning based upon projects, simulations and numerical applications, tutoring and student independent work. this methodology is currently been applied within the mechanical engineer degree of the escuela técnica superior de ingeniería del diseño at the universitat politècnica de valència. results have been critically analyzed, paying attention to both the performance of the students as well as to some inconsistencies that have been found when applying the developed methodologies to large groups. the main result to be noted is the high comprehension level of the course concepts that has been attained by the students. keywords active learning methods; combustion; teaching methodologies; european higher education area multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 2 1. introduction the present contribution reports on the methodology implemented to deliver a combustion course within the bachelor degree in mechanical engineering at the escuela técnica superior de ingeniería del diseño (etsid) of the universitat politècnica de valència. before showing this approach, the importance of combustion science for engineering will be addressed, together with the difficulties involved in teaching the course in a curriculum that is within the framework of the european higher education area. the study of combustion science within the mechanical engineering degree is justified due to the fact that more than 90% (spanish ministry, 2011) of worldwide energy transformations involve oil, coal, natural gas or biomass as fuels. a combustion process that releases the energy stored in such fuels is therefore necessary. consequently the knowledge of the phenomena involved in combustion must be acquired by students during, as the probability to be faced with a combustion-related problem during their professional career is very high. although there are exceptions, it was not very usual to find a specific course on combustion in old university engineering curricula. such contents were delivered rather within other courses, such as chemistry to introduce fundamental concepts, or thermal machines or thermal power stations to deal with more practical applications of combustion. this can be due to the fact that combustion is a strongly multidisciplinary subject (turns, 2000), for which one needs many concepts developed in other courses found in university curricula (chemistry, thermodynamics, fluid mechanics, mass and energy transport phenomena, etc.), and when designing such curricula, singledisciplinary courses were preferably chosen (desantes et al., 2009). however, the combustion course was already part of some of the previous curricula at the universitat politècnica de valència, in particular at the industrial engineering degree, hosted by the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 3 escuela técnica superior de ingenieros industriales (etsii), and at the aeronautics engineering degree, hosted by the etsid. as a matter of coherence with new curricula for bachelor degrees in energy engineering (etsii), mechanical engineering and aeronautics engineering (etsid), a specific combustion course has also been included. taking into account the current change process in the university system, which is due to a large extent to the recent modification in curricula (4-year bachelor degree) for harmonization with the european system, it is necessary to rebuild the teaching and learning methods. the student should have a more active role within this process, and he/she should not be a mere recipient of information (uk-he 2007). new teaching methods focus on self-learning, guided and cooperative work, projects, case studies, etc. so that a constructive approach for the student’s curriculum is achieved. to develop a course on a topic such as combustion, where abstract concepts are traditionally developed with the help of more enjoyable laboratory exercises, new methodologies have been undertaken to make the student an active element for the development of the course. however, it must be noted that this course has more than 180 enrolled students, who are divided into 2 groups for theoretical classes and 25-student groups for laboratory exercises, which makes the application of such methodologies to enhance the student’s participation highly complicated. the present contribution starts from the experience of the authors in delivering combustion courses in the industrial and aeronautical engineering degrees of the universitat politècnica de valència. the main objective of the present work is the selection and combination of different methodologies, within the ones currently available, with the purpose of combining teaching methods with high implications for both students and lecturers, which motivate students and presents them both the fundamentals of such a subject, as well as its concrete applications. it must be noted that such methodology will be incorporated into a group with a large number of students. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 4 after the present introduction, the document follows with the description of the methodology. after that, a discussion of the obtained results is presented, and lastly the conclusions. 2. proposed methodology: integration of different teaching methods the current methodology is the result of combining different teaching methods which imply the lecturer and the student in a similar way. independently on the selected methods, the interaction between them is prioritized. the selection of the different methodologies is based on the carpio’s work and the experience (carpio, 2008). 2.1 methods with high implication of lecturer in the next paragraphs, the selected methodologies will be presented. the following information will be defined: the main objective of the methodology, its implementation, the competences that the student is intended to achieve and, finally, the evaluation system that is applied. 2.1.1 master classes their main goal is to perform a clear and systematic presentation of the essential theoretical contents for a proper development and comprehension of the rest of activities in which the course is scheduled (biggs, 2011). other important objective that master classes pursue is to help the lecturer become aware of the comprehension level that students are acquiring with his/her explanations. particularly, classes are developed in a completely interactive framework in which short numerical applications are solved as well as the student participation is enhanced by means of questions. as a consequence, an multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 5 environment with a continuous dialogue between student and lecturer is created; therefore, a previous preparation of the student becomes necessary (collier, 1983). some of the competences that the student can develop thanks to master classes would be: ability to allocate combustion in the frame of the energetic transformations and capability to provide a combustion process definition. capability to describe laminar and turbulent premixed combustion phenomena. capability to describe laminar and turbulent mixing controlled combustion phenomena. ability to identify physical and chemical phenomena which govern pollutant emissions generation. regarding the evaluation system, it is worthy to note that at the end of each of the 4 units of the complete course, a multiple choice test is performed by means of a digital platform developed at the universitat politècnica de valència (poliformat). tests are available for students during 60 hours. the number of questions varies from 5 up to 15 depending on the unit under evaluation. incorrect answers do not introduce any penalty. once the student decides to start the test, he/she has 2 minutes per each question. once the time is over, the test is considered to be finished, and answers together with the total grade of the test are sent to lecturers via poliformat. the 4 tests associated to the evaluation of acquired knowledge during the master classes represent a 10% of the total grade of the course. 2.2.2 guided work lecturer directs and guides students work by means of activities and exercises in which the student implements contents learned during the master classes (carpio, 2008). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 6 learning based upon projects and simulations: along each unit, the students performed 2 or 3 practical applications. these applications can be experimental or based on simulations. during these sessions, different abstract concepts are introduced to the students. these concepts will be treated in detail during the development of the master classes. the knowledge acquired by students is evaluated considering short exam answers provided by them during the own session as well as considering multiple choice tests that students carry out via poliformat during the next weekend after each practical session. learning based on numerical applications: group of questions/exercises adapted for each student. boundary conditions are defined considering each student identification number. these questions/exercises are solved after each of the 4 course units. all documents needed and required are exchanged via poliformat. for each exercise, students have two weeks to complete it. time starts when the unit is finished. if the student presents the exercise during the first week, lecturers provide feedback to the student giving guides to solve the possible errors which are made by the student. thus, the student is able to send again the exercise before the final deadline (2 weeks). by contrast, if the student presents the exercise after the first week has gone by, it is not possible to resend it again. numerical applications (4) represent a 30% of total grade of the course. the acquired capabilities by means of the guided work could be: capability to quantify the premixed combustion phenomena. capability to quantify the mixing controlled combustion phenomena. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 7 2.2.3 tutoring its main objective consists of completing the master classes. the tutoring is thought like a personalized meeting between lecturer and student. it can be performed individualized for a student or in small groups (collier, 1983). in any case, in the tutoring framework, the dialogue is enhanced and also a previous preparation for both parts is required, especially from the student side (anderson et al., 1997). with the aim of avoiding lecturer or students availability problems, the tutoring is performed on student demand via e-mail and it takes place in the lecturer’s office. nevertheless, if the doubt is generalized for a group of students, a classroom with the needed audiovisual media is fitted out to assist the students properly. the most relevant capabilities that can be achieved with the tutoring could be: capability to assume prominence and responsibility from the student side. capability to start reflection mechanisms in the student. 2.2 methods with high implication of the student. independent work. in contrast to master classes, which are basically carried out by the lecturer alone, the independent work make up the particular way to be followed by each student, in which learning is adapted to his/her particular circumstances and needs (carpio, 2008; posada alvarez, 2004). the aim behind this method is to promote a learning based on the search and evaluation of new information that allows solving problems out of the text books, to perform a better diagnosis of them, being less dogmatic and more open minded to consider alternative possibilities (abercrombie, 1980). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 8 2.3 work load per teaching method once the different selected methodologies have been defined, the time scheduled for each of them is described. the whole combustion course has 4.5 ects and therefore it implies that the total numbers of hours that can be spent in the different methodologies are 112.5 hours. formal hours for face-to-face activities: 22.5 h (master classes); 22.5 h guided work (projects and simulations), 3 h exam. formal hours for not face-to-face activities: 14 h (tutoring). independent work: 33.75 h (study); 16.75 h guided work (numerical applications). 3. discussion of the proposal and results on the one hand, the methodology presented previously will be compared to that already used in two equivalent courses that were taught in the industrial (at the etsii) and aeronautics (at the etsid) engineering degrees. since these two other subjects were delivered to students after choosing their major, both had a reduced number of students (<30). this last fact is in contrast with the context of the current course, where more than 180 students are enrolled. on the other hand, it is necessary to take into account that the proposed methodology is on the way of being introduced right now, as the subject is taught this year for the first time (academic year 2012-2013), and during the spring semester. consequently, the results that will be presented and discussed will be, up to now, incomplete. finally some observations, which seem to show some disadvantages or difficulties that may appear during the evaluation process, together with some proposals to overcome them, will be mentioned. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 9 3.1. comparison with its application to much smaller groups in a previous work (desantes et al., 2009), the methodology designed to be used in two courses equivalent to that of the present work but taught in the industrial and aeronautics engineering degrees was presented. as mentioned above, the main difference between these two subjects and the subject now under analysis is the number of students enrolled in them, both having a single group of less than 30 students. when comparing the introduction of both methodologies, the following remarks can be done: the thematic units are almost equivalent, since both the contents and the guidance of the master classes are very similar. consequently, despite its more difficult application, a more interactive character is intended to be given also now to the master classes, since this increases the student involvement. the study of laboratory practical applications (both experimental and numerical) is still maintained, since this helps not to lose the realism and applicability of the theoretical concepts studied. even if with smaller groups some team works were performed, with a continuous and careful monitoring of its development, this activity is completely not viable in a big size group, since it will consume all the lecturer’s time and energy. the alternative is to propose some individual works (customized, as mentioned before), which allow the student to face a real problem, to propose a tentative solution, to receive some feedback from the lecturer to know whether he/she is on the right way or not, and, finally, to propose a definitive solution with higher probability of success. it was observed that this procedure improves significantly the student response, as will be demonstrated in the next section. finally, in the context of a subject with such a big number of students, the “true” continuous evaluation that could formerly be applied with a smaller group was multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 10 given up. however, an effort was done to impose a working rhythm to the student by introducing some evaluation activities (with a limited weight in the final grade), in an attempt to evaluate the student in a more continuous way. lastly, removing the final exam of the course wasn’t considered appropriate, because it allows to ensure that the student has acquired all the basic knowledge to pass the course. after this comparison, it can be concluded that the transition from a small to a big group requires some unavoidable adaptations, not changing the spirit, but the style, to harmonize an evaluation with a high reliability degree with a reasonable amount of work for the lecturer. 3.2. obtained results as mentioned previously, the results that can be presented right now are incomplete, since this subject is currently being taught, and it is new this academic course (20122013). up to now (at the moment of writing this paper) the evaluation activities that have already taken place (the results of which are already available) are: 2 evaluation tests for the theoretical contents, 4 tests to evaluate the completion of the laboratory sessions, and 1 customized individual work. even if the final results of the course are not yet available, there are some aspects that are worth mentioning: the results of the different tests are excellent (see table 1). the most unfavorable case has an average grade of 7.6. even if these results are not fully reliable (even more if the observations described in section 4.3 are taken into account), they show that the student has done an effort to work out the different parts of the subject. this small (or big!) effort will surely have a beneficial impact on the final result. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 11 table 1. results obtained in the tests already performed. theory 1 theory 2 lab. 1 lab. 2 lab. 3 lab. 4 max. grade 10 10 10 10 10 10 min. grade 3 3.3 0 0 2 4 avg. grade 7.6 8.2 8.7 8.0 8.8 7.9 concerning the customized individual work, the fact of giving the student (if he/she wishes it) the opportunity to receive some feedback about what is wrong and what is correct, has significantly improved the obtained results, as demonstrated in figure 1. in this figure it can be observed how the histogram of grades for the 61 students that sent a preliminary result of their work (figure 1a) is improved when they deliver their final result (figure 1b). regarding this group of students, initially the average grade is 5.1, whereas the final one is 9.3. for the rest of students (98 students), who delivered only their final results, the average grade is 8.2. it can be observed, then, that the students having some feedback from the lecturer have achieved higher grades compared to those not taking benefit from this possibility. hence, it can be concluded that better results can be enhanced with a reasonable effort of the lecturer: on the one hand, because a working rhythm is set to the student and, on the other hand, because of the fact of giving him/her some feedback. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 12 (a) (b) figure 1. variation of the histogram of grades for the preliminary hand out (a) and for the final hand out (b) of the first customized individual work of the subject. 3.3. observed difficulties the detailed analysis of the results obtained by the students showed some possible difficulties that may appear when trying to apply some of the evaluation techniques employed in this subject, especially concerning the tests done by the students in a remote way (not in-person exam) via the digital platform of upv (poliformat). as an example, in figure 2 the relationship between the grade obtained in the test and the time spent to complete it is shown (it corresponds to the evaluation test of the first block of the subject, done by 166 students). in the figure a linear fit to the data is also plotted, to show the trend between these two parameters. surprisingly, this plot shows that the grade is higher as the time spent to do the test is shorter. even if this result can be, somehow, logical (it shows that the cleverest students take less time to complete the test), it is really strange taking into account the following consideration: the test consisted of 10 questions, and assuming that half a minute is required to read, understand and answer a question, a minimum of 5 minutes is needed to complete the test. however, some students have multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 13 obtained a grade of 8 or 9 doing the test in less than 5 minutes (see colored area in the figure). with the goal of better understand the previous result, some other information that will help for this purpose is shown in figure 3: on the left (figure 3a), the relationship between the grade and the moment at which the test was done is shown (it is worthy to know that the tests are open during two and a half days -60 hours-, and the students can do it at anytime during this period; this time is given in hours, starting from the time that the test is available for the students). it can be observed that, curiously, the grade is higher as time goes by… and, on the right (figure 3b), the relationship between the time spent to complete the test and the time when it was done is shown. in this case it is observed that, as time goes by, the students need less time to complete the test (and, based on what was said before, they obtain a higher grade). these observations raise the suspicion that the information “flows” among the students, which seem to learn from each other in such a way that the students doing the test later seem to recognize the questions that their 0 5 10 15 20 2 4 6 8 10 g ra d e [0 1 0 ] duration [min] figure 2. relationship between the grade obtained in the test and the time spent to complete it. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 14 colleagues have already answered, and they are able to complete the test faster and better at the same time. probably this fact does not affect all the students but, for sure, it seems to affect some of them. for this reason it is strongly advisable not to give too much weight to this type of evaluation, otherwise this will lead to an unfair overall evaluation. in fact, this weight needs to be defined based on a delicate equilibrium: big enough to motivate the student for studying the subject, but small enough to avoid passing the course unfairly. this is the reason of the moderate weight of this type of evaluation that was chosen in the context of the present course. (a) (b) figure 3. correlation between the grades obtained in the test and the moment at which it was done (a), and between the time spent to complete the test and when it was done (b). 4. conclusions different teaching methods have been evaluated and chosen within the framework of the european higher education area to be implemented into a combustion course delivered at the etsid of the universitat politècnica de valència. such methodology is based upon multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 15 the interaction between the lecturer and the student, and is being applied for groups with large number of students. among the main results, the high grades obtained by students have to be noted, which are independent of the area that is being evaluated (theoretical lectures, guided work…). in spite of the fact that such an estimator is not definitive to determine the success of the methodology, it can be considered as a first good approximation. however, certain deficiencies have been detected due to the dissemination of information among students when using the evaluation system by means of the upv digital platform, which questions some of the grades obtained by some students. such information dissemination has been quantified numerically. when eliminating such results, grades are still high, showing a high degree of comprehension of the concepts by the students. references abercrombie m.l.j (1980). aims and techniques of group teaching. london, society of research in higher education. anderson c. (1997). enabling and shaping understanding through tutorials, en f. marton et al., the experience of learning. edinburg: scottish university press. biggs j. and tang c. (2011). teaching for quality learning at university. mcgraw – hill education, open university press. carpio c. (2008). methods of teaching-learning applied to graduate in teaching in the framework of the european higher education area. docencia e investigación: revista de la escuela universitaria de magisterio de toledo, 18, 11-23. collier g., (1983) the management of peer-group learning: syndicate methods in higher education. guildford: society for research into higher education. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2190 social and technological sciences molina et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 1 (2014): 1-16 | 16 desantes j.m., garcía-oliver j.m., novella r. and garcía a. (2009). methodology of continous teaching-learning applied to combustión science. xvii conference on educational innovation in technical sciences, valencia. ministry of industry, energy and tourism (2011). energy in spain 2011. http://www.minetur.gob.es/energia/es-es/documents/energia_espana_2011_web.pdf turns s. r. (2000). an introduction to combustion: concepts and applications. united states, mcgraw-hill science. posada álvarez r. (2004). higher studies based on competences, formación superior basada en competencias, interdisciplinarity and student autonomous work. revista iberoamericana de educación. www.rieoei.org/deloslectores/648posada.pdf. uk-he europe unit guide to the bologna process (2007). http://www.ihep.org/assets/files/gcfpfiles/guide_to_the_bologna_process_edition_2.pdf edition 1&2. multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 1 application of learning analytics to improve higher education carlos llopis-albert1 , francisco rubio1 1 instituto universitario de ingeniería mecánica y biomecánica (i2mb). universitat politècnica de valència – camino de vera s/n, 46022 – valencia, spain corresponding author: carlos llopis-albert, e-mail address: cllopisa@upvnet.upv.es received: 10 january 2021; accepted: 17 august 2021; published: october 2021 abstract in the digital era, the teacher assumes very diverse roles among which are to be an adviser, a generator of multimedia content, and more recently a data analyst. big data analytics may play a major role in higher education for all the agents involved, the teachers and educators, the students themselves and the managers or heads of university centers. this paper applies learning analytics to the subject of theory of machines and strength of materials of the bachelor's degree in chemical engineering at universitat politècnica de valència (spain). the aim of analyzing the available information is to improve teachers’ actions and communication, to enhance resource efficiency, to assess classroom procedures, the achievement of transversal competences, the student typology and their results, or the attitudes and commitment they acquire with the subject taught. results show the existence of niches with competitive advantages, improvements in the quality and performance of the teaching-learning experience. keywords: learning analytics; big data; transversal competences; information and communications technology, e-learning, blended learning to cite this article: llopis-albert, c., rubio, f. (2021). application of learning analytics to improve higher education. multidisciplinary journal for education, social and technological sciences, 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://orcid.org/0000-0002-1349-2716 http://orcid.org/0000-0003-3465-702x https://doi.org/10.4995/muse.2021. multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 2 1. introduction learning analytics is an emerging research field that is growing in a significant way. they involve the measurement, collection, analysis, and reporting of data on the students and their contexts, in order to understand and optimize learning and environments in which they take place. the remarkable emergence of this field is attributed to several factors (e.g., dollár and steif, 2012; ferguson, 2012):  big data: the generalization of institutional databases and virtual learning environments (vle) involves the management of large data sets by institutions educational institutions, with the aim of using them to improve teaching-learning process.  online learning: the increasing presence of big data in education is accompanied by an increase in online and blended teaching-learning environment (btle), as well the widespread use of open educational resources (rea). they are any type of educational material that is in the public domain. they cover textbooks, lecture notes, syllabi, assignments, tests, videos, screencasts, and massive open online course (moocs). they are published under open licenses, such as creative commons (cc), that stipulates how materials may be used, reused, shared, adapted, and modified in agreement with specific needs.  the worldwide exponential development of information and communication technology (ict) over the past decade.  national interests: countries and international organizations are showing a greater interest in measuring, demonstrating, and improving educational outcomes and optimizing learning for the benefit of society. 2. material and methods the european higher education area (ehea) reoriented the focus attention of educational system towards the implementation of student-centered learning, facilitating that they are active, autonomous, critical, and reflective. there are multiple learning environments established that coexist with total naturalness, with the ultimate goal that students necessarily acquire the specific https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 3 knowledge of their degree and develop a corpus of competence required in the professional environment (oei, 2019). the main objective of learning analytics is to understand and improve learning and the environments in which it occurs. however, learning is a complex process in which multiple factors intervene. for this reason, it is necessary to use tools, techniques, and methods from various fields of research to study it. the collection of data from the students, when they are in a learning context, opens the door to new interpretations about what happens during the execution of the learning processes. it is possible to establish a classification of the uses of data in learning analytics, for example, referring to the prediction of student performance, providing feedback for instructors, grouping of students by learning profiles, detecting atypical behaviors or differentiated rhythms, analysis of uses in social networks, analysis of the development of competencies, planning and scheduling of courses, etc., so that the field of empirical research it is paid for future generations of teachers. the potential benefits of learning analytics are analyzed through a case study. specifically, it is applied to the subject of theory of machines and strength of materials (tm&sm), which belongs to the third year of the bachelor's degree of chemical engineering. it was taught in the first semester of previous academic year at universitat politècnica de valència (upv, spain). it is a compulsory subject that consists of 6 ects (european credit transfer and accumulation system), three of which are theory lessons and the other three are practical classes. the subject was taught under a blended learning (bl) methodology because of the coronavirus pandemic. a blended learning environment entails in-person classes, which are complemented with online learning. in addition, resources like videos, articles, screencasts, podcasts, and more are meant to enhance inperson classes and create an enriched learning experience. the classes were broadcasted live through the communication platform microsoft teams®, thus allowing the student to choose between attending the classes in-person in the university classroom, or by teams completely online. in addition, since the classes were recorded students could also consult them asynchronously, which is undoubtedly a great advantage for students. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 4 it is made up of two differentiated parts taught by two departments, with a weight of 50% on the final grade for each part. the department of continuous medium mechanics and theory of structures teaches the first part, which is related to the strength of materials, while the department of mechanical and materials engineering, to which i belong, teaches the second part related to theory of machines. in the 2020-2021 academic year there were only one group with seventy-seven students enrolled. classroom theory is taught by two professors, each one from a different department. additionally, there are four groups of practical lessons, which are taught by four teachers. two groups are taught by the theory lessons teachers, while two additional teachers are needed for the other two groups. this subject, which belongs to the common block to the industrial branch, is related to other subjects of the syllabus degree of chemical engineering, such as industrial equipment design or construction and industrial architecture. the syllabus regarding to the two different parts of the subject is subsequently presented:  strength of materials: the purpose of this part of the course is to introduce the student to the study of mechanics of deformable solids. at the end of the course, students will be able to identify and calculate the internal forces that occur in the solid because of the application of external forces, calculate the stresses in elementary structural systems of bars (beams and gantries) and properly dimension structural elements subjected to static loads.  theory of machines and mechanisms: it is a basic discipline in all engineering degrees of the industrial branch and occupies a fundamental place in very diverse fields of application. this subject is intended to provide a first introduction to the analysis and design of machines. to perform their function, machines transform movements and actions through mechanisms to adjust them to the useful work to be done. starting from the concepts of rigid solid dynamics, studied in physics, these are applied to the kinematic analysis (relation between entry and exit movements) and dynamic (relation between applied actions and resulting movements) of mechanisms. the syllabus covers the basic concepts of the theory of machines and mechanisms, the kinematic analysis https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 5 of plane mechanisms, the dynamic problem (including the newton's laws and the principle of virtual powers), and the design of anti-vibration systems. there are three practical classes, in which students must solve kinematic and dynamic problems using a specific software and to design an anti-vibration system. this research also aims to evaluate the transversal competences (tc) as defined by the upv. they try to synthesize a competency profile for its students, assuring a reference framework of all degrees. they encompass a set of cognitive skills and metacognitive and instrumental and attitudinal knowledge of significant value for the knowledge society. they are related to a set of attitudes and procedures that can be translated from one specific professional field to another. moreover, they are crucial and transferable in relation to a wide variety of personal, social, academic, and work contexts throughout of the life. upv defines thirteen transversal competences (upv, 2020). in this subject three transversal competences (tc) are evaluated:  tc-03. analysis and problem solving. analyze and solve problems effectively, identifying and defining the significant elements that constitute them.  tc-04. innovation, creativity, and entrepreneurship. innovate to respond satisfactorily and in an original way to personal, organizational, and social needs and demands with an entrepreneurial attitude.  tc-05. design and project. design, direct and evaluate an idea effectively until it is concretized in a project. the theory of machines and mechanisms part of the subject is taught by professors llopis-albert and rubio, which have an extensive experience in this field. to be able to transmit knowledge effectively, especially in the last years of the degree, it is important that teachers keep up to date with technological advances in the subject. they have published many articles closely related with the subject in the last five years. for instance, llopis-albert et al., 2015-2021; rubio et al., 2015-2021; valero et al., 2015-2021. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 6 3. results and discussion learning analytics and statistics are applied to enhance the teaching-learning process. on the one hand, figure 1 presents the online resources visits to the subject site, which belongs to the upv institutional platform (www.upv.es). on the other hand, figure 2 depicts the online resources visits made by each student to the subject site. it shows the most active user, i.e., the user who has consulted more resources. a total of 12470 visits from the seventy-seven students enrolled in the subject to the platform were received for the whole term. the average length of stay per visit is 31 minutes. the average time statistics show that there is a high correlation between the student activity and their score. the site is only available to members of the course and includes all the learning materials: announcements and organizational tasks, syllabus, schedule, teaching guide, audio-visual materials generated over the classes (both recorded videos of the classes and annotations on the virtual whiteboard), screencasts, and pdfs with the theory, practical lessons material, solved exams from past years, solved. it is observed that the greatest number of visits occurs in the weeks when each of the two partial exams are performed. 0 100 200 300 400 500 600 700 2 8 -f e b -2 0 2 1 1 4 -f e b -2 0 2 1 0 4 -f e b -2 0 2 1 2 8 -j a n -2 0 2 1 2 1 -j a n -2 0 2 1 1 4 -j a n -2 0 2 1 0 7 -j a n -2 0 2 1 3 1 -d e c2 0 2 0 2 4 -d e c2 0 2 0 1 7 -d e c2 0 2 0 1 0 -d e c2 0 2 0 0 3 -d e c2 0 2 0 2 6 -n o v2 0 2 0 1 9 -n o v2 0 2 0 1 2 -n o v2 0 2 0 0 5 -n o v2 0 2 0 2 9 -o ct -2 0 2 0 2 2 -o ct -2 0 2 0 1 5 -o ct -2 0 2 0 0 8 -o ct -2 0 2 0 0 1 -o ct -2 0 2 0 2 4 -s e p -2 0 2 0 1 7 -s e p -2 0 2 0 1 0 -s e p -2 0 2 0 0 1 -s e p -2 0 2 0 v is it s date online resources visits visits unique visitors https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://www.upv.es/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 7 figure 1. online resources visits to the subject site, which belongs to the upv institutional platform. figure 2. online resources visits by each student during the term. table 1. final grades of the subject theory of machines and strength of materials. grades alumni exam attendance total distinction (merit with distinction) 2 2.60% 2.60% merit (9 to 10) 10 12.99% 12.99% good (7 to 8.9) 31 40.26% 40.26% pass (5 to 6.9) 27 35.06% 35.06% fail (<5) 7 9.09% 9.09% exam attendance 77 100.00% no attendance 0 0.00% average score standart deviation maximum score minimum score 7.03 1.9 9.9 0.60 0 100 200 300 400 500 600 700 800 900 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 o n li n e r e so u rc e s v is ti s student online resources visits by each student during the term https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 8 table 1 and figure 3 present the final grades of the subject theory of machines and strength of materials (tm&sm) and also the final grades for the practices of the part of theory of machines ™, while table 2 shows the final grades of the practical classes of the part of tm. table 2 illustrates the final grades of the practical classes of the part of theory of (tm). a wide dispersion of scores has been observed in the scores, which is highly correlated with the different time devoted to the subject as presented in figure 2. results show a high academic performance, in which only 5 students failed the subject out of 77 (table 1). it is notable that all the students enrolled in the subject attended the exam. the average score is remarkable and most of them achieved a score between 7 and 9 (40.26% of the students for the subject and 65.06% for the tm practices). however, a higher average score is achieved in the practices. this is because they are easier an received the help of the teachers. moreover, only one student failed the practices lessons and there were 12 students that obtained a score between 9 and 10 (table 2). the standard deviation of the scores is slightly higher for the practices (2.35 if compared to the 1.9 of the score of the subject). table 2. final grades of the practical classes of the part of theory of (tm). tm practices grades alumni exam attendance total merit (9 to 10) 12 15.58% 14.46% good (7 to 8.9) 54 70.13% 65.06% pass (5 to 6.9) 10 12.99% 12.05% fail (<5) 1 1.30% 1.20% exam attendance 77 92.77% no attendance 6 7.23% average score standart deviation maximum score minimum score 7.5 2.35 10 3.09 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 9 figure 3. final grades of the subject theory of machines and strength of materials (up) and final grades for the practices of the part of tm (down). 0 10 20 30 40 distinction (merit with distinction) merit (9 to 10) good (7 to 8.9) pass (5 to 6.9) fail (<5) n u m b e r o f st u d e n ts grades tm&sm grades 0 10 20 30 40 50 60 merit (9 to 10) good (7 to 8.9) pass (5 to 6.9) fail (<5) n u m b e r o f st u d e n ts grades tm practices grades https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 10 figure 4. standard deviation of the marks of the theory of machines (tm) subject regarding theoretical lessons (tm-theory) and practical lessons (tm-practices). figure 4 displays the standard deviation of the marks of the theory of machines (tm) subject regarding theoretical lessons (tm-theory) and practical lessons (tm-practices). since the exam of the theory part is more difficult, its standard deviation is higher. figure 5. theory of machines (tm) grades with regard to the three practical lessons (pr_i) and four existing groups (3aq_i). https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 11 figure 5 shows the marks of the 3 practices of the part of theory of machines (tm) for the four existing groups (3aq_i). it is observed that the second practice is the one with the lowest notes because it is the most complex. figure 6. theory of machines (tm) grades regarding the kinematic and dynamic part of the subject. figure 6 shows the theory of machines (tm) grades regarding the kinematic and dynamic part of the subject. it is observed that the average score of the dynamics problem is quite low and should be corrected. however, the explanation for this may be due to the fact that it is taught at the end of the term and students have less time to prepare it. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 12 figure 7. theory of machines (tm) grades regarding the four existing practice groups (3aq_i). figure 7 depicts the theory of machines (tm) grades regarding the four existing practice groups (3aq_i). it is clear that students belonging to the 4 groups obtain similar marks, although the 3aq2 group stands out from the rest. figure 8. final mark of practices of the part of theory of machines (tm) for the four existing practice groups (3aq_i). https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 13 figure 8 presents the final mark of practices of the part of theory of machines (tm) for the four existing practice groups (3aq_i), which allows to determine discrepancies between them. it is observed that the students belonging to the 3aq1 group are the ones with the worst results. the teacher responsible of such group should detect what was the problem to correct it. figure 9. grades for the two parts of the subject, theory of machines (tm) and strength of materials (sm). figure 9 presents the grades for the two parts of the subject, theory of machines (tm) and strength of materials (sm). students obtain higher marks in the part of strength of materials, but the difference is reasonable. figure 10. theory of machines (tm) final grades regarding the theory and practical evaluation. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 14 figure 10 presents the theory of machines (tm) final grades regarding the theory and practical evaluation. students obtain higher marks in practices in comparison with the rest of the evaluation acts (open-response exams), since they are easier and receive help from teachers during the lab sessions. in view of the experience gained and the results, most of the learning-teaching process is satisfactory, but there is a room for improvement:  the academic performance of the students has been satisfactory with only 9% failures.  the discrepancy in the scores of the tm and sm parts should be corrected.  the practice scores are very high and more or less equal between the different groups.  students get lower marks in the dynamics part, perhaps because it is taught at the end of the term and students have less time to prepare it.  the practice marks are significantly higher than those of the theory exam.  the dispersion in the marks of the theory and problems exam is greater.  it is worth mentioning that the attendance to classes in-person was around 45% of the students, while the rest of students attended the classes through teams. only a couple of students did not attend the classes neither in-person at university classroom or by teams. eventually, these students failed the course. as an improvement for next academic courses, these students should be contacted before the end of the course to correct this situation.  attendance to practices either in-person or online was 100% of the students and the average grade was high (around 7 points) and only 3 failed.  it has been observed that the students who are more active in the site platform are the ones with the best grades. thus, a great correlation is observed between academic performance and devoted effort.  it is estimated that the time spent to obtain the average grade for the subject is adequate (7.03 points) and only 9% fail.  it is appreciated that the resource with the most accesses in the part of theory of machines is the one that presents a summary of the subject with the basic equations. this leads to the belief that students prefer material with the fundamental ideas and concepts of the subject than much material that takes a long time to study. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 15  the possibility of including other materials, such as learning games, could be investigated.  the largest number of visits is made on dates immediately prior to the partial exams. the student should be made aware of the importance of continuous work.  from the dashboard data, it has detected that there is a group of practices with significantly lower marks than the other three. thus, there should be better coordination between the different practice teachers to correct this aspect.  the practices have been carried out by means of an online exams tool embedded in the site platform. it has been observed that the grades have been very high, even though students were connected online and had to work more autonomously.  it is observed that the average score of the dynamics problem is quite low in comparison with the kinematic problem and it should be corrected.  it is observed that the second practical lesson of the tm part is the one with the lowest score because it is the most complex. the level of difficulty of the practices should be equalized.  with regard to qualitative data, based on the students' comments, they were satisfied with the development of the subject, the available material, etc. they suggested that the basic equations should be available during the exam.  it is observed that the students obtain higher marks in the part of strength of materials (sm). this could be corrected with greater coordination between the teachers of the department of mechanical and materials engineering and the department of continuous medium mechanics and theory of structures in order to equalize the level of evaluation acts.  it is observed that students obtain higher marks in practices with respect to the rest of the evaluation acts (open-response exams). the level of difficulty of the practices could be increased by making students work more autonomously and without so much help.  it is considered correct that the students belonging to the four practice groups obtain similar marks. https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 16 4. conclusions the sudden outbreak of the coronavirus pandemic forced a quick adaptation of the traditional in-person educational model to blended or hybrid models by means of ict tools. this paper applies learning analytics and statistics with the aim of improving the quality and performance of the teaching-learning experience. results have shown the existence of niches for improving the students’ performance. the detected gaps are intended to be solved in the following academic year. the proposed blended model has been proven to be suitable for the achievement of the transversal competences, and to improve both the students’ performance in a mechanical engineering subject, and their level of satisfaction. in fact, the student’s satisfaction survey regarding the subject, evaluation activities and teachers are very positive. results have shown an excellent pass rate and a high correlation among the evaluation of the open response written answers, the objective tests, the practices, and the final grade for the subject. author contributions: conceptualization, c.l.a; methodology, c.l.a and f.r.; validation, c.l.a and f.r.; formal analysis, c.l.a and f.r.; investigation, c.l.a and f.r.; resources, c.l.a and f.r.; data curation, c.l.a and f.r.; writing original draft preparation, c.l.a and f.r.; writing review and editing, c.l.a and f.r.; visualization, c.l.a and f.r. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references dollár, a., steif, p. s. (2012). web-based statics course with learning dashboard for instructors. computers and advanced technology in education. https://doi.org/10.2316/p.2012.774-025 ferguson, r. (2012). learning analytics: drivers, developments and challenges. in international journal of technology enhanced learning, 4, (5–6), 304–317. https://doi.org/10.1504/ijtel.2012.051816 llopis-albert, c., rubio, f., valero, f. (2015). improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research, 68 (7), 1429-1431. https://doi.org/10.1016/j.jbusres.2015.01.027 llopis-albert, c., rubio, f., valero, f. (2018). optimization approaches for robot trajectory planning. multidisciplinary journal for education, social and technological sciences, 5(1), 1-16. https://doi.org/10.4995/muse.2018.9867 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.jbusres.2015.01.027 https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 17 llopis-albert, c., rubio, f., valero, f. (2019). fuzzy-set qualitative comparative analysis applied to the design of a network flow of automated guided vehicles for improving business productivity. journal of business research, 101, 737-742. https://doi.org/10.1016/j.jbusres.2018.12.076 llopis-albert, c., rubio, f., valero, f., liao, h., zeng, s. (2019a). stochastic inverse finite element modeling for characterization of heterogeneous material properties. materials research express, 6(11), 115806. https://doi.org/10.1088/2053-1591/ab4c72 llopis-albert, c., valero, f., mata, v., pulloquinga, j.l., zamora-ortiz, p., escarabajal, r.j. (2020). optimal reconfiguration of a parallel robot for forward singularities avoidance in rehabilitation therapies. a comparison via different optimization methods. sustainability, 12(14), 5803. https://doi.org/10.3390/su12145803 llopis-albert, c., valero, f., mata, v., zamora-ortiz, p., escarabajal, r.j., pulloquinga, j.l. (2020a). optimal reconfiguration of a limited parallel robot for forward singularities avoidance. multidisciplinary journal for education, social and technological sciences, 7(1), 113-127. https://doi.org/10.4995/muse.2020.13352 llopis-albert, c., rubio, f. (2021). methodology to evaluate transversal competences in the master's degree in industrial engineering based on a system of rubrics and indicators. multidisciplinary journal for education, social and technological sciences, 8(1), 30-44. https://doi.org/10.4995/muse.2021.15244 llopis-albert c., rubio f., valero f. (2021a). modelling an industrial robot and its impact on productivity. mathematics, 9(7):769. https://doi.org/10.3390/math9070769 llopis-albert, c., rubio, f., valero, f. (2021). impact of digital transformation on the automotive industry. technological forecasting and social change, 162, 120343. https://doi.org/10.1016/j.techfore.2020.120343 llopis-albert, c., palacios-marqués, d., simón-moya, v. (2021). fuzzy set qualitative comparative analysis (fsqca) applied to the adaptation of the automobile industry to meet the emission standards of climate change policies via the deployment of electric vehicles (evs). technological forecasting and social change, 169, 120843. https://doi.org/10.1016/j.techfore.2021.120843 llopis-albert, c., rubio, f., valle-falcones, l.m., grima-olmedo, c. (2020). use of technical computing systems in the context of engineering problems. multidisciplinary journal for education, social and technological sciences, 7(2), 84-99. https://doi.org/10.4995/muse.2020.14283 oei (2019). learning analytics and education. revista iberoamericana de educación. organización de estados iberoamericanos para la educación, la ciencia y la cultura (oei). monográficos, volumen 80(1), 217 pages. https://rieoei.org/rie/issue/view/learning%20analytics/vol%2080%281%29 rubio, f., llopis-albert, c., valero, f., suñer, j.l. (2015). assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering, 10 pages. article id 931048. https://doi.org/10.1155/2015/931048 rubio, f., llopis-albert, c., valero, f., & suñer, j. l. (2016). industrial robot efficient trajectory generation https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.jbusres.2018.12.076 https://doi.org/10.1088/2053-1591/ab4c72 https://doi.org/10.3390/su12145803 https://doi.org/10.4995/muse.2020.13352 https://doi.org/10.4995/muse.2021.15244 https://doi.org/10.3390/math9070769 https://doi.org/10.1016/j.techfore.2020.120343 https://doi.org/10.1016/j.techfore.2021.120843 https://doi.org/10.4995/muse.2020.14283 https://rieoei.org/rie/issue/view/learning%20analytics/vol%2080%281%29 https://doi.org/10.1155/2015/931048 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 llopis-albert and rubio (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 1-18. https://doi.org/10.4995/muse.2021.16287 18 without collision through the evolution of the optimal trajectory. robotics and autonomous systems, 86, 106–112. https://doi.org/10.1016/j.robot.2016.09.008 rubio, f., llopis-albert, c. (2019). viability of using wind turbines for electricity generation in electric vehicles. multidisciplinary journal for education, social and technological sciences, 6(1), 115-126. https://doi.org/10.4995/muse.2019.11743 rubio, f., valero, f., & llopis-albert, c. (2019a). a review of mobile robots: concepts, methods, theoretical framework, and applications. international journal of advanced robotic systems, 16(2), 172988141983959. https://doi.org/10.1177/1729881419839596 rubio, f., llopis-albert, c., valero, f., besa, a.j. (2020). sustainability and optimization in the automotive sector for adaptation to government vehicle pollutant emission regulations. journal of business research 112, 561-566. https://doi.org/10.1016/j.jbusres.2019.10.050 upv, 2020. proyecto institucional competencias transversales. universitat politècnica de valència (upv). valencia. spain. https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf valera á., valero f., vallés m., besa a., mata v., llopis-albert c. (2021). navigation of autonomous light vehicles using an optimal trajectory planning algorithm. sustainability. 2021; 13(3):1233. https://doi.org/10.3390/su13031233 valero, f., rubio, f., llopis-albert, c., cuadrado, j.i. (2017). influence of the friction coefficient on the trajectory performance for a car-like robot. mathematical problems in engineering, 9 pages. article id 4562647. https://doi.org/10.1155/2017/4562647 valero, f., rubio, f., llopis-albert, c. (2019). assessment of the effect of energy consumption on trajectory improvement for a car-like robot. robotica, 37(11), 1998-2009. https://doi.org/10.1017/s0263574719000407 valero, f., rubio, f., besa, a.j. (2019a). efficient trajectory of a car-like mobile robot. industrial robot: the international journal of robotics research and application, 46(2), 211–222. https://doi.org/10.1108/ir10-2018-0214 https://doi.org/10.4995/muse.2021. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.robot.2016.09.008 https://doi.org/10.4995/muse.2019.11743 https://doi.org/10.1177/1729881419839596 https://doi.org/10.1016/j.jbusres.2019.10.050 https://www.upv.es/entidades/ice/info/proyecto_institucional_ct.pdf https://doi.org/10.1155/2017/4562647 https://doi.org/10.1017/s0263574719000407 https://doi.org/10.1108/ir-10-2018-0214 https://doi.org/10.1108/ir-10-2018-0214 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 1 data mining techniques for the study of online learning from an extended approach sánchez-sordo josé manuel* universidad nacional autónoma de méxico, fes iztacala, departamento de psicología, área de procesos estadísticos. * corresponding author: email: jose.sordo@ired.unam.mx received: 2019-07-01; accepted: 2019-04-23 abstract in the latest years information technologies have impacted society changing the way human beings learn, and because of that it is necessary to study the intimate relationship between humans and their technological tools. on this path the extended mind thesis posits human cognition as a process that occurs in conjunction between biological and non-biological components, furthermore connectivism is stated as a learning theory for the digital age. based on such approaches this work presents a summary of a research whose objective was to know how people extend their cognitive processes with the aim of learning through the internet. methodologically, an artificial intelligence algorithm for supervised learning (j48) was used to analyze the data of 336 participants with the aim of obtaining classification rules (patterns) of internet use. finally, the results show that people who report visiting specialized websites, read electronic books and take into account the spelling of the resources they are looking at on the internet are the ones with optimal strategies for learning online. keywords cognition, connectivism, data mining, e-learning, extended mind, machine learning. https://doi.org/10.4995/muse.2019.11482 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 2 1. introduction in recent years, information technologies have massively impacted society, and the new generations for marqués (2012), have been naturally assimilating this new culture attached to the digital. derived from this, international organizations such as unesco (2006), have suggested development plans based on informatics focused on the educational, which according to the “knowledge society” aim for a sustainable development (unesco 2013). however, current technologies have not only transformed the education system or the economic model, since the use of technological tools or instruments far from being just a matter of the “digital age” or of the current times, is contemplated as foundational of the human genre, because the use and generation of technology allows the modification of nature by human beings, situation that brings with it profound implications on the plane of the evolutionary and particularly of the cognitive, insofar as human activity beyond the biological combines in its core the integration of external tools that allow it to continue transforming the world and transforming itself, since these, tools as vygotski (1995) stated, are artificial organs and it is because of this that it is proposed that the intimate relationship between human beings and their technologies for learning purposes should be addressed nowadays from novel theoretical and methodological approaches from various disciplines such as psychology, philosophy and computer sciences. in the same way, one of the main approaches or models that currently addresses the interaction of tools in a deeper sense is the extended mind thesis, which posits human cognition as a process that occurs in conjunction between biological and non-biological components as proposed by clark and chalmers (2011). this supposes the integration of tools as a part of the cognitive process, thus giving origin to the notion of extension of the mind —mind as understood from the behaviorist approach of ryle (2009), that beyond how criticizable it is, his vision of the mind as something that connects with other notions prevents the quick assimilation of itself as an organ, either material or immaterial, as mentioned by calvo in clark et al. (2011)— so their thesis (clark 2008; clark et al. 2011), multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 3 states broadly that the brain, the objects and the world coordinate and extend as one thanks to cognitive action. in respect to this, the authors emphasize that said integration of the world’s objects as a fundamental part of the cognitive process is achieved thanks to the functional parity that exists between the functions of both the organism and the object (clark 2001), that is to say that the tools (whether digital devices or a pencil) are assimilated as part of cognition not because of their physical or material constitution but by the functions they perform. likewise, there are proposals similar to the extended mind, such as hutchins’s (1995), distributed cognition, which postulates that the mind is in the world, in contrast with the notion that the world is in the mind. because for this approach cognition, as also proposed by clark (2011), is distributed amongst the people, objects and tools that belong to certain contexts. by way of example, hutchins (1995) mentions that the knowledge needed to operate a naval ship does not exist only inside a person's head, but that the process is distributed through objects, people and tools in the environment itself. being then the objective of distributed cognition (2000), to know and describe the distributed units that coordinate to perform cognitive action, as well as the contextual framework in which the activity is performed. based on such postulates, this work presents a research whose goal was to know, with the aid of artificial intelligence algorithms, how people, from the perspective of clark (2008) interact with information networks and the internet to extend their cognitive processes to generate learning in conjunction with software and electronic devices such as web browsers, search engines, computers and smartphones, which within this work’s theoretical framework should be understood as cognitive extensions, because by using them we can overcome our natural limitations, thus harboring the cognitive process in a complex composed of biology and technology. for which the study of learning from an extended approach becomes important nowadays since it is necessary to know how learning occurs multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 4 in the digital age in conjunction with the tools, and not addressing it more only under the popular but somewhat obsolete concept of ict for education, that far from reflecting about the psychological processes that are implied and modified with the use of technologies for learning, it seems in part to only open the market for a few applications and artifacts that are believed to place the classrooms and educational institutions at the “vanguard” as suggested by marqués (2012). 1.2. extensions and learning in the digital age: as stated, in this approach human cognition is conceived as an extended process that does not occur entirely inside the skull (clark et al. 2011), which implies the “fusion” of man with his tools in a psychological sense. the acceptance of such philosophical proposal suggests then a different way of approaching the study of learning and education, being in the sphere of the educational connectivism (siemenes 2006), a proposal that is stated as a learning theory for the digital age, that is, a referential and explanatory framework of the learning process that occurs in formal or informal digital educational environments (sánchez 2014), and not a psychological theory that attempts to explain human development or behavior in its entirety, but a theory that provides or tries to provide an answer as to how people learn and increase their current state of knowledge in a specific context: computer networks. this clarification is pertinent given the criticisms that connectivism has received as a learning theory, such as zapata-ros’s (2015), which adhering to old theoretical conceptions refuses to accept siemens’s (2004), proposal that “knowledge can reside in non human appliances”, approach that is in accord with what clark (2011) proposes, and that reflects the underlying social context that gives life to these new theories: technology as an active agent in the learning process. at a conceptual level connectivism integrates principles of networks, complexity, selforganization and aspects related to the extended mind, since it emphasizes that a large part of the knowledge and learning that is generated occurs outside of people’s heads. for multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 5 downes (2011), connectivism is the thesis in which learning is distributed throughout a network of connections, and therefore learning consists of the skill to build and navigate those networks. regarding this, redecker (2009), states that such networks exist both externally and internally, in the external they are the structures that we create in order to be up to date and to continuously create and connect with new knowledge; and their nodes are the entities (people, online encyclopedias, websites, wikis, forums, applications, etc.) with which individuals connect to form a network. internally, learning networks can be perceived as structures that exist in our minds in the connection and creation of comprehension patterns, given that as siemens (2006) affirms we psychologically adapt our brain’s connections to process the environment in which we move, given that the brain restructures its neural connections with the use of technology, which clearly implies a direct relation between the organization and functioning of our brain and what we can learn about the world. for this approach then the value of the study of learning is to comprehend the capacity of individuals to generate connections and networks in informal educational environments on the internet that promote the specialized connection between sets of information that allow individuals to increase their knowledge, thus emphasizing the skill and autonomy of the individual to obtain and store information in structures external to him (mostly computer networks or digital devices), that from the approach proposed here perform as cognitive extensions. concerning the above, the study of strategies or ways in which people interact with technology to generate connections and networks for learning becomes necessary since it indicates the use and perception that people have of the learning that is generated in a distributed way on the web. on this path garay et al. (2013), state as cognitive strategies related to learning on the web 2.0 the following: search (of information), compilation, management, reflection and practices, which at the same time are associated with particular online tools like forums, blogs, wikis and social networks. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 6 although from connectivism the term cognitive strategies is not used, downes (2009) mentions that knowledge is no longer monopolized by official instances, but that it is diversified in positions, facts and opinions, being then the skill or strategy to differentiate between reliable and unreliable sources of knowledge something of great importance within connective learning. 2. methodological perspective objective: to know and describe with artificial intelligence algorithms, how people extend their cognitive processes when interacting with computer networks with the goal of generating knowledge from a connectivist logic. type of study: a non-experimental study with a quantitative approach using artificial intelligence techniques for data mining was carried out. sample: for this work a non-probability sample of 336 volunteers was required. the majority (71%) were women from mexico city belonging to the professional areas of biological sciences (50%) and humanities (21%). 2.1. procedure participants were asked to fill the online questionnaire titled “strategies for the selection and use of information for online learning” developed for this study (content validity index of .95), and which is composed of 60 items (attributes) that address the possible strategies involved in the online learning process. based on the bibliography (garay, lujan, and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 7 etxebarria 2013; head and eisenberg 2010; hernández 2010) it was decided that the dimensions addressed by the questionnaire were the following: 1. sociodemographic data: gathers information about the participant, like age and place of residence. 2. connectivity: aims to know aspects related to the users’ internet access. 3. type of sources consulted: evaluates the number and format of the online resources that the participant consults (what do participants consult on the web?). 4. search and access to information: provides a list of options to know how the participants search for and access information on the internet (how do they find the information?). 5. validation of the information: gathers information about what criteria are used by the participants to determine the consulted information as true or reliable. 6. storage and recovery: inquires about what the participants do to store the resources they consult on the web in order to be able to access them in the future. 7. self-evaluation: allows the participant to rate their strategies of internet use in relation to the generation of knowledge. 2.2. data analysis for the data analysis, the kdd (knowledge discovery in databases) model was used, it is a multistep process for the discovery of knowledge in large data collections. nigro, xodo, corti and terren (2004), mentioned that the kdd process is iterative by nature, and depends on interaction for a dynamic decision making. one of the more complete definitions of kdd is proposed by fayyad (1997), who assured: knowledge discovery in databases are rapidly evolving areas of research that are at the intersection of several disciplines, including artificial intelligence, statistics, machine learning, pattern recognition and visualization for extracting knowledge automatically from databases. (p.2) multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 8 to perform the kdd the following steps are required: 1. data selection. in this stage data sources and the type of information to be used are determined. 2. pre-processing. this stage consists of the preparation and cleaning of data extracted from the various data sources in a manageable form, necessary for the following phases. 3. transformation. consists of the preliminary treatment of data, transformation and generation of new variables from already existing ones with an appropriate data structure. 4. data mining. it is properly the modeling phase, in which intelligent methods are applied with the objective of extracting patterns previously unknown, valid, new, potentially useful and comprehensible that were contained or “hidden” within the data. 5. interpretation and evaluation. the truly interesting patterns obtained are identified based on certain measures and the obtained results are evaluated and interpreted. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 9 figure 1. kdd steps outline. 2.2.1 description of the performed analysis and processes: regarding the objective set for this research, and using the previously described kdd model, quantitative analyses were carried out through data mining with the j48 supervised learning algorithm and descriptive statistics in order to be able to interpret the gathered information as it is explained:  the data derived from the application of the online questionnaire “strategies for the selection and use of information for online learning” to the 336 participants was analyzed with the goal of finding classification rules (patterns) in the data for the subsequent classification of the subjects according to their internet use strategies (good, regular and poor) using the j48 algorithm. subsequently descriptive data were obtained according to the participants’ perception of their internet use. the algorithm used to perform the data analysis is described below: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 10 decision trees (j48): it is a free version of the commercial c4.5 algorithm developed by quinlan (sancho 2018) and has the objective of decreasing the entropy of the data by using information gain. this is to find the attribute that better divides or “arranges” the data according to the categories in which they are to be classified using the function: e(s)=∑ci=1−pilog2(pi)7 therefore, in order to obtain the attribute that generates more homogeneous branches within the decision tree sancho (2018), indicates the following: 1. total entropy is calculated. 2. the data set is divided according to the different attributes. 3. the entropy of each branch is calculated and then they are added proportionally to calculate total entropy: e(t,x)=∑c∈xp(c)e(sc)e(t,x)=∑c∈xp(c)e(sc) 4. this result is subtracted from the original entropy, obtaining as a result the information gain (entropy decrease) using this attribute. gain(t,x)=e(t)−e(t,x)gain(t,x)=e(t)−e(t,x) 5. the attribute with the highest gain (of information) is selected as a decision node, it is to say, the attribute through which the data classification will begin. finally, this process is repeated with all the nodes that do not perform as outputs until null entropy levels are obtained (leaves). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 11 3. results and findings a. classification rules (patterns) of internet use for online learning: next, the results obtained from the application of the j48 algorithm for the discovery of internet use classification rules (patterns) are shown according to what was reported by the 336 participants in the “strategies for the selection and use of information for online learning” questionnaire, it should be mentioned that the correlation attribute eval (weka 3.8.3) filter was applied in order to estimate correlations between questionnaire attributes and in this way select the most pertinent to analyze with the algorithm. the model generated by the algorithm to classify the data according to the participants’ internet use strategies showed very high levels of area under the roc curve (roc), which means that the patterns shown below are accurate indicators to know and predict the participants’ behavior on the internet for education or learning purposes. table 1. statistical details of the j48 classifier tree “internet use strategies for learning” kappa: 0.842 correctly classified instances 91.6667% roc area: .966 number of leaves 35 shown on the following image is the decision tree (pruned) product of the model generated with the j48 algorithm, shown in said tree are the nodes (attributes) that better arranged the distribution of all the data for their classification: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 12 diagram 1. decision tree model obtained for the “strategies for the selection and use of information for online learning” questionnaire data (n=336). objective: to find patterns of internet use for learning and with it classify the participants according to their strategies in “poor, regular, and good”, and infer possible patterns of cognitive extensions (see appendix for better image quality). as it can be observed in the diagram above, 35 patterns to classify the subjects (n=336) were found according to the internet use they reported and the resources they access, store and recover to generate learning. as it can be observed, the tree includes rectangles (leaves) colored green, yellow and red, each corresponding to a classification respectively; green for good (strategies), yellow for regular (strategies) and red for poor (strategies). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 13 table 2. patterns and classifications of the use of internet for learning classification: poor regular good number of patterns: 4 patterns (classification rules) 22 patterns classification rules) 9 patterns classification rules) number of instances (persons): 21 (19 correct, two wrong) 205 (187 correct, 18 wrong) 110 (102 correct, eight wrong) as it can be observed on table 2, the largest number of patterns obtained was for the regular strategies category (22), which means that the majority of the participants have regular strategies of internet use for academic purposes. a. patterns with more classified instances: poor: 1 (13 persons), 2 (5 persons). regular: 1 (62 persons), 2 (37 persons). good: 1 (65 persons), 2 (13 persons). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 14 figure 1. shown by the graph are the two most representative patterns of each category according to the number of classified instances in their interior. then, of the 35 patterns obtained with the algorithm (diagram 1) individual patterns are shown and interpreted for each category that contains the most instances and that better classifies and predicts the participants’ actions. with them we can infer how people extend their cognitive processes on the internet, since these individual patterns show the combinations of attributes that the majority of the participants perform in relation to what tools they use and how they do it to connect (sánchez 2019), and in this way generate knowledge and online learning. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 15 pattern no.1 of classification rules for good use of the internet) diagram 2. main pattern of good strategies for the use of internet for learning. interpretation: based on this pattern (no.1) we can induce that most of the participants (65/110) with good strategies of internet use for learning employ digital tools in the following manner: they are people who almost always take into account the quality of the spelling and writing of the resources they consult online to consider them valid; on occasion they make summaries or take notes of the information they consider relevant, they consult plenty of specialized websites and almost always look up the bibliographical references that are cited within the sources they consult. they also use wikipedia and check many (4-5) books or book chapters in electronic formats and use many of google’s specialized search options and almost always take into account the quality of the images and graphics that are included on the online documents they go over. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 16 with this information it can be said that this particular combination of tools and activities is how the participants extend their cognitive processes in conjunction with tools to generate connections on the internet in an optimal way with the purpose of learning. pattern no.2 of classification rules for regular use of the internet) diagram 3. main pattern of regular strategies for the use of internet for learning. pattern (no.2). in this pattern the participants (62/187) with regular strategies of internet use for learning employ digital tools in the following way: sometimes they consider writing and grammar as something important to select internet resources, also on occasion they make summaries of the relevant information, but they never or almost never visit specialized websites. based on this interpretation it can be said that the pattern predicts that this combination of tools and activities is how most of the participants extend their cognitive processes in conjunction with online learning tools in a way that can be considered as regular, because they take into account spelling and make summaries or notes, but they do not consult specialized websites. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 17 pattern no.3 of classification rules for poor use of the internet) diagram 4. main pattern of poor strategies for the use of internet for learning. pattern (no.3). in this pattern the participants (11/19) with poor strategies of internet use for learning employ digital tools on the following way: they never or almost never take into account the quality of the writing or spelling of the online resources they consult, as well as they never or almost never look up the bibliographical references that are cited and also visit none or very few specialized websites. this way we can predict that people that do not take into account grammar nor cited references in online sources are the ones that make a deficient use of online tools (extensions) to connect with new learning. b. descriptive analysis: shown in this section are a couple of graphics with results obtained from the analysis of the frequencies of the participants’ opinions in relation to how technology influences some of their cognitive processes. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 18 the first aspect addressed was: “by being able to store the information you consult online with academic or learning purposes inside virtual folders, your ‘internal’ memory processes were amplified and/or modified.” figure 2. memory amplified or modified by technology. overall, 84% of the participants agree or totally agree that their “internal” memory processes have been modified by the use of current information technologies. situation accordant with the ideas proposed by sánchez et al (2019), about the extension of the mind, that state that cognitive processes, in this case memory, are shared or distributed between neural/biological processes and electronic devices. likewise, the participants were asked if: “when using computer networks or digital devices to learn or generate knowledge, they consider that their learning is occurring not only ‘inside’ their heads but occurring in conjunction with the devices or networks”. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 19 figure 3. learning distributed between “inside the head” and technological devices. we can observe that most of the participants consider that their learning processes occur not only inside their heads, but that they are distributed with devices and computer networks, situation that in some way reflects what is mentioned by siemens (2004) and clark (2008), about learning and cognition that occurs “outside” the heads of individuals. 4. conclusions the findings of this research show in some way how cognition is shared or extended when we use digital tools, based on the fact that we psychologically adapt our neural connections with the use of technology (siemenes 2006), which clearly implies a direct relationship between the organization and functioning of our brain and the tools we use (maravita and iriki 2004). with this we show that connectivism is a theoretical framework that shares postulates with the philosophical thesis of the extended mind proposed by clark & chalmers (2011), so addressing cognition from these approaches together gives a current and complex view of the phenomenon of learning in digital educational environments. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 20 in relation to this, it is considered that the results of this research provide data of interest to people involved in non-formal and online education, because interesting facts were found about which tools favor online learning, being some of the most favorable abilities or strategies: to check the spelling of online resources, to make summaries of reliable information, as well as the use of some specialized search engines. also, other relevant result is that most of the participants agree that ict have modified their “internal” cognitive processes such as memory. also, the kdd model allowed us to discover from the data relevant information about how the participants of this study use certain technological tools on the internet, allowing us to then generate usability profiles that allowed to classify the subjects as having optimal, regular and poor use of the internet with learning purposes, contributing thus with relevant data for learning and educational psychology in the digital age. in addition, the kdd model and the use of artificial intelligence algorithms such as the ones used in this research proved to be effective tools for the discovery of new knowledge within psychological research, because its use approaches psychology to different methodological paradigms that allow the analysis of human behavior in an innovative way (fayyad 1997), allowing also the analysis of large amounts of data. finally, we can mention that this work’s contribution is to address education from psychology and philosophy with artificial intelligence in a digital environment, areas of knowledge that are giving a fundamental shift to education nowadays. basically, the proposal of this work is based on an evaluation of the learning process using artificial intelligence techniques for the classification of patterns in the use of information and tools on the internet, thus giving basic clues to support the concept of extended mind, which should be addressed in future educational research starting from these bases and having other considerations as there are other types of learning and ways in which people interact with tools (apud 2014). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 21 5. references  apud, i. (2014). ¿la mente se extiende a través de los artefactos? algunas cuestiones sobre el concepto de cognición distribuida aplicado a la interacción mente-tecnología. revista de filosofí. 39 (1), 137-161.  castañeda, l., & adell, j. (2013). entornos personales de aprendizaje: claves para el ecosistema educativo en red. alcoy: marfil.  clark a. (2001) mindware: an introduction to the philosophy of cognitive science. oxford: oxford university press  clark, a. (2008). supersizing the mind: embodiment, action and cognition extension. oxford: oxford university press.  clark, a., & chalmers, d. (2011). la mente extendida. oviedo: krk ediciones.  downes, s. (2009). the new nature of knowledge. online [available] https://www.downes.ca/cgi-bin/page.cgi?post=53404  downes, s. (2011). connectivism and connective knowledge [online] available: https://www.huffingtonpost.com/stephen-downes/connectivism-andconnecti_b_804653.htm  fayyad, u. (1997). data mining and knowledge discovery in databases: implications for scientific databases. ssdbm '97 proceedings of the ninth international conference on scientific and statistical database management , 211.  garay, u., lujan, c., & etxebarria, a. (2013). el empleo de herramientas de la web 2.0 para el desarrollo de estrategias cognitivas: un estudio comparativo. porta linguarum. 20, 169-186.  head, a., & eisenberg, m. (2010). how college students evaluate and use information in the digital age [project information literacy progress report]. washington, d.c.: university of washington. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 22  hernández, s. (2010). uso de estrategias de aprendizaje con apoyo en internet considerando el género y el nivel educativo. viii congreso iberoamericano de ciencia tecnología y género. curitiba.  hutchins e. (1995), cognition in the wild, bradford book.  hutchins, e. (2000) distributed cognition, online [available]: http://comphacker.org/pdfs/631/distributedcognition.pdf  maravita, a., & iriki, a. (2004). tools for the body (schema). trends in cognitive sciences. 8 (2), 79-86.  márqués p., (2012). impacto de las tic en la educación: funciones y limitaciones, revista 3ctic, 1, (3).  nigro, xodo, corti and terren, (2004), online [available]: http://sedici.unlp.edu.ar/handle/10915/21220  redecker, c. (2009). review of learning 2.0 practices: study on the impact of web 2.0 innovations on education and training in europe. bruselas: joint research centre.  ryle g., (2009). the concept of mind, rouledge  sánchez-sordo, j (2019). redes y cognición; abordando la mente extendida en ambientes conectivistas de aprendizaje, revista digital internacional de psicología y ciencia social, to be published.  sánchez-sordo, j. (2014). conectivismo y ecologías para la educación a distancia en la web 2.0. revista mexicana de bachillerato a distancia, 6(12), 11.  sancho, f. (2018). aprendizaje inductivo: árboles de decisión. obtenido de universidad de sevilla: http://www.cs.us.es/~fsancho/?e=104  siemens, g. (2004). conectivismo: una teoría de aprendizaje para la era digital. madrid: ediciones nodos ele.  siemens, g. (2006a). conociendo el conocimiento. madrid: ediciones nodos ele. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 23  unesco, (2006). hacia las sociedades del conocimiento: informe mundial de la unesco. [available]: https://unesdoc.unesco.org/ark:/48223/pf0000141908 .  unesco, (2013). education for sustainable development goals: learning objectives. [available] https://unesdoc.unesco.org/ark:/48223/pf0000247444  vygotsky, l. (1995). historia del desarrollo de las funciones psíquicas superiores. madrid: editorial visor.  zapata-ros, m. (2015). teorías y modelos sobre el aprendizaje en entornos conectados y ubicuos. bases para un nuevo modelo teórico a partir de una visión critica del “conectivismo”. revistas vsal. 16 (1), 1-49. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11482 social and technological sciences issn: 2341-2593 sánchez (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 1-24 | 24 appendix: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 134 the power of ethics: rethinking leadership roles at workplaces mitashree tripathy, asst prof. dept of humanities, orissa engineering college, bhubaneswar, odisha * corresponding author: email: mitashreetripathy84@gmail.com; phone: +91-9437114432. received: 11 july 2018; accepted: 19 september 2019 abstract in the past leadership roles at workplaces looked only into bringing profits to organizations, compete with each other and create a brand image come what may. the emergence of incorporating moral and ethical codes developed when both employees and leaders of organizations big and small got involved in unethical practices which lead to a tremendous loss in business and involved legal cases. not only the reputation of such companies got tarnished but gradually they started losing their authenticity in the business world. surprisingly, moral codes and ethical standards always existed in the organizations’ policies but their relevance was recognized only in the twenty-first century when business organizations began to merge and expand worldwide because of globalization and the system changed forever. organizations today simply cannot compromise with unethical practices and values. today, organizations believe that good leadership is not merely competing, bringing profits and creating an image but also transforming workplaces and bringing changes in the lives of people through ethics. studies reveal that the integration of ethics within leadership roles can bring significant benefits at workplaces. morally upright leaders foster good business, encourage excellent task performance, make powerful decisions and nurture corporate vision. this paper studies the relevance of blending leadership with ethics with a purpose to fetch substantial impact at workplaces. keywords ethics, morals, leadership, ethical dilemmas, workplaces multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 135 1. introduction the word ‘leadership’ suggests different connotations in different domains. some understand leadership is managing a team either at schools or colleges or in any sports team. others mean leadership as voluntarily initiating with some work and the perceptions are many that diverge differently in different contexts. at workplaces, however, leadership defines a position or a role to carry out some goals associated with the betterment of the organization. a typical model categorizes leadership as not what leaders do rather what they are like. to understand it better let us consider an example. john is the team leader leading a group of employees in his company. he does his regular duties, he is technically sound, intelligent, trains his employees well. but he lacks compassion, avoids problematic conversations, does not know how to deal with complaints, cannot accept challenges readily, and is not enthusiastic in solving problems. comparing what john as a leader does and what he is characterized with as a person, one can easily conclude that simply fulfilling job roles of leadership does not make anyone a leader. rather there is so much more to it. leadership both as a role and a trait is one of the most essential tools at workplaces to deliver performances that generate effectiveness and accomplish organisational goals. the process begins by setting vision combined with values and moving ahead with pleasure inspiring as many people in the path of success. it is however important to also understand that being a leader is not a designation instead it is a role and it can be at any level at a given workplace. but are values that important? today innumerable companies around the world believe in values, morals, and ethics as an important means of workplaces of any kind. working with ethics makes a leader an ethical leader. collins in his book essentials of business ethics: creating an organisation of high integrity and superior performance describes “ethics is the set of principles a person uses to determine whether an action is good or bad. ethics permeates every stakeholder interaction involving owners, customers, employees’ lenders, suppliers, and government officials” (collins, 2009, p.4). today leaders at workplaces have to do tremendous activities everyday without fail. but sometimes leaders get deviated from their track and mismanage situations at workplace. therefore, leaders are supposed to be conscious while performing duties. self-aware is a must. chamberlain in his book conscious leadership: a guide to making a difference one person at a time discusses “individuals within organizations have to navigate corporate culture and often take their cues from the individuals around them. effective leaders have to lead and set the tone for that culture and that team. both have to take charge and determine how to do this. it is multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 136 an individual responsibility. each of us needs to see where we are living up to our values and where we are sabotaging our achievement to reach our goals” (chamberlain, 2016, p. 6-7). ethical leaders must realise on what grounds the values of leadership get incapacitated in the process of achieving goals. the business world is hypnotizing and studies reveal that people in higher roles often get carried away by practices that devalue morals and ethics. hence, it becomes easy to fall prey to the dishonourable and disreputable preparations in exchange for some temporary yielding. leadership is, therefore, knowing the inner self first and finding out the ways that drive towards such practices and then knowing the people around. leaders who view the existence of ethics in their characteristics traits and job roles as crucial often come across situations that put them into dilemma. thoughts like serving clients, customers and members of the organizations uniquely without any misrepresentation, achieving desired results without being dishonest, delivering work performance without harming the position, providing satisfaction to customers without discomforting staff and finance soundness, leading through uprightness without being misguided and many more. however, doing so is not a cakewalk. langlois in the book the anatomy of ethical leadership: to lead our organizations in a conscientious and authentic manner peers being courageous in such situations. langlois writes “in such a context, making an ethical decision can become a perilous obstacle course though amaze of reflection and organizational rules. the path is fraught with pitfalls and may require a certain moral fortitude” (langlois, 2011, p. 35). everyday leaders face a number of challenges related to professional and personal structures and come across complex circumstances when trapped between their morals and professional demands in order to resolve problems that bypass their state of affairs. workplaces embody a large mass of people both internally and externally including staff members, managers, executives, customers, clients, policyholders, stakeholders, government officials and many more. an effective leader is in charge of maintaining codes of conduct with each of these parties through a set of codes of conduct. the codes of conduct that the efficient leaders create, shape the future of the organization. an efficient leader is looked upon by others. employees tend to follow under the principles of leaders. hence, if the leaders wish to induce honesty among the people they meet and do business with, they will have to venture values like honesty, truthfulness with everyone they deal. creating an ethical organisation first begins from self. raichur in his book timeless principles for the workplace writes “honesty, integrity and other core values come top multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 137 down. if the leader is honest, personally demonstrates honesty, emphasizes it, empowers people to be honest and celebrates honesty, the rest of the people within the organization will follow” (raichur, 2015, p. 174). leaders are responsible for a number of operations at the workplace from taking the right decisions to choosing the right method to solve a problem from appropriate behaviour with others to act as per the circumstances. however, responsibility and accountability are two different sides of the same coin. responsibility embodies sharing of actions, workloads, decisions and the like, while accountability means being solely answerable to decisions, actions, and behaviour. effective leadership does not believe in simply winding up responsibilities but also being accountable or answerable to the consequences of the responsibilities covered. if things go upside down ethical leaders do not blame others rather find and fix answers to their problems. bustin describes accountability as “a contract, a commitment, a personal promise. practised effectively, accountability is way of thinking and acting all the time and ultimately trumps any financial, intellectual, structural, or technological ability. the reason is simple: accountability is not based on circumstances but rather on an attitude of accomplishing a task or achieving an objective despite circumstance” (bustin, 2014, p. xiv). one of the most powerful yet difficult attribute to practice today at workplaces is patience. today many leaders believe in deciding fast and not bearing time to wait for the results which at times can be devastating. however, the relevance of patience in leadership is hard to describe but it produces results far beyond pleasant. “by demonstrating patience, leaders reinforce the importance of focusing on the long-term outcomes. patience does not mean ignoring the interim milestones or short-term deliverable. it does mean keeping them in context” (kolodinsky, et al., 2012). patience in itself takes time to practice but is considered to be one of the most essential tools for leaders at workplaces. in leadership, fairness is a desirable quality. studies reveal that the decisions made by ethical leaders are based on justice and fairness. effective leaders are known for their fair evaluation of situations after studying and comprehending precise and complete information. beugre in his book managing fairness in organizations mentions about “meindl who studied the leadership style of several managers in relations to perceived fairness in 1989. results showed that high task oriented leaders preferred equity-based allocation rules. they were likely to distribute rewards on the basis of individual performance” (beugre, 1998, p.70). http://www.ifyouwilllead.com/leading-with-patience-the-will-to-wait multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 138 this paper studies the importance of ethical leadership at workplaces at the outset and then discusses the ethical dilemmas that sometimes leaders face and the best and convenient options to tackle such dilemmas. further, the paper goes to study some of the most important ethics that leaders of today ought to incorporate in their roles namely, honesty, accountability, fairness, courage, and patience with an aim to deliver performance and achieve success. 2. the significance of ethical leadership leadership is an art that is either inherited or that can be acquired through formal education, training, and coaching. in most of the education curricula in the recent past leadership has been an important part of students’ lives but today as we move rapidly towards personal development to meet the needs and demands of everyday life, education curricula in most parts of the world has incorporated ethics in their subjected and knowledge regarding ethical leadership has been imparted to teach the students of today to develop strategies to administer effective leadership qualities in parallel with ethics that will guide and shape their career and also provide them ways to handle ethical dilemma. leadership is a complex task and at workplace, today leaders find themselves in a position where the consequences of their actions can either amplify or reduce their power to make strong decisions and influence others. in many such situations, their intentions are not considered to be the driving factor instead their actions are judged over. leigh in his book ethical leadership: creating and sustaining an ethical business culture discusses “to be an ethical leader is indeed to be different. this kind of leader acknowledges the complexity of running a responsible business, yet tries to do it anyway. ethical leaders embody the purpose, vision and values of the organisation and of the constituents, within an understanding of ethical ideals. they connect the goals of the organisation with that of the internal employees and external stakeholders” (leigh, 2013, p. 6). it is because of ethical leadership that the entire business organization functions on a daily basis while at the same time empowering and setting the organisation’s vision. ethical leaders set values and standards at the workplace in building and generating an ethical organization. for example, setting workplace values like equality and fairness ensures each one of the staff members to understand how business transactions are handled and so on. also, they realize what expectations the employees are going to fulfil towards organization, liking dealing with customer, behaving righteously and so on. the purpose is multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 139 simply to define an ethical message with clarity to implement ethical means while taking any step towards organizational benefit. ethical leadership fosters a higher level of veracity that further leads towards a deeper sense of fidelity and provides confidence to other staff members to recognize, believe and follow the same. when individuals join at the workplace, with them comes the deeprooted personal values which get blended with the ethics and values of organization. this combination becomes a strong establishment for other strong personality traits that generate guidelines to think, act and decide at the workplace. doohan asserts “two attitudes are critical to leadership, personal integrity in relation to one’s vision of life, and integrity in relation to the organization’s primary values, issues and loyalties” (doohan, 2007, p.65). leaders have a crucial responsibility to convey their feelings, speech, and personality to their followers. but at first, the leaders must hold true to themselves in what they need to convey regarding the integrity or setting a vision or any other value as the followers want to keep their confidence and faith in their leaders as they believe that if leaders are honest they would live their vision. an interesting feature of leadership is to always stay beside people they and their organization is associated with. hence, leaders are mostly people-oriented and the decisions they make, or the power they use to dole out the goodness for the people concerned and provide interests both to employees and organizations. “this modelling serves as a guide and motivator for others to put the needs and interests of the group ahead of their own. such engagement creates an intellectual and emotional commitment between leaders and their followers that make both parties equally responsible in the pursuit of common goals” (parsons, 2017). 3. ethical dilemma and leadership every day leaders come across a number of ethical situations and difficulties that are not sufficiently expert of providing a win-win solution or in other words a situation that involves an option between equally insufficient substitutes. this complex situation is an ethical dilemma where at a given situation, leaders have not only to choose between right and wrong or good or bad instead between right and right and good and good. apparently, leaders end up in mental conflicts between moral alternatives and which are equally important and where choosing one would result in bypassing another. however, at workplaces, ethical dilemma can have a massive impact on the nature of work. fernando describes “a business dilemma exists when an organizational decision maker faces a choice multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 140 between two or more options that will have various impacts on the organization’s profitability and competitiveness and its stakeholders” (fernando, 2010, p. 9-1). let us understand ethical dilemma in leadership roles at workplaces and how ethical leaders find a way to find solutions through one of the real business life example. binta niambi brown who is the ceo and co-founder of fermata entertainment ltd and who by profession is also a lawyer. in a certain situation, brown was about to close her three billion dollar asset when she found that some information could have deliberately destroyed the deal. she could not reach her business partner and had two choices: “either tell her client and risk losing the deal or keep quiet until the papers were signed” (giang, 2015). finally she took up the decision to tell the client. in her own words “even if the deal had been blown up for good, honest reasons rooted in decent integrity and morality, there’s always the fear that you are going to become the associate whose deal blew up, and now everybody’s talking about how the senior person wasn’t around and you are being goody two-shoes and you ruined the deal” (giang, 2015). the inability to make proper decisions at the proper time is often the result of fears that are associated with our thoughts and the consequences are perilous. however, at times when we simply give up our fears and take a decision, we find ourselves in celebrated moments. the next section studies in details some of the most important ethical tools required in performing leadership roles. however, it must also be understood that most of the leaders perform roles giving prior importance to ethics in performing their tasks because ethics as a virtue have been embedded within them since childhood. also, there are leaders who believe that doing their tasks without ethics is simply doing injustice to their job roles. hence, such leaders have successfully learned the basics of ethics and their importance in workplaces through training, education and coaching. in both the cases, the incorporation of the principles of ethics and their relevance remain unparalleled and the results are outstanding and satisfying. 3.1 honesty honesty is considered to be one of the most positive aspects of leadership. leadership roles without honesty can put leaders into trouble if not instantly though but a while later. let’s understand the case of bernard madoff who is infamous for the largest investment scandal ever and another good number of criminal offenses like securities scam and falsehood. this caused him to be sentenced to prison for a period of 150 years. the investment scandal robbed the life’s savings of many people causing enormous downfall to many charities and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 141 foundations and destruction to pension funds. the leadership traits in madoff like being positive and confident attracted uncountable investors; however, he was not detected with the ethical principles like honesty to do his business. hence, leadership traits and abilities without leadership ethics are although purposeless yet devastating. daft claims “one aspect of being an ethical leader is being honest with followers, customers, shareholders, and the public, and maintaining one’s integrity. honesty refers to truthfulness and nondeception. it implies an openness that followers welcome” (daft, 2014, p. 40). the need to implement honesty as an ethical value in leadership is to build trust among the followers. the outrage of corruption, fraud, scandals, and greed among the top level executives in the past is gradually devaluing admiration and respect towards leaders. hence, the need of the hour at the work places today around the globe demands trust or rebuilding it to promote better work atmosphere and workplace relationships. trust is a difficult and vulnerable phenomenon to be achieved. lymn and adler describe trust as “a bond that is created between and among people. trust is an emotional and a cerebral connection, characterised by an ability to rely on someone to act in ways that will be of benefit to one’s own health and wellbeing” (lymn and adler, 2011, p. 10). followers often tend to start trusting their leaders during bad times or during times that cause stress or during situations that have negative outcomes. ethical leaders during each of the situations stand by their followers, provide solutions, bring strength and provide comfort thereby building trust and security. 3.2 fairness at the workplaces the discernment of fairness comes to employees when they find their leaders doing justice in making the right decisions. fairness in judgments and decisions is calculated when a leader does not count culture, caste, creed, sex, and age while evaluating a work performance instead only focuses on the quality of work as delivered by the employees. ethical leaders by being fair during decisions and judgments are unbiased and make their people feel satisfied and calm. on this note burchell and robin suggest leaders to “make tough calls while considering multiple variables, and in some cases those variables cannot be made known. even when the variables can be made known the decisions often need to be made quickly and cannot be explained thoroughly beforehand” (burchell and robin, 2011, p. 98). hence, whatever may be the requirement or methodologies or values, the insight regarding their leader being fair to make decisions and judgments remain convinced. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 142 fairness is one of the tough ethics to be mastered and it requires quite an open mind to accept and evaluate situations before jumping up to any decision or judgment. employees are workplaces are needed to be highly empowered and encouraged by their leaders. this is required so as to achieve goals faster and also employees remain motivated to welcome more tasks. while being fair with the employees, the leaders tend to leverage confidence among the employees and boost their self-esteem at the same time. employees feel respected and start working towards organizational goals. however, on the other hand when leaders do not play fair and implement one-man-ships or get biased, employees lose respect and trust towards their leaders which directly affects the reputation of the leader and also the organization. 3.3 accountability people have a notion in their mind that being a leader means simply to give instructions commands and make people work under them. however, only a few people understand that leadership is a big challenge and in addition to that leadership with accountability is yet another big challenge that only a few leaders willingly accept. leadership and accountability is a very rare combination. however, this rare combination has unique traits that have the potentiality to uplift the standard of any given workplace. alston describes some of the important qualities of an accountable leader. he writes “accountable leaders hold themselves accountable for their actions and decisions and are able to hold others accountable for the same” (alston, 2017, p. 23). ethical leaders with accountability do not blame others for the consequences of their choices or actions. accountable leaders take steps to be responsible not only of their choices and actions but also of their team’s. they carry out all their responsibilities and also take the conscientiousness to answer, explain or take any blame when things go wrong. they are real solvers and fixers of any problem. the importance of being an accountable leader is vital for any organization. dive asserts “accountability is vital to organisations for many reasons. lack of true accountability causes excessive cost, both economic and psychological, de-motivation in those who work for the organisation, dissatisfaction in those served by it, and sub-optimal performance in general” (dive, 2008, p. 12). in other words, accountability provides freedom to leaders to take up an action or come to a decision. if the leaders have the liability to answer to a particular consequence of the decision they have made or action they have undertaken, that clearly suggests that the decision or the action must be overall beneficial for the organization. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 143 a typical accountable leader generally uses terms of ‘i’, ‘me’, ‘my’ rather putting it to someone else like “what would be my responsibility?”, “please let me know if i can help you with anything”, “it was i who suggested on the project” and so on. accountable leaders take ownership of their duties and do it wholeheartedly to fulfil both their responsibilities and of the organization. 3.4 courage ethical leaders have to perform various roles and take severe decisions at times that require speculations and considerations. most of the decisions are based on ethical dilemmas where the leaders have to choose between right and right. in such situations leaders to select one alternative keeping in mind what consequences not selecting the other alternatives can bring. for this leaders need the courage to choose and take decisions without fear sticking to principles and ready to face the challenges and difficulties. cottrell and harvey suggest “to become a courageous leader, you must have an undeniable, indisputable and unwavering commitment to be the best you can be. the true measure of your leadership is the ability to look in the mirror and know that you had the courage to do what you felt was the right thing to do” (cottrell and harvey, 2004, p. 9). however, there is still a difference between doing the right things and doing things rightly that requires thorough research and a deep evaluation and insight of both the alternatives keeping in mind the consequences and further evaluating the consequences before arriving into a decision. often leaders get de-motivated from exhibiting courage in many situations as they have their preconceived thoughts regarding the consequences or desired results. in other words, leaders likely underestimate the significance of courage and the outcomes attached to it. aristotle, however, studied that courage is the first virtue and everything else is attached to it and is secondary. in the words of foster, nollette, and frank “for aristotle, courage resides in the realm of confidence and fear. the excess of confidence is rashness, or foolhardiness and its deficiency is cowardice; courage, therefore is the median point between these two extremes” (foster, nollette and frank, 2012, p. 106). courage is thus the calibre to comprehend one’s strengths and constraints in a given situation. there are two ways courage can be perceived by others in a workplace; courage as a virtue and courage as a vice. leaders while choosing to be a courageous need to understand the fine line of difference between both. while courage applied as a virtue brings long-term success and meets the demands of time, as a vice it can become along with a false ego that multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 144 paves ways for devastation. today workplaces demand courage as a virtue to unlock both the unseen opportunities and challenges keeping intact the values. comer and vega rightly write “leaders must consistently and proactively model exemplary behaviour by applying the personal governance practices as they build a climate that supports moral strength” (comer and vega, 2015, p.137). 3.4 patience ethical leadership is embodying various morals and principles and patience is one of them. patience is all about self-control in adverse situations. patience as a personality trait has been unrecognized and if at all recognized undervalued. the business world is travelling at a faster rate and in such a rapid pace implementing patience often goes overlooked. patience is required at every step especially while dealing with situations that are likely to cause a negative outcome to organization. for example, the workplace may demand extra working hours due to insistent issues or at times situations may cause impulsive reactions. also, at times dealing with unmanageable people can cause losing temper. however, in all the situations leaders are expected to practice patience and develop tolerance. practising patience can provide unwavering support towards the development of self and also achieve organizational excellence. leaders have to work in different fields at workplaces having different goals and yet different resources. however, a few strategies would assist in working with patience successful. leaders can start with understanding the organizational culture at first and then each one of the employees. the next strategy involves listening to viewpoints of others first and presenting self-view points after that. ethical leaders need perseverance and not giving up. ethical leaders must understand that innovative ideas always take a little more time to be nourished. not only this communicating about the same to people of different opinions and personalities and at different time periods also require lots of time. once, the ideas are imparted and in safe hands, convincing is the next strategy. although the process is an elongated one yet it breeds results. 4. conclusions leaders are the futures of any given organization. practicing ethics along with their duties will not only encourage employees to follow them but also provide satisfaction to the heads of the organization that their company is in safe hands. this paper studied the structure of leadership and the significance of ethical leadership at workplaces. ethical leadership does not merely recognize and appreciates the positive skills or personality traits but also demands the integration of ethics into roles, responsibilities, and duties for mutual benefits both of self and workplaces. ethics play wonderful roles in motivating employees and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 145 leaders at workplaces to build strategies and principles to go by it. ethical leaders must be encouraged, supported, and polished to achieve the maximum of abilities that generate the maximum of accomplishments. this paper also studied some of the basic yet requisite ethics that leadership roles must implement in their daily activities at the workplaces in order to receive excellent outcomes. it can thus be concluded that ethics such as honesty, courage, fairness, accountability, and patience are few among many workplace ethics which require practicing and exercising. 5. references alston, f. (2017). lean implementation: applications and hidden costs. boca raton: crc press, taylor & francis group. https://doi.org/10.1201/9781315154497 beugré, c. d. (1998). managing fairness in organizations. westport (connecticut): quorum books. burchell, m., & robin, j. (2011). the great workplace: how to build it, how to keep it, and why it matters. san francisco: jossey-bass/wiley. bustin, g. (2014). accountability: the key to driving a high-performance culture. new york: mcgraw-hill education. chamberlain, r. (2016). conscious leadership in the workplace: a guidebook to making a difference one person at a time. new york, ny: morgan james publishing. collins, d. (2009). essentials of business ethics: creating an organization of high integrity and superior performance. hoboken, nj: john wiley & sons. https://doi.org/10.1002/9781118386163 comer, d. r., & vega, g. (2011). moral courage in organizations: doing the right thing at work. armonk, ny: m.e. sharpe. cottrell, d., & harvey, e. l. (2004). leadership courage: leadership strategies for individual and organizational success. walk the talk. daft, r. l. (2014). leadership experience. new york: cengage learning. dive, b. (2008). the accountable leader: developing effective leadership through managerial accountability. london: kogan page. doohan, l. (2007). spiritual leadership: the quest for integrity. new york: paulist press. https://doi.org/10.1201/9781315154497 https://doi.org/10.1002/9781118386163 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11594 social and technological sciences issn: 2341-2593 tripathy (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 134-146 | 146 fernando, a. c. (2012). business ethics and corporate governance. chennai: dorling kindersley (india), licensees of pearson education in south asia. foster, d. t. (2012). emergency services leadership: a contemporary approach. burlington, ma: jones & bartlett learning. giang, v. (2015, june 03). 7 business leaders share how they solved the biggest moral dilemmas of their careers. retrieved july 15, 2018, from https://www.fastcompany.com/3046630/7-business-leaders-share-how-they-solved-thebiggest-moral-dilemmas-of-their kolodinsky, b. k., gomez, i., lewinson, s., arneson, s., allen, g., senator-john, s., . . . wahl, c. (2012, august 29). leading with patience – the will to wait. retrieved july 14, 2018, from http://www.ifyouwilllead.com/leading-with-patience-the-will-to-wait langlois, l. (2011). the anatomy of ethical leadership to lead our organizations in a conscientious and authentic manner. edmonton: athabasca university press. leigh, a. (2013). ethical leadership: creating and sustaining an ethical business culture. london: kogan page. lyman, a., & adler, h. (2011). the trustworthy leader: leveraging the power of trust to transform your organization. san francisco, ca: john wiley & sons. parsons, m. (2013, march 6). the importance of ethical leadership. retrieved july 15, 2018, from http://www.theworkplacecoach.com/the-importance-of-ethical-leadership/ raichur, a. (2015). timeless principles for the workplace. bangalore: all peoples church. https://www.fastcompany.com/3046630/7-business-leaders-share-how-they-solved-the-biggest-moral-dilemmas-of-their https://www.fastcompany.com/3046630/7-business-leaders-share-how-they-solved-the-biggest-moral-dilemmas-of-their http://www.ifyouwilllead.com/leading-with-patience-the-will-to-wait http://www.theworkplacecoach.com/the-importance-of-ethical-leadership/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 32 the influence and benefits of technology as a strategy in organizations josé g. vargas hernándeza, maría teresa alcántara salcedob auniversity center for economic and managerial sciences, university of guadalajara periférico nte.799 edif g201-7 núcleo universitario los belenes, zapopan, jal. méxico. corresponding author. email: jvargas2006@gmail.com bcentro universitario de ciencias económico administrativas, universidad de guadalajara. periférico nte.799 núcleo universitario los belenes, zapopan, jal. méxico. email: msalcedo@cucea.udg.mx received: 19 march 2019; accepted: 05 january 2020; published: april 2020 abstract the article presents a description of "the influence and benefits of technology as a strategy in organizations." technology is considered as a competitive advantage, which allows organizations to benefit, this technology means that these companies can maintain themselves and acquire a position interesting within the market, now organizations are interested in the acquisition of technology, this acquisition takes value because of the economic importance that can come from having the best technology within the organization. the document presents information about technology as a key strategy, which can help companies to create a competitive advantage within the market. keywords: technology; strategy; acquisition of technology, organizations, competitive advantage. knowledge management https://doi.org/10.4995/muse.2020.10693 mailto:jvargas2006@gmail.com multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 33 1. introduction in the article, on the one hand, it is talked mainly about the influence and benefits of technology as a strategy in organizations, and for this, it is necessary to make a small description of where organizations come from. the organizations are born from the need of people to group themselves, these groups, gather certain characteristics such as: individuals with common interests, the same objective, the need for investment together, among other characteristics. the organizations demand an order that allows to reach their objectives. this order is related to the structure, administrative processes, and the constant technological updating within the organization. on the other hand, the article presents a description of the main characteristics to consider for the acquisition and integration of new technology. one of the strategies of companies is to constantly seek ways to remain within the market, so it is necessary to perform different processes such as, the update of human resources with which the organization has and the acquisition of technology. the combination of these two it can result in the creation of a competitive advantage. as porter (2009) says, today organizations of all kinds are forced to compete to provide value, understood as the ability to meet or exceed the needs of customers effectively, any organization needs a strategy to offer superior value to its customers. it is said that only organizations that are competitive are those that survive in the market. this document is structured as follows. in the first place: there is the approach to the problem, which is what gives direction to the investigation. second, the justification is located, which is what gives reason to the investigation. third, there is the objective, the question and the hypothesis, these give meaning to the research process. fourth, structure, which allows to show information about the influence and benefits of technology as a strategy in organizations. in fifth place, is the theoretical framework, which contains the review of the theorists that support the concept of research. in sixth place is the https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 34 methodology used to obtain the information. in eighth place the results are presented, to reach the conclusions, and possible lines of future research. the document is presented as follows, in a first part the problem statement, with the justification of the study, objectives, questions and hypotheses. in a second part, the structure of the document with its theoretical framework and methodology. to finish in the third part, with the results and conclusions. 2. statement of the problem it is said that the adoption and use of technology influence a key element within organizations. this contributes to a competitive advantage strategy that allows companies to be placed in some position within the market. in order to achieve this competitive advantage, it is necessary to have a well-prepared and competent team in the use and management of the technology, as well as in the process of updating the necessary processes. to achieve the above, it is necessary to plan different strategies, that allow to create awareness and to create in the personnel a positive attitude with respect to the adoption and updating of the technology, within their daily activity. according to stern, (2002) technologies are a tool that allows different strategies to be developed within organizations, with the aim of making the organization more competitive. 3. justification of the study currently, technologies are part of the daily life of most individuals and organizations (businesses, schools, churches, among others), are seen as necessary for the case of https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 35 companies, the stay and permanence within the market. therefore, organizations must include constant training and the acquisition of technologies that allow them to be more competitive. the acquisition and use of technologies, is favorable to organizations, can reduce the possibility of being out of the market, increases the possibility of being more competitive. the adoption of the use of technology, leads to the change within the organization. it is necessary to have sufficient trained personnel in the new practices of the company with respect to the adoption of technology, with the sole objective of being more competitive within the market. the use of technology increases the possibility of being able to carry out new business, optimize processes and then increase the economic growth of the organization. hence the need to investigate the influence and benefits of technology as a strategy in organizations. 4. study objective the objective of this article is to identify, through documentary information, the influence and benefits of technology as a strategy in organizations. this can be a support for the companies, because it can allow them to establish the relationship between the influence, adoption and acquisition of technology, with respect to the benefits and economic gain obtained from the application of the use of technology. 5. research questions is there a relationship between the acquisition and use of technology with respect to the benefits obtained from this application in the organizations? https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 36 6. hypothesis h1: the acquisition and use of technology in organizations are a positive influence that is reflected in economic benefits. h2: the acquisition and implementation of technology is a competitive strategy of organizations. h3: the acquisition and implementation of technology positively influences organizations that allows them to stay within the market. 7. structure of the research work the purpose of the document is to offer information to facilitate decision-making in the acquisition of new technology, through documentary evidence about the processes to be followed. the processes of acquisition of new technology in organizations, are a key piece for these, because they are considered as the option that allows to keep organizations as competitive companies within the market. in the publication the time (2014) in an interview with brennan, vice-president at that time of the smes worldwide, affirms that the mission of the companies that provide technology of any kind, is to help smes to get more results in the business from its investments in technology. technology allows organizations to offer different products, solutions and services, among others. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 37 for belloch, c. (2012) icts are the set of technologies that allow access, production, processing and communication of information presented in different codes (text, image, sound, leavit, (1965) define ict in three categories, a. processing techniques b. application of statistical and mathematical methods for decision making c. simulation of higher order thinking through computer programs. icts are expressed in the use of computers, equipment that allows storing, retrieving, transmitting and manipulating data in the context of business. ict also includes television, telephones, hardware, software, the internet, e-commerce and computer services. the acquisition or investment of technology is generally related to different processes that are expected to positively influence companies and how it should be adopted to obtain maximum benefits. it is presumed that the integration of technologies in companies allows: a. improve administrative processes b. facilitate information management. c. reduces costs d. increase competitiveness. especially in the current environments that have become very competitive, organizations are looking for more options that allow them to obtain this advantage with respect to other companies. porter (2003) calls competitive advantage the value that a company is able to create for its customers. the organizations in order to achieve the economic advantage, are able to adopt and modify anything within the company, from the structure, administrative processes, personnel update, technological advances, in short, adopt any change according to the preferences according to their consumers. villaseca and torrent (2001), cite the oecd (1998), which https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 38 mentions that technological innovation has changed the economic structure of advanced economies from several dimensions and that in turn fosters the synergy that it generates over the rest of the economy. thasos (2018), argues that technology has been an important factor in economic expansion throughout the history of mankind. feeney, (2007), affirms that it is necessary to develop a technological leadership strategy that allows the creation of an economic and institutional regime that allows providing an efficient creation and application of knowledge to the economic process of the company. however, technology alone cannot achieve success, it is necessary to make a series of adjustments within the company, rely on all possible tools to achieve the efficient use of technologies. the organization has structure, administrative processes, goals and objectives to be achieved, which are directly linked to the constant updating of technology. a consequence of the acquisition, implementation and use of technology allows to improve the administration processes. therefore, the organizations are aware of the need to invest in the acquisition of state-ofthe-art technology that allows them to appear in the market in a good position and at the same time allows them to perfect their administrative processes with the ultimate goal of achieving greater success. from the above it is argued that it is advisable to rely on the following theories. 8. theoretical framework a. dynamic capacities: according to rivera and figueroa, referring to ahenkora and adhei (2012), dávilas (2013), ellonen et al., (2011) and argue that companies develop dynamic capacities that https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 39 allow them to generate sustainable competitive advantages. they also say that dynamic capacities are developed as a route that allows the continuous exploration of competencies according to the transformations in the environment. miranda, (2015), presents the model of dynamic capabilities in organizations, where he cites oregan and ghobandian, (2004) as those who argue that resources are defined as the set of knowledge, physical assets, human capital and others tangible and intangible factors that organizations own and control, which allow them to produce efficiently and effectively to offer their products and services to the market. and he says that these resources serve as a source of competitive advantage in the long term, when they can be evaluated, rare, difficult to imitate and not substitutable, and dynamic capabilities allow the resources of the organization to have these attributes and be superior to those of the competition (eisenhardt and martín, 2000). garzón, (2015) in his article the dynamic capacities in the organizations, he mentions that the dynamic capacities show common characteristics for several organizations, likewise he maintains that the dynamic capacities will be source of competitive advantage of the organizations. it also says that the theory of dynamic capabilities is aimed at achieving greater understanding about why some organizations build and maintain a competitive advantage, despite the changes that occur today. b. neo-institutionalist theory. according to vargas, (2015) neo-institutionalism studies the features of the economic institutional structures that enable the development of peoples. according to arias pineda (2008), he mentions that neo-institutionalism helps to study administrative phenomena https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 40 systemically and in context, establishes relationships between cultural and social factors, as well as the various forms of human association, in order to understand the process organizational. while tijerina (2008) in his article of "the neo institutionalism of duglas ..., states that organizations are conceived as groups of individuals with common identities and objectives for-profit companies, political bodies, economic bodies, social bodies, educational bodies. it talks about the transaction costs defined by the cost of acquiring information. he also mentions that they are jointly determined by incentives, the behavior of state agencies and technology, among others. esquinca (2011), states that neo-institutionalism aims to carry out actions according to reality without departing from the legal norm. these are to give organizations a certain degree of freedom without falling into legal contradictions. c. network theory salazar lópez (2016), suggests that the modeling of networks allows the resolution of multiple mathematical programming problems through the implementation of special algorithms. presents the following basic concepts in network theory. 1) graphic; represented by a series of points called nodes, joined by lines called branches. 2) network; graph with some type of flow in its branches. symbols representing the nodes and branches are used. 3) chain; series of branch elements 4) route; the nodes that constitute a chain. 5) cycle; a cycle corresponds to the chain that links a node to itself. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 41 6) tree; it is a graph in which there are no cycles. although the conception of network theory focuses on solving mathematical programming problems, the existence of different types of networks is known: 1) social. 2) transportation. 3) electric. 4) biological. 5) internet, and 6) information. the theory of social networks according to gonzález (2014), who says that the idea of social network began to be nuanced in the social sciences, since the thirties of the last century. he points out that social networks arise from the process of social interaction and cites madariaga & sierra, (2000), who defines it as the process by which a group of people are oriented towards others and act accordingly to the behaviors of one another. the importance of networks is the possibility of establishing connections to any part of the world with any person that is of our interest. at this time in the world there is a mega conglomerate of networks, which allows, facilitate and promote connectivity between different organizations, people and devices that may or may not be in different countries. the theory of networks takes importance within organizations, by the evolution in the way that organizations, organizations that have introduced technologies, which allow a better interaction between people, with greater speed and efficiency, this in turn generates relationships that allow the creation of a support network between organizations. however, the introduction of technologies has become a challenge for companies that have a rigid structure, and if they want to survive they must adapt to this new social environment, where https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 42 it is very important to know how communication mechanisms work between people, how to access information, the necessary change in decision making, transmission and dissemination of messages, ideas, promotion of collaboration, drive innovation to compete in the market. therefore, it is important to recognize the existence of types of networks, and consider the way in which information circulates in these networks. first, the type of computer networks according to apser (2015) specialized in platform design is presented: table 1: types of computer networks types of computer networks according to its use shared networks unites a large number of users. exclusive networks connect two or more points exclusively. (security, speed or absence of other types of networks). according to your property, private networks they are managed by companies, individuals or associations. (owners only) public networks belong to state agencies and are open to anyone who requests it. depending on your location and service coverage local area networks (lan): connected computers are at small distances, for example, those that connect home or office equipment metropolitan area networks (man) more extensive than the previous ones, they are formed by several lans connected to each other. wide area networks (wans) cover a large area, often even a whole country or continent. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 43 depending on the type of access wired: computers connect to the network through electromagnetic waves transmitted by air (wi-fi). wireless: computers connect to the network through electromagnetic waves transmitted by air (wi-fi). combination of the previous two: the network has both wap wireless access points and cable connections. source: own elaboration, based on the information of apser (2015). for this case, it is important to present the classification of type of business tools, presented by antenna communications (2016), and are centralized and decentralized networks. centralized: the communication revolves around a single person, who is in charge of directing the whole process and works as a link or main axis for the other decentralized members: the participants interact constantly without there being a leader or prominent member in the chain. from the above it is necessary to consider robinson's social network analysis (2014), who says that it is a methodology imported from the field of sociology and widely used in social sciences as well as in natural and exact sciences, which allows to identify underlying structures of the relationship between different actors and the situations of power and subordination that exist between them. it is based on the theory of networks that conceives the object of analysis not as independent units, but as members of a network in which relationships are established among themselves. and in this sense, the analysis allows the identification of collaboration groups, the power relations among the collaborators themselves. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 44 for this case it is important to present the classification of type of business tools, presented by antenna communications (2016), and are centralized and decentralized networks. d. knowledge management theory nonaka (1999) argues that knowledge management is a system that facilitates the search, coding, systematization and dissemination of individual and collective experience of the human talent of the organization, to turn them into knowledge, with the objective of generating competitive advantage in a dynamic environment. rodríguez rovira, (1999) affirms that information professionals can help today, the top management of companies and all types of corporations, to understand the determining role that knowledge management can play to guarantee the future of their organizations through the influence and competitive advantages. competitive advantages of knowledge management: 1) improvement of quality in products and services. 2) improvement of customer service. 3) improvement of relations with suppliers. 4) create conditions to improve the work environment. 5) better interpersonal communication. 6) it improves the information and the communication, it stimulates the participation of the workers. 7) reduction in the number of management / production processes. 8) simplification of management / production processes. 9) it allows greater efficiency in the use of resources. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 45 10) it provides better tools for managing management. knowledge management involves different components, mainly learning, human capital, intellectual, all this aims to raise and strengthen the areas of companies through increased learning, with the ultimate goal of increasing competitiveness. with this, being leaders in the market, to have new business opportunities, improvement in communication between the members of the organization, generation of a network of collaborators. however, to achieve an efficient knowledge management it is necessary to consider the possibility of having documentary evidence, for constant consultation and to be able to share it with the members of the organization. it is of no use if it is not distributed. this is where the technologies take an important role, because they provide the possibility of facilitating access to information through different tools, such as 1) creation of databases. 2) software. 3) internal platform and everything that facilitates access to this information, which allows a timely consultation for its application and administration. for organizations, knowledge management promotes communication among collaborators. something that cannot be ignored, is that globalization covers all areas, markets and technological progress, said globalization forces economies to seek cost reduction in the production of goods and services offered to the customer. as a result of the above requires companies to an increase in the use, application and implementation of technology, to be able to figure within the market. from the above it was considered necessary to consult the 2014 economic censuses of inegi, to obtain information that may reflect at least the heading of the introduction of technologies, specifically the use of computer equipment in several sectors and it was obtained that in the manufacturing area, business and services; with respect to the items of: 1) banking and financial operations. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 46 2) procedures or governmental procedures. 3) purchase-sale of services or products. 4) information search. 5) perform business management. with the following results expressed in percentages: in greater application of computer equipment in the heading of manufactures: • search for information with 91.2% • business management with 65.4% • baking and finacial operations 62.7% • the two remaining items below 50% in greater application of computer equipment in the commerce area: • search for information with 83.2% • business management with 65% • banking and financial operations 51.2% • the two remaining items below 40% in greater application of computer equipment in the field of services: • search for information with 89.9% • business management with 62.9% • the three remaining items below 42% these three items are those that make it possible to demonstrate the use of technology in activities within organizations. inegi (2014). this gives us a small idea of how it is present, the use of technology in the life of organizations. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 47 9. methodology (method) it is based on documentary information obtained mainly from studies conducted about organizations, technology, strategy, technology acquisition, competitive advantage, which are the main variables, as well as knowledge management and different theories that can allow a review and analysis, about the influence and benefits of technology as a strategy in organizations (hernández, baptista and fernández, 2010). 10. results an own elaboration scheme is presented based on the theories addressed in the document, which allows to visualize, how these theories intertwine and show interest in the central variables, technology and competitive advantage, mainly. hence, it is considered that technologies are a positive influence in organizations and that they are used as a strategy, to figure and remain valid and within the market. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 48 figure 1: technology and its relation to competitive advantage source: own elaboration. tecnology and competitive advantage dynamic capabilities companies develop dynamic capabilities that allow model of dynamic capabilities in organizations: resources the resources of the company, tangible: physical and financial non-human, technologica l and intangibles: non-human and human neoinstitutionalism groups of individuals with common objectives and for linked to the acquisition of information, through the use of knowledge management networks problem solving, stocks of different types of networks, in this case in question, social networks and collaboration evolution in the way organizations currently operate through the introduction of technology it allows to identify underlying structures of the relationship between different actors and the situation of power among collaborators dissemination of individual and collective experience of the human talent of the organization, to turn them into knowledge, with the aim of generating competitive advantage in a dynamic environment. the determining role that knowledge management can play to guarantee the future of their organizations through influence and competitive advantages technologies take an important role, because they offer the possibility of facilitating access to information through different tools https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 49 1. discussion, conclusions and future lines of research miranda, (2015) affirms that the competitive advantage is one of the most studied topics in the field of strategic management, and cites teece, pisano and shuen (1997), which explains how the combination of skills and resources can be developed by employees and oriented to achievement of competitive advantage. it is concluded that, with the documentary information obtained from different sources, it allowed to show where, as the technologies and the competitive advantage are manifested and that is evidently a strategy and influence within the organizations, as well as the benefits and economic gain obtained from the application of the use of technology. the acquisition and implementation of technology in organizations generate a positive influence, because it allows its members to develop communication skills. the introduction of technologies in organizations allows developing new strategies that increase competitiveness. in conclusion, the acquisition and implementation of technology has a positive influence on the organizations that allow them to stay within the market. 2. proposal for future studies the analysis of information about the influence and benefits of technology as a strategy in organizations will serve as a framework for conducting deeper theoretical and empirical studies on the specific relationships between economic benefits and technology. https://doi.org/10.4995/muse.2020.10693 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 50 1) identify the level of technological management in organizations and consider the characteristics of the organization in which the study will be conducted. 2) clearly identify the dimensions of technology adoption capacity. 3) apply measurements to each variable. 4) apply qualitative methodologies in the case study. in summary, the document is an important contribution that presents an extensive review of specialized bibliography. references ahenkora, k.; adjei, e. (2012). a dynamic capabilities perspective on the strategic mana gement of an industry organisation. journal of management and strategy, 3 (3), pp. 21-27. https://doi.org/10.5430/jms.v3n3p21 antena comunicaciones (2016). tipos de redes de comunicación empresarial, consultado el 02 de mayo de 2018. recuperado de http://www.antennacomunicaciones.com/tipos-redes-comunicacion-empresariales/ apser. (2015). las redes informáticas: que son, tipos y topologías, cloud computing para negocio y ceo`s, servicios gestionados. consultado el 02 de mayo de 2018, recuperado de http://www.apser.es/blog/2015/06/20/las-redes-informaticas-queson-tipos-topologias/ arias, p. a. a. (2008). el neo-institucionalismo y sus aportes a la teoría de la organización, gestión y región. universidad católica de pereira num.6. consultado el 1 de mayo 2018. recuperado de http://biblioteca.ucp.edu.co/ojs/index.php/gestionyregion/article/view/902 belloch, c. (2012) las tecnologías de la información y la comunicación (tic) en el aprendizaje, universidad de valencia. dávila l., j. c. (2013). capacidades organizacionales: dinámicas por naturaleza. cuadernos de administración, bogota, colombia, v. 26 no.47. pp. 11-33. https://doi.org/10.4995/muse.2020.10693 https://doi.org/10.5430/jms.v3n3p21 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 51 consultado el 26 de mayo. recuperado de: http://www.redalyc.org/html/205/20531182002/index.html eisenhardt, k. & martin, j. 2000. dynamic capabilities: what are they?. strategic management. strategic management journal. no.21 https://doi.org/10.1002/10970266(200010/11)21:10/11<1105::aid-smj133>3.0.co;2-e el tiempo. (2004). la importancia de la tecnología en las empresas. consultado el 30 de abril de 2018. recuperado de http://www.eltiempo.com/archivo/documento/mam1532000 ellonen, h.; jantunen, a.; kuivalainen, o. (2011). the role of dynamic capabilities in developing innovation-related capabilities. lappeenranta university of technology. https://doi.org/10.1142/s1363919611003246 esquinca, a. j. (2011). el neo-institucionalismo. gestiopolis. consultado el 26 de mayo de 2018. recuperado de: http://www.gestiopolis.com/el-neoinstitucionlismo/ farfán, b. d.y. y garzón, c. m.a. (2006). la gestión del conocimiento. documentos de investigación. no. 29, pp 107. consultado el 28 de abril de 2018. recuperado de https://www.researchgate.net/publication/27627446_la_gestion_del_conocimient o feeney, r. (2007), innovación y desarrollo económico en latinoamérica, boletín de estudios económicos, bilbao, v.62, iss. 190, pp. 61-80 garzón, c.m.a. (2015). modelo de capacidades dinámicas. revista dimensión empresarial. vol. 14, no.1, pp.111-131, consultado el 25 de mayo de 2018, recuperado de: www.scielo.org.co/pdf/v13n1/v13n1a07.pdf https://doi.org/10.15665/rde.v13i1.341 gonzález, b.n. (2014). la teoría de redes sociales y las políticas públicas. una aproximación al debate teórico y a las posibilidades de intervención en realidades sociales. forum. revista departamento de ciencia política. no. 6. consultado el 15 de abril de 2018, recuperado de file:///c:/users/may/downloads/dialnetlateoriaderedessocialesylaspoliticaspublicasunaapr-6119901.pdf hernández, s. r., baptista, l. p. y fernández, c. o, (2010) metodología de la investigación. 5a ed. méxico, mcgraw-hill inegi (2014). uso de equipo de cómputo e internet en las empesas, censos económicos. consultado el 08 de mayo de 2018, recuperado de internet.contenidos.inegi.org.mx https://doi.org/10.4995/muse.2020.10693 https://doi.org/10.1002/1097-0266(200010/11)21:10/11%3c1105::aid-smj133%3e3.0.co;2-e https://doi.org/10.1002/1097-0266(200010/11)21:10/11%3c1105::aid-smj133%3e3.0.co;2-e https://doi.org/10.1142/s1363919611003246 https://doi.org/10.15665/rde.v13i1.341 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 52 leavitt, h. j. (1965) applied organizational change in industry: structural, technological and humanistic approaches in en j. g. marzo (ed), handbook of organizaciones. pp.11441170. editores rand mcnally madariaga, c., & sierra, o. (2000). redes sociales y pobreza. psicología desde el caribe(5), 127-156. miranda, t.j. (2015). el modelo de las capacidades dinámicas en las organizaciones. investigación administrativa. no.116. pp 81-93. consultado el 28 de abril de 2018. recuperado de: http://www.redalyc.org/pdf/4560/456044959005.pdf nonaka, ikujiro y takeuchi, hirotaka (1999). la organización creadora del conocimiento, editorial oxford press, méxico d.f. o regan, n. & ghobadian, a. (2004). the importance of capabilities for strategic direction and performance. management decision. 42: 292312. https://doi.org/10.1108/00251740410518525 ocde (1998). relaciones con los medios de comunicación. consultado el 28 de abril de 2018. recuperado de: http://www.oecd.org/centrodemexico/46440894.pdf porter, m. e. (2003). ventaja competitiva: creación y sostenimiento de un desempeño superior méxico: continental, 2003. pp. 165-201. porter, m. e. (2009) ser competitivo, ed. actualizada y aumentada, edit. deusto s. a. ediciones. 624 p. rivera, r. h.a., figueroa, g. l. s. (2003). capacidades dinámicas, una fuente de ventaja competitiva, criterio libre journal. v. 11, no. 19., bogotá colombia. pp. 245-261. https://doi.org/10.18041/1900-0642/criteriolibre.2013v11n19.1110 robinson, g. n. (2014). el análisis de redes sociales. definición y aplicación, consultado el 2 de mayo de 2018. recuperado de https://nrobinsongarcia.wordpress.com/2014/01/13/el-analisis-de-redes-socialesdefinicion-y-aplicaciones/ rodríguez r. j.m. (1999). la gestión del conocimiento: una gran oportunidad. el profesional de la información. consultado el 10 de mayo de 2018. recuperado de http://www.elprofesionaldelainformacion.com/contenidos/1999/marzo/la_gestion_ del_conocimiento_una_gran_oportunidad.html salazar, l.b. (2016). teoría de redes. ingeneriaindustiralonline.com. consultado el 18 de abril de 2018, recuperado de https://www.ingenieriaindustrialonline.com/herramientas-para-el-ingenieroindustrial/investigaci%c3%b3n-de-operaciones/teor%c3%ada-de-redes/ https://doi.org/10.4995/muse.2020.10693 https://doi.org/10.1108/00251740410518525 https://doi.org/10.18041/1900-0642/criteriolibre.2013v11n19.1110 http://www.elprofesionaldelainformacion.com/contenidos/1999/marzo/la_gestion_del_conocimiento_una_gran_oportunidad.html http://www.elprofesionaldelainformacion.com/contenidos/1999/marzo/la_gestion_del_conocimiento_una_gran_oportunidad.html multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10693 social and technological sciences issn: 2341-2593 vargas and alcántara (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 32-53 | 53 stern, c. (2002). a strategy for development. the world, washington, d.c. https://doi.org/10.1596/0-8213-4980-5 tasos, v. (2018). las ventajas y desventajas de la tecnología en nuestra economía, consultado el 17 de mayo de 2018, recuperado de: http: //www.cuidatudinero.com teece, d., pisano, g. & shuen, a. (1997). dynamic capabilities and strategic management. strategic management journal. 18: 509-533. https://doi.org/10.1002/(sici)1097-0266(199708)18:7<509::aidsmj882>3.0.co;2-z tijerina, g. e. (2008). el neoinstitucionalismo de douglass c. north: una exposición crítica. universidad autónoma metropolitana. unidad iztapalapa consultado el 10 de abril de 2018. recuperado de http://www.laisumedu.org/desin/fronteras/tijerina.pdf vargas, h. j. g. (2015). teoría institucional y neo-institucional en la administración internacional de las organizaciones. sistema de información científica, red de resvistas científicas de américa latina y el caribe, españa y portugal, http://www.uaem, redalyc.org. villaseca, r. j. and torrent, s. j. (2001). tecnología y economía: una aproximación a la interpretación económica del conocimiento, consultado el 28 de abril de 2018. https://doi.org/10.4995/muse.2020.10693 https://doi.org/10.1596/0-8213-4980-5 https://doi.org/10.1002/(sici)1097-0266(199708)18:7%3c509::aid-smj882%3e3.0.co;2-z https://doi.org/10.1002/(sici)1097-0266(199708)18:7%3c509::aid-smj882%3e3.0.co;2-z multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 42 social distance through music in efl students maría gemma sánchez gonzález university of extremadura, muñoz chaves, 3 cáceres, 10003, spain. sanchez_gemma@hotmail.com received: 26 january 2021; accepted: 19 august 2021; published: october 2021 abstract music is constantly around us and, if in english, it fosters social distance in efl students, consequently increasing the level of affinity with the language, however, can social distance help with other personal aspects? music lyrics provide different perspectives that can become enriching on an academic level, as music brings cultural aspects of the country of origin and directly affects emotions, therefore song lyrics may become a very powerful instrument when combining social distance and the level of english of students in order to improve academic and future professional performance, among others. the aim of this study was to measure the impact of social distance on the improvement of academic performance, emotional states and future prospects in efl students. in order to measure those variables, a questionnaire, based on the likert scale, was conducted amongst 82 students from different educational centres (spain). resulting data analysed with the use of the structural equation modeling (sem-pls) revealed the relevance of social distance through music in order to improve learning processes, to feel emotions and to consolidate hopes for future prospects. key words: efl students; emotions; future prospects; music lyrics; social distance to cite this article): sánchez gonzález, m.g. (2021). social distance through music in efl students. multidisciplinary journal for education, social and technological sciences, 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014. https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0003-1254-9933 https://doi.org/10.4995/muse.2021.15014 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 43 1. introduction music is an academic resource that seems to meet all purposes and tick all boxes, since it is one of the most important resources of intellectual, cultural, artistic as well as technological and economical strength nowadays. the fact that young individuals devote a considerable amount of time to listening to music, an activity always on the increase, is worth mentioning. apart from that, music reflects emotions, and emotions are aroused by different types of music (sloboda & juslin, 2001), and if matched with lyrics, emotions increase, directly affecting human nature, for instance, pulse rate, facial expression, movement or skin conductance (ménard et al., 2015). a suitable and most convenient scenario is therefore provided by music to engage people in different activities, frames of mind, ethical values (nicolás & azorín, 2013), which can benefit relationships of all kinds. music has the ability to engage and must not be neglected as an academic tool to assess cultural aspects such as values portrayed by music in today´s world (sánchez gonzález, 2020). on the other hand, it needs to be mentioned that the limbic system (known as the emotional brain), which is a part of the brain in charge of emotions and moods resulting in various connections, directly affects processes such as memory and learning skills. the acquisition of a language involves psychological representations of the world around us, but also the combination of emotional factors, either in a conscious or non-conscious manner. learners of a language create a relationship with that specific language because it is a representation of themselves and the social group they belong to. the language has the ability to transmit sociocultural, existential and emotional identity, which is a motivational reason to learn it (gardner & lambert, 1972). the comprehension of another language provides an easier interpretation of social situations in different settings and the increased ability in digital literacy comes as a plus to be updated with technologies. all in all, linguistic meanings and emotions are the foundations of attitude change. it can be stated that language is a cognitive ability that is unique to human beings, as it involves sensory and motor systems as well as memory and attention mechanisms supporting social cognition (fedorenko, 2014). that social cognition involves how we perceive people and the surrounding world, which then can be influenced by social distance in regards to foreign languages. https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 44 according to alcaraz varó (2000, p. 14), efl "is the first choice in primary and secondary education curriculums of nearly every country". having a command of another languages drives students to become independent in both ideas and actions with a more critical outlook, as the language provides an approach of diverse cultural aspects and social standards, resulting in different ways of thinking. in addition, once the compulsory secondary education stage has been completed by students, the option to enroll in vocational training education is available. this educational branch offers some courses where english as a foreign language is compulsory, which would help, among other subjects, to prepare students to enter the world of work (niittylahti et al., 2019), which is linked to vocational areas such as business, economics and/or commerce (fuller, 2015) in order to give them a change to complete their practical education and facilitate their transition from being a student to being an independent professional with a critical perspective. bearing in mind that globalisation has affected all corners of life and the exchange of information through english is an undeniable fact brings the concept of social distance. as claimed by schumann (1994), social distance as one of a range of social factors influencing language learning. migration or intellectual connections are different characteristics that cause social distance in different groups. now, can music foster social distance? competence in a foreign language is required to be able to understand the lyrics of songs in order to detect structure and meaning of words. according to abbott (2002), feelings or ideas can be portrayed in songs, as they utter cultural aspects, which can then be used in listening activities in the classroom for the enjoyment and excitement of learners of a second language or foreign language, who moved by social distance, and wishing to be part of what they listen to, can improve their level of the language among other aspects. when music lyrics in a foreign language are listened to carefully and are felt as own, social distance becomes key to understand and embrace the culture portrayed in that piece of music. this fact could help vocational training students in their daily working performance as well as in their future prospects as music can be a multifunctional tool not only to relax or motivate while working but also to learn. https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 45 2. literature review encouraged by the research question: can social distance through music, assisted by emotions, improve academic and professional prospects in efl students? 2.1 advantages of learning english as a second language and music in english (me) it is broadly accepted that english is the common language used nowadays in every branch of knowledge. it has become an asset needed for success. the reasons and scopes to use it are endless, from academic to professional fields. flowerdew and peacock (2001, p. 10) state that "the international language of research and academic publication is english" in the sense that this language is necessary to access research fields (alcaraz varó, 2000; flowerdew & peacock, 2001). english has become an international language to exchange all kinds of information (gonzález davies & celaya villanueva, 1992; alcaraz varó, 2000; flowerdew & peacock, 2001). as graddol (1997, p. 50) states, the english language plays a very important role as a lingua franca, which is now backed up by the use of the internet and rapid technological changes. on their part, richards et al. (1992, p. 187) refer to english as a second language, as a language for international communication. all around the globe english is being taught and learnt at different academic levels (gonzález davies & celaya villanueva, 1992; kachru and nelson, 2001). as postulated by north & hargreaves (1997, p.1) "music has many different functions in human life, nearly all of which are essentially social", therefore any experience related to music can have an impact on personality and due behaviour. music can be and assisting tool (tarrant et al., 2000) for pronunciation skills and other acquisitions aspects where emotions are involved, including the activation of memory and recall (fonseca-mora, 2000). it is appropriate to mention the use of music to promote motivation and engagement with students, which will facilitate the learning of a different language. it is one of the main recreational activities for young people, which influences their identities and preferences in life. music affects health and well-being as it can change behaviours in beneficial ways (macdonald et al., 2012). when immersed in the language, the social distance factor has an impact on them as they need to adapt to another culture and new contexts to be able to understand and embrace them (sam & berry, 2010). having command of another language becomes an asset not only to individuals but https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.tandfonline.com/doi/full/10.1080/02673843.2011.650182?src=recsys multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 46 also to society in general, as it enables the person to have different perspectives in life, better understanding and approaches of the surrounding world and more specifically, of other specific cultures. in the vocational training classroom, when practicality should reign, the use of music no doubt would offer a scenario for better performance and enhancement of the social distance factor. the above statements lead to the following hypotheses: h1: music in english (me) positively influences social distance (sd) h4: music in english (me) positively influences the improvement of academic and professional prospects of young people (iappyp) 2.2 social distance (sd) in learning processes as claimed by sam & berry (2010), the acculturation process involves modification and sharing of social norms and patters of another group of reference, therefore, social distance can have an relevant impact on learning processes related to foreign languages. these processes are related to emotions created in the limbic system and those emotions may easily come from listening to music, emerging from cognitive and affective approaches. according to the social distance statement by schumann (1994), should attitudes be considered, the feelings and emotions of reference groups regarding the language in question can be improved by music in that language and directly have a relevant positive impact on learning skills. the size and relevant of the english language as a lingua franca is a reason to perceive the social distance factor in learning. music is connected to affective and unconscious factors (fonseca-mora, 2000) and seem to settle in our memories. having command of another language becomes an asset not only to individuals but also to society in general, as it enables the person to have different perspectives in life, better understanding and approaches of the surrounding world and more specifically, of other specific cultures. on their part, do and schallert, (2004) affirmed that linguistic meanings are associated with emotions and emotional processes, influenced by stimuli, and can have a impact on perceptions of those who are learning. as pointed out by mclean (1980, p. 17) “learning is most effective when https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 47 the learner is the initiator of the learning process”. this process can be facilitated by the fact that music is an enjoyable activity commonly used for learning processes. in the case of english in efl students in spain, it must be pointed out that, to some extent, every student has some knowledge of the language and when emotions are involved in the listening of a song, translating comes into play. translating emotions (izard, 2002) is an unprecedented fact in lyrics of most preferred songs. in spain, a country highly influenced by music in english, young people are updated with music trends and enjoy listening to music in their devices everywhere they go, feeling connected with their favourite artists. this information leads to the following hypotheses: h2: social distance (sd) positively influences emotions (e). h5: social distance (sd) positively influences the improvement of academic and professional prospects of young people (iappyp) 2.3 emotions (e) as a key factor in learning music is well-known for being a form of expression and a way to produce feelings and combine emotions through the words used in them, with a subsequent impact on people´s perceptions of different aspects of life. diverse types of music create different emotions (sloboda & juslin, 2001) and, if accompanied by lyrics, which represent an array of realities we may be familiar with, feelings and emotions do increase leading to learning processes, which, influenced by the limbic system, may modify beliefs and attitudes in people (aragão, 2011; do & schallert, 2004). the role of emotions is paramount in these processes as they are involved in neurological changes in the brain. affective factors need to be considered regarding language learning (gardner & lambert, 1972) where the control of emotions becomes paramount. besides, memories that occur during a particular emotional state will be recalled easier should a similar emotion state take place. the limbic system is involved in classifying experiences that are stored in the brain either consciously or unconsciously. going deeper into the topic, krashen (1982) proposed the so-called "affective filter” hypothesis stating that the level of anxiety of a learner is key for language input, therefore when https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 48 anxiety is low, the accessibility to language input increases and language acquisition then takes place. the role played by emotions in learning processes actually depends on several cognitive and motivational mechanisms; motivation is one of them (pekrun et al., 2002). listening to music can be a calming as well as an exciting activity for academic purposes leading to language acquisition. this "affective filter" being influenced by motivation and selfconfidence can boost language input and acquisition. schumann (1994) postulates that "the brain is the seat of cognition, that cognitive processes are neutral processes, and that, in the brain, affect and cognition are distinguishable but inseparable therefore providing a concept of affective filter related to the brain´s stimulus regarding the appraisal system involved with cognition in order to promote or inhibit second-language acquisition". music becomes a very valuable tool, academically speaking, to create a positive environment in the classroom for different purposes: mood enhancement, coping with distress, identity construction and social identity formation. music’s consoling effects were reported as resulting particularly from the sound and texture of the music itself, from attribution of personal meaning to music’s lyrics, and, to a lesser extent, from perceptions of closeness to artists and other listeners. (bogt et al., 2011, p. 147). the understanding and assignation of emotions to a foreign language becomes paramount to develop attitudes. feelings and emotions then become of special interest in learning processes related to the english language in order to be able to understand song lyrics. the combination of variables such as motivation and emotions may have different meanings for each individual as well as different outcomes when motivation comes into play (do & schallert, 2004). according to punset (2012, p. 110), "emotions are the result of how we experience, physically and mentally, the interaction between our inner world and the outer world. for a human, emotions are expressed through behaviors, expressions of feeling, and physiological changes. although the basic emotions are universal, the emotional experiences, or feelings, are more personal in as much as they infect each person's mood, temperament, personality, disposition, and motivation". a field of imagination and feelings are provided by music thanks to the emotional factor, which materialises in lyrics. it is somehow a force that unifies music listeners and that is unique https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 49 (levitin, 2006). emotional experiences are key in language learning processes (shoaib & dörnyei, 2004), since emotions of any nature may have a impact on the motivational behaviour of language learners (pekrun et al., 2002). emotional states are key in learning, since learning is one of the major sources of human emotions nowadays (pekrun et al., 2002) leading to both academic and professional success. emotions are key in human nature leading to different present and future behaviour and actions. if emotions are raised by music while working or studying, performance may be bettered. these statements lead to the following hypothesis: h3: emotions (e) positively influence the improvement of academic and professional prospects of young people (iappyp) 3. methodology with the purpose to provide a context, meet the aim of the study (measure the impact social distance through music in english on students), and to be able to both predict and comprehend the phenomenon of the study (creswell, 1994), the quantitative method by means of a questionnaire was considered the best option for this type of study. the smartpls programme (ringle et al., 2015) was employed with the sem-pls (structural equation model-partial least squares) multivariate technique in order to study the relationships among the proposed variables (either unobservable or latent). sem models are frequently used in social sciences branches and are highly recommended to test hypotheses. constructs are variables that were extracted from the literature review in order to draft the items of the final questionnaire. primary and secondary sources of data collection were employed and implemented, as recommended by cooper and schindler (2006) with the aim to answer the previously posed research question. the first stage of this study focused on searching for related literature on music and impact on young people, in particular on how efl students. then a connection with social distance with the motions raised by music was sought in order to prove the benefits of music in english for future academic and professional prospects. 3.1 justification of variables and items https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 50 the design of the constructs aimed at defining the items for the questionnaire; previous literature concepts by different authors were considered. table 1. justification of variables constructs and items indicators me: music in english 1. i translate and try to understand songs in english me1: relaxed attitude to understand another language if interested in the song lyrics. (krashen,1982) 2. understanding english makes me enjoy songs in that language more me2: interpretation of linguistic input of a known language. (fedorenko, 2014) 3. musical activities in english helps improve my level of english me3: musical activities to increase command of a foreign language. (fonseca-mora, 2000; fonseca-mora et al., 2011) sd: social distance 1. speaking in english makes me feel part of the culture of that language sd1: social distance linking and motivating l2 (sam & berry, 2010; schumann, 1994) 2. english is an asset to have for academic and professional projects sd2: english as a lingua franca. (graddol, 1997; alcaraz varó, 2000) 3. i believe the english language is essential for both personal and affective needs sd5: english as a cohesion factor. (sam & berry, 2010) e: emotions 1. i believe music can encourage me to be more productive and affective in my relations e1: motivation to encourage learning and efficiency. (gardner & lambert, 1972) 2. i believe music is the perfect channel to express my feelings and emotions e2: emotions and feelings through music. (sloboda & juslin, 2001) iappyp: improvement of academic and professional prospects of young people 1. music engages people from all corners of life iappyp1: music, an engagement instrument. (nicolás & azorín, 2013) 2. i believe music raises awareness of different realities in the world iappyp2: music to assist different purposes (tarrant et al., 2000) considering the information above, five hypotheses can be drawn, whose direction is detailed in the model (see figure 1). h1: music in english (me) positively influences social distance (sd) h2: social distance (sd) positively influences emotions (e) h3: emotions (e) positively influence the improvement of academic and professional prospects of young people (iappyp) https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 51 h4: music in english (me) positively influences the improvement of academic and professional prospects of young people (iappyp) h5: social distance (sd) positively influences the improvement of academic and professional prospects of young people (iappyp) the arrows in figure 1 represent the paths of the dependent variables leading to the independent variable and the strong connection among them. figure 1. model designed. source: own https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 52 3.2 instrument a likert scale-based questionnaire, with 10 items related to music in english, social distance and emotions, previously drafted by means of google forms, was sent to three different groups of vocational training students, by email, with a direct link to the questionnaire. the first draft of the questionnaire was previously tested with 5 random students by means of whatsapp groups previously arranged by the researcher using a mobile phone. 3.3 population a total of 82 (n=82) participants specialising in oral hygiene, dental prosthetics, and telecommunications from two vocational training centres in extremadura, and ages ranging from 19-36, were randomly selected. first a brief introduction of the topic was given to them to provide some basic guidelines related to confidentiality, time frames and honesty in their responses. they were encouraged to take part in the study by offering research credit. the participants, who claimed to have a lower intermediate level of english, duly responded to the questionnaire. 4. results table 2 shows the evaluation of the measurement model (henseler et al., 2015), which exhibits the construct reliability performed to ensure internal consistency of the indicators of the study regarding the concept they represent. on his part, chin (1998) suggest that the model is considered acceptable if the loads are greater than 0.5.this is measured by means of the cronbach’s alpha, whose measure criteria should be around 0.70, also by the rho a2, by the composite reliability (cr), with also a measure range of 0.70, and the convergent validity of the latent variables, assessed by the average variance extracted (ave); this means that a group of items is actually representing a construct. convergent validity refers to the level at which two measures of the same concept do correlate. table 2 displays how all latent variables are about 0.50, a condition recommended in https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 53 order to justify half of the variance of its indicators (henseler et al., 2009). this condition is fulfilled by all values in this particular case. the measure proposed by fornell and larcker (1981) shows the variance obtained by a construct from its indicator in regards to the variance, due to the measurement error. the values of the composite reliability through the extracted mean variance should stay above the recommended limits of 0.7 and 0.5 respectively, which is actually verified in the study. here, the four constructs show an ave with values ranging from 0.598 to 0.620, which actually confirms the suitability of the convergent validity used in the model. as for the internal consistency reliability, the composite reliability of each construct, which in this case exceeds the value of 0.7, reveals a measurement model with adequate internal consistency reliability. the confidence intervals here, 2.5% and 97.5% respectively, showed strong relationships in variance in regards to the constructs mentioned in the study. table 2. reliability and validity variable cronbach alpha rho_a cr ave me 0.821 0.821 0.820 0.603 iappyp 0.859 0.862 0.858 0.603 e 0.833 0.832 0.830 0.620 sd 0.812 0.819 0.817 0.598 5. discussion up to the present moment, not much research has been found on how social distance in second language users ́perception of the meaning of song lyrics can have a positive impact on their future professional lives. according to abbott (2002), songs utter cultural aspects, among which feelings or ideas can be mentioned and these can be properly received through listening activities for the enjoyment of learners of a foreign language. after all, music is a multi-purpose tool for learners, as it comes with cultural baggage and new trends that can improve the understanding and perception of those. https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 54 having in mind that the aim of the study was to measure the impact of social distance on the improvement of academic performance, emotional states and future prospects in efl students, and that the path modeling validated the hypotheses (within 2.5% and 97.5% confidence intervals), results then confirmed the robust relationships in variance among the constructs that have been previously designed. vocational studies are known for having a more practical approach for future work perspectives. in the present study, the variable of the level of english in vocational training students was considered due to the fact that english is compulsory in some of its courses, which have a relevant load of practical activities and also the fact that music in english is continuously being played, including work settings, making english as a foreign language as well as an international language relevant in their lives. the model introduced showed how social distance through english music may have a relevant impact on the understanding of the language and cultural aspects by means of emotions raised with music lyrics, and consequently improve future prospects in their professional and personal lives. this study has combined different variables that are present in most societies to show how useful music can be in order to change perceptions, increase motivation and hopes for professional prospects of the participants. in general terms, results confirmed the relationships of the variables previously posed to demonstrate a correlation between music in english, social distance and emotions resulting in better understanding and future expectations, as english is always a useful resort in all fields of knowledge. participants would pay attention to lyrics in a foreign language and try to translate them if they liked the song; that would make them enjoy them with more intensity and improve their command of the language. music would be seen as encouraging for personal and professional prospects and as a channel for emotions as well as to understand the culture of other realities. t he combination of emotions through words in song lyrics has a subsequent impact on emotions, productivity and engagement with other realities, therefore the appropriateness of the variables and hypotheses can predict the suitability of the model proposed. the external variables music in https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 55 english (me) social distance (sd) and emotions (e) then converge into the internal variable improvement of academic and professional prospects of young people (iaeppyp). all in all, being the research question: can social distance through music, assisted by emotions, improve academic and professional prospects in efl students? the analysis of the variables and results provided a positive answer. 6. conclusions nowadays, society is immersed in a musical globalisation process, however not much research has been conducted on the field of music in english as extra reinforcement for other academic fields such as the world of emotions or to improve future prospects in young people studying english as a foreign language. an understandable and most suitable connection between social distance regarding music in english has been proven in this study and their impact on the perceptions of efl students about personal paths. the social distance phenomenon, as stated by schumann (1994), is a factor with direct impact on language learning processes. this study confirms that. results have shown that music in english can play a relevant role in the education of students and that awareness of that fact can actually provide a different outlook for the teaching community in terms of using an enjoyable activity to complement and reinforce important concepts. considering all of the statements above and the results of the study, four main conclusions can be drawn: first, highly similar responses enabled to quantify social distance in the participant´s perceptions of english as a useful resort for their future prospects, considering this language essential in their lives. data obtained predicts attitude change. second, social distance (sd) can be considered as a determining factor in the world of music in a foreign language, as a language comes with culture and social beliefs and, as proven by the https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 56 study. the combination of emotions through words in song lyrics has a direct impact on perceptions such as english as a valuable asset for future prospects. third, the model has been presented to propose music in english as a tool to use when teaching young people, as it opens the door to new pedagogy and the world of emotions in a different language and, as proven by results, students are open and encouraged to learn through music. fourth, music in english (me) leads to social distance (sd), which involves emotions (e) resulting in an improvement of academic and professional prospects of young people (iappyp) as provided by the very similar responses given by students. from results obtained, some theoretical as well as practical implications can be drawn. on the one hand, this study backs up previous research conducted on the field of music in the english classroom and benefits thereof. the methodological utility can not be denied, as it becomes an assisting tool for teachers of all fields as music is always a tool that entertains and helps retain aspects of knowledge. besides, emotions and values from those emotions connected with music are worth a mention as they are socially relevant showing feelings and prediction of a particular generation of students. awareness is one of the practical implications provided by this study, as it allows the academic community to make use of musical resources in english, a language known by most students in spain, considering the cultural baggage of music to explore other fields of knowledge and improve teaching methods. the findings of this study will be an important source of information for academic purposes and also be the basis for future lyrics research. regarding limitations, the sample size (82 subjects), the location (mainly extremadura-spain) and the cultural bias of the researcher (an english teacher in spain) needs to be considered as limiting conditions that could have slightly affected results, however this fact can set the base for similar scenarios involving efl students. as a recommendation, further research on how songs in english may affect efl learners in their ethical and social standards is advised as it could broaden literature of foreign languages and https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 57 impact on l2 students and give light to yet an undiscovered inner world of emotions and music in english as a tool for learning and improvement of professional prospects. funding: this research received no external funding. conflicts of interest: the author declares no conflict of interest. references abbot, m. (2002). using music to promote l2 learning among adult learners. tesol journal, 11(1). alcaraz varó, e. (2000). el inglés profesional y académico. alianza editorial. aragão, r. (2011). beliefs and emotions in foreign language learning. fuel and energy abstracts,39(3), 302-313. https://doi.org/10.1016/j.system.2011.07.003 bogt, t., mulder, j., raaijmakers, q.a.w., & gabhainn, s.n. (2011). moved by music: a typology of music listeners. psychology of music, 39(2), pp. 147-163. https://doi.org/10.1177/0305735610370223 chin, w.w. (1998). the partial least squares approach to structural equation modeling, in modern methods for business research, marcoulides ga [ed]. lawrence erlbaum associates: mahwah n. j., 295-336. cooper, d.r., & schindler, p.s. (2006). business research methods (9th edition). mcgraw-hill. creswell, j.w., & clark, v.l.p. (2017). designing and conducting mixed methods research. sage publications. do, s.l., & schallert, d.l. (2004). emotions and classroom talk: toward a model of the role of affect in students’ experiences of classroom ciscussions. journal of educational psychology, 96(4), pp. 619-634. https://doi.org/10.1037/0022-0663.96.4.619 fedorenko, e. (2014). the role of domain-general cognitive control in language comprehension. frontiers in psychology, 5, p. 335. https://doi.org/10.3389/fpsyg.2014.00335 flowerdew, j., & peacock, m. (2001). research perspectives on english for academic purposes. cambridge university press. http://dx.doi.org/10.1017/cbo9781139524766 fonseca-mora, m.c. (2000). foreign language acquisition and melody singing. elt journal, 54(2), 146 152. fornell, c., & larcker, d.f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. https://doi.org/10.2307/3151312 https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/j.system.2011.07.003 https://doi.org/10.1177/0305735610370223 https://doi.org/10.1037/0022-0663.96.4.619 https://doi.org/10.3389/fpsyg.2014.00335 http://dx.doi.org/10.1017/cbo9781139524766 https://doi.org/10.2307/3151312 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 58 fuller, a. (2015). vocational education. international encyclopedia of the social & behavioral sciences (2nd ed), 232-238. elsevier. https://doi.org/10.1016/b978-0-08-097086-8.92091-9 gardner, r.c., & lambert, w.e. (1972). attitudes and motivation in second language learning. newbury house publishers. gonzález davies, m., & celaya villanueva, m.l. (1992). new teachers in a new education system: a guide book for the reforma. ppu. graddol, d. (1997). the future of english? british council. henseler, j., ringle, c.m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115-135. https://doi.org/10.1007/s11747-014-0403-8 izard, c.e. (2002). translating emotion theory and research into preventive interventions. psychological bulletin, 128(5), 796-824. https://doi.org/10.1037/0033-2909.128.5.796 kachru, b.b., & nelson, c.l. (2001). world englishes. in a. burns & c. coffin (eds.). analysing english in a global context, 9-25. routledge. krashen, s.d. (2003). explorations in language acquisition and use. heinemann. levitin, d.j. (2006). this is your brain on music: the science of a human obsession. dutton/penguin books. macdonald, r., kreutz, g., & mitchell, l. (2012). what is music, health, and wellbeing and why is it important? music, health, and wellbeing. oxford scholarship online. https://doi.org/10.1093/acprof:oso/9780199586974.003.0001 mclean, a.c. (1980). destroying the teacher: the need for a learner-centred teaching. e.t.forum, 28:3, 16-19. ménard, m., richard, p., hamdi, h., daucé, b., & yamaguchi, t. (2015). emotion recognition based on heart rate and skin conductance. in proceedings of the 2nd international conference on physiological computing systems, vol. 1: phycs, 26-32. https://doi.org/10.5220/0005241100260032 nicolás, g., & azorín, c. (2013). música y valores. una relación educativa ineludible. música y educación: revista trimestral de pedagogía musical, issn 0214-4786, año nº 26, nº 93, 16-25. niittylahti, s., annala, j., & mäkinen, m. (2019). student engagement at the beginning of vocational studies. nordic journal of vocational education and training, vol.9, n.1, 21-42. https://doi.org/10.3384/njvet.2242-458x.199121 north, a.c., & hargreaves, d.j. (1997). music and consumer behaviour. in d.j. hargreaves & a.c. north (eds.), the social psychology of music, 268-289. oxford university press. pekrun, r., & perry, r.p. (2014). control-value theory of achievement emotions. in r. pekrun & l. linnenbrink-garcia (eds.), international handbook of emotions in education, 120-141. routledge/taylor & francis group. https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1016/b978-0-08-097086-8.92091-9 https://doi.org/10.1007/s11747-014-0403-8 https://doi.org/10.1037/0033-2909.128.5.796 https://doi.org/10.1093/acprof:oso/9780199586974.003.0001 https://doi.org/10.5220/0005241100260032 https://dialnet.unirioja.es/servlet/revista?codigo=975 https://dialnet.unirioja.es/servlet/revista?codigo=975 https://dialnet.unirioja.es/ejemplar/323146 https://doi.org/10.3384/njvet.2242-458x.199121 http://refhub.elsevier.com/s0346-251x(19)30115-0/sref95 http://refhub.elsevier.com/s0346-251x(19)30115-0/sref95 http://refhub.elsevier.com/s0346-251x(19)30115-0/sref95 http://refhub.elsevier.com/s0346-251x(19)30115-0/sref95 http://refhub.elsevier.com/s0346-251x(19)30115-0/sref95 http://refhub.elsevier.com/s0346-251x(19)30115-0/sref95 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 sánchez gonzález (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 42-59. https://doi.org/10.4995/muse.2021.15014 59 punset, e. (2012). una mochila para el universo. ediciones destino. richards, j.c., platt, j., & platt, h. (1992). dictionary of language teaching and applied linguistics. longman. ringle, c.m., wende, s., & becker, j.m. (2015). smartpls 3. smartpls gmbh, boenningstedt. http://www.smartpls.com sam, d.l, & berry, j.w. (2010). acculturation: when individuals and groups of different cultural backgrounds meet. perspectives on psychological science, 5(4), 472-481. https://doi.org/10.1177/1745691610373075 sánchez gonzález, m.g. (2020). adolescents and values portrayed in music nowadays. multidisciplinary journal for education social and technological sciences, v. 7, 2, 45-60. https://doi.org/10.4995/muse.2020.14001 schumann, j. (1994). where is cognition? emotion and cognition in second language acquisition. in studies in second language acquisition, vol. 16, issue 2, 231-242. cambridge university press. shoaib, a., & dörnyei, z. (2004). affect in lifelong learning: exploring l2 motivation as a dynamic process. in p. benson, & d. nunan (eds.), learners’ stories, 23-41. cambridge university press. sloboda, j.a., & juslin, p.n. (2001). handbook of music and emotion: theory, research, applications, 605642. oxford university press. tarrant, m, north a.c., & hargreaves, d.j. (2000). english and american adolescents’ reasons for listening to music. psychology of music, 28(2), 166–173. https://doi.org/10.1177/0305735600282005 https://doi.org/10.4995/muse.2021.15014 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1177/1745691610373075 https://doi.org/10.4995/muse.2020.14001 https://doi.org/10.1177/0305735600282005 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 1 dimensions of critical thinking in workplace management & personal development: a conceptual analysis mitashree tripathy sri sri university, cutack, india * correspondence: mitashreetripathy84@gmail.com received: 1 january 2020; accepted: 04 september; published: october 2020 abstract globalization has caused a massive impact on the speed and nature of the business all around the world. the workplace setups have grown extremely complicated and job roles have become complex. employees suffer a hard time at the workplace as communication patterns have become increasingly intricate. decisions made by others are no more relied on and problems have turned more crucial than ever. judgments are expected to be drawn faster and communication crystal clear. in such a rapidly moving business world, employees are provided with only limited space and opportunities for personal growth and development. in short, the exit gate remains open for the one who does not prove to be befitting each of the complexities in the business world. at this expository point, employees are required to view different problems differently and create variable approaches towards finding solutions to the problems. the need for such a powerful skill set is required not only to create a powerful vision at the workplace but also to develop other sub-skills set in oneself. this paper studies developing critical thinking skills to make careful judgments, make proper decisions, solve complex problems and wipe out the basic inability to communicate appropriately especially at the workplace. this paper provides ways to foster better the workplace management and personal development especially for the employees at workplace through critical thinking. keywords: critical thinking, problem solving, judgement, decision making, communication https://creativecommons.org/licenses/by-nc-nd/4.0/ https://orcid.org/0000-0002-1181-6369 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 2 1. introduction an analysis of communication has necessarily teemed with delicacies and difficulties. the process of communication is intricate, inconstant and instantaneously influenced by several factors that are intertwined. this directly supports the fact that if scrutinized meticulously, some form of construing structure is usually denoted. the perspective of communication is guided by the exchange of information, thoughts, ideas, opinions, expressions, emotions and instructions between people. a very basic human tendency is to think unquestioningly and indiscriminately to various situations. people easily tend to draw conclusions and make quick judgments. more often people do not tend to do thorough research or think much before solving a problem. the reasons behind any of such thought process are because of the tendency to get biased or one-sided towards personal interests, motives and insensible emotions that hinders them to develop any ability or skill that throws light towards different situations having different approaches or evaluate situations before making decisions and judgments and so on. critical thinking does just that. critical thinking provides definite and controlled means as an advisable guide before taking any actions. “according to the national council for excellence in critical thinking, it is the intellectually disciplined means of aggressively conceptualizing, applying, analyzing, synthesizing and assessing information collected through experience, observation or reflection, as a guide to taking actions” (jones, 2016). kallet discusses the basis of our everyday life which is based on thinking. each action is guided by our thought process like eating, meeting someone, interacting, even sleeping. kallet, however, writes that “critical thinking is thinking but in a different way. (...) most would agree critical thinking is not our everyday, automatic, not-really-thinking-about-it thinking” (kallet, 2014, p. 1) chatfield is very positive that developing such thinking capabilities like no other would furnish us to continue learning all our lives. chatfield presents critical thinking as “when we are thinking critically, we are setting out actively to understand what is going on by using reasoning, evaluating evidence and thinking carefully about the process of thinking itself” (chatfield, 2017, p.6). he further discusses scepticism and objectivity which are attached with critical thinking and which is strong skill sets to respond open-mindedness and simplistic thinking to further an expert and a deeper assessment of situations. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 3 developing critical thinking skills can guide sensible decision making at the workplace. to this gambrill and gibbs understand that critical thinking assists in taking proper decision making which is useful and also helps in delaying decisions if needed for better management. they write “critical thinking is a unique kind of purposeful thinking in which we use standards such as clarity and fairness. it involves the careful examination and evaluation of beliefs and actions to arrive at well-reasoned decisions” (gambrill and gibbs, 2017, p. 5). a key aspect of critical thinking is that it views problem differently by abandoning logical thinking and bias attached to it and seeking to find out solutions through different approaches. kallet and kallet provide a framework of critical thinking components like clarity, conclusions, and decisions in their book think smarter: critical thinking to improve problem-solving and decision making skills. they believe that “within each of the framework components of clarity, conclusions, and decisions, there are numerous critical thinking tools and techniques to guide your thinking. as you use and practice these tools, your problem-solving and decision-making skills will improve” (kallet and kallet, 2014, p. 20). the significance of critical thinking is also about making fair judgments. dwyer writes “critical thinking considers possible viewpoints and results in interpretation, analysis, and evaluation of evidence and the conclusions inferred from the evidence. it is a purposeful, selfregulatory judgment that occurs when an issue is raised, the problem needs to be solved, opinions are reconsidered or experiences carefully reflected upon. critical thinking is more than personal opinion”(dwyer, 2012, p. 379). critical thinking is an art and subtle art to evaluate and consider a concept independently without falling for bias or discrimination and conclude to being logical, sensible and ethical. the workplace has a plethora of job roles, management and many more activities that entirely depend on the medium of communication. in fact, moisescu and golomoz believe that “competition is one of the most important factors that affect the competitive strategies of organisations” (moisescu and golomoz, 2018, p. 54). and in the world of competition an in order to bring efficiency to the workplace, proper communication is very essential. as a matter of fact, each communication at the workplace is based on critical thinking. communication without critical thinking is simply hazardous and the consequences can never be repaired or recovered. dyer in her book critical https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 4 thinking for business students writes “an important outcome of developing your critical thinking skills is that you can improve your own arguments when you write or speak. you will know how to state your views clearly and provide appropriate justification for them. you will be able to avoid fallacies in reasoning, explore your own underlying assumptions, and deal effectively with evidence that runs counter to your views. in general, you will develop further your appreciation for the use of language” (dyer, 2006, p.8). critical thinking has become the most required skill in the 21st century workplace. “when more than 400 senior hr professionals were asked in a survey to name the most important skill their employees will need in the next five years, critical thinking ranked the highest surpassing innovation o the application of information technology” (chartrand, ishikawa, and flander, 2013, p.1). it has become a pivotal and key element for the workplace success as it enables employees to view situations from diverse angles and offers response from multiple surfaces. different authors in their own way and thorough research have penned down developing critical thinking skills through various steps, strategies, principles, and stages. these steps are not obscured but are misleadingly simple as listed by bareham. in his book think well and prosper: a critical thinking guide bareham highlights six steps namely “decode, interpret, analyse, reason, inference and synthesize” (bareham, 2012, p.12-13). similarly, cottrell in the book critical thinking skills: effective analysis, argument and reflection writes “critical thinking assumes abilities in a range of skills such as categorizing, selection and differentiation, comparing and contrasting” (cottrell, 2017, p.5). likewise, ritchhart, church, and morrison made their own interpretations regarding building critical thinking skills. they viewed “observing closely, building explanations and interpretations, reasoning with evidence, making connections, considering different viewpoints and perspectives, and capturing the heart and forming conclusions” (ritchhart, church and morrison, 2011, p.11). in the same way, schiering detects “the thinking involves some of the following skills: organisation, evaluation, advanced decision making and problem-solving, reflection and selfactualization” (schiering, 2016, p.4). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 5 this paper studies in detail the benefits of critical thinking and the various aspects of critical thinking for effective workplace management and also to self-improvement. in addition to that referring to already developed ideas regarding stages to improve critical thinking, this paper provides additional inputs and explanations to deliver the approach. 2. benefits of critical thinking at the workplace global economy is aggressively unpredictable so much so that leaders find it struggling to maintain their calm and quotient and in doing so that may lead to a more judicious communication. tripathy writes “in this extremely unpredictable global economy when there is a speedy change in almost every aspects of organisations, it has been increasingly complicated for the leaders of the organisation to have a clear picture on getting their visions right” (tripathy, 2019, p. 110). the growth or fall of an organisation largely depends on how the leaders see through things. critical thinking assists in doing just that. as a part of soft skills critical thinking which is easily one of the countless required skills for top positions in organisations, recent studies reveal that critical skills are pivotal for every employee and are an essential component for every action in the workplace starting from communication to building team, from leadership to creativity, from making crucial decisions to forming a conclusion, from making judgments to solving problems and many more. overall, critical thinking is the basis of meeting every demand at the workplace. the section below is categorized into two subsections. the first section studies some of the many benefits critical thinking provides at the workplace and the second section studies the benefits critical thinking provides to self. 2.1 the critical thinking fosters effective communication workplaces are large platforms where diversities collaborate to work for a common goal under a common roof. employees from varied cultures, backgrounds, experiences, and distinctions communicate with each other to get the best of profits for the organisation and fulfilling the growing demands of the business world. every day the communication process travels from desks to desks, roles to roles and from positions to positions. it may be thereby concluded that the workplaces survive because of communication. however, keeping in mind the diversities and the need to achieve goals, employees are required to communicate effectively which in turn demands https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 6 thinking but not the usual way. instead, it applies the ability to analyse situations by discarding the conventional thought processes and applying to beyond conventional thought processes. let us understand the above point with an example. there are different personalities of employees in the workplace. however, there are many who do not feel comfortable communicating with people from different cultures and backgrounds. even if they communicate they develop ethnocentrism and generalizations towards them which in turn become barriers to communication. some employees also tend to bully and cause disturbances and nuisance thus causing an unpleasant ambiance at the workplace. such are the characteristic features of uncritical thinkers who would generally tend to avoid communicating with people who do not match their minds. on the other hand, critical thinkers are good communicators. they understand the importance of cross-cultural communication in business and have a propensity towards building a successful relationship with everyone. they critically think regarding the consequences of distasteful relationships and endeavour to break through the barriers thereby creating a working atmosphere. in the words of chase and shamo in the book elements of effective communication “critical thinking is the basis of those good communication skills that employers seek and democracies need. as a listener, critical thinking will help you recognise a speaker’s unstated assumptions. as a speaker, it will help you form precise statements that embody your thoughts. overall, critical thinking will place ideas into a broader context, showing how they relate to other things that you already know or believe” (chase and shamo, 2014, p. 213). critical thinking enables employees to think, observe and analyse situations and also the consequences and then proceed to draw a steadfast conclusion necessary for effective communication. apart from this critical thinking skills also help to choose the right words for a better language presentation especially during presentations in boardrooms. 2.2 critical thinking enables better decisions making each day both employees and managers have to make countless decisions for the benefit and future of the company. examples of decisions making can be listening to various outputs from the employees regarding an issue which is significant for the company and making a decision to choose the best suggestions. other examples can be identifying a fault in newly manufactured products and deciding to stop further production despite bearing tremendous loss for the raw https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 7 material of the production. still, some examples can be deciding to chop off salary as cost-cutting measures keeping in mind the impact on employees and many more. employees may be replete with great ideas but sometimes the ideas are generalized and simplified and not put forward through a thorough evaluation which results in perilous outcomes. besides this, the process of conversion of ideas into decisions is at times accompanied by fallacies and predetermined assumptions. this restricts moving further with appropriate decisions. at this point, the role of critical thinking becomes of pivotal connotation. critical thinking enables employees to make good decisions after arriving at a clever, clear and creative conclusion. decisions at the workplace are often stuffed with imperfections and actions resulted from such flawed decisions at times fail to deliver the breakthrough performance as expected. decisions making is a stressful task especially the important ones most likely at the workplace where one has to choose the best alternative from a myriad perspective. the fact as to which alternative would turn out to be the finest depends only after the consequences. as a matter of fact, all humans lead more than one realm of life like with self, family, friends, the workplace, neighbours, society, strangers and others. when it comes to taking important decisions in these realms they are likely to build their thinking processes in similar ways as they build in the other domains of life they live. this ultimately results in having common characteristics with the decisions making process and what remains is a generalization towards selecting alternatives and thus no rational decision is made. however, when decision making is accompanied by critical thinking the results are outstanding as critical thinking skills enable a decision-maker to analyse and evaluate situations a number of times before comparing the theories with past experiences and finally considering or coming up to a rational decision. paul and elder view “critical thinking when applied to decision making, enhances the rationality of decisions made by raising the pattern of decision making to the level of consciousness and deliberate choice” (paul and elder, 2013, p.184). apart from this critical thinking also provides guidance to modify decisions as and when required. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 8 3. critical thinking assists in problem solving the workplaces comprise different sorts of problems like conflicts, ethical issues, executive decisions, job role ambiguity and many more. certain problems are easy to solve without giving much stress. however, there are some problems that are difficult to solve and that cause disappointment, dissatisfaction and create annoyance. let us understand this with an example. many times employees have clashes of opinions and personalities which result in negative outcomes like poor delivery of performance, negative work atmosphere, and the kind. solving disputes is one of the most difficult jobs for managers as going in favour of the favourite employee leads to further disputes and not doing so results in strained relationships with the employee. such clashes will be endless until managers who practice a high degree of critical thinking come ahead to solve the matter. other examples include cases of sexual harassment that target not only women but men. but agreeing too easily that only a man can sexually harass a woman and not the other way round is not the style of solving a problem for critical thinkers. rickards discusses the nature of problems and distinguishes them into two varieties namely close-ended and open-ended. he writes “closed-ended problems have answers that are the logical consequences of the nature of the problem givens. open-ended problems do not have correct answers, because, for example, they are not precise enough defined or because there is disagreement about the nature of the problem, with different people seeing it in different ways” (rickards, 1997, p.11). although problems can be of different natures, however, to solve it thinking beyond logical method is required which is fulfilled by critical thinking. critical thinking includes a number of ways to solve a problem. the first step begins with identifying or recognizing a problem which is accessed by gathering as much background information about it. the second step begins with the approach towards the problem. butterfield opines the approach of solving problems can be through two ways “intuitively or systematically. when you solve a problem intuitively, you react immediately and instinctively, without following a particular procedure and systematic problem-solving method is to adopt a solution from a prior problem and apply it to your current situation” (butterfield, 2016, p.2). while the former approach is meant to take fast decisions the latter is adopted in rather an organized manner and proceeded systematically. the next step involves brainstorming or attempting to find possible solutions to https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 9 determine the problem. this step is often carried forward hastily as problem-solvers are often misguided with various possibilities at the same time. although it is good to have more than one possible solution, not being able to find the best solution is often frustrating. for every possible solution, it is required that problem solvers get a clear idea about the negative and positive outcomes of each possible alternative. this is the next step which is assessing each alternative and also the risks and merits attached to them. then comes evaluating or selecting the best solution for the problem and which dominates all other alternatives keeping in mind the maximum benefits from the solution and negligible or no risks with it. the final step involves monitoring solutions which ultimately results in the solving of the problem. goetsch in his book taking your faith to work concludes “critical thinking will make you a better problem solver. this is important because problem-solving is essential to success in a competitive the workplace” (goetsch, 2008, p. 155). 4. critical thinking generates better judgments fuller, munro and rainbird in their book the workplace learning in context write “when situations are complex or information is sparse, judgment becomes a critical aspect of decisionmaking: judgment of people; judgement of the quality of products, practices and processes; judgment of the relative significance of, and interaction between different factors; judgment of priorities, options, and strategies” (fuller, munro and rainbird, 200, p. 207). the workplaces are often loaded with difficult judgement to make for example, choosing the right candidate between two best candidates during an interview, prioritizing between focusing on employees’ efficiency and work ethics, prioritizing on putting efforts on achieving short term goals or long term goals. judgments at workplace often get deranged when the process collides with ethics. good judgments are based on ethics and abiding by them. in such situations not taking right judgment may lead to severe consequences the extreme being failure of business deals ultimately affecting organisations. hence, employees must adopt critical thinking while reviewing or making any value judgments. applying critical thoughts to ethics based judgment can produce miracle outcomes. critical thinking generates better ways of applying judicious evaluations to crucial situations. siddiqui describes it to be a skilful thinking that involves applying reasoning and logic to ideas one finds and opinions one forms and situations one experiences” (siddiqui, 2018). https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 10 drawing conclusions and making judgments is not an easy task. implementing critical thinking helps recognizing conjectures, evaluating debates based on the assumptions and finally making a judgment are the salient features of critical thinking that support to make good judgments or modify them within a stipulated time span without much harming situations. a good judgment depends on splitting reality from belief. while judging at times it becomes easy to fall for information that is provided without digging deep and getting insights. at this first stage, critical thinking helps in finding the assumptions and putting forward questions to the nature of the situation. the second stage is pivotal as this is where most of the times judgments are done based on pure bias or which matches one’s own viewpoint or else allowing emotions to penetrate the mind. critical thinking helps to remain controlled, unbiased and objectively while dealing in this second stage which assists in making better judgments. the final stage is where a judgment is to be taken by bringing and comparing dissimilar information together and finally arriving into conclusions through a logical, rational, sensible and quality decision. “when presented with information, people typically alternate between recognising assumptions and evaluating arguments. critical thinking is sequential in that recognising faulty assumptions or weak arguments improves the likelihood of reaching an appropriate conclusion” (chartrand, ishikawa, and flander, 2013, p. 5). the above points discussed what benefits critical thinking can bring to the workplace management. the sub-section below studies the advantages of mastering critical skills bring to selfdevelopment. 4.1 critical thinking instils self reflection critical thinking adds meaning to our existence. simply doing actions without having the knowledge of the potentialities one has is as good as wondering aimlessly without any purpose. taking time from the daily life, one can begin to think beyond the usual duties and think insightfully to evaluate the strengths, weaknesses, fears, and calibre that one carries and how one can make the strengths strong, weaken the weaknesses, discard fears and make use of the calibre to get opportunities. this helps in avoiding mistakes and motivating self to be a quick learner by developing good judgemental and decisionmaking skills. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 11 critical evaluation of self helps in analysing and planning for commitments in various situations and domains of life and also provides a vent to unacknowledged and countless abilities an individual may own. suter in his book introduction to educational research: a critical thinking approach provides briefly a list of the qualities critical thinkers possess. he writes “critical thinkers are able to display inquisitive ideas, consider defensible conclusions, remain sceptical, reason with clarity, self-correct, justify conclusions, and seek multiple sources of information” (suter, 2011, p.11). an individual can add more and more skills and traits to his/her personality through critical thinking. one can be able to deeply and in an organised manner structure one’s life as thinking critically enables in polishing the best of qualities one has and by bringing one to close connection with self. it also prepares an individual not to be dependent on others and taking complete possession of duties and responsibilities. hoaglund expresses “critical thinking is reflective in that it observes its own progress, evaluates each step to decide whether it is justified and corrects its own. the critical thinker is self-reliant and aspires to a degree of independence or autonomy in their thinking” (hoaglund, 1999, p. 2). self-reflection is largely influenced by the values, beliefs and morals an individual carries and has a massive impact in the workplace too as employees get a tight hold on themselves and control their feelings and behaviour. it empowers employees to pause, think and analyse situations before taking any action which is way beneficial. lot of issues both minor and major gets solved. the management association and information resources propose “the definition for values that emphasizes the need and benefit of self-reflection as part of the method by which an individual sets personal limitations and understands the consequences of his/her behaviour. through self-reflection we are able to determine the best course of action in our personal lives and within an organisation” (management association and information resources, 2018, p. 100). 4.2 critical thinking builds compassion one of the most important criteria one must infuse within self is the ability to be empathetic or compassionate in order to develop a personality. as it is there are tremendous circumstances that bring sufferings, failure, and disappointment to life. at such a point what one requires is a shoulder to lie on or kind words of compassion. at the workplace too employees expect care and consolation https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 12 “yet many workers say their employers remain silent and uncaring in the face of suffering, perhaps because they believe work life and home life should be kept separate, or because they fear making a mistake in offering support” (how to awaken compassion at work, 2018). critical thinking enables compassion as putting one’s foot into other shoes and feeling the pain and suffering of others as own. compassion stems from kindness or an unfathomable consciousness towards others. gilbert writes “compassion can be defined in many ways, but its essence is a basic kindness, with a deep awareness of the suffering of oneself and of other living things, coupled with the wish and effort to relieve it” (gilbert, 2010, p. xiii). critical thinking smoothes the progress of thoughts, wisdom, and understanding towards being compassionate. it first encounters the intensity of understanding with self and finds out when what and how an individual is affected by various emotions and then gradually proceeds to understand the emotions of others. however, the same understanding gets clouded with feelings of jealousy, depression, and anxiety. the emotions of others would start seeming vague and there would be focus only on self and not on others thereby damaging relationships. critical thinking builds the emotional gap and restores relationships. at the workplace too critical thinking and compassion go hand in hand if attempted to bring these feelings together. worline and dutton write “facing challenges from turbulent environments, regulatory changes, or customer complaints, leaders and managers can easily dismiss the need for compassion. but when seeking to build high-performing organisations that meet the challenges of a twenty-first-century work environment, compassion matters more than most people recognise” (worline and dutton, 2017, p. 13). as a matter of fact, critical thinking coupled with compassions fosters the workplace relationships and information sharing with each other. the combination can have a desirable impact both on work performance and interpersonal relationships. 4.3 critical thinking augments creativity critical thinking is useful in developing creative and innovative ideas to solve a problem, to do a task, to write and to bring out change. only inquisitive people who question, investigate, search enquire and are curious for ideas are passionate to alter ideas into action are likely to be creative in their thoughts (tripathy, 2019, p. 43). creativity rejects a wide range of applicable https://creativecommons.org/licenses/by-nc-nd/4.0/ https://greatergood.berkeley.edu/article/item/how_to_awaken_compassion_at_work multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 13 deviations. it encourages an individual to dismiss the assumptions which are common and biased or any other approach to the thoughts development process which narrows down intelligence and inventiveness. creativity constitutes intellectual subsets like questioning, cognizance and insightful means to foster new ideas and problem-solving methods. in the words of goodwin and sommervold “creativity is the ability to invent or repurpose, to make something novel that has value. the value can be social, political, financial, or personal, but in order for an act to be considered creative, it must produce something of value” (goodwin and sommervold, 2012, p. 50). creativity is transferable which assists in solving many problems every day. critical thinking fosters a creative approach towards problems in order to solve it and also ensure that the problem does not arise again. individuals, as creative people learn to learn various things around them and at the same time are bearing to mistakes and take pleasure in insightful thinking. also simply developing ideas is not the only thing that creativity encourages; finding out the link between ideas is equally significant. however, it is the critical thinking earned by severe efforts that enable creativity over and over again generating new ideas, answers and solutions to causes, questions and problems respectively. lau states “a crucial step in the creative cycle is to actively explore the connections between ideas. the emergence of a new idea might only take seconds, but the thinking habits that allow the idea to emerge can take years of hard work and discipline to cultivate” (lau, 2011, p. 201). at the workplaces too individuals with creativity accompanied by critical thinking can solve various problems, make decisions and draw conclusions without fallacies. the above section discussed the significance of critical thinking in personal development. at this point it is important to understand that for personal development an individual requires many skills and abilities. however, those skills and abilities begin with the basis of critical thinking which is the centre for learning, behaving, dealing and leading. it is also mandatory to comprehend and follow certain steps that are required to be followed in order to develop critical thinking skills. the section below studies the important steps one by one essential to develop critical thinking. 5. steps to develop critical thinking studies reveal that developing critical thinking skills at a very early stage in life fosters better opportunities in life and boosts the way an individual sees life through https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 14 thought processes. often at workplaces “leaders face a number of challenges related to professional and personal structures and come across complex circumstances when trapped between their morals and professional demands in order to resolve problems that bypass their state of affairs” (tripathy, 2019, p. 135). most of the times they fail to use critical thinking to differentiate between these structures. instead, they select only a few domains where they can use critical thinking thereby ignoring other fields of life. in order to master critical thinking an individual needs to follow step by step technique that would promote goodness at workplace management and develop an overall personality. this would foster an overall development in all spheres of life but simply a countable few. the stages below discuss in details the step by step procedures to inculcate critical thinking. 5.1 gather knowledge the first stage of critical thinking is to gather as much knowledge as possible regarding a concept or a subject. one cannot think without acquiring knowledge. not having sufficient knowledge would lead to confusion and vague thinking. only having corrected and sufficient knowledge can make us fair in our judgments and decisions. paul and elder write “our false knowledge, misconceptions, prejudices, and illusions stand in the way of the possibility of our being fair”(paul and elder, 2013, p. 29). hence, one needs to carefully receive the ideas, information and adequate knowledge that would further assist in thinking failing which reaching the next stage is meaningless. 5.2 understand the next stage is to understand the knowledge acquired. once sufficient knowledge and information is gathered it’s time now to understand it specifically. a thorough research is to be done on the existing knowledge and the acquired knowledge. implementation of understanding means going in exactly the direction our thinking is taking us to but also ensuring the fact that personal interpretations are discarded. however, it must also be kept in mind that the consequences of the interpretation and understanding can have a significant impact if the sense of understanding carries flaw. at this stage one can start reasoning and questioning and can receive insightful answers. this stage provides limitless combinations of solutions to solving various problems. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 15 moore writes “the framework of structured reasoning frames questions to help decide the best means or best combination of means that are suited to solving specific intelligence problems” (moore, 2010, p. 18). 5.3 contextualize the third stage is to contextualize or finding out any relative or related situation or context with the present one. contextualization is beneficial as it provides results and outcomes which were earlier not achieved and hence furnishes with new ways one can move ahead to find better solutions. in this stage there is a comparison between the present situation with the past experiences and thereby moving ahead with the results. bareham suggests “if we can fit new information into an accurate context by associating it with our existing knowledge, we can acquire some semblance of understanding by blending the new with the old if we can comprehend, we can express meaning and significance” (bareham, 2012, p. 13). the constraint to combine collected information and already existing information elucidates the reason behind the impossibility of learning when dealing with knowledge which we are completely ignorant of. hence, when there is no substantial related knowledge or information, links and connections cannot be formed and the base to differentiate between information, truth, reality, intention, meaning or meaningless. 5.4 evaluate and analyse the next stage involves evaluation and analysis of the facts and results in hand. this further seeks to endeavour efforts to find the right answer to the solution. hence, for this one needs to in a much-organised manner dismantle and break apart the total with an aim to comprehend the single parts or pieces. people most of the time because of hurry and worry do not evaluate or analyse at all. however, the need to evaluate and analyse will tell the need to scrutinize comprehensions, understandings and interpretations. therefore it takes time. evaluation and analysis include thorough attention towards fine points without being diverted by other diverting agents. cottrell views “when critically evaluating arguments, it is important to remember that one can find an argument to be good or effective even if one does not agree with it” (cottrell, 2017, p.12). evaluation and analysis require teaching, learning, and extensive concentration and regulation to do it greatly fitting. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 16 5.5 synthesize after a thorough analysis that is the meticulous study of breaking down its components to smaller parts and exploring their connections with those smaller parts and a constituent as a whole, it is time now to combine those smaller parts and form new information by eliminating the unwanted information and keeping back the important ones. black suggests “within critical thinking, synthesis is the term given to the construction of new and original arguments out of appropriately interpreted and organised evidence and other raw materials for argument construction (black and thompson, 2012, p. 166). studies reveal that the act of synthesizing is faster and more than analysing as critical thinkers quickly want to come to a conclusion or solution. however, both analysis and synthesis require a deep understanding of the broken parts, only then it is possible to reconstruct and infuse meaning so that they can become more than the total constituent. this would be helpful in drawing conclusions and taking actions. 5.6 take actions after going through all the stages of critical thinking the final stage is to take actions, or conclude. however, this stage also upholds another stage of modification or alterations if required depending on the situation and the emergence of taking any further action. the action has to be fulfilled with clarity and has to be evidence and research-based. many conclusions fair and just are made after coming through the afore discussed stages. effective decisions are made and man complex problems are solved. 6. conclusion this paper studied the essence of thinking systematically, rationally, logically and sensibly. critically evaluating and formulating thoughts and beliefs through organised procedures. critical thinking is relevant to personal development and the workplace management as it determines our actions, decisions and choices we make. critical thinking governs our automatic thinking and gives us power to learn, explore and defend against biases, fallacies, influences and errors. critical thinking demands non-acceptance of believing things which we see or hear and putting efforts to categorize truth as viewed in general and not prejudiced. the paper studied critical thinking as the most important skill set in the present era and the countless benefits it furnishes for the workplace management like effective communication, powerful decision making, substantial judgments, https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 17 problem-solving and also improves personal management by instilling indispensable abilities like self-reflection, compassion and creativity for a better living and successful life. the paper also studied in detail the various stages of developing critical thinking so as to fully understand the process and work accordingly. this paper is not only useful for professional lives but also for personal lives overall to bring excellence in all domains of life. references black, b., & thomson, a. (2012). an a to z of critical thinking. london: continuum. braeham, s. (2012). think well & prosper: a critical thinking guide. bookbaby. butterfield, j. (2017). problem solving and decision making. boston, ma: cengage learning. chartrand, j., ishikawa, h., & flander, s. (2013). learn to apply and develop the new #1 the workplace skill [pdf]. pearson. chase, r. s., & shamo, w. (2013). elements of effective communication. washington, ut: plain and precious publishing. chatfield, t. (2017). critical thinking: your guide to effective argument, successful analysis & independent study. los angeles: sage. cottrell, s. (2017). critical thinking skills: effective analysis, argument and reflection. basingstoke: palgrave macmillan. dwyer, j. (2016). communication for business and the professions: strategies and skills. melbourne, vic.: pearson australia. dyer, l. (2016). critical thinking for business students. brantford, ontario: w. ross macdonald school resource services library. gambrill, e. d., & gibbs, l. e. (2017). critical thinking for helping professionals: a skills-based workbook. new york, ny: oxford university press. gilbert, p. (2009). the compassionate mind: a new approach to life's challenges. canada: new harbinger publications. goetsch, d. l. (2008). taking your faith to work: how christians can succeed in secular careers. powder springs, ga: american vision press. goodwin, m., & sommervold, c. (2012). creativity, critical thinking, and communication: strategies to increase students' skills. lanham: r&l education. https://creativecommons.org/licenses/by-nc-nd/4.0/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 18 hoaglund, j. (1999). critical thinking. newport news, va: vale press. how to awaken compassion at work. (n.d.). retrieved july 8, 2018, from https://greatergood.berkeley.edu/article/item/how_to_awaken_compassion_at_work jones, a. (2018, april 22). the importance and benefits of critical thinking and reflection in the workplace. retrieved july 5, 2018, from https://www.careermetis.com/benefits-criticalthinking-the workplace/ kallet, m. (2014). think smarter: critical thinking to improve problem-solving and decisionmaking skills. hoboken, nj: john wiley & sons. kallet, m., & kallet, m. (2014). think smarter: critical thinking to improve problem-solving and decision-making skills. john wiley & sons. lau, j. y. (2011). an introduction to critical thinking and creativity: think more, think better. hoboken, nj: john wiley & sons. moore, d. t. (2011). critical thinking and intelligence analysis. place of publication not identified: lulu com. moisescu, f., & golomoz, а. (2018). effects of business combinations on the competitive environment. multidisciplinary journal for education, social and technological sciences, 5(2), 51-72. https://doi.org/10.4995/muse.2018.10164 paul, r., & elder, l. (2014). critical thinking: tools for taking charge of your professional and personal life. upper saddle river, nj: pearson education. rainbird, h., fuller, a., & munro, a. (2004). the workplace learning in context. london: routledge. rickards, t. (1997). creativity and problem solving at work. aldershot: gower. schiering, m. s. (2016). teaching creative and critical thinking: an interactive workbook. lanham: rowman et littlefield. siddiqui, f., & individual member of institute of management consultants of india. (n.d.). fareed siddiqui. retrieved july 8, 2018, from https://careerandeducation.expertscolumn.com/benefits-critical-thinking-the workplace social issues in the workplace: breakthroughs in research and practice. (2018). hershey: business science reference, an imprint of igi global. https://creativecommons.org/licenses/by-nc-nd/4.0/ https://greatergood.berkeley.edu/article/item/how_to_awaken_compassion_at_work https://www.careermetis.com/benefits-critical-thinking-workplace/ https://www.careermetis.com/benefits-critical-thinking-workplace/ https://doi.org/10.4995/muse.2018.10164 https://careerandeducation.expertscolumn.com/benefits-critical-thinking-workplace multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12925 social and technological sciences issn: 2341-2593 tripathy (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol.7 nº 2 (2020): 1-19 | 19 suter, w. n. (2012). introduction to educational research: a critical thinking approach. thousand oaks, ca: sage. tripathy, m. (2019). the power of ethics: rethinking leadership roles at workplaces. multidisciplinary journal for education, social and technological sciences, 6(2), 134-146. https://doi.org/10.4995/muse.2019.11594 tripathy, m. (2019). the impact of ethical leadership on organisational behaviour: a conceptual analysis. pacific business review international 11( 10), 109-114. tripathy, m. (2019). overcoming the major challenges in new entrepreneurship: an orientation through soft skills. smart journal of business management studies, 15(2), 38-46. https://doi.org/10.5958/2321-2012.2019.00013.7 worline, m. c., & dutton, j. e. (2017). awakening compassion at work the quiet power that elevates people and organisations. oakland, ca: berrett-koehler. https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.4995/muse.2019.11594 https://doi.org/10.5958/2321-2012.2019.00013.7 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 81 music as a tool for ethics maría gemma sánchez gonzález university of extremadura, c/muñoz chaves, 3 10004, cáceres, spain email: sanchez_gemma@hotmail.com received: 9 april 2022; accepted: 26 august 2022; published: october 2022 abstract the current musical situation nowadays reflects a fast-technological change that has contributed to main social issues, such as a tangible crisis of values globally. the impact of music lyrics, nevertheless, could be a powerful useful tool to engage people in different ethical values leading to an improvement of relationships of any nature. can music be that magic wand? this study aims at measuring the impact of lyrics on values of people and the power to improve norm activation. a questionnaire based on the likert scale was conducted among 281 subjects either at university or with university studies. results were analysed using the structural equation modelpartial least square (sem-pls), which showed the relevance of items and connected hypotheses and a significant prediction capacity (r 2 = 0.458). conclusions exposed that music can predict attitude change and become a tool for ethics. keywords: behaviour; ethics; music; lyrics to cite this article: sánchez-gonzález, m.g. (2022). music as a tool for ethics. multidisciplinary journal for education, social and technological sciences, 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/%200000-0003-3053-0646 https://doi.org/10.4995/muse.2022.17497 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 82 1. introduction facts speak for themselves and music being one of the preferred leisure activities is a fact in today´s world. everywhere we go, music is always there being combined with any situation. music facilitates suitable environments to engage individuals in universal ethical values (nicolás & azorín, 2013), leading, no doubt, to the improvement of relationships. regarding emotional states (hariri, bookheimer, & mazziotta, 2000) the connection between values involving emotions to be developed and music arising and representing emotions is undeniable. ethical values of people positively will be reflected in the way they act and behave on both personal and social levels, since personal beliefs dictate moral obligation (schwartz & howard, 1981) and successive norm activation (liere & dunlap, 2006). as mentioned before, the global crisis people are currently experiencing (sortheix, parker, lechner, & schwartz, 2017) and the fact that music is powerful enough to portray moods and feelings lead us to the hope that music could be the ideal tool for ethical purposes. music can be used to pump values of all sorts to people and subsequently promote attitude change as lyrics offer the perfect scenario for mostly reflection and feelings (christenson & roberts, 1998). it is well-known that lyrics mirror social trends, personal perceptions of life, worries or concerns (christenson, de haan-rietdijk, roberts, & bogt, 2019) and have an impact on listeners. on a wider note, music in a foreign language could serve different purposes, not only to channel emotions and build values but also to improve skills on a more academic level. schumann (1994) coined social distance as a factor easily found in society that contributes to the learning of a language. besides, mclean (1980, p.18) stated “a clear need for the content of language-teaching materials to involve the learner -to relate to his needs, interests and moral concerns”, therefore the potential of music in english may not be ignored. music as a perception and value changer can not only amuse individuals but also increase quality of life (juslin, liljeström, västfjäll, & lundqvist, 2010) and subsequent attitudes. all in all, music can be proposed as a valuable tool that can improve quality of life. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 83 2. literature review considered a constant factor around us, music can be employed for the promotion of ethics and attitude change, which may definitely lead to an enhanced world where personal as well as social respects is a must, not forgetting nature and the environment (dietz, stern, & rycroft, 1989). the change to use music for prosocial advantages cannot be missed but take advantage; and delving into lyrics and the message they send needs to be considered to achieve meaningful changes in society. 2.1 music with personal values (mpv) creativity provided by music is one main factor to consider (hallam, 2015), since it triggers communication and realiation of own concerns and achievement (dewall, pond, campbell, & twenge, 2011). lyrics, no doubt, may lay the floor for personal reflection and therefore reinforce values that set robust behavioural standards in any situation or context (maslow, 1954; perrinjaquet, furrer, marguerat, usunier, & cestre, 2007). when music with selfcare values takes over, it definitely improves guidance in life and help overcome personal issues as well as provide gratification (de la rosa herrera & publiese, 2017), which may have an impact on social feelings and skills due to the connection individuals feel when they listen to music, thus becoming one and sharing identity patterns. humans are in a constant search for discovering and developing own abilities and potential in order to match their self-concept, image and self-esteem (rokeach, 1973). on the other hand, music can actually change perceptions and provide meaning to experiences, making them last in time and making them transcend by means of values (rokeach, 1973; schwartz, 1994). music may reflect types of personality and provide benefits such as cognitive, physical or motor skills which can suddenly improve our response drive and act in a specific way (aubrey, 2006; schellenberg & von scheve, 2012), therefore influencing human attitudes. the statement above leads to hypothesis 1: music with personal values influence (mpv have an impact on actions of people (ap). https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 84 2.2 music with family/transcendental values (mftv) it is a fact that songs can actually acquire the role of communicators and disseminators of messages with values since they express ideas and feelings connected to different lyrics and rhythms. family being the first social group is the promoter of values that are intimately connected to personal experiences and circumstances (hallam, 2015). songs are also part of family celebrations which are linked to values that will endure through a life time (ilisko & kravalepaulina, 2015). a well-known fact is that social bonding is primarily created by music lyrics, which drive individuals to behave in a particular way, enhancing prosocial behaviour (greitemeyer, 2009a; greitemeyer, 2009b; gentile, anderson, yukawa, ihori, saleem, ming, & sakamoto, 2009). besides, learning processes are facilitated by listening to music, which in turn, will help make decisions on both personal and social levels, since it functions as a promoter of acceptance, tolerance and justice, among others (hallam, 2015). in addition, transcendental music do have the potential for engagement regarding idealism, divinity or/and individualism (jackson, 2014), therefore influencing actions of people. the statement above leads hypothesis 2: music with family and transcendental values (mftv) have a positive impact on actions of people (ap). 2.3 music with environmental values (mev) music being considered as a very powerful means for the dissemination of thoughts, ideas or concerns may actually contribute to the preservation of the environment and everything surrounding environmental issues. environmental lyrics may not be undermined but the opposite. music can help spread very different and powerful message depending on musical tastes (zillmann & gan, 1997). on the other hand, lyrics containing ethical messages offer personal as well as social interaction resulting in the enhancement of values related to the environment (sagiv, roccas, cieciuch, & schwartz, 2017) since they create that special connection and empathy with the world around us. songs being short and repetitive help with efficiency when it comes to disseminate https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 85 messages to others and may have a direct impact feelings and emotions (taruffi & koelsch, 2014), influencing behaviour change. the above information leads to hypothesis 3: music with environmental values (lev) have a positive impact on behaviour change (bc). 2.4 social distance (sd) the use of the english language has touched every corner of the world and, needless to say, every field of knowledge, consequently learning english has become a must; there is always a reason to learn it. according to kachru and nelson (2001), english is the language that is internationally being taught and learnt at all levels (gonzález davies & celaya villanueva, 1992; alcaraz varó, 2000; flowerdew & peacock, 2001) and which is necessary to achieve success, therefore, it plays a main role as a lingua franca (graddol, 1997). many a country, including spain, includes english as a second language in their educational system, in their curriculum, therefore it is learnt as a foreign language (gonzález davies & celaya villanueva, 1992); and recently group learning is in high demand as a resource for second language learning processes (adiantika & purnomo, 2018). due to the era of globalisation we are experiencing, the concept of acculturation comes along; it is a concept involving adjustment/modification of norms established by society, that is, social norms, attitudes and values embraced by a particular culture (sam & berry, 2010). to have control of a language somehow portrays more sociability (albino, 2017). thus, the social distance phenomenon develops from the cognitive and affective approach when two different cultures come in contact with each other and individuals start to adapt to the new culture by different reasons (emigration, work, learning, acceptance). on a global level, the enormous influence that english music has all over the world is a fact the same as the fact that it is an engaging tool related to learning processes of all kinds (sánchez gonzález, 2021). acquiring a new language may bring unexpected benefits from structural and motivational properties that music has (fonseca-mora, toscano-fuentes, & wermke, 2011). music as a potential instrument for learning, if combined with social distance may positively influence performance of different nature (sánchez gonzález, 2021). https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 86 mclean (1980, p. 17) states that, “learning is most effective when the learner is the initiator of the learning process”. since listening to music is such a pleasurable activity that can be employed for many a purpose, individuals, with no much effort, become learners of the message in the lyrics and those lyrics may have an impact on their attitude and actions afterwards (sánchez gonzález, 2021). the statements above lead to hypothesis 4: social distance (sd) has a positive impact on actions of people (ap). 2.5 actions of people (ap) schwartz and howard (1981) detailed that beliefs of personal nature did imply moral obligations in order to carry out actions to basically avoid consequences that are unwanted. to be aware of those consequences drives the individual to act from a responsible angle in order to obtain established objectives (han, 2014; schwartz, 1977). actually, emotions have the power to positively activate a wide range of experiences and attitude change in people (sánchez gonzález, 2021) of all ages, especially in those moments when personality swings do take place (coyne, stockdale, nelson, & fraser, 2011). when people are wrapped up in music, this fact generates emotions, something innate to humans, acting as a common channel for communication, togetherness (sanchez gonzález, 2021) and, of course, understanding and respect for the world and all its elements surrounding us. compromise, then, is born to then promote change in actions and attitudes of people leading to the development of personal rules (sánchez gonzález, 2021). the information above leads to hypothesis 5: actions of people (ap) have a positive impact on the development of personal rules (dpr). 2.6 behavioural changes (bc) socialisation is paramount for the survival of the human race. paying attention to the general learning model, the impact and effects of songs with different lyrics can play a relevant role in social behaviour (greitemeyer, 2009b). music is an indispensable element, increasingly being used https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 87 in the education of children (do & schallert, 2004; fonseca-mora, 2000) for reinforcement and enhancement of positive attitudes. the openness of young people to give anything a chance is something praiseworthy; being open to change also involves adopting the role of collaborators of the world. although people generally act upon rules learnt from family and personal experiences, to be open to change also affects norm activation (de groot & steg, 2009) and this fact may influence other peers. this statement leads to hypothesis 6: the development of personal rules (dpr) has a positive impact on behaviour change (bc). 2.7 music as a tool for ethics (mte) music is a pleasurable activity able to relax and bring back memories from any time in the past; it has the ability to bond people, not only friends and family members but also any individual that shares the same kind of music; it fetches a sense of identity and belonging (bogt, mulder, raaijmakers, & gabhainn, 2011) within a joyful atmosphere. it can be said that music with prosocial lyrics can actually influence prosocial behaviour (greitemeyer, 2009a) as well as personal and social interaction (sagiv et al., 2017) , which may change the world for the better. al kandari & al qattan (2020) state that task-based learning along with new technologies can effectively influence group learning. music can be measured as an extremely significant and persuasive instrument that can engage people in different scenarios and therefore enhance values related to personal, environmental, family and transcendental values. then, the statement that music is a tool for ethics can be affirmed. these statements then lead to hypothesis 7: behaviour change (bc) has a positive impact on the understanding of music as a tool for ethics (mte) https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 88 figure 1. hypotheses according to pls path modeling h 7 behaviour change (bc) h 6 development of personal rules (dpr) music with family and transcendental values (mftv) (pgt) social distance (sd) h 5 actions of people (ap) h2 music with personal values (mpv) h1 music with environmental values (mev) music as a tool for ethics (mte) h3 h4 https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 89 3. methodology the methodology of this study is based on main pillars taken from the literature review previously exposed, which leads to the following research question: can music be a tool for ethics? in order to write and develop a report on how music and the words of the lyrics can engage with people and change their attitude for then to become an ethical tool for different purposes and as to facilitate the researcher with the capacity for prediction of the fact in consideration (creswell & clark, 2017), the quantitative research method was chosen to also meet the aim of this study. this method was believed to be the most suitable considering the population sample being relatively small. the methodology followed some phases: phase 1 covered a thorough online search for a wide range of song lyrics containing values that would meet the purpose of the study to then analyse them at a later date. phase 2 was performed at different universities. volunteers took part after a brief introduction of the topic by lecturers who collaborated with the researcher. a focus groups had been previously arranged to analyse the questions of the questionnaire to be completed by the students. all participants were of legal age and therefore no ethical considerations were needed to be addressed. in phase 3, a pilot survey was conducted with the intention to double check the level of understanding of the questions and the reliability of possible responses. after some changes, the questions became clearer and shorter with a simplistic design, as suggested by hoinville and jowell (1978), as to allocate contents and safeguard top cooperation. the questionnaire was built upon the likert-scale model, using multiple-choice answers with a total of with thirty-five (35) items. participants would take an average time of 4 minutes to complete the questionnaire. after gathering all answers, the analysis thereof followed by using the partial least square (pls) method (it program). 3.1 variable definition the variables use in this research were justified by the literature review previously stated (see table 1). every construct of the study was defined by five (5) items acting as measurement models. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 90 table 1. variables justification constructs indicators mpv: music with personal values mpv 1: modification in personal values in the socialisation process. (schwartz, 2011; rokeach, 1973) mpv 2: value predilection affects affect different values for attitude change. (rokeach, 1973; tooby & cosmides, 1990) mpv 3: motivation as vital in efficient learning. (fonseca-mora et al., 2011) mpv 4: fondness and and emotions leading to value development. (perrinjaquet et al., 2007) mpv 5: music connected to personality kinds. (aubrey, 2006; schellenberg & von scheve, 2012) mftv: music with family/transcendental values mftv 1: language input, learning to cause attitude change. (schäfer et al., 2013) mftv 2: music, a bond for education. (do & schallert, 2004; fonseca et al., 2011) mftv 3: value reinforcement by emotions. (jackson, 2014) mftv 4: music to increase prosocial behaviour. (greitemeyer, 2009a; greitemeyer, 2009b; gentile et al., 2009) mftv 5: family values as guiding values. (ilisko & kravale-paulina, 2015) mev: music with environmental values mev1: environmental doings to sponsor positive values. (gatersleben et al., 2008) lev2: music, shared ground for creativities. (gatersleben et al., 2008) mev3: cohesion to be boosted by music. (kachru & nelson, 2001; gatersleben et al., 2008) mev4: social bonding by emotions provoked (juslin et al., 2010). mev5: social awareness/altruism by values. (juslin et al., 2010). sd: social distance sd1: social distance to drive learning and togetherness. (sam & berry, 2010; schumann, 1994) sd2: english as part of the globalisation process. (graddol, 1997; alcaraz varó, 2000) sd3: a related factor for own learning (schumann,1994; mclean, 1980) ap: actions of people ap1: music to relax. (napier & shamir, 2018) ap2: music to motivate. (christenson et al., 2019) ap3: music, always suitable (jackson, 2014). ap4: music to create (schellenberg & von scheve, 2012) ap5: music to lay ground for reflection (rentfrow & gosling, 2003) dpr: development of personal rules https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.tandfonline.com/doi/full/10.1080/02673843.2011.650182?src=recsys multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 91 dpr1: sense of identity, relaxation and belonging by music. (rentfrow & gosling, 2003; napier & shamir, 2018) dscr2: connection of people leading to attitude change. (sortheix et al., 2017) dpr3: the omnipresence of music to drive emotions (taruffi & koelsch, 2014) dscr4: music encouraging creativity. (schellenberg & von scheve, 2012) dpr5: music to reflect on social values. (sortheix et al., 2017) bc: behaviour change bc1: emotions leading to attitude change. (rokeach, 1973) bc2: music, common ground for identity (bogt et al., 2011) bc3: music always suiting a purpose. (gardner & lambert, 1972) mte: music as a tool for ethics mte1: music as an inspiring tool for mental processes (fonsecamora, 2000) mte2: music to connect individuals with shared interests. (sagiv et al., 2017) mte3: emotions, imperative in attitude change. (taruffi & koelsch, 2014) mte4: actions and awareness (schwartz, 1977) mteeyp5: music to connect and spread messages. (liere & dunlap, 2006) 3.2 research population the participants of the study were mostly university students or subjects with university experience of any kind and branch, since they were considered the most appropriate subjects as they were directly connected with the problem of study and were mature enough to answer with honesty and veracity to the questions given to them. they shared their views about the influence of music in their lives and predicted future changes in behaviour. a total of two hundred and eightyone (281) participants took part in survey within a period of two to three months in autumn of 2020. the research method chosen allowed the collection of data to be easy and cost-effective, as most of the phases were conducted through internet applications and tools such as google forms and instant message available options. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 92 3.3 data analysis and statistical treatment the method for data analysis and statistical treatment used was the smartpls programme (ringle, wende, & becker, 2015) which made use of the pls multivariate technique (structural equation modeling) to detect and analyse the relations amongst the variables employed in the study, either unobservable or latent (hair et al., 2013). structural equation modeling was chosen in order to quantify behavioural patterns for later statistical treatment (fornell & bookstein, 1982). it is a model suitable for proposals related to social sciences and to deeply test the hypotheses of the anticipated model. 3.4 data analysis and measurement model the main objective of pls is to predict the dependent variables in order to maximise the explained variance (r 2 ) that belongs to the dependent variables. in other words, the pls model (barclay, thompson, & higgins, 1995) encompasses assessment and reliability factors of the measurement model and assessment of the structural model in order to measure and monitor the weight and extend of relations between the variables in the study. to assess both the validity and reliability of the model employed, consistency reliability, convergent validity and discriminant validity parametres were used. as anticipated by carmines and zeller (1979), in order to secure indicators, these should have a load equal or above 0.707, that is, the variance that a construct and its indicator share should be higher than the error variance. in this particular case, all loads (λ) were above 0.7, therefore the analysis thereof is then proper and accepted. all outer loadings showed reasonable reliability. table 2 displays the construct reliability and convergent validity figures, conducted in order to guarantee internal consistency of the indicators in regards to the construct. this was measured by the cronbach’s alpha, which states that all figures should be about 0.70, also by the rho a 2 , by the composite reliability (cr), with a measure range of 0.70 as well; the convergent validity of the latent variables was assessed by the average variance extracted (ave. convergent validity reflects the level of correlation between two measures of the same concept. on the other https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 93 hand, latent variables should always stay above 0.50 as to properly defend half of the variance of its indicators (henseler, ringle, & sinkovics, 2009). this condition is present in this study. constructs have an ave fluctuating from 0.596 to 0.683 (table 2), approving that the measurement model has appropriate convergent validity. in regards to internal consistency reliability, the composite reliability (cr) of each construct surpasses the value of 0.7 therefore the consistency is justified. table 2. construct reliability and convergent validity cronbach's alpha 1 rho_a 2 composite reliability (cr) average variance extracted (ave) ap 0.714 0.715 0.895 0.635 bc 0.764 0.809 0.898 0.638 dpr 0.830 0.865 0.876 0.619 mev 0.723 0.766 0.855 0.518 mftv 0.720 0.713 0.817 0.541 mpv 0.811 0.820 0.860 0.617 mte 0.716 0.743 0.818 0.596 sd 0.764 0.847 0.880 0.683 the assessment of convergent validity is performed by the mean extracted variance (ave), a measure proposed by fornell and larcker (1981), which shows the variance obtained by a construct from its indicator in relation to the variance due to the measurement error. table 3 shows how following the fornell-larcker’s criterion, a latent variable is required to share more variance with its indicators than with other variables (fornell & bookstein, 1982). values in the study fulfill this requirement. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 94 table 3. fornell-larcker criterion3 ap bc dpr mev mftv mpv mte sd ap 0.813 bc 0.325 0.839 dpr 0.540 0.335 0.790 mev 0.487 0.437 0.462 0.783 mftv 0.568 0.520 0.576 0.687 0.772 mpv 0.588 0.486 0.542 0.585 0.768 0.782 mte 0.625 0.388 0.545 0.510 0.560 0.522 0.762 sd 0.439 0.373 0.418 0.358 0.401 0.495 0.364 0.842 the last step to take was regarding discriminant validity, which was taken into account in order to validate the measurement scale used in the study and hence define an improved approach of the scale in question. discriminant validity measure how each and every variable differs from the others. in the heterotrait-monotrait (htmt) factor, measures are far from related.this factor should never go higher than 1 to show accuracy in the distinction between two factors the htmt value should never exceed 0.90 for the constructs to show correlation and be significative. all results obtained from the model showed a ration below 0.90 (henseler, ringle, & sarstedt, 2015). 3.6 data analysis and inner model table 4 shows values corroborating the variance explained (r 2 ) and results prove the strong connection among the constructs used. as stated by chin (1998), the coefficient of determination (r 2 ) can be classified into different categories, those being weak (0.19), modest (0.33) or significant (0.67) depending on the results from variables. this main figure clearly shows how the dependent variable, music as a tool for ethics (mte) had predictive capacity by showing r 2 = 0.458 (> 0.33 = modest). the model used the bootstrapping measuring technique to generate tstatistic values by processing 5,000 random samples from 281 responses obtained. the stone–geisser (q²) test (stone, 1974; geisser, 1974) states that values can be classified as follows: 0.02 for small, 0.15 for intermediate and 0.35 for pertinent predictive relevance. in this research, all constructs showed that prediction and relevance, as q² values showed 0.02 or above. there is actually relevance prediction if q² > 0, which is the case in the https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 95 present study, whose dependent variable music as a tool for ethics (mte) has proven to have predictive weight (0.248). table 4 shows constructs confirm that requirement. table 4. r2= coefficient determination) and q² =stone-geisser test r 2 adjusted r 2 q² ap 0.439 0.431 0.257 bc 0.099 0.096 0.053 dpr 0.291 0.298 0.176 mte 0.458 0.453 0.248 in order for path coefficients to be validated and be reliable, they should remain above 0.100 in order to create power in the model employed. the structural model proves to be reliable and satisfactory. as huber et al (2007) corroborate, figures exceeding 0.05 (huber et al., 2007) are to be suitable and valid. taking a look at table 5, confidence intervals as well as t-values are evidenced to deliver the evaluation of the path coefficient. after the analysis of every interval, to obtain 0 is not possible. on the other hand, the worth and weight of the latent variables is to be measured by path coefficients after analysis their relations. this figure should always remain above 0.2 to be suitable. for this study, an additional technique was employed, that is, bootstrapping, in order to estimate the suitability of pls and calculate standards errors of parametres. (in this precise case, 5,000 samples were used). table 5. confidence intervals original sample (o) average sample (m) 2.5% 97.5% ap -> bc 0.325 0.326 0.225 0.399 ap -> dpr 0.540 0.539 0.419 0.633 ap -> mte 0.456 0.447 0.344 0.543 bc -> mte 0.160 0.164 0.072 0.272 dpr -> mte 0.270 0.269 0.165 0.372 mev -> ap 0.138 0.175 0.005 0.259 mftv -> ap 0.213 0.221 0.042 0.362 mpv -> ap 0.281 0.297 0.087 0.423 sd -> ap 0.160 0.177 0.041 0.329 https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 96 every construct was proven to be suitable as well as indicators established, which were duly interrelated. the model of variables and hypotheses was exposed and analysed according to the pls path modeling which predicts the suitability of the constructs (hair et al., 2014). regarding the external variables, music with personal values (mpv), music with family/transcendental values (mftv), music with environmental values (mev), social distance (sd), actions of people (ap), development of personal rules (dpr) and behaviour change (bc) do actually and definitely converge into the internal variable, music as a tool for ethics (mte). endnotes 1 ought to exceed 0.70 for individual reliability (nunnally, 1994). 2 ought to exceed 0.70 for individual reliability (dijkstra & henseler, 2015). 3 values ought to exceed 0.50 to evaluate shared variance between cr & ave (fornell & larcker, 1981; henseler et al., 2015). 4. discussion and conclusions this research analyses and assessed the influence and prediction capacity of music and its intimate connection with ethics and engagement with people. after analysing 281 replies, a model was implemented to demonstrate how music could become a tool by connecting all variables of the model introduced to predict attitude change. by means of participative methodology and the model built, the constructs proved to have explanatory capacity (r 2 = 0.458) to show the relevant power of music, which is a tool to build emotions and reinforce ethics towards society and ourselves. firstly, all outer loadings were validated as they showed (λ > 0.7) for relevance. in addition, all constructs showed pertinent capacity (r 2 = 0.458). regarding confidence intervals (2.5% and 97.5%) these confirm robust relations in variance in all constructs of the model. the predictive capacity music as a tool for ethics (mte), the dependent variable, must be taken into account (q2= 0.248) as it fuses expected results. also, actions of people (ap) demonstrated better attitude towards behaviour change, towards personal, family, transcendental and environmental values. therefore, all hypotheses met the main aim of this research. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 97 the findings of this research coincide with those mentioned in the literature review which state that young people are music bond to such stage that social changes do actually take place and therefore these affect emotions and ethical values. the reliability provided by the model used show accurate responses in a way, confirming that music can actually influence different situations in the life of people, impacting on their ethics. the results presented how music can transform perceptions and promote attitude change in people. the following conclusions may be drawn: 1. responses obtained allowed the researcher to quantify social impact due the predicted attitude change after analysis. 2. being exposed to music with values may enhance emotions leading to attitude change on personal and social levels. 3. the social distance (sd) hypothesis proved to have a positive impact on actions of people (ap), leading to behaviour change (bc), and therefore determining that languages come with social beliefs that can be shared with music. 4. the engagement provided by music results in positive responses from those listening to music with ethics in the lyrics. 5. music can improve and change perceptions of issues in the world promoting behavior change and norm activation. 6. the significant level of acceptance of music as a consistent powerful instrument by those taking part in the questionnaire can help formulate strategies in various academic fields. 7. the model proposed offers a wide range of possibilities related to pedagogy strategies based on ethics connected and applied to subjects in any educational system. 8. music with personal values (mpv), music with family/transcendental values (mftv) and music with environmental values (lev), including the social distance (sd) factor, have a positive impact on actions of people (ap), leading to the development of personal rules (dpr) and affecting behaviour change (bc), therefore, the fact that music can be a tool for ethics (mte) can be confirmed. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 98 this research can be ended by posing the following question: can music be used for ethics and become an effective tool? certainly, it can. theoretical and practical implications as a theoretical implication, this research can be considered an instrument to corroborate past work carried out using same or similar variables, namely, music, youngsters and ethics. as a practical implication, it needs to be mentioned that awareness created in all the parties involved in the study is the main factor, however it could be of interest for any other branch of knowledge, as the variables used are general variables that come be intrinsic to human beings, therefore it would offer transcendence for some problems of practical nature: english music could be used in any educational system and age along with the ethics subject as it would improve not only language skills but also values to deal with any situation in life and make you feel good. limitations the limitations found in this study are related to personal self-belief values as a parental approach has been used all throughout. besides, intangibles as hypotheses are used and prediction levels are to be seen in the future. in addition, the location where the study was carried out was based solely in spain. recommendations for future research it would be recommended for future research to delve into different social and linguistics fields that deal with the use of profanity in music and the different paths and impact that this may have on young people and their behavior when dealing with ethical values. funding: “this research received no external funding”. conflicts of interest: “the author declares no conflict of interest”. authors must identify and declare any personal circumstances or interest that may be perceive. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 99 references adiantika, h., & purnomo, h. (2018). the implementation of task-based instruction in efl teaching speaking skill. indonesian efl journal, 4(2), 12-22. https://doi.org/10.25134/ieflj.v4i2.1371. al kandari, a., & al qattan, m. (2020). e-task-based learning approach to enhancing 21st-century learning outcomes. international journal of instruction, 13(1), 551-566. albino, g. (2017). improving speaking fluency in a task-based language teaching approach: the case of efl learners at puniv cazenga. sage open. https://doi.org/10.1177/2158244017691077 alcaraz varó, e. (2000). el inglés profesional y académico. alianza editorial. aubrey, j.s. (2006). effects of sexually objectifying media on self-objectification and body surveillance in undergraduates: results of a two-year panel study. journal of communication, 56, 366-386. https://doi.org/10.1111/j.1460-2466.2006.00024.x barclay, d., thompson, r., & higgins, c. (1995). the partial least squares approach to causal modeling: personal computer use as an illustration. technology studies, 2, special issues on research methodology, 2, 284-324. bogt, t., mulder, j., raaijmakers, q.a.w., & gabhainn, s.n. (2011). moved by music: a typology of music listeners. psychology of music, 39(2), 147-163. https://doi.org/10.1177/0305735610370223 carmines, e.g., & zeller, r.a. (1979). reliability and validity assessment, (vol. 17). sage publications. chin, w.w. (1998). the partial least squares to approach to structural equation modeling. modern methods for business research, 295(2), 295-336. christenson, p.g., de haan-rietdijk, s., roberts, d.f., & bogt, t.f.m. (2019). what has america been singing about? trends in themes in the u.s. top-40 songs: 1960-2010.psychology of music, 47(2), 194-212. https://doi.org/10.1177/0305735617748205 christenson, p.g., & roberts, d.f. (1998). it´s not only rock & roll: popular music in the lives of adolescents. hampton press. coyne, s., stockdale, l., nelson, d., & fraser, a. (2011). profanity in media associated with attitudes and behavior regarding profanity use and aggression. pediatrics. https://doi.org/10.1542/peds.2011-1062 creswell, j.w., & clark, v.l.p. (2017). designing and conducting mixed methods research. sage publications. de groot, j.i.m., & steg, l. (2009). morality and prosocial behavior: the role of awareness, responsibility and norms in the norm activation model. journal of social psychology, 149, 425-449. https://doi.org/10.3200/socp.149.4.425-449 de la rosa herrera, k., & publiese, r. (2017). the uses and gratifications of music among emerging adults. international journal of arts & sciences, 10(1), 351-364. dewall, c.n., pond, r.s., jr., campbell, w.k., & twenge, j.m. (2011). tuning in to psychological change: linguistic markers of psychological traits and emotions over time in popular u.s. song lyrics. psychology of aesthetics, creativity, and the arts, 5(3), 200-207. https://doi.org/10.1037/a0023195 dietz, t., stern, p.c., & rycroft, r.w. (1989). definitions of resources: the case of environmental risk. sociological forum, 4, 47-70. https://doi.org/10.1007/bf01112616 dijkstra, t.k., & henseler, j. (2015). consistent partial least squares path modeling. mis quarterly, 39(2). https://doi.org/10.25300/misq/2015/39.2.02 do, s.l., & schallert, d.l. (2004). emotions and classroom talk: toward a model of the role of affect in students’ experiences of classroom ciscussions. journal of educational psychology, 96(4), 619-634. https://doi.org/10.1037/0022-0663.96.4.619 https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.25134/ieflj.v4i2.1371 https://doi.org/10.1177/2158244017691077 https://doi.org/10.1111/j.1460-2466.2006.00024.x https://doi.org/10.1111/j.1460-2466.2006.00024.x https://psycnet.apa.org/doi/10.1177/0305735610370223 https://doi.org/10.1177/0305735617748205 https://doi.org/10.1177/0305735617748205 https://doi.org/10.1542/peds.2011-1062 https://psycnet.apa.org/doi/10.3200/socp.149.4.425-449 https://psycnet.apa.org/doi/10.1037/a0023195 https://doi.org/10.1007/bf01112616 https://www.researchgate.net/deref/http%3a%2f%2fdx.doi.org%2f10.25300%2fmisq%2f2015%2f39.2.02 https://psycnet.apa.org/doi/10.1037/0022-0663.96.4.619 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 100 flowerdew, j., & peacock, m. (2001). research perspectives on english for academic purposes.cambridge university press. http://dx.doi.org/10.1017/cbo9781139524766 fonseca-mora, m.c. (2000). foreign language acquisition and melody singing. elt journal, 54(2), 146-152. fonseca-mora, m.c., toscano-fuentes, c., & wermke, k. (2011). melodies that help: the relation between language aptitude and musical intelligence, vol. 22, no. 1, 101-118. fornell, c., & bookstein, f.l. (1982). two structural equation models: lisrel and pls applied to consumer exit-voice theory. j. mark. res. 19, 440-452. https://doi.org/10.1177/002224378201900406 fornell, c., & larcker, d.f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. https://doi.org/10.2307/3151312 geisser, s. (1974). a predictive approach to the random effects model. biometrika, 61, 101-107. gentile, d.a., anderson, c.a., yukawa, s., ihori, n., saleem, m., ming, l.k., & sakamoto, a. (2009). the effects of prosocial video games on prosocial behaviors: international evidence from correlational, experimental, and longitudinal studies. personality and social psychology bulletin, 35, 752-763. https://doi.org/10.1177/0146167209333045 gonzález davies, m., & celaya villanueva, m.l. (1992). new teachers in a new education system: a guide book for the reforma. ppu. graddol, d. (1997). the future of english? british council. greitemeyer, t. (2009a). effects of songs with prosocial lyrics on prosocial thoughts, affect, and behaviour. journal of experimental social psychology, 45, 186-190. https://doi.org/10.1016/j.jesp.2008.08.003 greitemeyer, t. (2009b). effects of songs with prosocial lyrics on prosocial behaviour: further evidence and a mediating mechanism. personality and social psychology bulletin, 35(11), 1500-1511. https://doi.org/10.1177/0146167209341648 hair, j.f., ringle, c.m., & sarstedt, m. (2013). partial least squares structural equation modeling: rigorous applications, better results and higher acceptance. long range planning, 46(1-2), 1-12. https://doi.org/10.1016/j.lrp.2013.01.001 hallam, s. (2015). the power of music: a research synthesis of the impact of actively making music on the intellectual, social and personal development of children and young people. ed. institute of education university college han, byun-c. (2014). psicopolítica: neoliberalismo y nuevas técnicas de poder. herder. hariri, a.r., bookheimer, s.y., & mazziotta, j.c. (2000). modulating emotional responses effects of a meocortical network on the limbic system. neuroreport, vol. 11, issue 1, 43-48. https://doi.org/10.1097/00001756-200001170-00009 henseler, j., ringle, c.m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115135. https://doi.org/10.1007/s11747-014-0403-8 henseler, j., ringle, c.m., & sinkovics, r.r. (2009). the use of partial least squares path modeling in international marketing. advances in international marketing, 20, 277-319. https://doi.org/10.1108/s1474-7979(2009)0000020014 hoinville, g., & jowell, r. (1978). survey research practice. heinemann educational books. ilisko, d., & kravale-paulina, m. (2015). sustainability of family values through the generation as viewed by the bachelor program students. society, integration, education. proceedings of the international scientific conference. https://doi.org/10.17770/sie2013vol1.1541 izard, c.e. (2002). translating emotion theory and research into preventive interventions. psychological bulletin, 128(5), 796-824. https://doi.org/10.1037/0033-2909.128.5.796 jackson, s. (2014). how to respect yourself and others. good choices, good life. https://tinyurl.com/y8j6ecw5 https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://dx.doi.org/10.1017/cbo9781139524766 https://doi.org/10.1177/002224378201900406 https://psycnet.apa.org/doi/10.2307/3151312 https://doi.org/10.1177/0146167209333045 https://doi.org/10.1016/j.jesp.2008.08.003 https://doi.org/10.1177/0146167209341648 https://doi.org/10.1016/j.lrp.2013.01.001 https://doi.org/10.1097/00001756-200001170-00009 https://psycnet.apa.org/doi/10.1007/s11747-014-0403-8 https://doi.org/10.1108/s1474-7979(2009)0000020014 https://doi.org/10.17770/sie2013vol1.1541 https://doi.org/10.1037/0033-2909.128.5.796 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 101 juslin, p.n., liljeström, s., västfjäll, d., & lundqvist, l.o. (2010). how does music evoke emotions? exploring the underlying mechanisms. in p. n. juslin & j. a. sloboda (eds.), series in affective science. handbook of music and emotion: theory, research, applications, pp. 605-642. oxford university press. kachru, b.b., & nelson, c.l. (2001). world englishes. in a. burns & c. coffin (eds.). analysing english in a global context, 9-25. routledge. liere, k., & dunlap, r. (2006). moral norms and environmental behavior: an application of schwartz's norm‐activation model to yard burning1. journal of applied social psychology, 8, 174-188. https:/doi.org/10.1111/j.1559-1816.1978.tb00775.x maslow, a.h. (1954). motivation and personality. harper & row. mclean, a.c. (1980). destroying the teacher: the need for a learner-centred teaching. e.t. forum, 28:3, 16-19. nicolás, g., & azorín, c. (2013). música y valores. una relación educativa ineludible. música y educación: revista trimestral de pedagogía musical, issn 0214-4786, año nº 26, nº 93, 16-25. nunnally, j.c. (1994). psychometric theory 3e. tata mcgraw-hill education. perrinjaquet, a., furrer, o., marguerat, d., usunier, j.c., & cestre, g. (2007). a test of circumplex structure of human values. journal of research in personality, 41, 820-840. https://doi.org/10.1016/j.jrp.2006.10.006 ringle, c.m., wende, s., & becker, j.m. (2015). smartpls 3. smartpls gmbh, boenningstedt. http://www.smartpls.com rokeach, m. (1973). the nature of human values. free press. sagiv, l., roccas, s., cieciuch, j., & schwartz, s.h. (2017). personal values in human life. nature human behaviour, 1. https://doi.org/10.1038/s41562-017-0185-3 sam, d.l, & berry, j.w. (2010). acculturation: when individuals and groups of different cultural backgrounds meet. perspectives on psychological science, 5(4), 472-481. https://doi.org/10.1177/1745691610373075 sánchez gonzález, m.g (2021). la música como herramienta de formación ética en los jóvenes. universidad de extremadura. cáceres. https://dehesa.unex.es/handle/10662/12659 sánchez gonzález, m.g (2021) social distance through music lyrics in efl students. multidisciplinary journal for education social and technological sciences 8(2), 42-59 https://doi.org/10.4995/muse.2021.15014 schellenberg, e.g., & von scheve, c. (2012). emotional cues in american popular music: five decades of the top 40. psychology of aesthetics, creativity, and the arts, 6(3), 196-203. https://doi.org/10.1037/a0028024 schumann, j. (1994). where is cognition? emotion and cognition in second language acquisition. in studies in second language acquisition, vol. 16, issue 2, 231-242. cambridge university press. schwartz, s.h. (1977). normative influences on altruism. in l. berkowitz (ed.). advances in experimental social psychology, vol. 10, 221-279. academic press. schwartz, s.h. (1994). are there universal aspects in the structure and contents of human values? journal of social issues, 50, 19-45. https://doi.org/10.1111/j.1540-4560.1994.tb01196.x schwartz, s.h., & howard, j.a. (1981). a normative decision-making model of altruism. in j.p rushton & r.m. sorrentino (eds.), altruism and helping behavior, 89-211. erlbaum. sortheix, f., parker, p., lechner, c., & schwartz, s. (2017). changes in young europeans' values during the global financial crisis. social psychological and personality science, 10(1). https:/doi.org/10.1177/1948550617732610 stone, m. (1974). cross-validatory choice and assessment of statistical predictions. journal of the royal statistical society, 36, pp. 111-147. https://doi.org/10.1111/j.2517-6161.1974.tb00994.x https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1111/j.1559-1816.1978.tb00775.x http://refhub.elsevier.com/s0191-8869(13)01389-5/h0130 https://dialnet.unirioja.es/servlet/revista?codigo=975 https://dialnet.unirioja.es/servlet/revista?codigo=975 https://dialnet.unirioja.es/ejemplar/323146 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0140 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0140 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0140 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0140 https://www.researchgate.net/deref/http%3a%2f%2fdx.doi.org%2f10.1016%2fj.jrp.2006.10.006 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0145 https://doi.org/10.1038/s41562-017-0185-3 https://doi.org/10.1177/1745691610373075 https://dehesa.unex.es/handle/10662/12659 https://www.researchgate.net/journal/multidisciplinary-journal-for-education-social-and-technological-sciences-2341-2593 https://www.researchgate.net/journal/multidisciplinary-journal-for-education-social-and-technological-sciences-2341-2593 http://dx.doi.org/10.4995/muse.2021.15014 https://psycnet.apa.org/doi/10.1037/a0028024 http://refhub.elsevier.com/s0191-8869(13)01389-5/h0160 vol9n2/human%20values?%20journal%20of%20social%20issues,%2050,%2019-45 vol9n2/human%20values?%20journal%20of%20social%20issues,%2050,%2019-45 https://psycnet.apa.org/doi/10.1111/j.1540-4560.1994.tb01196.x https://doi.org/10.1177/1948550617732610 https://doi.org/10.1111/j.2517-6161.1974.tb00994.x multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17497 social and technological sciences e-issn: 2341-2593 sánchez-gonzález (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 81-102. https://doi.org/10.4995/muse.2022.17497 102 taruffi, l., & koelsch, s. (2014). the paradox of music-evoked sadness: an online survey. plosone. https://doi.org/10.1371/journal.pone.0110490 taylor, s.j., & bogdan, r. (1984). introducción a los métodos cualitativos. ediciones paidós. zillmann, d., & gan, s.-l. (1997). musical taste in adolescence. in d. j. hargreaves & a. c. north (eds.), the social psychology of music, 161-187. oxford university press. https://doi.org/10.4995/muse.2022.17497 https://doi.org/10.4995/muse.2022.17497 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1371/journal.pone.0110490 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 54 can we improve academic performance and student satisfaction without additional time cost for teachers? evidence from a blended methodology in microeconomics anett erdmann, esic business and marketing school, anett.erdmann@esic.edu, alfonso jesús torres marín, esic business and marketing school, alfonso.torres@esic.edu abstract purpose: the main goal of this paper is to determine if the use of a blended methodology can improve performance and satisfaction of the students, with no additional time cost for teachers. as a second objective, the article attempts to explain observed differences across students in the effect of the methodology on study time based on the theory of optimal decision making. finally, we sketch a simple cost benefit analysis for the digital learning platform (dlp) used. design: the teachers combined the traditional classes methodology with the adoption of a dlp in two courses of microeconomics at an undergraduate level at esic business & marketing school. subsequently it is analyzed the impact of this methodology on student satisfaction and performance, as well as on student’s study time using different analytical tools. findings: students’ grades, at the final exam, increased in a significative way as they spent more time with the dlp and/or when they do more digital assignments at home. their satisfaction with the blended methodology, and the use of the dlp was quite high for most of students. their feedback on working time relative to traditional methods showed two mailto:anett.erdmann@esic.edu mailto:alfonso.torres@esic.edu multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 55 extremes, either studying much more or much less. we provide a theoretical explanation for this observation, based on microeconomic theory. a cost-benefit analysis of the dlp tool at an institutional level suggests that its economic costs are more than justified by the economic benefits of the tool in terms of student’s satisfaction, brand reputation and teachers time saving. contribution: this document provides a methodology to measure the benefits of an innovative learning methodology using relevant indicators and employing advanced statistical techniques as regression analysis. it also helps us to understand student’s behavior in the face of an educational innovation based on technology. the findings are in line with economic theory. keywords: blended learning, technology-enhanced learning, virtual labs, educational innovation, computer software on education. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 56 1. background despite the importance of innovation in the pre-university stage (sangrá, 2016) (rodríguez miranda, 2009) (tornero & pi, 2013) and in certain university areas such as engineering, mathematics or biology (beard, 2017) (bravo, guerrero and lópez, 2011), in economics there are not many contributions of innovation in the university environment. table 1 lists elements that may affect blended education in economics. as shown in table 1, there are new elements to consider in economics education, that can be used to improve results from a blended perspective. among them we can consider: (1) various types of instruments, (2) simulations, (3) transversal applications (4) use of different places. table 1: new elements to consider in economics teaching and the learning outcomes of esic students instruments that can be used for economics training in spanish universities and specifically in esic. new teaching and learning tools and new technologies. physical means of support for learning adaptation of the classrooms. use of multimedia tools, social networks, blogs and virtual platforms. new teaching methodologies such as gamification, metacognition, flipped classroom. simulations in the learning of various disciplines of economics learning supported by professional scenography’s. experiences of professional praxis exercises based in the resolution of problems business games simulators multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 57 transversalities to improve the learning outcome of economics teaching synergies with other areas of knowledge such as sociology, geography, business, demography, law, etc. teacher synergies with other public and private institutions. teacher synergies with professors or students from other spanish and foreign universities. places of learning about the economy. classrooms, libraries, homes and any other place with internet access. source: (torres, gonzalez & bordonado, 2018) as shown in table 1, there are new elements to consider in economics education, that can be used to improve results from a blended perspective. among them we can consider: (1) various types of instruments, (2) simulations, (3) transversal applications (4) use of different places. new tools and learning technologies should be considered among the new instruments to be used. multimedia tools, social networks, virtual platforms, as well as methodologies such as gamification, metacognition or flipped classroom stand out in this first category. in simulations in the learning, it can be included learning systems based on professional scenography, professional practice experiences, problem-based learning or business games simulators. cross-cutting techniques should also be considered to improve learning outcomes in economics. for this, can be used synergies with other areas of knowledge such as sociology, geography, business administration, demography, advocacy, etc. -, with other institutions -publics or privatesor with professors or students from other spanish or foreign universities. finally, different places can be used to learn about economics – classrooms, libraries, homes, and any other place with internet access. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 58 in many occasions some subjects in economics are not the most preferred ones for students, they find them difficult and consequently they are not very much motivated to study (becker & greene, 2001). sometimes these courses are criticized for being overly theoretical, uncreative, non-innovative and far from the real world (liu & xie, 2019). when this happen students simply get lost and fail. they do not acquire the knowledge and skills they should. in other cases, the courses become too mechanic, like receipts that people apply without understanding what they are really doing. to improve the previous situation, it is very helpful the use of innovative methodologies and techniques, that help students to improve their learning results and scoring through a learning by doing methodology using digital interactive tools (lisitsyna, efimchik, & izgareva, 2017). the use of new technologies allows students to better understand concepts that are difficult to explain, such as the level of confidence of the impact of an advertising campaign on sales, the sensibility of the return of one stock to the changes in the market returns or the impact of education over salaries (hattinger, engeström, & sannino, 2018). blended learning is a method commonly used by students from all over the world, either because the professor proposes it in class or autonomously as a support to the teacher's explanations (medina, medina, & rojas, 2016). in any case, the use of digital platforms, which contain specific material by areas or blocks, has spread globally, with the use of information and communication technology (ict) as a key element for learning (pérez, urbano, & onías, 2017). the easy access to the different digital platforms, the volume of contents they provide, the immediacy of the information, as well as the guided learning and the support and control of the teacher, make the mixed methodology very effective for an achievement of the objectives of learning established in any course (aguirre, quintana, & miranda, 2015). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 59 currently only in moodle (moodle.org, 2018) there are 130,037,486 registered users, of which 8,197 are registered in spain, and in miriadax (the largest online platform of spanish language courses worldwide) there are 4,071,438 registered students worldwide (miríadax_, 2018). nowadays we also observe the implementation of other learning management systems in spain, like for instance canvas, implemented in the current academic year by a business school (cunef). on the other hand, publishers have gradually adapted to the new reality by incorporating digital materials into their catalogue, although in many cases, this adaptation usually includes the digitalization of the manual following a traditional structure, enriched with self-evaluating activities and additional resources (castro -rodríguez, de castro calvo, & hernández rivero, 2017). the editorial pearson for instance uses different learning management systems. while the english platform mylab – which we use in this teaching experiment – has been developed 100% by pearson, the spanish version of mylab (imi) is based on sakai (source: pearson office). another characteristic of this type of training is the degree of involvement required by the students. they can organize and plan their work, identify and solve problems, decide their work time, as well as evaluate their own progress (fernández, 2005). this is significant considering that the labor market in spain calls for university education, not only a more practical training aimed at the company, but a proactive and decisive attitude towards work, with notable disinterest and the tendency to mechanize the tasks assigned to young graduates (alonso, fernández rodríguez, & nyssen, 2009). the dlp is a teaching innovation initiative that apply the technological advances to interactive teaching based on the learning by doing methodology (efimchik, ivaniushin, kopylov, & lyamin, 2017). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 60 for all the previous reasons, it is not surprising that the use of this type of methodologies is increasingly implemented in the university sphere (brown, 2016). in this research we focus on a practical case of enabling and enhancing blends methodologies (bonk & graham, 2012) carried out at esic. the theory of optimal decision making and the impact of new educational technologies on study time as we will see in the findings of this research, there are strong evidences about two extreme behaviors of students with respect to their working time. some of them suggest that they have work much more, meanwhile some others have work less than under traditional methods. how can we explain this pattern? in this section, we will base on the theory of optimal decision making to explain how the impact of new methodologies on study time may differ that much across individuals study effort is in general difficult to measure. one possibility to think of working effort is in terms of the time spend on the studying for a subject. an alternative way of quantifying working effort is the amount of work, in terms of exercises, that have been realized, to which we will come back later. for the moment let us focus on the study time. in the following paragraphs we will give a short overview on the existing theory and empirical evidence on working time and performance, and later we will formulate our hypothesis about the effect of the dlp based on the theory of optimal decision making. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 61 first, let us consider evidence based on standard teaching methodology and the corresponding theory. in general, the literature on the effect of study time on grades find only a week relationship that sustains the common perception that studying more translates into a higher grade (gortner lahmers & zulauf, 2000). for a detailed overview of these studies see plant et al. (2015), who analyze the effect of the choice on study time of university students at florida state university on the official university records. the authors identify study-time as a significant predictor of the grade, when controlling for other variables like attendance, study environment or planning which are supposed to determine deliberate practice. in general, the relationship between practice and performance is sketched by ericcson (2002) as a process of practice that takes time to yield effects in performance. figure 1 illustrates the expected performance of professionals, e.g. musicians, increasing with practice quite fast but once a certain experience has been acquired additional practice increases performance at a lower rate until a certain limit and may even turn down once physical maturity has been reached. while the representation by ericsson refers to the career path over the working age, a similar relationship holds when considering the performance within the university career or at an even smaller scale at the course level. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 62 figure 1. relationship between performance and practice source: ericcson (2002), figure 2.2, p. 25 figure 2 represents an adaptation of the relationship between performance and practice to the academic performance within a university course, in line with the model of optimal study time by the core team (2017) and applied to our research question. the figure sketches the expected relationship between the exam score and the time studied under standard teaching methodologies. if students dedicate some time to study, the learning effect is large and is expected to be reflected in the exam score. however, as students dedicate more time to study, the effect of studying an additional hour on the exam score becomes smaller as dedicated hours increase, which is a standard learning effect. this outlined relationship between exam score and study time or number of assignments realized can be conceptually interpreted as a “score-production function”, that is, the production of exam score as a function of the study time or an alternative measure of study effort. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 63 figure 2. adaptation to academic performance based on study time source: self-made note that there are limits of improving performance. for instance, studying a few hours in a row without break, you will notice how your concentration decline and reach a point where you cannot assimilate more content. likewise, the innate capability of each students for a particular task determines the limit where studying additional time will not be reflected in the performance. second, based on the existing empirical and theoretical evidence on the trade-off between study-time and exam grade as well as the self-reported information by the students, we set up our hypothesis about the effect of the introduction of mylab as complementary learning tool. in the next section we will test the hypothesis with objective data. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 64 figure 3. adaptation to academic performance based on study time: expected effect of dlp method source: self-made consider working with a dlp tool, such as mylab, which provides direct feedback and offers walk-through options such that a certain performance can be reached with in less time. analogue individual tutorial hours. figure 3 illustrates this argument through a shift of the score-production function upwards, which is represented as a dashed line. in this context, the expected effect of working with the dlp tool mylab is interpreted as a technology improvement in learning techniques which allows to improve performance. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 65 this hypothesis is in line with the recent literature on deliberate practice. for instance, ericsson (2002), who analyzes the effect of the amount of high-quality practice accumulated during individuals’ careers on the performance, with reference to fields like practice of musicians, sportys, mathematicians etc. third, given the relationship between work effort and exam score, let us consider the decision a student face. we all have a time constraint of 24 hours a day and we have to make a choice how to employ our time to achieve our personal objectives. given that time is scarce, this implies for students that spending more time on a subject in order to increase performance – as outlined before – students have to scarify time for other activities, for instance studying for another course or simply enjoying free time. this tradeoff between study time for a subject and free time (or alternative activities) is a key concept in economics in optimal decision making. an intuitive and narrative introduction of this trade-off as a first model of optimal decision making can be found in the opensource economic book by the core team (2017). the optimal decision about effort of studying which translates according to the “scoreproduction function” into a higher exam score is an individual cost-benefit analysis by the students. while the potential benefits (e.g. improved understanding, complementary contents, better performance in exams) are easy to identify, there have been no monetary costs for students to use the platform. however, given the time constraint of students, they incur implicit costs known as opportunity cost, the best alternative activity they give up when dedicating time on studying with mylab. this implies that the implementation of the platform has two effects on students: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 66 1. “time-win effect”. the efficiency of learning liberates time. this additional free time can be employed to study more and climb up the learning curve and improve performance. 2. “substitution effect”. the increased efficiency of studying with the platform implies an increase of the opportunity costs of study time. the substitution effect leads to less study time achieving the same performance as under traditional methods (“least-effort”). which of the two effect dominates is an empirical question, that we analyze in the findings section of this article. 2. purpose the main goal of this paper is to determine if the use of a blended methodology can improve performance and satisfaction of the students, with no additional time cost for teachers. as a second objective, the article attempts to explain observed differences across students in the effect of the methodology on study time based on the theory of optimal decision making. finally, we sketch a simple cost benefit analysis for the digital learning platform (dlp) used. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 67 3. methodology we adopted the digital learning platform mylab from pearson for two groups of the course of microeconomics in the degree of business administration at esic business and marketing school. this tool helps to create hybrid courses which require online homework as well as digital content delivery (ahmed, ohkubo, limaye, & ballard, 2017). it provides students with a larger number of practical exercises, with simultaneous guidance and in time feedback during the learning process. simultaneously, it generates other benefits as dedicate more time in class to some other activities with more added value for students – as discussions of current topics, real-world application videos, simulations, experiments, critical thinking, work group and group discussions and questions and to free more time for professor to dedicate to other professional activities. the tools offered by the platform are very flexible, and could be personalized by the professor so that: a) students may dedicate a personalized number of hours to work and solve exercises, according to their needs. b) you can offer them different level of help, modify the deadline, or the scoring of the different problems. c) you may use them for practice, to consolidate some ideas (without scoring the students) or for testing and scoring them. d) you may do it at home, or in class, e) you have complete control: you may know individual results, and group result for individual questions, sections or topics. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 68 f) you can implement different help programs directed to individuals that do not understand something (they had poor results in some questions) or to solve general doubts when an important proportion of the class is having a poor performance in some specific questions or topics. students have immediate feedback in case they need help. (“walk through options, “help”) as a final consequence of all the above-mentioned advantages, we expect students to be better oriented and learn more in quantity and/ or in quality, and this way improve their results, their learning experience and satisfaction and all this without increasing the time dedicated by the professor. respective the time dedicated by students, we were not sure about the effect of mylab on the time dedication of students. on the one hand, different features like the instantaneous feedback, the walk-through options and the help option allow students to solve problems and avoid being stuck, such that they may engage with this feature and use the additional free time to study more and improve performance – that could be denominated “time effect”-. on the other hand, a more efficient learning increases the opportunity costs of the time dedicated to learning and the substitution effect leads to less effort achieving the same result as under traditional methods. which of the two effects dominate depends on student individual preferences. we adopted mylab in 2 different groups, in total 50 second year students, during the first semester of 2018-2019, taught by different professors. in group 2 the professor assigned 6 compulsory homework’s in mylab and two partial exams were realized in the platform under observation in computer rooms, such that 95% of the evaluation of the course was realized through mylab. in group 1 the professor assigned only 4 compulsory multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 69 homework’s, but students were motivated to realize additional training using the individual study plan offered by the platform before doing the assignments. in the latter group the exams were realized on paper. implementation the software is used by students working at home (and in class occasionally) on a personal computer on the cloud. students use my lab for understanding content, applying principles to the real world, homework assignments, and testing. our goals by assigning tasks through my lab are to complement the teaching of new concepts in the classroom by providing homework and practice opportunities, help students assess their own understanding of the course material, offer the possibility to deepen knowledge in topics they are especially interested in, and track their progress, and finally identify students who are at high risk of failing (hinerasky, fahr, & sliwka, 2017). as course instructors our role is to assign content, homework and assessment in my lab and provide support and remote monitoring to students using the program at home while the feedback and walk-through explanation of concrete exercises is directly and instantaneously provided by mylab. while students are often working remotely, the course is not self-paced as defined due date are firm. we encouraged students to be responsible for their own learning experience. after finishing each topic students were supposed to do some study planning (practice and quiz with no scoring), and subsequently the assignments that were scored. to encourage students to do assignments, their evaluation represents 10% of the final course scoring. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 70 4. findings to analyze the results two kinds of dataset have been used: (1) a first group of data has been directly extracted from mylab and (2) a second group proceed from an individual student survey at the end of the semester. 4.1 the students experience this section presents the qualitative results from the individual student survey passed to all the students at the end of the semester. responses from a voluntary survey of students indicate (see table 1) that most students recognize the value of my labs which reinforces the hypothesis established in this paper about the satisfaction of the students with the methodology and the tool. specifically, we want to highlight the following overall results from that survey respective the dlp: a) 78% of students would recommend the use of the platform to other students of microeconomics. b) 80% of students strongly agree or agree that they liked the direct and instantaneous feedback for the digital assignments. c) 76% of students strongly agree or agree that the additional resources of the platform have helped them to learn more than in a traditional course. d) 80% of students agree or strongly agree that their understanding of the teacher presentation has increased because of using the digital platform. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 71 the detailed results of the survey are provided in table 2. table 2: students opinion about the use of the digital platform in microeconomics questions strongly disagree disagree agree strongly agree 1.my digital understanding of the teacher presentations has increased because of using the digital platform 12,2% 7,3% 61,0% 19,5% 2.because of working with the digital platform i have worked more 14,6% 17,1% 24,4% 43,9% 3.using the digital platform provided additional resources to learn more than i would have from a traditional course 7,3% 17,1% 39,0% 36,6% 4.i found it easy to get used to mylab (e.g. web navigation, drawing tools for assignments) 12,2% 22,0% 29.3% 36,6% 5.i liked the direct and instantaneous feedback for the digital assignments 2,4% 17,1% 29,3% 51,2% 6.because of working with the digital platform i have liked the subject more 22,0% 31,7% 26,8% 17,1% 7.the use of pearson digital platform has impacted positively my exam score 22,0% 14,6% 24,4% 39,0% 8.i would recommend using pearson digital platform to other students taking this course 9,8% 12,2% 34,1% 43,9% source: self-made in general, the results from the survey suggest that students are satisfied with the dlp tool, which should not be confused with the fact whether they like the subject, which in turn doesn’t seem to be altered by the use of the digital tool. especially the direct and instantaneous feedback by the platform seems to be valued by the student, which is reinforced by the student comments on the additional question what they liked most about using the digital platform. in the following we provide some of the representative student statements: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 72 • “i like that when some question is wrong, it is clearly explained with instant feedback” • “explanations when exercises are not done in the correct manner” • “i could work whenever i wanted and that if i had a question it could be answered whenever i had the question” • “available to give us the correct answer and explain it in the moment. easier and more handful” all these observations are in line with pérez et al. (2017), who state that some of the advantages of digital platforms is the immediacy of information and guided learning. our survey highlights these two aspects when using mylab in terms of direct and instantaneous feedback for assignments and emphasize the “walk-through” help function. respective the usage of the dlp in terms of working time, the results from the survey suggest differences across groups. at the overall level, 43,9% of all students strongly agree that they have worked more using the platform and 14,6% of all students strongly disagree. considering the results at the group level, it calls attention that especially in group 1 the answers are polarized, which is represented in graph 1. while 42% of the students disagreed or strongly disagreed on having worked more, 58% agreed or strongly agreed to have worked more. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 73 graph 1. answer to question: because of working with the digital platform i have worked more. group 1. source: self-made additional to the scale-questions in table 2, we asked for the student’s willingness to pay for the use of the platform (which is analyzed in detail in section 4) and about the usage of book references. respective the literature used in the course, the answers reveal that 66% of the students accessed the online book provided in mylab often (at least 5 times during the term) while only 10% of the students had a look at the printed version of the book. all the previous results strongly suggest that our hypothesis about the positive effect of new educational technologies (dlp) on student performance may be confirmed. 0,00 0,10 0,20 0,30 0,40 0,50 i strongly disagree i disagree i agree i strongly agree multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 74 4.2. the student performance as said before, one of the main goals of this paper is to determine if the use of a blended methodology can improve performance of the students. more specifically we claim as a relevant hypothesis that the average score of exams depends on the training time spend in the study plan or the number of assignments realized. in order to test our hypothesis about the score-production function and to identify the opportunity cost, we make use of the data extracted from mylab, in particular, we analyze the number of assignments that the students did during the semester – as a means of quantity worked – and the time spent in the study plan – as a means of the time dedicated to study. table 3 provides some summary statistics. given that there are many potential differences between the two groups, the analysis is realized at a group level. the analysis with two groups can be understood as sensitivity analysis and is expected to reinforce the results based on small groups. note that the average exam grade in both groups is similar, as well as the average pearson score (score obtained in the mylab assignments). however, this is not the case for the way students made use of the platform. in group 1, 25 out of the 28 students that composed it made active use of the study plan and realized on average 2,96 out of 4 compulsory assignments. in group 1, only 3 of the 22 students made use of the study plan but students were assigned more compulsory exercise sets and they realized on average 4,73 out of 6 assignments in total (including a first practice assignment) and 3,90 compulsory assignments out 5. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 75 table 3. summary statistics mylab. variable group 1 group 2 number of enrolled students 28 22 average time spend in the study plan 7,96 hours (12,21) na average number of compulsory assignments realized 2,96 (1,45) 3,90 (1,26) average pearson score (over 10) 5,69 (2,54) 6,30 (2,29) average exam score (over 10) 7,10 (1,34) 7,36 (1,14) *standard deviations in brackets. source: self-made the graph 2 shows the observations of time spent in the study plan and the corresponding exam grade (and pearson grade respectively) and the adjusted score-production function as a fractional polynomial prediction plot for group 1. the data suggest a positive relationship between the time spent in the study plan by every student and the score in the final exam, with a steep initial increase of performance as study time increases but only a small performance effect if the time spend in the study plan is larger than 14 hours. note that these observations are in line with the theoretical framework from the previous section suggesting a decreasing marginal effect of study time on the exam grade. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 76 graph 2. relation between score and time dedicated previously to do exercises in the study plan. group 1 source: self-made given the small class size of 25 students, the results have to be interpreted with caution but can be understood as a first experimental result. in order to quantify the impact of the time spent in the study plan (for students of the group 1) on the student performance or which is the same, the opportunity costs of study time using mylab -, we provide a simple ols analysis, which main results we explain in the following lines. the model that was estimated respond to a simple regression as reflected in the next equation: 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 = 𝛽𝛽0 + 𝛽𝛽1(𝑇𝑇𝑇𝑇𝐸𝐸𝑠𝑠 𝑇𝑇𝑖𝑖 𝑀𝑀𝑀𝑀𝑀𝑀𝐸𝐸𝑀𝑀 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑀𝑀 𝑝𝑝𝑝𝑝𝐸𝐸𝑖𝑖) + ԑ equation 1.1 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 77 the results obtained are (with standard errors in brackets): 𝐸𝐸[𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠] = 6,74 + 0,04(𝑇𝑇𝑇𝑇𝐸𝐸𝑠𝑠 𝑇𝑇𝑖𝑖 𝑀𝑀𝑀𝑀𝑀𝑀𝐸𝐸𝑀𝑀 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑀𝑀 𝑝𝑝𝑝𝑝𝐸𝐸𝑖𝑖) (equation 1.2) (0,31) (0,02) the coefficients of the model show us that the time spent in the study plan significantly increases the exam grade. spending one additional hour in the study plan increases the score in the exam by 0,04 points. the determination coefficient of the model (r2) is 15,1%, which may be interpreted as follow: the time dedicated to the study plan explains 15.1% of the variations in the final grade of the students with respect to the average. in none of the groups we find a significant effect of the score in the assignments on the exam score, which suggests that it is really the training that matters – in terms of the number of problems solved or the time spend in the study plan – rather than a matter of general analytical ability. complementary, to verify the sensitivity of our results, we consider the relationship between the number of assignments realized during the course and the performance in the exams. as indicated previously, due to differences in the implementation of the dlp approach by the different professors, in group 2 almost no student used the study plan, but students were assigned more homework’s such that they practiced directly with the scored online assignments. hence, for this group we measure work effort in terms of the number of assignments realized before the exam. we expect a positive relationship multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 78 between the number of exercise sets realized and the exam score. to test this hypothesis, likewise we use a simple ols regression as follows: 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 = 𝛽𝛽0 + 𝛽𝛽1𝐴𝐴𝑠𝑠𝑠𝑠𝑇𝑇𝐴𝐴𝑖𝑖𝐸𝐸𝑠𝑠𝑖𝑖𝑠𝑠𝑠𝑠 + ԑ (equation 2.1) the previous equation helps us to quantify the impact of number of assignments done by students of group 2 during the semester on their performance the model that was estimated respond to a simple regression as reflected in the equation below. the figures between brackets represent the standard deviation . 𝐸𝐸[𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠] = 5,46 + 0,46𝐴𝐴𝑠𝑠𝑠𝑠𝑇𝑇𝐴𝐴𝑖𝑖𝐸𝐸𝑠𝑠𝑖𝑖𝑠𝑠𝑠𝑠 (equation 2.2) (0,84) (0,19) we found that the exam score (average of the two partial exams in mylab) significantly improves with the number of assignments realized, which confirms our motivation of the implementation of mylab. the coefficients of the model show us that on average, the realization of an additional compulsory assignment improves the exam score in 0,46 points (out of 10). the determination coefficient of the model (r2) is 29,5%, which may be interpreted as follow: the number of assignments done explains 29,5% of the variations in the final grade of the students with respect to the average. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 79 the comparison between the determination coefficients of equation 1.2 and 2.2, (15,1% versus 29,5%) seems to suggest that the explanatory power of the number of assignments performed exceeds the correspondent to time dedicated to carrying out study plans), although we cannot discard some differences due to the diverse methodological approach followed by each one of the teachers. 4.3 the effect of the digital learning platform mylab on study time this section presents the effect of the digital learning platform mylab on study time, helping us to verify the hypothesis about the different impact of the new educational technologies on study time across individuals. the descriptive data of the student survey, represented in graph 1, suggest that, as mentioned between our hypothesis, there is heterogeneity between students, that is, preferences about study time and scoring differ across students. moreover, given the differences between groups we believe that the instructors can influence the dominance of one effect or other through the implementation of the features offered in one way or another, which remains to be investigated further. the answers to the survey made to students – already mentioned in epigraph 4.1 suggest two extremes of reported working time: (1) strongly agree with having worked more or (2) strongly disagree with having work more (which means working the same or less) than under traditional methods, which is especially pronounced for group 1 (see graph 1). around one fourth of the students in the group 1 indicated that they strongly disagree with the statement that said that they have work more using the new technological multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 80 solution and additional 15% more disagree-. these results can be explained over the bases of the optimal decision-making theory. in case the time-win effect is dominating to the substitution effect, students will dedicate a part of their additional free time to study more and climb up the learning curve to improve their performance. however, if the substitution effect is the dominant, the increased efficiency of studying with the platform will lead to less study time achieving the same performance as under additional methods. which of the two effect dominates is an empirical question? it is also convenient to mention that there is a small group of students that dedicate less time to study, simply because they are averse to the use of new technological educational platforms and they simply get discourage using it. in these circumstances they do the minimum effort to pass the course. 4.4. cost benefit analysis from the university perspective virtual platforms may either be directly adopted by the institutions, used as external services for application or integrated in existing platforms. in general, the implementation or transition of such platforms implies a high fixed cost for the institutions. hence, in some respects, it may be profitable to rely on third party providers for student applications which merely imply variable costs in terms of student licenses. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 81 it could be interesting for any university to conduct a cost-benefit analysis (cba) of a the third-party provided virtual platform, and for that reason in this section it is sketched one cba of mylab for esic. in this article we defined the net benefit or economic rent of the use of mylab as the sum of the net benefits for students, which is interpreted as student satisfaction, the net benefits for professors and the institutional costs. net benefit = (benefitsstudent – coststudent) *students + net benefits prof – monetary costs esic 1.students. the survey results evaluate several potential benefits and costs of the complementary use of the digital platform. between the direct benefits we are considering the following: (1) the students are increasing their understanding of the subject and learning how to better apply it; (2), they are learning more and contextualizing better; (3) the learning impact positively on their performance, generating better scores. to these benefits we should add the implicit ones, as for example, the need of working less time to pass the course, as one of the possibilities already mentioned in the epigraph 4. we cannot forget the implicit cost (or opportunity cost for the students). they will need some time to get used to it, which reduce their time for other activities (either study or leisure), and, they may have more working time (in this case normally to increase their scoring). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 82 based on these features and others, students make their judgement to calculate their economic rent (or net benefit), which implies the willingness to pay for the mylab access. the survey data suggest that the most accepted price is 10 € (see graph 3), following a value-based pricing approach. it does not imply that a license should be priced at 10€. the institution needs to be considered as well. graph 3.willingness to pay for mylab use in microeconomics source: self-made 2.esic. we consider that the main implicit benefits for the institution and the professors are the time saving, that can be used for the creation of class material, check of evaluation, grading, for other subjects or in general for other activities of high added value such as research, or congresses attendance. other benefits for the institution may be better brand reputation, or image because of the use of new technologies in class, that can generate 0 1 2 3 4 5 6 7 8 9 0 5 10 15 20 40 60 80 100 group 1 group 2 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 83 new student want to come esic in the future. however, we are not including it in this simplified cba, given the complexity to estimate those values. the costs included in this analysis are the direct monetary costs of the licenses and the implicit costs of other activities (e.g. visiting professors or practioneers). respective the monetary cost, the cost of licenses is calculated in 24,5€ (before vat) per student, amounting a total 1.227,4€ for 50 licenses. hence the net benefit, or economic rent, from an institutional point of view can be formulated as follows: economic rent = 10€ * 50 + net benefit prof * 2 – 1227,40€ to be profitable, the net benefit of each professors (without considering esic brand reputation or image) has to be at least 363,75€. considering only the time saving in grading of the assignments and exams, we feel that this amount is justified. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 84 5. limitations we need to consider, as the main limitation, that this project was conceived as a pilot with a total number of 50 students which implies that the results need to be interpreted with caution and give rise to further research in this area. 6. conclusions the main purpose of this article is to provide an analysis to determine if the implementation of a blended methodology could improve performance and satisfaction of the students, with no additional time cost for teachers. as a second objective, the article attempted to explain observed differences across students in the effect of the methodology on study time, based on the theory of optimal decision making. third, a simple cost benefit analysis for the digital learning platform (dlp), is used to analyze the implementation from a perspective of profitability for institutions. to address these questions, teachers at esic business & marketing school combined the traditional classes methodology with the adoption of dlp in two courses of microeconomics at an undergraduate level. to analyze the results two kinds of dataset were used: (1) a first group of data extracted from mylab and (2) a second group proceeding from an individual student survey at the end of the semester. the followings are the most remarkable conclusions obtained from this research. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 85 we find evidence that student`s grades at the final exam increase in a significative way as they spent more time with the dlp and/or when they do more digital assignments at home. spending one additional hour in the study plan increases the score in the exam by 0,04 points and the realization of an additional compulsory assignment improves the exam score in 0,46 points (out of 10). this suggests that it is really the training that matters rather than a matter of general analytical ability. there is no additional cost for the teacher to offer additional guided learning time, since the use of the study plan of the dlp and the “walk through option” substitutes individual tutorial hours. standard tutorial hours with the professors were offered but almost not used at all. likewise, the correction dlp method allows to increase the number of assignments implying for the teacher only a marginal cost in selecting a set of exercises and keeping track on performance, since the feedback and grading is automatized by the platform. hence, we conclude that the use of the dlp provides students with opportunities to increase their academic performance without additional time cost for teachers. however, we find that students make use of this opportunities in different ways. the student feedback on working time in contrast with traditional methods showed two extremes, either studying much more or much less. a theoretical explanation for this observation, based on microeconomic theory is provided. we argue that those students who work more than under standard methods take advantage of the more efficient multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 86 learning with the dlp to climb up the learning curve, while those students who work less take advantage of the more efficient learning time to reduce working time on the subject and liberate time for other activities. which effect predominates depend on individual preferences. given our experience with two groups, taught by different professors, our conjecture is that instructors can influence the dominance of one effect or other through the implementation of the features offered in one way or another, which remains to be investigated further i any case, we find that most students show a high level of satisfaction with the blended methodology and the use of the dlp. responses from a voluntary survey of students indicate that most of them recognize the value of my labs. students recognize that the dlp help them to improve their understanding of the teacher presentation, and that the additional resources of the platform have helped them to learn more than in a traditional course. as a consequence, they liked more the subject, and it has impacted positively in their exam score. for all these reasons, most of them would recommend the platform to other students of microeconomics. all the previous results strongly suggest that our hypothesis about the positive effect of new educational technologies (dlp) on student satisfaction may be confirmed. a sketch of a cost-benefit analysis of the dlp tool at an institutional level suggests that its economic costs are more than justified by the economic benefits of the tool in terms of student’s satisfaction, brand reputation and teachers time saving, that can be dedicated to other activities with higher added value. as a summary, this article contributes with the provision of a methodology to measure the goodness of an innovative learning tool, using relevant indicators and employing multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 87 advanced statistical techniques as regression analysis. it also helps us to understand the key factors for the success of this blended methodology and why students may behave differently in the face of an educational innovation based on technology. the findings are in line with economic theory. references aguirre, a.c., quintana h.p. & miranda, r.t. (2015). aplicación de las tic en la educación superior como estrategia innovadora para el desarrollo de competencias digitales. campus virtuales, 3 (1), 88-101. ahmed, n., ballard, a., ohkubo, s., & limaye, r. (2017). global health elearning: examining the effects of blended learning models on knowledge application and retention. alonso, l. e., fernández, c. j., & nyssen, j. m. (2009). el debate sobre las competencias. una investigación cualitativa en torno a la educación superior y el mercado de trabajo en españa. madrid: aneca. beard, l. h. (2017). ‘incentivised reading’-using an online vle to measure engagement and attainment in student learning. international journal for innovation education and research, 5(11), 74-86. https://doi.org/10.21125/edulearn.2017.0006 https://doi.org/10.21125/edulearn.2017.0006 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 88 becker, w. e., & greene, w. h. (2001). teaching statistics and econometrics to undergraduates. journal of economic perspectives, 15(4), 169-182. https://doi.org/10.1257/jep.15.4.169 bonk, c. j., & graham, c. r. (2012). the handbook of blended learning: global perspectives, local designs. john wiley & sons. bravo, l. e. c., guerrero, k. g., & lópez, h. j. f. (2011). uso de las tic y especialmente del blended learning en la enseñanza universitaria. revista educación y desarrollo social, 5(1), 151-160. brown, m. g. (2016). blended instructional practice: a review of the empirical literature on instructors' adoption and use of online tools in face-to-face teaching. the internet and higher education, 31, 1-10. https://doi.org/10.1016/j.iheduc.2016.05.001 castro-rodriguez, m., de castro calvo, a., & hernandez rivero, v. m. (2017). analysis of spanish commercial digital educational platforms for primary education. revista latinoamericana de tecnologia educativa-relatec, 16(2), 49-62. efimchik, e. a., ivaniushin, d. a., kopylov, d. s., & lyamin, a. v. (2017). a technique for applying rlcp-compatible labs on open edx platform. in e-learning, e-education, and online training (pp. 12-18). springer, cham. https://doi.org/10.1007/978-3-319-49625-2_2 ericsson, k. a. (2002). attaining excellence through deliberate practice: insights from the study of expert performance. https://doi.org/10.1257/jep.15.4.169 https://doi.org/10.1016/j.iheduc.2016.05.001 https://doi.org/10.1007/978-3-319-49625-2_2 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 89 fernández march, a. (2005). nuevas metodologías docentes. valencia: universidad politécnica de valencia, valencia. gortner lahmers, a., & zulauf, c. r. (2000). factors associated with academic time use and academic performance of college students: a recursive approach. journal of college student development, 41, 544-556 hattinger, m., engeström, y., & sannino, a. (2018). from contradictions to transformation: a study of joint work-integrated elearning between industry and university. journal of engineering education. hinerasky, c., fahr, r., & sliwka, d. (2017). learning outcomes, feedback, and the performance effects of a training program. universitätsbibliothek. lisitsyna, l. s., efimchik, e. a., & izgareva, s. a. (2017, june). rlcp-compatible virtual laboratories with 3d-models and demonstration mode: development and application in e-learning. in international conference on smart education and smart e-learning (pp. 7581). springer, cham. https://doi.org/10.1007/978-3-319-59451-4_8 https://doi.org/10.1007/978-3-319-59451-4_8 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 90 liu, y., & xie, t. (2019). machine learning versus econometrics: prediction of box office. applied economics letters, 26(2), 124-130. https://doi.org/10.1080/13504851.2018.1441499 medina, j. m. c., medina, i. i. s., & rojas, f. r. (2016). uso de objetos virtuales de aprendizaje ovas como estrategia de enseñanza–aprendizaje inclusivo y complementario a los cursos teóricos–prácticos. revista educación en ingeniería, 11(22), 4-12. miríadax (2018, 3 de mayo). nuestros números. recuperado de https://miriadax.net/web/guest/nuestros-numeros moodle.org. (2018, 3 de mayo). course and content. recuperado de https://moodle.net/stats/ mosbeck, m., markum, g., platschek, a. & jantsch, a. (2018). e-learning system for automatic generation and evaluation of per-student customized tasks for hardware modeling courses, world academy of science, engineering and technology, international jour. https://doi.org/10.1109/norchip.2018.8573455 sangrá, l. m. (2016). la nueva brecha digital. el futuro de las nuevas tecnologías en primaria desde la formación del profesorado. revista electrónica interuniversitaria de formación del profesorado, 19(2), 301-313. https://doi.org/10.1080/13504851.2018.1441499 https://doi.org/10.1109/norchip.2018.8573455 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11869 social and technological sciences issn: 2341-2593 erdmann and torres-marín (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 54-91 | 91 pérez, p., urbano, j. & onías, a. (2017). los recursos didácticos digitales en la calidad del aprendizaje significativo en el área de matemáticas, (bachelor's thesis, universidad de guayaquil). plant, e. a., ericsson, k. a., hill, l., & asberg, k. (2005). why study time does not predict grade point average across college students: implications of deliberate practice for academic performance. contemporary educational psychology, 30(1), 96-116. https://doi.org/10.1016/j.cedpsych.2004.06.001 rodríguez miranda, f. d. p. (2009). los coordinadores y coordinadoras de centros tic: su importancia como asesores internos para la calidad y el desarrollo del currículo en la educación primaria de andalucía. the core team (2017). the economy: economics for a changing world. oxford university press. recuperado de http://www.core-econ.org/the-economy/index.html tornero j.m. & pi, m. (2013). la integración de las tic y los libros digitales en la educación. españa: grupo planeta. torres, a.t., gonzalez e. y bordonado, j. (2018, 2-4, july). using a digital interactive labs educator tool to improve the scoring and learnig results of marketing students of econometric in esic business & marketing school. 10th international conference on education and new learning technologies. palma spain. isbn: 978-84-09-02709-5. doi: 10.21125. https://doi.org/10.1016/j.cedpsych.2004.06.001 4. findings liu, y., & xie, t. (2019). machine learning versus econometrics: prediction of box office. applied economics letters, 26(2), 124-130. https://doi.org/10.1080/13504851.2018.1441499 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.10383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 73 public infrastructures and livelihood strategies: the case of rural households in kersa district, jimma zone hassen nagesso1, tariku ayele2, birhanu nigussie3 1hasseng2013@gmail.com, sociology department (lecturer), jimma university 2ayeletariku@yahoo.com, sociology department (lecturer), jimma university 3brexnigussie@yahoo.com, psychology department (associate professor), jimma university received: 2018-04-18; accepted: 2018-08-14 abstract public infrastructures: roads, agricultural extension services, electricity, ict, protected water sources, irrigation, formal education centers, and formal health centers are essential for human-being in diversifying their livelihood strategies. the general objective of this study is to examine the possible effect of rural public infrastructures on the rural households’ livelihood strategies. the empirical assessments elsewhere in ethiopia and the circumstances on the rural livelihood in association with public infrastructures have conferred the paucity of sociological research. this study used the pragmatist research philosophy that advocates ontological and epistemological mixes in an effort to minimize the gaps noted on the empirical knowledge. accordingly, the research strategy employs the triangulation of quantitative and qualitative approaches. as mirror to the methodological triangulation, the analysis has followed a mixed design that combines descriptive and inferential techniques with the themes emerging through qualitative explorations. cross-tabulation descriptive statistics and binary and multinomial logistic regression were employed. consequently, the findings of the research revealed that public infrastructures have a significant influence on livelihood diversification strategies. specifically, there were significant associations whereby households who have access to assume infrastructures did more likely engage in mixed livelihood diversification strategies than households who don’t have access to respective rural public infrastructures. the findings from qualitative data also emphasize indispensability of given public infrastructures for diversification of livelihood strategies. thus, by including cultural elements of local people, responsible https://doi.org/10.4995/muse.2018.10383 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 74 bodies should increase the required resources for the purpose of upgrading and managing public infrastructures particularly on all-weather roads. keywords public infrastructures; livelihood; livelihood strategies; rural household; and livelihood diversification strategies 1. introduction 1.1 background of the study the beginnings of public infrastructures construction can be traced as far back as the roman empire two thousand years ago. the industrialization in europe of the 19 century brought rapid urbanization and expansion of public infrastructures such as transport (railways, tramways, metropolitan), water supply and sewerage and energy. nowadays cross the globe, infrastructure is the lifeblood of prosperity and economic confidence (phillips and roth 2013). access to public infrastructures is often identified as a key factor for sustained and rapid socio-cultural, economic and political development for rural people (mensah, bourdon and latruffe 2014). improved rural infrastructure also leads to expansion of markets, improvement of food security, social participation, female participation, and job opportunities. the development of rural infrastructure also helps to enlarge services with greater access to factors of production and productivity. the female labor participation rate increases as traditional taboos against it are overcome with public infrastructures enhancement (rahman 1993). easier access to rural public infrastructures allows diversification of livelihood diversification strategies (bryceson and bradbury 2008). because of well documented importance of rural public infrastructures to promote the above listed and other advantages for rural people, either national governments or international aid agencies seem to prioritize investments in the construction of new public infrastructures and maintenance of existing infrastructures. the un millennium project (2005) has re-emphasized the need for a ‘big push’ strategy in public investment to help multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 75 poor countries and groups of people break out of their poverty trap and meet the mdg goals. regarding public infrastructures in africa; infrastructure has been responsible for more than half of africa’s recent improved growth performance and has the potential to contribute even more in the future (foster and briceño-garmendia 2010). in addition, african governments need further scientific researches for the purpose of bringing a balanced socio-economic development in selection, funding, implementing, monitoring and evaluating the projects of rural public infrastructural development (adb 1999). ethiopia is the second-most populous country in africa with a population of more than 100 million. it is one of the world’s poorest countries. it has lowest level of public infrastructure particularly in rural areas of which an estimated 83 percent of the country’s population lives (csa 2015). nowadays, it is alleged that the fundamental causes of poverty, isolation, powerlessness, vulnerability, unemployment, and high income inequality are insufficient and also unequal access and custody of public infrastructure (escobal 2005). as a result, international community in general and ethiopia government in particular are promoting basic services program at a national and local levels to improve access to and quality of public infrastructures such as education, health, water supply, rural roads, agricultural extension services, electricity, ict, irrigation, and credit services (fao 2014). researchers (such as baron 2010; dubale 2010; tirkaso 2011; mogues 2011; assefa, bienen, and ciuriak 2012; deribe and roda 2012; kahssay and mishra 2013; demenge, rossella, katharina, alemu, and kebede 2014; derso, mamo and haji 2014; fao 2014; shiferaw, söderbom, siba, and alemu 2015;) studied the role of a given specific infrastructures in improving the life of rural people in socio-economic and political aspects. they all come up with the findings that improvement in a given public infrastructure improves the livelihood outcomes (augmentation of household incomes, boost of production and productivity, improvement of human and social well-being, decreasing poverty, increment of natural resources conservation and management, decrement of vulnerability and increment of working days) through diversifying livelihood strategies. although many studies were done so far on the impacts of public infrastructure on the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 76 livelihood outcomes, there are still gaps that this study anticipated to fill. most of the researches didn’t show the correlation among livelihood strategies and combination of public infrastructures in the context of accessibility of public infrastructures. thus, this study attempted to fill this gap. 1.2 objectives of the study 1.2.1 general objective the general objective of this study is to investigate the possible effect of accessibility to public infrastructures on rural households’ livelihood strategies in the case of kersa district, jimma zone. 1.2.2 specific objectives • to identify the nature and types of livelihood strategies along with an enlargement of public infrastructures • to examine an association of public infrastructures towards rural households’ livelihood strategies • to find out the possible effect of accessibility of public infrastructures on livelihood diversification strategies 1.3 significance of the study this study will provide a solid document that might be used as a source of information regarding, the impact of rural public infrastructures on rural livelihood strategies, for various actors (readers, students, researchers). it will also help the responsible bodies and stakeholders of an area, in which the study was conducted, to get information, to improve multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 77 strategic plans and to reconsider social policy. this can be possible through publishing and providing a finding document to responsible bodies. 1.4 limitations of the study several limiting contexts present challenges to the level of progresses required in this study. the scantiness of empirical research, unavailability of forums and scientific papers exclusively or primarily dedicated to the effects of public infrastructures on livelihood strategies created challenges to the research’s endeavour. 2. literature review 2.1 conceptual framework the selected conceptual framework among the definitions of public infrastructures is the conceptualization of the conventional theory on public goods that recognize public infrastructures as goods that are typically technical indivisible, have low excludability, long life and are rarely traded (escobal 2005). ahmed and donovan (1992) recognize that with the increasing importance of the role of agriculture in economic development, the literature started including agricultural research, extension services, financial institutions or/and irrigation as part of a much broader concept of infrastructure. a lot of researchers envisage that infrastructure investments may have macroeconomic and microeconomic impacts. at the macroeconomic level, improved access to new infrastructure services may change the marginal rate of return of the main infrastructure, but it may also affect the marginal rate of return of other public infrastructure as well as the returns to those private assets that are already in the hand of the poor. on the other hand, microeconomic effects can be traced through changes in market specific relationships or household specific behavioral changes (escobal 2005). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 78 2.2 the livelihood impact of public infrastructure: theory sustainable livelihood outcomes approaches are based upon evolving thinking about poverty reduction, the way the poor live their lives, and the importance of structural and institutional issues. the twin influences of the policy framework and governance, which have dominated much development thinking since the early 1980s, are also reflected in sustainable livelihood, as is a core focus on the community. community-level institutions and processes have been a prominent feature of approaches to natural resource management and are strongly emphasized in sustainable livelihood approaches, though in sustainable livelihood the stress is on understanding and facilitating the link through from the micro to the macro, rather than working only at community level (ashley and carney 1999). 2.3 the impacts of infrastructure on livelihood sustainability several studies conducted and show that rural infrastructure (both physical and institutional) such as irrigation, watershed development, rural electrification, roads, markets, credit institutions, rural literacy, agricultural research and extension together play a key role in determining the people livelihood (narayanamoorthy and hanjra 2006). in this regard, the work made by chaya (2007) argued that the existing poor transportation and communication outlets limited provisioning of basic health services for societies residing in rural and remote areas especially on times of emergencies needed. 2.4 policy framework of infrastructures the development of public infrastructures enables all countries to achieve the mdgs, there should be identification of priority public investments to empower poor people, and these should be built into mdg-based strategies that anchor the scaling-up of public investments, capacity-building, resource mobilization, and official development assistance. seven main investment-and-policy clusters are identified in the areas of rural development; urban multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 79 development; health systems; education; gender equality; environment; and science, technology and innovation. this ‘big push’ strategy is designed to set low-income economies on a growth path that will become self-sustainable, as core investments in infrastructure and human capital will enable poor people to join the global economy and establish the basis for private-sector-led diversified exports and economic growth (anderson, renzo and levy 2006). 3. research methods 3.1 study setting and population the study area is in the oromia national regional state (onrs) of ethiopia, jimma zone administrative area. according to the csa (2015) census, the oromia regional state has a population of 33, 692,000 of which 4,880,000 is urban dwellers and 28, 812,000 is rural dwellers (csa 2015). jimma zone is purposively selected from the zones of oromia region. the total population of jimma zone is 2,986,957 of which 1,498,021 are male and 1,488,936 are female. kersa is one of the woredas in the jimma zone of the oromia region of ethiopia. it is bordered in south by dedo, southwest by seka chekorsa, west by mana, north by limmu kosa, northeast by tiro afata and southeast by omo nada. the altitude of this woreda ranges from 1740 to 2660 meters above sea level; mountains include sume, gora, kero, folla and jiren. 3.2 research design the study at hand deployed a mix of both quantitative and qualitative designs. the philosophical foundation of the study is pragmatism. the reasons for the selection of pragmatic approach are: to use variety of data sources, to use multiple methods in the study at the same time or one after the other and to use multiple perspectives to interpret the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 80 results. approximating longitudinal survey with cross-sectional design was employed. the researchers selected this study design because there were no baseline data in the study area. 3.3 methods of data collection a. household survey the data which were collected through household survey are: demographic and socioeconomic data (age, sex, religion, marital status, educational status, household’s size); nature and changes of livelihood diversification strategies; and accessibility to a given infrastructures. b. in-depth interview in-depth interview was employed in collecting detailed information to substantiate quantitative data and to offer a complete picture of association among accessibility of public infrastructures and livelihood strategies. hence, kebeles’ elders – 16 individuals (four from each kebele) were purposively selected and deeply interviewed. the researcher selected the above participants assuming that they have experience on issues under study and can provide profound information on the changes of livelihood strategies as a result upgrading public infrastructures. c. key informant interview in the opinion of bernard (2006), key informants are groups of people with whom the researcher talks and communicates extensively over a lengthy of duration. the key informant interview was held with key individuals on all selected sectors of public infrastructures. accordingly, the head of all respective infrastructures (sectors) bureaus at district level – 16 individuals (two each) from office of: health; education; water, mineral and energy; irrigation and rural development; electricity (power); transportation; agricultural; and ict were interviewed about an associational changes of public infrastructures and rural livelihood strategies and its diversification. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 81 3.4 instruments of data collection in this study, the main data-generation instruments were structured questionnaires and semi-structured checklists. the primary objective of structured questionnaire was to elicit quantitative information from households’ heads. the preparation of structured questionnaires, i.e. the instruments followed a design that hastens enticing pertinent information from the target groups. items on each of the instruments communicated clearly the purposes of the study, shaded light on precautionary ethical issues and explained the powers of following instructions while filling out the questions. altogether, the contents of the questionnaire items covered issues on an association and extricable effect among an accessibility of public infrastructures and livelihood strategies. checklists used for the qualitative field research were semi-structured guides that elicited qualitative information (meanings, words and ideas) through deeper consultations from informants, key informants and discussants. 3.5 sampling and sample size since it was possible to access the lists of the residents from the respective study kebeles. this quality marked simple random sampling technique as the most appropriate to be used. from thirty one rural kebeles, four kebeles (tolikarso, bulbuli, babo and kallacha) were randomly selected. thereby 255 households were selected by simple random sampling; and lottery method of sampling was utilized among its strategies. the sample size was determined depending on the formula of yamane (1967:886) because it’s the simplified in the case of finite population. the formula considers 95% of confidence, and 5% margin of error. the formula is n = n / [1 + n (e) 2]; where n is the sample size, n is the population size, and e is the level of precision. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 82 3.6 reliability and validity the aspects of inter-rater technique of reliability proved its usefulness in the context of the pilot testing. the inter-rater reliability assessed the reliability of research instruments by utilizing four interviewers per site (tolikarso, bulbuli, babo and kallacha). internal consistency of instruments was assured by the split-half correlation. the two halves of an instruments provided similar result of (r = .88). subsequently, instruments had strong internal consistency. the researchers believed that the items on the instrument captured the concepts that are essential in the research. 3.7 methods of data analysis the analysis applies a mixed design. quantitative analysis uses the numeric data gathered through the sample household. the quantitative data applied both the techniques of descriptive and inferential statistics. the descriptive analysis emphasizes on percentages, central tendencies and graphic presentations. consequently, the interpretations followed presentations made through frequency tables portraying numeric facts in different chapters of the study. the results conferred the prevailing relationships among the variables compared through column percentages. in addition, the phi-coefficient, lambda-coefficient and spearman’s rho-coefficient help to explain the strength and direction of association wherever the data appeared apparent. the qualitative data were transcribed, categorized, interpreted and schematized based on their respective contents and themes. the meanings, words, symbols and argumentative texts have formed basic premises in the structures of reporting the sub-titles, sections and chapters. 3.8 ethical considerations in conducting this study, an ethical considerations and safety measures were made. accordingly, before going to the field the letter from jimma university, college of social multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 83 sciences and humanities research coordinate, was taken and given to the woreda administrative and other required bodies. after we went to the field and contacted with respondents, the purposes and importance of the study were explained for the participants of the study and informed consent was obtained from each of them. thus, participants were given the authority to permit or refuse in the collection of data in any form; full right was deserved to withdraw at any time: to change ideas or to edit recorded materials. besides, the privacy of the participants was promoted, and they were informed that whatever information they provide be kept confidential. that is, the confidentiality and anonymity of information were strongly maintained. 4. results and discussion this chapter deals with data analyses and presentation of the study and attempts to answer the research’s objectives concerned with possible effect of public infrastructures on the rural livelihood strategies in four kebeles of kersa district. specifically, it includes about the presentation of: (a) demographic and socio-economic information of respondents; (b) nature and types of livelihood diversification strategies (mixed livelihood strategies, only non-farm strategy, merely off-farm strategy, purely farming strategy and others) along with accessibility of public infrastructures; and (c) possible effects of accessibility to public infrastructures (all season road, formal health centers, formal education centers, ict, protected water sources, agricultural extension services, electricity, and irrigation) on livelihood strategies diversification. 4.1 demographic and socio-economic characteristics of sample respondents this section presents demographic and socio-economic variables such as sex, age, religion, educational status, and marital status. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 84 table 4.1: sex, age, religion, educational status, and marital status of respondents variables category frequency percent sex male 218 55.5 female 37 14.5 age <30 3 1.2 31-40 20 7.8 41-50 99 38.8 51-60 33 12.9 61-65 80 31.4 >65 20 7.8 religion orthodox 31 12.2 islam 199 78.0 protestant 17 6.7 other 8 3.1 educational status can't read and write 99 38.8 grade 1-8 96 37.6 grade 910 45 17.6 grade 11 12 9 3.5 diploma holder 3 1.2 degree and above holder 3 1.2 marital status married 192 75.3 divorced 16 6.3 widowed 47 18.4 source: household survey 2016 table 4.1 shows that the majority of the respondents were male (55.5 percent) and followed by (14.5 percent) of female. concerning age, majority of respondents were fall under a category of 41-50 (38.8 percent) and followed by 61-65 (31.4 percent), 51-60 (12.9 percent), 30-40 (7.8 percent), > 65 (7.8) and <30 (1.2 percent) respectively. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 85 regarding religion of respondents the majority of respondents were muslims (78.2 percent), followed by orthodox (12.2 percent), protestant (6.7 percent), and other (3.1 percent) respectively. on the subject of educational status of the survey respondents, the majority of the respondents 99 (38.8 percent) were can’t read and write. the second largest were those between grade one and eight 96 (37.6 percent). an accumulation of respondents below grade eight were 76.4%. the smallest were those who hold diploma and who hold degree and above each of 3 (1.2 percent). to conclude the mainstream of the sample households 192 (75.3 percent) were married; followed by widowed 47 (18.4 percent) and divorced 16 (6.3 percent) respectively. 4.2 accessibility to public infrastructures and livelihood diversification strategies the people in the study area have involved in numerous livelihood diversification strategies. in so doing, describing about the nature and types of these livelihood diversification strategies has a lion share in presenting the possible effect of an assumed infrastructures on it. diversification as a livelihood strategy is defined as a process in which the person or the rural family unit builds a group of activities and goods looking for better ways of living (ellis 2000). one of interviewed respondents stated that, “diversification is our norm. very few people manage their life by single source, hold all their wealth in the form of any single asset, or use their assets in just one activity.” this implies that almost all of rural households diversify their life. for the purpose of this study, the researcher has grouped it into: (1) mixed livelihood diversification strategies – messing one or more of activities from either off farm activities, non-farm activities, farming activities and others; (2) only off-farm1 strategy – fetching from only one or more off farm activities such as land renting to other farmers, purchasing additional farm land, and employment on another farm; (3) 1some authors use off-farm strategy and non-farm strategy interchangeably. however, for the purpose of this study, the researcher used: off-farm strategy as activities made up of agricultural wage income; while non-farm refers to those activities that are not primary agriculture or forestry or fisheries. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 86 only non-farm strategy endearing in only one or more of non-farm activities such as daily employment out of farm (skilled, semi-skilled and non-skilled worker); and small business (charcoal production, quarrying and production of building materials, furniture making, carpentry, painting, pottery, baskets making and selling); (4) only farming strategies (specialty crops, organic and biomass production, and crop harvesting); and (5) others appealing with either getting social help, family and friends help, or begging. reardon et al. (2007) found that the development of public infrastructures have significant effect in increasing non-farm activities besides agricultural activities. in other words, the percentage of respondents who affianced in mixed strategies were (60.4 percent) followed by only farming strategy (19.6 percent), only non-farm strategy (9.8 percent), and 5.1 percent each of only off-farm strategy and other. this is equivalent to saying that the probability of engaging in mixed, only farming, only non-farm, only off farm and other strategies in sample were 0.604, 0.196, 0.098, 0.051, and 0.051 respectively. majority of respondents claimed that accessibility to public infrastructures suggestively initiates them to engage in mixed livelihood diversification strategies. consistently, one respondent argued that, “an access to a given infrastructures can easily expand our means of income generation.” majority of key interviewed respondents from different sectors also contended as an upgrading of a given public infrastructures is a pull factors for rural households in diversifying their income generating activities. consistently, fernando and porter (2002) found that facilitating mobility can empower women to gain greater control over their own lives by increasing their access to markets and their exposure to education, training, and information and by offering them more opportunities for political participation. likely paudel (2014) found that transportation facilities were significantly aided to increase in participation of women in social and income generating activities. of 113 who have an access to all-weather roads, 62.8 percent mixed their livelihood diversification strategies while the remaining fell in either one of a categorized livelihood diversification strategies. the chi-square test shows that there was significant and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 87 positive association between all-weather roads and livelihood diversification strategies with (χ2 (4), 255 =131.881, p=0.000) at (α=0.05). the phi coefficient reported that there is strong association between variables with (φ=0.719) value. expert from transport office indicated as, “all-weather road helps rural people particularly youth in diversifying their income generating activities.” expert from agricultural and development office also added, “all-weather road helps rural people to expand their production from that of only for consumption to that of for market.” likely gibson and oliva (2009) argue that there is growing interest in the rural non-farm sector in developing countries as a contributor to economic growth, employment generation, livelihood diversification, and poverty reduction. regarding public education centers, of 137 respondents who have an access to it, 78.1 percent encompassed in mixed strategies while the remaining incorporated in either one of a given strategies. the chi-square test found positive and significant association between an access to formal education centers and livelihood diversification strategies, with (χ2 (4) =40.992, p=0.000) at (α=0.05). the lambda coefficient with the value (0.347) also reported the strength of the association to be moderate. likely, officer from education office claimed that: formal education centers are where knowledge is produced. the existence of education centers at nearby of households home initiate them to learn for themselves and send their children to school. thus education increases farmers’ ability to use their labor and other assets effectively and efficiently. regarding public health centers, of 111 respondents who have an access to it, 80.2 percent mixed their livelihood diversification strategies. the chi-square test also shows positive and significant association with (χ2 (4) =36.44, p=0.000) at (α=0.05). the lambda coefficient with the value (0.216) reports that the association had weak ties. in addition, officer from a district’s health office contended that: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 88 the main objective of health posts are to keep healthy of community. if they are healthy, they engage in whatever they want. this means, there is no predecessors of health. thus over all, through keeping health of society, health centers increases livelihood diversification for rural community. the majority of respondents; who have an access protected water sources (91.7 percent) engaged in mixed strategies and those who have not an access (37.4 percent) betrothed in only farming activities. the association was tasted significant and positive with (χ2 (4) =116.2, p=0.000) at (α=0.05). the lambda coefficient with the value (0.642) reports that an association had moderate influences. an expert from water, energy and mineral office also claimed this as: in areas where water is not available, women and children travel tens of kilometers to fetch water. this is seen through queues in water points during dry seasons. disease associated with water affect the poor with greater margins as compared to those who have an access with a burden of ill health that creates a vicious cycle of poverty and sickness. the majority of respondents who have an access to ict (87.7 percent) encompassed in mixed strategies while those who have not an access (32.8 percent) engaged in farming activities. the association between an access to ict and livelihood diversification strategies was tasted significant and positive with (χ2 (4) =55.1, p=0.000) at (α=0.05). the lambda coefficient with the value (0.552) states that there was moderate association between ict and livelihood diversification strategies. an interviewed respondents also witnessed, “an access to ict components particularly to mobile is highly helping us to diversify our livelihood diversification strategies.” of 159 respondents who are getting agricultural extension services, 88.5% mix their livelihood diversification strategies. getting agricultural extension services and diversifying livelihood diversification strategies were significantly and positively associated with (χ2 (4) =51.2, p=0.000) at (α=0.05). however, the strength of the association between the variables was weak (φ=0.448). one of interviewed agricultural extension workers also added as, “we are teaching and showing people about the nature multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 89 and advantages of diversifying their livelihood diversification strategies by using the resources they have effectively and efficiently.” all of respondents who have an irrigation access managed their life by engaging in mixed livelihood diversification strategies. the chi-square test found the significant and positive association between an access to irrigation and livelihood diversification strategies with (χ2 (4) =15.1, p=0.000) at (α=0.05). however, cramer’s v coefficient2 with value (0.243) reports that there was very little strength of an association. finally, an access to electricity had also a significant and positive association with livelihood diversification strategies with (χ2 (4) =50.9, p=0.000) at (α=0.05). however, (φ=0.348) reports shows weak association between variables. unswervingly, beyene and muche (2010) found that development interventions aiming at increased income diversification will immensely and significantly contribute to the attainment of food security. gachassin, najman and raballand (2015) also found that better road access increases the number of activities within households. this corresponds to a ‘pull’ factor that draws people into greater earnings opportunities. by connecting places, people, and opportunities, tarred roads can act as a development tool in rural areas of africa. in addition, table 4.8 below presents the parameter estimates of the level of livelihood selection consequences of an access to public infrastructures. 5. conclusion and recommendations this chapter begins by offering a glimpse on the core foundations of the inquiry. it draws the conclusions based on the presentations, analysis and discussions made in the previous chapter. it also forwards a range of implications to public infrastructural development, policy practice, institutional operations and casts light on public infrastructures and rural livelihood research that seeks perfection. 2a measure of association independent of sample size. this statistic is a modification of the phi statistic so that it is appropriate for larger than 2 × 2 tables. v ranges between 0 (no relationship) and 1 (perfect relationship). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 90 5.1 conclusion conclusions entail empirical knowledge generated on the two inter-related key components of the study: an accessibility of public infrastructures and rural livelihood diversification strategies. regarding demographic and socio-economic backgrounds, majority of rural household’s head are male, can’t read and write, marriage, and have greater than five households members respectively. public infrastructures coverage in rural parts of ethiopia is at infant stage. however, large number of people live in rural areas. in other words, the demand of rural people about public infrastructures couldn’t be answered. they are using traditional means of life in place of these infrastructures. as a result, they can’t get an access to compulsory livelihood strategies easily. the levels of public infrastructures and livelihood diversification strategies are directly related with each other among rural communities. as there is low level of access to public infrastructures, there is low level of livelihood diversifying livelihood strategies. all public infrastructures are significantly and positively associated with an engagement of rural people in mixed livelihood diversification strategies. they move from a single strategy to multiple strategies proportionally to access to a given specific infrastructure. all-weather roads, electricity, ict and agricultural extension services might initiate rural people to start pity trade, engage in skilled and unskilled labor wage, share farming land with other, and etc. for instance, if there is no road there is no production for market but only for consumption. so, an advancement of public infrastructures leads rural households to diversify their livelihoods which inextricably intimates to achieving improved rural livelihood. this is also consistent with ‘multi-voicedness’ principle of activity theory which argue that livelihood diversification is a multiple role division among household’s members that leads to livelihood outcomes’ components improvement. finally, an advancement of public infrastructures and rural livelihood strategies are intractably associated with each other. this is consistent with the conclusion of ellis multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 91 (2000), who argued, “livelihood includes natural, physical, human and financial goods, and social capital. facilities to access these goods determine rural families´ livelihood and well-being.” similarly, an access to public infrastructures improves rural livelihood strategies simultaneously. analyzing one of them separately from the others is problematic. in other words, to get a full map of rural people with its objective oriented activities, multivoicedness, historicity, contradictions and transformation; studying instantaneously about the possible effects of public infrastructures on the complex and changing rural livelihood assets, strategies and components of outcomes have a decisive role. 5.2 recommendations the implications of the research call for ways to merge the theoretical claims and the practical actions pertaining to effect relationships between rural public infrastructures development and livelihood improvement. in this respect, the research forwards a range of intertwined implications to development, research, policy and institutional operations in view of promoting the practices associated with the sociology of rural sociology. all sectors of government should keep on in constructing new public infrastructures and maintaining the existing ones. lagging of public infrastructures coverage resulted in lagging of rural people livelihood improvement. the more public infrastructures constructed the more the more rural people diversify their livelihood strategies. local communities’ culture is not at the center of public infrastructures development and by implication pushed away to the fringes of socio-cultural development. locals are in the margins of the wider interactional scenarios and benefits. a countervailing initiative, taken by responsible organs, should reconsider ways to educate, train, re-orient and abridge direct stakeholders who give services to the rural community about a local cultures and mores. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 92 upgrading public infrastructures is not inclusive. for example rural people could not use the languages by which messages are sent for them from telecommunications. urban people treat them again as they are far from information. these lead them to fearing of asking for their rights in utilizing public infrastructures. thus responsible bodies should work on raising awareness on the rights and duties of rural people in accessing and utilizing public infrastructures. public infrastructures intervention programs need to pursue a more clearly defined gender strategy to ensure participation by women in infrastructures resource management and decision making in all aspects. public infrastructures users, especially women and children, should be among those consulted during the planning stage before any decisions are taken about public infrastructures improvement. finally, the aim and purpose of construction should be vibrant and unambiguous. in other words, the sectors working on specific public infrastructures should be transparent and accountable. acknowledgements above all, we would like to express our gratitude to jimma university for its material support in general. we also thank the university for covering the financial expenses to carry out the fieldwork particularly. we are grateful for the time and information shared by the public institutions during the fieldwork. we owe special respect to kersa woreda offices of transportation, education, health, ict, energy and water, and development and irrigation. we thank them for opening their doors and enabling me to access relevant data, archives and documents. our special gratitude is also due to the respondents of household survey, key informants and interviewees. finally, we are also grateful for all our friends who have suggested and commented for the successful accomplishment of this study. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 93 references anderson e. paolo de r. and stephanie l. 2006. the role of public investment in poverty reduction: theories, evidence and methods. overseas development institute, 111 westminster bridge road london se1 7jd uk. anderson p. and shimokawa s. 2006. rural infrastructure and agricultural development. paper prepared for presentation at the annual bank conference on development economics, tokyo, japan, may 29-30, 2006. ashley c. and carney d. 1999. sustainable livelihoods: lessons from early experience. london, uk. assefa h., bienen d., and ciuriak d. 2012. ethiopia investment prospects: a sectorial scan. addis ababa. bryceson f., bradbury a. & bradbury t. 2008. “roads to poverty reduction? exploring rural roads' impact on mobility in africa and asia.” development policy review 26(2):459-482. bryman, alan. 1998. social research methods. 2nd ed. oxford: oxford university press. babbie, earl. 2008. the basics of social research. 4th ed. australia: wadsworth. baron d. 2010. the impact of telecommunications services on doing business in ethiopia; private sector development. hub/addis ababa chamber of commerce and sectorial associations. bhatta b. 2004. “socio-economic transformations and road accessibility: evidence from northern ethiopia.” a thesis submitted to partial fulfilment of the degree of master science in development and resource economics: norwegian university of life sciences. bernard, russell h. 2006. research methods in anthropology: qualitative and quantitative approaches. 4th ed. oxford: altamira press. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 94 bowser w. 2015. the long and short of returns to public investments in fifteen ethiopian villages. 3ie replication paper series executive editor: annette n brown. chaya n. 2007. poor access to health services: research commentary. pai, washington dc, usa. creswell, w. john and l. plano clark. 2011. designing and conducting mixed methods research. 2nd ed. sage publications inc. corbetta, piergiorgio. 2003. social research: theory, methods, and techniques. london: sage publications. cohen, l., lawrence manion and keith morrison. 2000. research methods in education. 5th ed. london: routledge flamer. creightney, c. d. 1993. transport and economic performance: a survey of developing countries. washington, d.c.: world bank. csa. 2015. section bpopulation demenge j., rossella a., katharina w., alemu a., and kebede m. 2014. multifunctional roads: the potential effects of combined roads and water harvesting infrastructure on livelihoods and poverty in ethiopia. journal of infrastructure development. deribe k., roda a. 2012. the potential of mobile phone technology for public health practice in ethiopia. derso d., mamo and y., and haji j. 2014. “analysing socio-economic factors affecting the use of information and communication technologies among farmers in central highlands of ethiopia.” african journal of agricultural science and technology (ajast) 2(8):163171. (http://www.oceanicjournals.org/ajast), issn 2311-5882. dubale t. 2010. multi-year expert meeting on services development and trade: the regulatory and institutional dimension, telecommunication in ethiopia. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 95 ellis f. 2000. rural livelihoods and diversity in developing countries. oxford: oxford university press. escobal j. 2005. the role of public infrastructure in market development in rural peru. wageningen university. available at: http://mpra.ub.uni-muenchen.de/727/ fan s., hazell p. and thorat s. 1999. “linkages between government spending, growth and poverty in rural india.” research report 110. washington, dc: ifpri. fao. 2014. analysis of public expenditures in support of food and agriculture in ethiopia, 2006-2012.www.fao.org/in-action/mafap foster v. and briceño-garmendia c. 2010. africa’s infrastructure: a time for transformation. the international bank for reconstruction and development. foster v. and morella e. 2010. “ethiopia’s infrastructure: a continental perspective.” the world bank. washington, dc 20433 usa. fosu, k. j., n. heerink, e. k. ilboudo, m. kuiper, and a. kuyvenhove n. 1995. "public good and services and food security: theory and modeling approaches with special reference to ghana and burkina faso.” accra, ghana: public goods and services projects. réseau de recherché sadaoc. kahssay, a. and mishra s. 2013. “community development through hydroelectric project: a case study of gilgel gibe iii hydroelectric power project in ethiopia.” international journal of community development 1(1):19-34. marvasti, amir b. 2002. qualitative research in sociology: an introduction. london: sage. publications. mensah j., bourdon h., and latruffe l. 2014. “infrastructure access and household welfare in rural ghana.” african development review 26(3):508–519. mogues t, 2011. “the bang for the birr: public expenditures and rural welfare in ethiopia” journal of development studies 47(5):735–752. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.0383 social and technological sciences eissn: 2341-2593 nagesso et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 73-96 | 96 narayanamoorthy a. and hanjra a. 2006. “rural infrastructure and agricultural output linkages: a study of 256 indian districts” ind. jn. of agri. economics 61(3). phillips p. and roth h. 2013. infrastructure: global priorities and global insights. urban land institute. satish p. 2007. “rural infrastructure and growth: an overview.” ind. jn. of agri. econ. 62(1). shiferaw a., söderbom m., siba e., and alemu g. 2015. road infrastructure and enterprise dynamics in ethiopia. jel code: o14, o18, h54. un millennium project. 2005. “investing in development: a practical plan to achieve the millennium development goals.” london: earth scan. world bank. 1994. “world development report 1994: infrastructure and development.” washington d.c: world development report. yerasework admassie. 2010. social research methods. college of social sciences department of sociology graduate program. << /ascii85encodepages false /allowtransparency false /autopositionepsfiles true /autorotatepages /none /binding /left /calgrayprofile (dot gain 20%) /calrgbprofile (srgb iec61966-2.1) /calcmykprofile (u.s. web coated \050swop\051 v2) /srgbprofile (srgb iec61966-2.1) /cannotembedfontpolicy /error /compatibilitylevel 1.4 /compressobjects /tags /compresspages true /convertimagestoindexed true /passthroughjpegimages true /createjobticket false /defaultrenderingintent /default /detectblends true /detectcurves 0.0000 /colorconversionstrategy /cmyk /dothumbnails false /embedallfonts true /embedopentype false /parseiccprofilesincomments true /embedjoboptions true /dscreportinglevel 0 /emitdscwarnings false /endpage -1 /imagememory 1048576 /lockdistillerparams false /maxsubsetpct 100 /optimize true /opm 1 /parsedsccomments true /parsedsccommentsfordocinfo true /preservecopypage true /preservedicmykvalues true /preserveepsinfo true /preserveflatness true /preservehalftoneinfo false /preserveopicomments true /preserveoverprintsettings true /startpage 1 /subsetfonts true /transferfunctioninfo /apply /ucrandbginfo /preserve /useprologue false /colorsettingsfile () /alwaysembed [ true ] /neverembed [ true ] /antialiascolorimages false /cropcolorimages true /colorimageminresolution 300 /colorimageminresolutionpolicy /ok /downsamplecolorimages true /colorimagedownsampletype /bicubic /colorimageresolution 300 /colorimagedepth -1 /colorimagemindownsampledepth 1 /colorimagedownsamplethreshold 1.50000 /encodecolorimages true /colorimagefilter /dctencode /autofiltercolorimages true /colorimageautofilterstrategy /jpeg /coloracsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /colorimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000coloracsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000colorimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasgrayimages false /cropgrayimages true /grayimageminresolution 300 /grayimageminresolutionpolicy /ok /downsamplegrayimages true /grayimagedownsampletype /bicubic /grayimageresolution 300 /grayimagedepth -1 /grayimagemindownsampledepth 2 /grayimagedownsamplethreshold 1.50000 /encodegrayimages true /grayimagefilter /dctencode /autofiltergrayimages true /grayimageautofilterstrategy /jpeg /grayacsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /grayimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000grayacsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000grayimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasmonoimages false /cropmonoimages true /monoimageminresolution 1200 /monoimageminresolutionpolicy /ok /downsamplemonoimages true /monoimagedownsampletype /bicubic /monoimageresolution 1200 /monoimagedepth -1 /monoimagedownsamplethreshold 1.50000 /encodemonoimages true /monoimagefilter /ccittfaxencode /monoimagedict << /k -1 >> /allowpsxobjects false /checkcompliance [ /none ] /pdfx1acheck false /pdfx3check false /pdfxcompliantpdfonly false /pdfxnotrimboxerror true /pdfxtrimboxtomediaboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxsetbleedboxtomediabox true /pdfxbleedboxtotrimboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxoutputintentprofile () /pdfxoutputconditionidentifier () /pdfxoutputcondition () /pdfxregistryname () /pdfxtrapped /false /createjdffile false /description << /ara <feff06270633062a062e062f0645002006470630064700200627064406250639062f0627062f0627062a002006440625064606340627062100200648062b062706260642002000410064006f00620065002000500044004600200645062a064806270641064206290020064406440637062806270639062900200641064a00200627064406450637062706280639002006300627062a0020062f0631062c0627062a002006270644062c0648062f0629002006270644063906270644064a0629061b0020064a06450643064600200641062a062d00200648062b0627062606420020005000440046002006270644064506460634062306290020062806270633062a062e062f062706450020004100630072006f0062006100740020064800410064006f006200650020005200650061006400650072002006250635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e0635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e> /bgr <feff04180437043f043e043b043704320430043904420435002004420435043704380020043d0430044104420440043e0439043a0438002c00200437043000200434043000200441044a0437043404300432043004420435002000410064006f00620065002000500044004600200434043e043a0443043c0435043d04420438002c0020043c0430043a04410438043c0430043b043d043e0020043f044004380433043e04340435043d04380020043704300020043204380441043e043a043e043a0430044704350441044204320435043d0020043f04350447043004420020043704300020043f044004350434043f0435044704300442043d04300020043f043e04340433043e0442043e0432043a0430002e002000200421044a04370434043004340435043d043804420435002000500044004600200434043e043a0443043c0435043d044204380020043c043e0433043004420020043404300020044104350020043e0442043204300440044f0442002004410020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200441043b0435043404320430044904380020043204350440044104380438002e> /chs <feff4f7f75288fd94e9b8bbe5b9a521b5efa7684002000410064006f006200650020005000440046002065876863900275284e8e9ad88d2891cf76845370524d53705237300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /cht <feff4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef69069752865bc9ad854c18cea76845370524d5370523786557406300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /cze <feff005400610074006f0020006e006100730074006100760065006e00ed00200070006f0075017e0069006a007400650020006b0020007600790074007600e101590065006e00ed00200064006f006b0075006d0065006e0074016f002000410064006f006200650020005000440046002c0020006b00740065007200e90020007300650020006e0065006a006c00e90070006500200068006f006400ed002000700072006f0020006b00760061006c00690074006e00ed0020007400690073006b00200061002000700072006500700072006500730073002e002000200056007900740076006f01590065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f007400650076015900ed007400200076002000700072006f006700720061006d0065006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076011b006a016100ed00630068002e> /dan <feff004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000620065006400730074002000650067006e006500720020007300690067002000740069006c002000700072006500700072006500730073002d007500640073006b007200690076006e0069006e00670020006100660020006800f8006a0020006b00760061006c0069007400650074002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /deu <feff00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f006300680077006500720074006900670065002000500072006500700072006500730073002d0044007200750063006b0065002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /esp <feff005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f00730020005000440046002000640065002000410064006f0062006500200061006400650063007500610064006f00730020007000610072006100200069006d0070007200650073006900f3006e0020007000720065002d0065006400690074006f007200690061006c00200064006500200061006c00740061002000630061006c0069006400610064002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /eti <feff004b00610073007500740061006700650020006e0065006900640020007300e4007400740065006900640020006b00760061006c006900740065006500740073006500200074007200fc006b006900650065006c007300650020007000720069006e00740069006d0069007300650020006a0061006f006b007300200073006f00620069006c0069006b0065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740069006400650020006c006f006f006d006900730065006b0073002e00200020004c006f006f0064007500640020005000440046002d0064006f006b0075006d0065006e00740065002000730061006100740065002000610076006100640061002000700072006f006700720061006d006d006900640065006700610020004100630072006f0062006100740020006e0069006e0067002000410064006f00620065002000520065006100640065007200200035002e00300020006a00610020007500750065006d006100740065002000760065007200730069006f006f006e00690064006500670061002e000d000a> /fra <feff005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f0075007200200075006e00650020007100750061006c0069007400e90020006400270069006d007000720065007300730069006f006e00200070007200e9007000720065007300730065002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /gre <feff03a703c103b703c303b903bc03bf03c003bf03b903ae03c303c403b5002003b103c503c403ad03c2002003c403b903c2002003c103c503b803bc03af03c303b503b903c2002003b303b903b1002003bd03b1002003b403b703bc03b903bf03c503c103b303ae03c303b503c403b5002003ad03b303b303c103b103c603b1002000410064006f006200650020005000440046002003c003bf03c5002003b503af03bd03b103b9002003ba03b103c42019002003b503be03bf03c703ae03bd002003ba03b103c403ac03bb03bb03b703bb03b1002003b303b903b1002003c003c103bf002d03b503ba03c403c503c003c903c403b903ba03ad03c2002003b503c103b303b103c303af03b503c2002003c503c803b703bb03ae03c2002003c003bf03b903cc03c403b703c403b103c2002e0020002003a403b10020005000440046002003ad03b303b303c103b103c603b1002003c003bf03c5002003ad03c703b503c403b5002003b403b703bc03b903bf03c503c103b303ae03c303b503b9002003bc03c003bf03c103bf03cd03bd002003bd03b1002003b103bd03bf03b903c703c403bf03cd03bd002003bc03b5002003c403bf0020004100630072006f006200610074002c002003c403bf002000410064006f00620065002000520065006100640065007200200035002e0030002003ba03b103b9002003bc03b503c403b103b303b503bd03ad03c303c403b503c103b503c2002003b503ba03b403cc03c303b503b903c2002e> /heb <feff05d405e905ea05de05e905d5002005d105d405d205d305e805d505ea002005d005dc05d4002005db05d305d9002005dc05d905e605d505e8002005de05e105de05db05d9002000410064006f006200650020005000440046002005d405de05d505ea05d005de05d905dd002005dc05d405d305e405e105ea002005e705d305dd002d05d305e405d505e1002005d005d905db05d505ea05d905ea002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e05d005de05d905dd002005dc002d005000440046002f0058002d0033002c002005e205d905d905e005d5002005d105de05d305e805d905da002005dc05de05e905ea05de05e9002005e905dc0020004100630072006f006200610074002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e> /hrv (za stvaranje adobe pdf dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. stvoreni pdf dokumenti mogu se otvoriti acrobat i adobe reader 5.0 i kasnijim verzijama.) /hun <feff004b0069007600e1006c00f30020006d0069006e0151007300e9006701710020006e0079006f006d00640061006900200065006c0151006b00e90073007a00ed007401510020006e0079006f006d00740061007400e100730068006f007a0020006c006500670069006e006b00e1006200620020006d0065006700660065006c0065006c0151002000410064006f00620065002000500044004600200064006f006b0075006d0065006e00740075006d006f006b0061007400200065007a0065006b006b0065006c0020006100200062006500e1006c006c00ed007400e10073006f006b006b0061006c0020006b00e90073007a00ed0074006800650074002e0020002000410020006c00e90074007200650068006f007a006f00740074002000500044004600200064006f006b0075006d0065006e00740075006d006f006b00200061007a0020004100630072006f006200610074002000e9007300200061007a002000410064006f00620065002000520065006100640065007200200035002e0030002c0020007600610067007900200061007a002000610074007400f3006c0020006b00e9007301510062006200690020007600650072007a006900f3006b006b0061006c0020006e00790069007400680061007400f3006b0020006d00650067002e> /ita <feff005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f00620065002000500044004600200070006900f900200061006400610074007400690020006100200075006e00610020007000720065007300740061006d0070006100200064006900200061006c007400610020007100750061006c0069007400e0002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /jpn <feff9ad854c18cea306a30d730ea30d730ec30b951fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e305930023053306e8a2d5b9a306b306f30d530a930f330c8306e57cb30818fbc307f304c5fc59808306730593002> /kor <feffc7740020c124c815c7440020c0acc6a9d558c5ec0020ace0d488c9c80020c2dcd5d80020c778c1c4c5d00020ac00c7a50020c801d569d55c002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /lth <feff004e006100750064006f006b0069007400650020016100690075006f007300200070006100720061006d006500740072007500730020006e006f0072011700640061006d00690020006b0075007200740069002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b00750072006900650020006c0061006200690061007500730069006100690020007000720069007400610069006b007900740069002000610075006b01610074006f00730020006b006f006b007900620117007300200070006100720065006e006700740069006e00690061006d00200073007000610075007300640069006e0069006d00750069002e0020002000530075006b0075007200740069002000500044004600200064006f006b0075006d0065006e007400610069002000670061006c006900200062016b007400690020006100740069006400610072006f006d00690020004100630072006f006200610074002000690072002000410064006f00620065002000520065006100640065007200200035002e0030002000610072002000760117006c00650073006e0117006d00690073002000760065007200730069006a006f006d00690073002e> /lvi <feff0049007a006d0061006e0074006f006a00690065007400200161006f00730020006900650073007400610074012b006a0075006d00750073002c0020006c0061006900200076006500690064006f00740075002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006100730020006900720020012b00700061016100690020007000690065006d01130072006f00740069002000610075006700730074006100730020006b00760061006c0069007401010074006500730020007000690072006d007300690065007300700069006501610061006e006100730020006400720075006b00610069002e00200049007a0076006500690064006f006a006900650074002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006f002000760061007200200061007400760113007200740020006100720020004100630072006f00620061007400200075006e002000410064006f00620065002000520065006100640065007200200035002e0030002c0020006b0101002000610072012b00200074006f0020006a00610075006e0101006b0101006d002000760065007200730069006a0101006d002e> /nld (gebruik deze instellingen om adobe pdf-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. de gemaakte pdf-documenten kunnen worden geopend met acrobat en adobe reader 5.0 en hoger.) /nor <feff004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d00200065007200200062006500730074002000650067006e0065007400200066006f00720020006600f80072007400720079006b006b0073007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /pol <feff0055007300740061007700690065006e0069006100200064006f002000740077006f0072007a0065006e0069006100200064006f006b0075006d0065006e007400f300770020005000440046002000700072007a0065007a006e00610063007a006f006e00790063006800200064006f002000770079006400720075006b00f30077002000770020007700790073006f006b00690065006a0020006a0061006b006f015b00630069002e002000200044006f006b0075006d0065006e0074007900200050004400460020006d006f017c006e00610020006f007400770069006500720061010700200077002000700072006f006700720061006d006900650020004100630072006f00620061007400200069002000410064006f00620065002000520065006100640065007200200035002e0030002000690020006e006f00770073007a0079006d002e> /ptb <feff005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020006d00610069007300200061006400650071007500610064006f00730020007000610072006100200070007200e9002d0069006d0070007200650073007300f50065007300200064006500200061006c007400610020007100750061006c00690064006100640065002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /rum <feff005500740069006c0069007a00610163006900200061006300650073007400650020007300650074010300720069002000700065006e007400720075002000610020006300720065006100200064006f00630075006d0065006e00740065002000410064006f006200650020005000440046002000610064006500630076006100740065002000700065006e0074007200750020007400690070010300720069007200650061002000700072006500700072006500730073002000640065002000630061006c006900740061007400650020007300750070006500720069006f006100720103002e002000200044006f00630075006d0065006e00740065006c00650020005000440046002000630072006500610074006500200070006f00740020006600690020006400650073006300680069007300650020006300750020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e00300020015f00690020007600650072007300690075006e0069006c006500200075006c0074006500720069006f006100720065002e> /rus <feff04180441043f043e043b044c04370443043904420435002004340430043d043d044b04350020043d0430044104420440043e0439043a043800200434043b044f00200441043e043704340430043d0438044f00200434043e043a0443043c0435043d0442043e0432002000410064006f006200650020005000440046002c0020043c0430043a04410438043c0430043b044c043d043e0020043f043e04340445043e0434044f04490438044500200434043b044f00200432044b0441043e043a043e043a0430044704350441044204320435043d043d043e0433043e00200434043e043f0435044704300442043d043e0433043e00200432044b0432043e04340430002e002000200421043e043704340430043d043d044b04350020005000440046002d0434043e043a0443043c0435043d0442044b0020043c043e0436043d043e0020043e0442043a0440044b043204300442044c002004410020043f043e043c043e0449044c044e0020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200431043e043b043504350020043f043e04370434043d043804450020043204350440044104380439002e> /sky <feff0054006900650074006f0020006e006100730074006100760065006e0069006100200070006f0075017e0069007400650020006e00610020007600790074007600e100720061006e0069006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b0074006f007200e90020007300610020006e0061006a006c0065007001610069006500200068006f0064006900610020006e00610020006b00760061006c00690074006e00fa00200074006c0061010d00200061002000700072006500700072006500730073002e00200056007900740076006f00720065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f00740076006f00720069016500200076002000700072006f006700720061006d006f006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076016100ed00630068002e> /slv <feff005400650020006e006100730074006100760069007400760065002000750070006f0072006100620069007400650020007a00610020007500730074007600610072006a0061006e006a006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b006900200073006f0020006e0061006a007000720069006d00650072006e0065006a016100690020007a00610020006b0061006b006f0076006f00730074006e006f0020007400690073006b0061006e006a00650020007300200070007200690070007200610076006f0020006e00610020007400690073006b002e00200020005500730074007600610072006a0065006e006500200064006f006b0075006d0065006e0074006500200050004400460020006a00650020006d006f0067006f010d00650020006f0064007000720065007400690020007a0020004100630072006f00620061007400200069006e002000410064006f00620065002000520065006100640065007200200035002e003000200069006e0020006e006f00760065006a01610069006d002e> /suo <feff004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f00740020006c00e400680069006e006e00e4002000760061006100740069007600610061006e0020007000610069006e006100740075006b00730065006e002000760061006c006d0069007300740065006c00750074007900f6006800f6006e00200073006f00700069007600690061002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a0061002e0020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /sve <feff0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400200073006f006d002000e400720020006c00e4006d0070006c0069006700610020006600f60072002000700072006500700072006500730073002d007500740073006b00720069006600740020006d006500640020006800f600670020006b00760061006c0069007400650074002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /tur <feff005900fc006b00730065006b0020006b0061006c006900740065006c0069002000f6006e002000790061007a006401310072006d00610020006200610073006b013100730131006e006100200065006e0020006900790069002000750079006100620069006c006500630065006b002000410064006f006200650020005000440046002000620065006c00670065006c0065007200690020006f006c0075015f007400750072006d0061006b0020006900e70069006e00200062007500200061007900610072006c0061007201310020006b0075006c006c0061006e0131006e002e00200020004f006c0075015f0074007500720075006c0061006e0020005000440046002000620065006c00670065006c0065007200690020004100630072006f006200610074002000760065002000410064006f00620065002000520065006100640065007200200035002e003000200076006500200073006f006e0072006100730131006e00640061006b00690020007300fc007200fc006d006c00650072006c00650020006100e70131006c006100620069006c00690072002e> /ukr <feff04120438043a043e0440043804410442043e043204430439044204350020044604560020043f043004400430043c043504420440043800200434043b044f0020044104420432043e04400435043d043d044f00200434043e043a0443043c0435043d044204560432002000410064006f006200650020005000440046002c0020044f043a04560020043d04300439043a04400430044904350020043f045604340445043e0434044f0442044c00200434043b044f0020043204380441043e043a043e044f043a04560441043d043e0433043e0020043f0435044004350434043404400443043a043e0432043e0433043e0020043404400443043a0443002e00200020042104420432043e04400435043d045600200434043e043a0443043c0435043d0442043800200050004400460020043c043e0436043d04300020043204560434043a0440043804420438002004430020004100630072006f006200610074002004420430002000410064006f00620065002000520065006100640065007200200035002e0030002004300431043e0020043f04560437043d04560448043e04570020043204350440044104560457002e> /enu (use these settings to create adobe pdf documents best suited for high-quality prepress printing. created pdf documents can be opened with acrobat and adobe reader 5.0 and later.) >> /namespace [ (adobe) (common) (1.0) ] /othernamespaces [ << /asreaderspreads false /cropimagestoframes true /errorcontrol /warnandcontinue /flattenerignorespreadoverrides false /includeguidesgrids false /includenonprinting false /includeslug false /namespace [ (adobe) (indesign) (4.0) ] /omitplacedbitmaps false /omitplacedeps false /omitplacedpdf false /simulateoverprint /legacy >> << /addbleedmarks false /addcolorbars false /addcropmarks false /addpageinfo false /addregmarks false /convertcolors /converttocmyk /destinationprofilename () /destinationprofileselector /documentcmyk /downsample16bitimages true /flattenerpreset << /presetselector /mediumresolution >> /formelements false /generatestructure false /includebookmarks false /includehyperlinks false /includeinteractive false /includelayers false /includeprofiles false /multimediahandling /useobjectsettings /namespace [ (adobe) (creativesuite) (2.0) ] /pdfxoutputintentprofileselector /documentcmyk /preserveediting true /untaggedcmykhandling /leaveuntagged /untaggedrgbhandling /usedocumentprofile /usedocumentbleed false >> ] >> setdistillerparams << /hwresolution [2400 2400] /pagesize [612.000 792.000] >> setpagedevice multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 the epistemic representation: visual production and communication of scientific knowledge. f. lópez cantos* dpto. de ciencias de la comunicación. universitat jaume i de castellón, avda. vicente sos baynat s/n, 12071 castellón de la plana, spain. * corresponding author: email: flopez@uji.es; phone: + 34 670333065 received: 2013-12-15; accepted: 2014-08-08 abstract despite its great influence on the history of science, visual representations have attracted marginal interest until very recently and have often been regarded as a simple aid for mere illustration or scientific demonstration. however, it has been shown that visualization is an integral element of reasoning and a highly effective and common heuristic strategy in the scientific community and that the study of the conditions of visual production and communication are essential in the development of scientific knowledge. in this paper we deal with the nature of the various forms of visual representation of knowledge that have been happening throughout the history of science, taking as its starting point the illustrated monumental works and three-dimensional models that begin to develop within the scientific community around the fifteenth century. the main thesis of this paper is that any scientific visual representations have common elements that allow us to approach them from epistemic nature, heuristic and communicative dimension. keywords illustrations; models; pictures; photography; images; simulation; epistemology; communication; science lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 152 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 “aristarchus of samos published hypotheses of whose foundations is that the universe would be much higher because it assumes that the fixed stars and the sun are still, as the earth rotates around the center and that the magnitude of the field is such that those the circumference of the circle representing the earth is described by the distance to the fixed stars as the center of the sphere to the surface, which is impossible because, lacking central magnitude, cannot have any reason to the surface sphere. “ archimedes. the sand reckoner. 1. introduction. the heliocentric theory of aristarchus of samos is known mainly through the reference made to it by archimedes (c. 287-212 bc) in his psammites or sandreckoner, who cites to refute it on the grounds that it is unfounded, as is clear from the argument that ends the quoted text. aristarchus (c. 310-230 bc) had a significant influence on his contemporaries for his revolutionary proposals but, until today, has reached to us only one of his works, and preserved “by joining the anthology astronomical tracts in the work of pappus of alexandria “(coronado, 2006: 4). that is, nowadays, his work is credited as authorship of this crucial, and now proven, scientific assertion, indirectly through to us by archimedes, and other authors including plutarch too, who compiled and discussed their daring theories, expanding its influence whenever his work communicating the original purpose to others transcending time and space in which the theory was developed. the reflections on the nature of the heavenly bodies occupying both aristarchus and copernicus or were based on the observation of astronomical phenomena and mere intuition and, later on with galileo, supported by data validation more or less accurate collected about their behavior depending on the measuring instruments used and illustrations drawn through the telescope. tycho brahe, although he had the most technologically advanced resources available at his time, was the last astronomer who observed the heavens with direct vision, without any optical instruments intermediating lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 153 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 it. the wealth of data collected only was useful for it to develop a mixed theory about the movement of the planets around the sun and the earth, that would be revised a few years later by his pupil johannes kepler, who tested heliocentrism and shared it with galileo, as shown in interesting that maintained communication through written correspondence (appelbom and gallows, 2001). galileo built his first telescope in 1609, and next year a microscope, developed from predesign of a prototype describing by dutch zacarias janssen, of which we have evidence of its existence since august 1595, and immediately began to make illustrations about his observations. in 1610 galileo himself also performed microscopic views of insects, specifically a bee, and also published his famous careful edited and detailed drawings of the moon and the phases of venus, siderius nuncius, summarizing his astronomical observations. tycho brahe, a few years earlier, worried about building of threedimensional cosmological models constructed as product of his observations of celestial bodies, as he explained with a function that “was primarily cognitive: to convey to another the main motions of a special scheme would be more than readly done with words or pictures “(mosley, 2006: 216). in other words, to represent in the best way possible the theoretical model of the solar system in order to facilitate dialogue with other astronomers, as purpose to develop astronomic knowledge in a scientific way that would describe celestial phenomena studied with the participation of scientific community. in this paper we are concerned precisely, but not with strictly historicist spirit, focusing our attention on epistemological, heuristic and communicative dimension, perspective that analyzes how ontological and relational features of different media for visual representations induce the significant changes in the practices and forms of understand the production and distribution of scientific knowledge over the past centuries, from the artwork to contemporary virtual simulations. our work, to sum up, addresses the knowledge representation primarily from a visual and communicative perspective, lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 154 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 without being impaired for over a text is necessarily brief we deal, but only tangentially, some other inescapable issues related with philosophical, sociological, psychological, or otherwise approach, realizing that have already been treated extensively and thoroughly, and much better, from other academic disciplines. in any case, our main thesis is that it has to review the role that visual representations are in the production, processing and distribution of scientific knowledge, and we will deal with it because we understand the science is up about visuals and creative visual thinking and communication are inherent in their development. 2. representation and models in science. in a recent work that addresses the current production of knowledge in contemporary laboratory environment from a micro-semiotic approach (allamel-raffin, 2011) clearly shows the complexity of the representation and the problems that we have been saying. in this study we analyze the conversational exchanges of people involved in the collection and validation of the images recorded by a transmission electron microscope (tem) at the institute of physics and chemistry of materials in strasbourg, and their initial assumptions and conclusions aren’t too far at all classical concerns regarding the representation and the critical arguments about we have outlined in the previous example. as the study shows how the tem image production in the laboratory concerning lacking external macroscopic, is an added difficulty makes very difficult to distinguish artifacts and epistemic object, unwanted effects introduced by the technical system contaminate the sample. but work explicit, especially, as the generalization of the meaning of the images obtained is revealed as a problem inherent to science, supporting this thesis in the detailed analysis of conversational interaction between researcher and technical that shows the difficulty of select and determine the adequacy of the obtained image, that acquires epistemic status only after a complex communicative argumentation that sanctions by consensus its validity. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 155 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 the tem image, meanwhile, has no visible reference but it presupposes, and becomes even more relevant the communicative exchange and argumentative discourse among scientists to determine suitability. the small planetarium that help brahe to represent his theories about the heavenly bodies were also cosmological models but, in this case, in three dimensions and sometimes with a certain movement to represent also its behavior in time. in any case, model that makes its referents recognizable in relation, preferentially establishing direct visual analogies or explicitly supported data whose visual representation is sanctioned only as epistemic object in his communicative dimension. in these simple examples we have the enormous complexity inherent in any representation and interpretation issues, the questions of meaning and development of knowledge and, also, it is clear their eminently dialogical nature and the importance of the communicative dimension. we have no way to compare what we see with what the image shows tem. to validate these representations we use a strategy of interpretation in which what matters is the relationship between authors and readers, establishing a dialog autoreflective within the scientific community participant facing a technological system that acquires relevance and importance in itself. the peculiarities of the statute and the relationship of the author and the reader in these types of communicative exchanges have been well identified in the essential text of umberto eco, 1979 lector in fabula, which accounts for the ability to text to mediate between both, autonomous and ideal on in the middle of author and reader expectations, and sums in his famous concept model reader from linguists work as jakobson, peirce or saussure in textual semiotics. not dwell too much on the specifics features of textual semiotics, theoretical approach in our opinion obsolete in many of his proposals, but we are interested, however, and we will take care of it right away carefully, the notion of model, a generic concept and used extensively in various areas of science in applications apparently as diverse, among many other examples, as approach to the nature of lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 156 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 knowledge and scientific creation, study the characteristics of human cognition, analyze sociological or cultural or other areas or, simply design and develop technological tools. before that review these scenarios, we should briefly emphasize the relevance of communicative dimension in contemporary samples for scientific knowledge production apparently misleading may reflect production strategy based on naive consensus, as to presume that the own evidence of represented the facts in its forcefulness imposed itself among researchers over any other consideration of psychological, moral, cultural or socio-political-economic. but nothing is further that this idea from reality, the examples clearly reveal the power relations inherent in any communicative interaction, and in all of them, are, among other tracks, arguments of explicit authority used by persuasion, or dissuasion, imposing as valid discourse intersecting with consensus. complex relationships across communicative interaction have been well studied extensively from various fields, especially from culture critical theories of the frankfurt school philosophers like marcuse, adorno, horkheimer, wittgenstein, foucault, derrida or habermas, or by authors with more recent works like mattelard, ramonet or chomsky, among many others. discursive techniques whose analysis is being a constant in the history of philosophy from plato’s gorgias or aristotelian poetics or de institutione oratoria of quintilian, a tradition of studies also addressed recently, with public communication or educational approach, among others, by knight (2006), vickery (2000), mazzolini (1993) or balex and carre (1985). as common practice in scientific activity in contemporary laboratories, temple of mediation and collective technical consensus among scientists for the production of knowledge in our time, have been critically addressed on numerous occasions over the past decades, including by latour and woolgar (1979), knorr-cetina (1981), lynch and woolgar (1990) and latour (1990). we shall not dwell more on all on the detail the cultural, sociological or educational approaches of visual representations of knowledge lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 157 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 and their derivations in this brief paper, but to advance the thesis statement with we heading this head we are going to discuss in some detail what we mean when we talk about models and their role in the representation of scientific knowledge. the three-dimensional representations were produced hundreds of years ago, as in the example of tycho brahe, with rudimentary materials available at the time, and progressively was made on technical and new media began to avoid more versatile and manageable use, growing as an entire industry to boost its production to extensive and strong demand for cosmological models by astronomers (de chadarerian and hopwood, 2004) of wax reproductions of the human body by naturalists (chen, 1999), or prototypes for use in mechanical engineering (linson, 2003). the models, in this sense, became the expression and / or demonstration of previous theories but also may were used as precursors to the development of new theories. it is clear, that models and theories are interrelated, although it is difficult to establish this relation, how as pointed frigg & hartmann (2012): “one of the most perplexing questions in connection with models is how they relate to theories. the separation between models and theory is a very hazy one and in the jargon of many scientists it is often difficult, if not impossible, to draw a line”. other authors adopt a stronger cognitive perspective to determine its nature, as wartofsky (1979) who does not hesitate to assimilate modeling to own inherent characteristics of cognition: “all models are one or another form of linguistic utterance, used to communicate and intended factually true description ... we begin modeling, therefore, with our first mimetic acts, and with our first use of language.” and conclude “we continue modeling by way of what, on various grounds, have been distinguished as analogies, models, metaphors, hypotheses, and theories “, following the track of phylysin work, pinker, kosslyn (block, 1981) or shepard & cooper (1982), was summarized well in this collection assertion in gardner’s seminal work on cognitive psychology and lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 158 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 visuality by kosslyn (1987: 354): “the information stored (in memory) as correspondence with the thing and not arbitrary represented “. nevertheless, and independent to the approaching perspective and the insurmountable obstacle to differentiate the models of the theories themselves, we should highlight the representational capacity of the models for the production of knowledge and their cognitive and communicative nature. especially interesting is its heuristic potential, and we take care of it a little after, as has rightly been pointed jordanova (2004: 443), for those models (in reference to material and dimensional) “have long been an important issue in the history of science, medicine, and technology, thanks to the concern of particularly philosophers and sociologist with models as heuristic devices for scientific thinking”. in a similar vein, morgan and morrison (1999: 10) have characterized mainly models as “autonomous agents” and “instruments of investigation”, and winsberg (2010: 8) follows this path to address contemporary cutting edge virtual simulations, but pointing the unique in that they are often made because the data systems that intend to study are limited, so apply with the aim of replacing experiments and observations as data sources themselves to provide potential models about the world. there is also a second meaning of the term that is specifically focused on the referent and is most common in artistic activity and we found everywhere in our media culture, but also explicit characterizes our tem image of the sample. we refer to the common form of so-called models, subjects or objects, to represent artistically and free of all artifacts, all imperfections that pollutes it, connecting deeply with the platonic proposal archaic operating on the binomial model / copy and based on the concept of ideal. this intentional idealization of representation of referent itself underlies what has come to be known in the classical tradition as canon, examples of which can be found easily in many contemporary cultural manifestations, but in what we want, too on how to address the scientific representation, contemporary creations with both cutting-edge science instrumentation as illustrated in the monumental works of the fifteenth century. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 159 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 as pointed timely kusukawa (2006: 85), the production of illustrations in the works of fuchs and vesalius was related to the recovery project works of dioscorides and galen and based on classical sources. these monumental works, also historia animalium gessner (kusukawa, 2010), were mostly illustrations with pedagogical purposes and maintained close relations with the written text, but the most important is their intention to be models for the rest of visuals knowledge that circulated at the time, appealing to the classics and his encyclopedic completeness of their publications, as vesalius, for example, who explicitly adopted idea of canon from sculptor polycleitus (450-420bc). the works of these authors were not save to criticism from his contemporaries, who expressed great reservations respect to the scientific validity of their artwork as faithful reproductions of nature, arguing strongly that “one can from live plants often recognize their pictures , but from plants pictured, one could never gain knowledge of new live plants”, or “eschewed the notion pointed canonical body of an altogether, and chose to depict individual, particular organs, with their subtle individual differences in size, shape, and configuration”(kusukawa, 2006: 92). there is an extensive philosophical tradition about the nature of the model since at least plato reaches contemporary textual semiotics as we have shown, is acceptable, at least in our opinion and non-deepen the complexity inherent in the term this brief example allows us to approach the concept from two perspectives. the expressed and instrumental character and we have been discussing and collects well griesemer keller, who distinguishes between models for and models of: “characterises as models for various kinds of tools scientific activities, intervention or materials such a concept and theory development, in addition to their role in representing objects (or phenomena) already in existence, models of things” (griesemer, 2004: 435). but above all, it should be stressed that these were idealized representations with communicative purpose, which epistemic and heuristic nature, as we’ll see below, does not depend on support. that is, both the illustrations on paper or haptic three-dimensional models are only representations, and representations are always models. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 160 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 3. the contemporary epistemic representation. when the director of the paris observatory, françois jean dominique arago, publicly presented daguerre’s invention on 1839, august 19, to the members of the académie des sciences and the école des beaux arts, vehemently justifying his interest for four reasons: the originality of the invention, its relationship to the arts, and it obvious practical uses, and above all, it great advantage to science. photographic techniques quickly joined the scientific work and, although there were a few short years of scientific illustrations coexisted with photographic reproductions, very soon those were considered little more than an oddity and were quickly were replaced by mechanical procedures inherent in the new technology. its inventors, niepce and daguerre, had expressed the same effect in the new image registration procedure that had been developed, and wanted to represent “real” nature, with a desire for progressive perfection of technique to achieve excellent levels of quality. with identical premises henry fox talbot, recognized worldwide as an expert botanist and mathematician with also extensive knowledge of optics and chemistry in addition to being a poet and politician, worked, around 1839, to develop procedures for image registration that lead to edit the first book illustrated with photographs of history the pencil of nature, in 1844. talbot’s purpose was to continue and complete with new techniques his invention in order to reproduce images of the taxonomic work that had opened the natural philosopher swedish carl von linne in the late eighteenth century, following the trail of the monumental works already mentioned that began to develop and distribute from sixteenth century. talbot was claiming, in short, with an encyclopedic collection of nature developing an image reproduction system that mimicked the botanical specimens with the utmost truth and fidelity, as years later similarly continue manifesting ramón y cajal about “the perfection that images had reached at the time lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 161 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 making it possible, finally, with the new advances for color photography” (ramon y cajal, 1912). the first astronomical photographs were made by daguerre himself commissioned by arago, but with limited success, which did not prevent the illustrious geographer and naturalist alexander von humboldt who received excited and slightly crisp imperfect image registered in the daguerreotype he could contemplate january of that year. humboldt started to publish a few years later, in 1845, his cosmos, including photographic images, and after the public presentation the new technique was soon adopted by scientists from different specialties, among the first ettinghausen, who attended the conference of arago and, on returning to his laboratory, performed the first photo of a cell under the microscope, early understanding that photography could even replace the specimen for research (thomas, 2008). the evidence, the occulata certidune or virtual witnessing, began to perform the final assault for the independence or representation from its referent, playing a fundamental role in the creation of epistemic objects themselves. an evidence overlapped to epistemic representation that was located between nature and technology with its own identity and autonomy. an epistemic representation that in the case of living organisms german philosopher nicole c. karafyllis has recently called biofact (karafyllis, 2003), that became itself the referent and replace it. at those first scientistsphotographers soon joined researchers from various areas such as roger fenton, who in 1850 published human primate skeleton; anna atkins, who began to build botanical taxonomies recorded on photographic paper collections of algae and ferns from 1843. a few years later, dr. jules luys was forced to resort to photography to save his reputation with the criticisms of an illustrated publication on the central nervous system of the human being, who had had to reissue in 1873 with the title of iconographie photographique des centres nerveaux including impeccable seventy photographic images and sixty-five lithographs that eliminate any subjectivity in presenting their lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 162 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 research results. the photographic representation showed their supremacy of the illustration, and even darwin himself included photographs and engravings in the expression of the emotions in man and animals, published in 1872. and showed also very useful for subsequent technological developments, such as those made by the physiologist chrono-photographer jules marey, cinema immediate antecedent, or those published in 1884 by ottomar anschütz representing a sequence of storks taking off, flying and landing, that were fundamental to develop modern aviation. the use of photography also expanded in other areas, such as medicine providing medical breakthroughs, such as those conducted by robert koch in bacteriology or, a few years later, ramon y cajal in neurophysiology, which definitely strengthened the validity of the evidence microscopic photography. the consolidation of photographic representations as evidentiary proof was, of course, far-reaching and won a double operation. on the one hand, definitely delegitimized any image or illustration that showed the slightest sign of artistry and, on the second hand, demanded autonomy from its referent while loyalty was such that it could perfectly replace as many dreamed, serve as an example the physiologist marey “who dreamed of a wordless science”, in words collected by daston and galison (1992:81): “there is no doubt graphical that expression will replace all others are whenever one has at hand a movement or change of state in a word, any phenomenon. born before science, language inappropriate often measures to express exact or definite relations“. with the photographic pictures the assignation of authority status at representation to serve the uses of neo-positivist science as epistemic and communicative tool was over. it had erased any suspicious about the validity of the proxy and had blurred its relationship with the theories and the context in which it was generated. this neo-positivism would only widened and deepened over the decades, remaining unchanged throughout most of the twentieth century, characterized by the development of x-rays, diffractive optics and lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 163 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 the radiosonde, and years later the electronic and nuclear technologies nuclear. but, above all, was expanded inside popular culture due to the development of new mass-media, popularizing as new technical images out the borders of scientific community, strongly reinforcing its status and authority. the photographic representation, also, as we pointed out in relation to illustration or haptic models, kept intact in its heuristic ability for the elaboration of scientific knowledge, in addition enhanced greatly by the strong evidentiary value attributed. furthermore, in the context in which it was first developed as cultural industry, photographic representation contributed significantly to the popularization of science. that is, further deepening and extending the communicative dimension of epistemic representations, spreading its scope abroad the bounds of scientific community to society, encouraging public expand neo-positivist concept has been called see and believe (hüppauf & weingart, 2008: 11). it became more than ever a epic narrative mythologizes scientific activity as the big history of science, which began to populate the discourse of mass media. you can find numerous example, from the popularization of dna photography registered by rosalind franklin with which watson & crick developed his famous discovery, collected in recent work on the popularization of scientific images (nikolow & bluma, 2008); or, more specific surveys about the relationship of science to cinema and its influence (kirby, 2011). or simply having a look at work of some famous nobel prizes as ramon y cajal or cecil f. powell, whose research were impossible to carry out without the use of visual representation techniques, both photographic, without doubt a essential support to develop and disseminate scientific knowledge along the twentieth century. but very soon, as we said, the new techniques of image production and critical theories of communication carry to new approaches from the philosophy and history of science, especially since mid-fifties of past century to revise complex relationship of the image with the reality and its implications on scientific models and theories, in brief, their lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 164 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 interdependence with the production of knowledge, and its former status started to redefine. began to reconsider the nature of representation of new imaging technologies explicitly elaborated views that had no analogy whatsoever with their referents, because were created from vast amounts of data, and these were not visible directly, only interposing instrumentation and technique. that new tools showed very clear the role of intermediation in the representations early, as we can see easily realize today having a fast look over multiple display systems based on different types of sensors and capture techniques that build digital representation with multiple applications (met, pet, satellite, etc.). these new representations were becoming a simply translation of theoretical models about inaccessible directly referent to a graphic language for easy understanding and with and heuristic purposes, again leaving bare the nature of the representation as a model that had been diluted in photographic technology neopositivism we commented, starting to be banish entirely by the serious challenge for the whole of science caused the already mentioned work of thomas s. kuhn. in our contemporary digital, our classical illustrations or images are rapidly being replaced by the term visualization in a multimedia universe of mapped representations (tufte, 2006: 13) that are moving toward virtual simulation, and whose artistry has been highlighted (manovich, 2002; luminet, 2009), and resemble the illustrations of sixteenth century we have been mention along the text. the heuristic nature of representation is now shown us explicitly its model features, in the words of humphreys (2004) collected in frigg & hartmann (2012) and referred to the virtual simulations: “when standard methods fail, computer simulations are often the only way to learn a dynamical model something about; they help us to ‘extend ourselves’, as it were. in situations in which the underlying model is well confirmed and understood, computer experiments may even replace actual experiments, which has advantages and economic, minimizes risk (as, for example, in the case of the simulation of atomic explosions).” lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 165 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 thus, to address new virtual simulations we need, as frigg & reiss (2008) assert, a new approach from a) metaphysics, as “simulations create some kind of parallel world in which can be conducted experiments under conditions more favorably than in the ‘real world’“, a new analysis from b) semantics ”of how models / theories relate to concrete phenomena“, and from the c) methodology because it is an activity ”that lies ‘in between’ theorizing and experimentation“, and, finally, from d) epistemology. on these four requirements humphreys (2009: 625) adds, in a critical article about, a “fifth aspect of simulations is that in the mathematically oriented sciences, progress is now inescapably linked to technological progress”, holding a controversial debate: “i have never subscribed to myself that metaphysical position. i have argued in this article that their second and third claims are incorrect. computational science requires a new non-anthropocentric epistemology and a new account of how theories and models are applied. these requirements are, to me, more than sufficient to justify the claim that significantly computational science is a new sui generis activity accompanied by new, recognizably philosophical, issues. these methods claims that lie ‘in between’ theorizing and experimentation are, i believe, best interpreted metaphorically“. the controversy is served and the debate is open, and undoubtedly present in the near future than difficult to resolve, even more so when we are entering what is being referred to the era of simulation / gaming and video game culture, in which new screens begin to try from the perspective of the interfaces (quaggiotto, 2012), but that’s another story certainly interesting to analyze in the near future but that exceeds the limits of this short paper. 4. discussion. in this work, no matter what we bring the following years and as a very modest contribution to broad and open-ended debate about it, we have analyzed the scientific lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 166 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 representations as epistemic and heuristic tools, especially focusing our attention on its communicative dimension as its scientific pretensions to become, in his demonstrative role, in independent witness to validate scientific research as a model of previous theories about real, all of them inherent features, in our opinion, to any scientific representation, regardless its manifestation in two or three dimensions and its materiality. in short, we can state that in our present is appropriate to understand the production, processing and distribution of scientific knowledge from their eminently visual and ideal character and we have to address the analysis of illustrations, haptic models, images and virtual simulations as epistemic, heuristic and communicative representations: • epistemic representations, as a result of its own mainly purpose: to pack in their materiality some scientific knwoledge. • heuristics representations, in so far as their use provide tools to promote from itself further inventions and to create new knowledge from; • communicative representations, finally, as eminently communicative artifacts that serves to scientific community for interchange knowledge. in order to present our findings with balance, we have to mention some limitations of this research. could be interesting for future research, and necessary, to deal with the basis of historical and contemporary ideas about representation concept, but this approach could leads us too far from our main argumentation and in this paper we opt to admit simply that images and models are representations of “something”, as commonly is being used in literature of scientific communities. we are aware about it but despite this limitation we think this work will be valuable to start new interesting approaches and encourage new researchers to develop further investigations to the fundamental role that scientific images plays nowadays. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 167 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 5. references allamel-raffin, c. (2011): “the meaning of a scientific image: case study in nanoscience a semiotic approach, nanoethics n. 5, pp. 165–173. appelbaum, v.; baldasso, r. (2001): “galileo and kepler on the sun as planetary mover”, largocampo filosofare. eurosymposium galileo 2001, pp. 281-390. balex, d. & carre, c. (1985): visual communication in science. cambridge: cambridge university press. burke, p. (2003): “images as evidence in seventeenth-century europe”, journal of the history of ideas, vol. 64, no. 2, pp. 273-296. chen, joseph (1999): “the development of anatomic art and sciences: the ceroplastica anatomic models of la specola”, surgery, october 1999 volume 45 issue 4. chomsky, n. y focault, m. (2006): the chomsky-focault debate. on human nature. new york: new press. coronado, g. (2006): “heráclides y aristarco. propuestas astronómicas no ortodoxas en el pensamiento griego”. coris. revista de ciencias sociales y humanidades, n. 4. daston, l. & galison, p. (1992). “the image of objectivity”. representations. 40(autumn), pp. 81-128. daston, l. & galison, p. (2007): objectivity. new york: zonebooks. de chadarerian s. y hopwood, n. (2004): models: the third dimension of science. stanford: stanford university press. eco, u. (1979): lector in fabula. milan: bompiani. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 168 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 ford, b. j. (1992): images of science: a history of scientific illustration. london: british library. friedhoff, r. m. (1989): visualization: the second computer revolution. new york. harry n. abrams. frigg, r. & hartmann, s. (2012): "models in science", the stanford encyclopedia of philosophy (fall 2012 edition), edward n. zalta (ed.) frigg, r., & reiss, j. (2008): “the philosophy of simulation: hot new issues or same old stew?” synthese, pp. 593–613. galileo galilei (1610): siderius nuncius, transl. a. van helden, chicago (1962). galileo galilei (1632): dialogue concerning the two chief world systems, trans. by stillman drake, university of california press, 1953 (revised 1967). galison, p. (1997): image and logic: a material culture of microphysics. chicago: chicago university press. griesemer (2004)”three-dimensional models in philosophical perspective”, models: the third dimension of science. stanford: stanford university press. habermas, j. (1981): theorie des kommunikativen handelns, 2 vols. frankfurt am main: suhrkamp, traduc. inglés en the theory of communicative action (vol i 1984, vol ii 1987). boston: beacon. harry, r. (1992): the scientific image: from cave to computer. new york: abrahams. humphreys, p. (2004): extending ourselves: computational science, empiricism, and scientific method. oxford: oxford university press. humphreys, p. (2009): “the philosophical novelty of computer simulation methods”, synthese, pp. 615-626. hüppauf, b. & weingart, p. (2008): science images and popular images of sciences. new york: routledge. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 169 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 jordanova, l. (2004): “materials, models and visual culture”, models: the third dimension of science. stanford: stanford university press. karafyllis, n. c. (2003): biofacts. essays on man between artefact and living entity. paderborn: mentis. keller, e. f. (2000): “models of and models for: theory and practice in contemporary biology”, philosophy of science, 67 (proceedings). kirby, d.a. (2011): lab coats in hollywood: scientists impact on cinema, cinema's impact on science and technology. cambridge, ma: mit press. knight, d. (2006): public understanding of science. a history of communicating scientific ideas. london, new york: routledge. korr-cetina, k. (1981): the manufacture of knowledge. an essay on the constructivist and contextual nature of science. oxford: pergamon press. kosslyn, s. (1987): “imágenes mentales: ¿una creación imaginaria?”, en gardner, h. la nueva ciencia de la mente. historia de la revolución cognitiva. barcelona: paidós. kuhn, t. s. (1962): the structure of scientific revolutions. chicago: chicago university press. kusukawa, s. (2006): “the use of pictures in the formation of learned knowledge: the cases of leonard fuchs and andreas vesalius”, transmitting knowledge: words, images and instruments in early modern europe, oxford. oxford university press, pp. 73-96 kusukawa, s. (2010): the sources of gessner’s pictures for the historia animalium, annals of science, vol. 67, no. 3, july 2010, pp. 303-328. kusukawa, s. (2011): “picturing knowledge in the early royal s(ociety: the examples of richard waller and henry hunt”, notes & records of the royal society. latour, b. & woolgar, s. (1979): laboratory life: the social construction of scientific facts. london: sage. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 170 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 latour, b. (1990) “drawing things together”, representation in scientific practice. cambridge, mass. ;london : mit press. linson, h. et alii. (2003) 3d-printing the history of mechanisms. cornell university. lópez cantos, f. (2010): “la imagen científica: tecnología y artefacto”, revista mediterranea de comunicación, n. 1, pp. 158-172. lópez cantos, f. (2010): “más lejos, más cerca, más color. imaginar la ciencia en las fronteras”. ii congreso internacional de imagen,cultura y tecnología:medios ,usos y redes. madrid: universidad carlos iii, pp. 152-158. luminet, j-p. (2009), « science, art and geometrical imagination », conferencia invitada en el iau symposium 260 the role of astronomy in society and culture , unesco, 1923 enero, paris, url = http://arxiv.org/ftp/arxiv/papers/0911/0911.0267.pdf lüthy, c. & smets, a. (2009): “words, lines, diagrams, images: towards a history of scientific imagery”, early science and medicine, 14 pp. 398-439. lüthy, c. (2006):”where logical necessity becomes visual persuasion: descarte’s clear and distinct illustrations, transmitting knowledge: words, images and instruments in early modern europe, oxford. oxford university press. lynch, m. & woolgar (1990): representation in scientific practice. cambridge, mass. , london: mit press. manovich, l. (2002): “the anti-sublime ideal in data art” url = http://www.manovich.net/docs/data_art.doc martz, e. & francoeur e. (1997): history of visualization of biological macromolecules url = http://www.umass.edu/microbio/rasmol/history.htm> mazzolini, r. (1993): non-verbal communication in science prior to 1900. firenze: instituto e museo di storia della scienza. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 171 http://arxiv.org/ftp/arxiv/papers/0911/0911.0267.pdf http://www.manovich.net/docs/data_art.doc multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 mcguinn, c. (2004): mindsight: image, dream, meaning. cambridge, ma: harvard university press. morgan, m. y morison, m. (1999): “models as mediating instruments”, “models and autonomous agents”, models as mediators. perspectives on natural and social science. cambridge: cambridge university press. mosley, a. (2006): “objects of knowledge: mathematics and models in sixteenthcentury cosmology and astronomy”, transmitting knowledge: words, images and instruments in early modern europe, oxford. oxford university press. nikolow, s. & bluma, l. (2008): “science images between scientific fields and the public sphere”, en hüppauf, b. & weingart, p. (2008): science images and popular images of sciences. new york: routledge, pp-33-51. polanyi, m. (1967): the tacit dimension, london: cox & wyman. pólya, g. (1945): how to solve it. garden city, ny: doubleday. quaggiotto, m. (2012): “images of knowledge. interfaces for knowledge access in an epistemic transition”, second international workshop on knowledge federation, dubrovnik, croatia, october 3-6, 2010. ramón y cajal, s. (1912): fotografía de los colores. bases científicas y reglas prácticas. zaragoza : ed. prames (2006). rocke, a. j. (2010): image & reality. chicago: chicago university press. roe, a. (1952): ‘‘a psychologist examines 64 eminent scientists.’’ scientific american 187.5 november, pp. 21-25. ronald e. mcroberts, r. (2011): “satellite image-based maps: scientific inference or pretty pictures?”. remote sensing of environment, 115 (2011) 715–724. rudwick, m. (1976): “the emergence of visual language for geological science,17601840”. history of science, 14:1, pp. 49-95. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 172 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2226 social and technological sciences eissn: 2341-2593 shapin, s. & schaffer, s. (1985): leviathan and the air-pump: hobbes, boyle and the experimental life, princeton; princeton university press. shepard, r. & cooper, l. (1982). mental images and their transformations cambridge, thomas, a. (2008): "la fotografía en pos del conocimiento", en el mundo descrito. madrid: fundación ico. thomas, nigel j.t., "mental imagery", the stanford encyclopedia of philosophy (winter 2011 edition), edward n. zalta (ed.). thoren, v. (1990): the lord of uraniborg: a biography of tycho brahe, cambridge. trumbo, j. (2010): “making science visible: visual literacy in science communication”, visual cultures of science: rethinking representational practices in knowledge building and science communication. hanover, n.h.: dartmouth collegue press. tufte, e. (2006): beautiful evidence. cheshire: graphic press ltd. van helden, a. (1989): siderius nuncius. chicago: chicago university press. vickery, b. (2000): scientific communication in history. boston: scarecrow press. wartofsky, m. w.c (1979): models: representation and the scientific understanding. dordretch, london: reidel. winsberg, e. (2010): science in the age of computer simulation. chicago, london: chicago university press. lópez cantos, f. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 152-173 | 173 humphreys, p. (2009): “the philosophical novelty of computer simulation methods”, synthese, pp. 615-626. multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 19 digital transformation of universities: the influence of covid-19 and students’ perception anett erdmanna,* , aurora estrada presedoa, maría de miguel valdésa aesic university, camino valdenigrales, s/n, 28223 pozuelo de alarcón, spain * correspondence: anett.erdmann@esic.edu received: 27 july 2021; accepted: 10 september 2021; published: october 2021 abstract on account of the disruptive nature of the covid-19 disease, the present paper aims to analyze the main repercussions of the pandemic for the field of education. the objective is twofold: (1) to describe the process of digitalization and digital transformation of educational institutions, (2) to analyze students’ insights regarding the implementation of hybrid methodologies of learning and identify potential differences in perception across university degrees. a structured literature review on the resonance of the covid-19 pandemic for education is performed. from this, the concept of the digital divide has risen to prominence, as education digitalization has managed to broaden global social inequalities. in this context, the prevailing e-learning methodologies are detailed, outlining the differences between asynchronous and synchronous format. the topic of blended learning is put forward, detailing the diverse hybrid education models present in the actual paradigm of education. in order to analyze and measure the perception of students regarding education digitalization, a survey based on validated scales is conducted among 305 university students in spain. a statistical analysis reveals that the most frequently implemented teaching model within the covid-19 scenario is the mixture of in-person and synchronous remote lessons. in addition to this, overall satisfaction, perception of workload and confidence in the professional future differs across branches of study. finally, several implications for educational framework are presented. keywords: digital divide; covid-19; education; perception; blended learning to cite this article: erdmann, a., estrada, a., & de miguel, m. (2021). digital transformation of universities: the influence of covid-19 and student’s perception. multidisciplinary journal for education, social and technological sciences, 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0001-6194-568x https://doi.org/10.4995/muse.2021.16007 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 20 1. introduction the rapid spread of the sars-cov-2 virus resulted in severe global health, economic and social repercussions across all sectors of the economy, on top of accelerating the digitalization process (crawford, 2021). as a result of the restrictions imposed by governments to counteract the coronavirus, educational institutions remained closed for almost the whole of 2020 (world bank group, 2020), and hence were pushed to rapidly adapt to remote learning pedagogies. the different educational settings which have emerged in this context have been described and conceptualized, but the effectiveness of the new learning environments requires an ex-post evaluation of the experience in the course year 2020-21 (nuñez-canal & de obesso, 2021). this motivates us to analyze, from a student’s perspective, the degree to which the covid-19 pandemic has affected and transformed the field of education and identify possible differences across branches of study. the covid-19 pandemic resulted in not only an economic crisis, but moreover, a severe health crisis. this resonated through the field of education by accentuating the digital divide, which makes reference to how technological expansion can induce different opportunities of development depending on the social context of each individual (oecd, 2001). in this framework, we analyze the evolution of e-learning over time, which reflects the increasing integration of technology into the learning process with the objective to enhance the learning progress in the light of the industrial revolution 4.0 (halili & sulaiman, 2021). since the arrival of the covid-19 pandemic, the term blended methodologies has become the protagonist within the context of education digitalization, making reference to the mixture of both in-person and remote lectures (cronje, 2020). the objective behind education digitalization has always been to universalize access to education. from this idea, the notion of open educational resources is conceived, making reference to released learning materials, mostly in digital format, that are open access at zero cost (unesco, 2020). massive online open resources originated from this stream of thought, allowing the expansion of the traditional borders of education towards new unexplored territories (downes & siemens, 2013). https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 21 as a way to analyze the perception of education digitalization amongst university students in spain, a self-elaborated online survey was conducted based on validated likert scales. to analyze the results obtained, a two-sample t-test was performed in order to identify potential differences in students’ perception across university degrees. results reveal that the most frequently implemented teaching model within the covid-19 scenario is the mixture of in-person and synchronous remote lessons, and that satisfaction, perception of workload and confidence in the professional future differ across branches of study. 2. literature review 2.1. background on the impact of crises on education 2.1.1. the socioeconomic divide traditionally, crises tend to highlight issues such as the social divide, a term used to describe the existing breach between low-income and high-income countries as well as within each nation (garcía, 2004). we are currently engaged in an unpredictable health crisis. with social distancing being the key prevention method against the spread of the virus, the world has had to restrict overall social contact (cambridge, 2020). this situation has highlighted social inequalities and accentuated the digital divide, defined as inequality regarding access to digital devices and the internet (unesco, 2020). in terms of how crises traditionally impact the educational outcomes of students, according to the investigation on education and crises developed by the university of indiana (shafiq, 2010), these are the main factors that affect educational outcome: negative impact: 1. a cut in household income may lead to a reduction in the budget dedicated to pay for education. 2. a reduction in adult salaries may lead to an increase in child labor rates. 3. reductions in hourly wages tend to make people work longer hours, leading to parents having less time to help children with school activities. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 22 4. if a crisis leads to a labor market that does not reward educational preparedness, people would be less interested in investing in education. positive impact: 1. a reduction in labor force salaries may make child labor less attractive, for which people would rather invest the time in educating themselves. 2. during any crisis, people with higher education tend to suffer less than those without it, which can lead to more people enrolling in school. 2.1.2. the covid-19 pandemic the main consequence induced by the covid-19 pandemic in the field of education has been the closure of all educational institutions. school closures will not only exacerbate negative educational outcomes, but also a worldwide economic crisis, which is very likely to continue to impact at a global level even when schools start to reopen. this double impact will become a threat to the quality of learning over time. if countries do not take immediate actions to respond, these impacts will imply long-term costs in terms of human capital and well-being (world bank group, 2020). in order to discuss the long-term effects of the covid-19 pandemic, we must consider its repercussion over the main macroeconomic indicators, as these will allow us to understand the current global economic situation, in addition to offering information on the future recovery. one of the key repercussions of the current economic situation for education is the impact on demand, with an average decrease of economic growth in the euro area of 6.8% in 2020 (oecd, 2021). as a consequence of this, the global increase in unemployment and the general loss of income will test the ability of thousands of households to pay for school enrollment fees. even for students who do not drop out of school, households will be less likely to pay for educational supplies and materials until the economy recovers (ocu, 2020). due to this, reenrollment campaigns and financial incentives, such as scholarships, will be key to motivate students to apply for tertiary education degrees. these incentives should be focused towards those students that suffer from vulnerable https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 23 positions, who will be mostly found at public institutions. when facing a situation of economic crisis, countries must take advantage of the circumstances in order to create intrinsic reforms that will help improve the overall performance of education and, moreover, build resilience to face future situations (world bank group, 2020). the covid-19 pandemic affected 90% of students globally through the closure of educational institutions (statista, 2020). as stated by unicef (2020), the school experience goes far beyond the acquisition of knowledge, as it helps conciliate family and work time, develop social relationships and compensate for overall social inequalities. according to a study on the effects of lockdown on children’s health, one out of every four minors between the ages of 3 and 12 have spent over 6 hours daily in screen time since being confined at home, which entails a greater exposure to risks associated with the prolonged used of the internet (universidad del país vasco, 2020). regarding the availability of digital tools and an internet connection, 89% of the children living within the oecd countries have access to a computer and the internet. it should be noted that in many homes technological devices are shared between family members, which complicates things even further (unicef, 2020). with schools closing down, children’s education becomes more reliant on family members and the household environment (escuela andaluza de salud pública, 2020). the oecd forecasts that the educational gap generated during this time will have long-term effects and will end up excluding students who have not been able to keep up with digitalization requirements (oecd, 2020). faced with the closure of schools, children from disadvantaged household units may find their learning process affected due to a lack of resources to adapt to these new learning methodologies (garcía, 2004). the oecd calculates that in a normal school year the gap between children belonging to different income levels can reach up to a month of delay in learning. during the pandemic, this effect is estimated to have increased up to six months (oecd, 2020). several international organizations that protect the application of human rights (unicef, oecd and save the children) proposed the following guidelines, summarized below, which had to be present in future education plans in order to guarantee the right to education for every child: https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 24 1. digitalization: educational administrations should be the ones in charge of developing a centralized, digital teaching platform to guarantee homogeneity, accessibility and protection of personal data. the state should provide support to teachers as a way to train them on the new digital tools available (unicef, 2020). 2. families’ role: ensure there is an open and fluent line of communication between schools and households, in order to provide support to those families that suffer from a vulnerable position (save the children, 2020). 3. equity: provide support and learning resources to students at risk of exclusion. distribute digital devices to families who cannot afford them, or do not have access to the internet (save the children, 2020). 2.1.3. the digital divide for a long time, the only “weapon” available to counteract the spread of the coronavirus was social distancing (dr. ali, 2020). as previously mentioned, this situation induced the closure of all educational facilities in over 190 countries, affecting over 1.2 billion students worldwide (unesco, 2020). as digitalization of education gains relevance, we should not forget that there are over four billion people around the world that do not have access to the internet (world economic forum, 2018). within this context the concept of the digital divide gains awareness. we understand by this term the existing socio-economic gap between individuals, households, and overall geographic regions, regarding their access to icts (oecd, 2001). if we choose to completely digitalize the field of education, what will be of those who do not have the means to access technological devices? based on article 26 of the un’s universal declaration of human rights, the purpose of education has always been to become a universally accessible service. therefore, education digitalization might go against the fundamental pillar on which this concept is based. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 25 the digital divide is a worrying issue at a global level. in spain, according to the spanish national institute of statistics, 99% of the population between the ages of 16 and 24 have regular access to the internet (ine, 2020). meanwhile, in africa, according to the world bank, only 22% of higher-income households have access to the internet (world bank, n.d.). this exposes a wider set of problems such as nations’ digital infrastructure and the need for each country to develop a specific social-distancing education plan accordingly to the resources accessible and available to their population. 2.2. digitalization of the educational field though it might seem like the idea of distance education is quite recent, it has actually been a topic present in the education debate ever since society reached a certain level of alphabetization. in fact, we can talk about four main stages of remote learning. first, by the end of the 19th century, mailing education appeared. at the beginning this method consisted of teachers mailing books to their students, and students mailing back doubts and homework. this method evolved throughout the 20th century, becoming a one-way mailing system by which the student received the necessary resources to be his/her own teacher (garcía, 2002). second, during the 1970s, television and radio allowed for educational institutions to make the educational process more dynamic, as audiocassettes and videocassettes became the new learning methodologies (bernal, 2012). third, the invention of the personal computer marked the beginning of digital learning as we know it today. since this time, computer education consisted of multimedia materials for the student to interact with, which allowed efficiency and flexibility (garcía, 2002). fourth, with the spread of the internet over the last 10–15 years, new online learning materials and platforms started to arise. these tools, like for instance digital learning platforms, have been used as substitute or complementary tools to traditional classroom teaching with the objective to provide a more efficient learning environment for both, students and teachers (erdmann & torresmarín, 2019). yet, it was not until the covid-19 pandemic that the world found in online learning the only resource available to keep education going. in fact, due to the pandemic, digital https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 26 transformation has speed up making six years of progress (clifford, 2020). however, “emergency remote teaching” that has emerged in the anomalous context of the pandemic is different from what is known as e-learning (nuñez-canal & de obesso, 2021). 2.2.1. e-learning and blended methodologies developing from the previously mentioned traditional remote education techniques, the term “e-learning”, which refers to the process of teaching through the use of the internet (downes, 2005) has become very familiar. two different types of e-learning methodologies can be distinguished (hrastinski, 2008): (i) asynchronous learning makes reference to deferred online education; the content is posted online, but without live lessons.  advantages: flexibility is key, as it allows students to adapt their studies to their own personal commitments. student contributions are considered more thoughtful, as they spend more time concentrating on their own development (hrastinski, 2008).  disadvantages: some students face difficulties to follow through in the absence of inperson educational guidance from an instructor (hughes, 2014). (ii) synchronous learning refers to online lessons in real time.  advantages: students are able to interact instantaneously (hrastinski, 2008).  disadvantages: set time schedules that could be incompatible with students lifestyles (hughes, 2014). within this paradigm, the concept of blended learning becomes more prominent, making reference to the combination of both online and in-person education (cronje, 2020). there are four models of blended learning within the actual paradigm of education:  rotation model: based on a fixed schedule, students rotate between different learning methodologies, one of these always being online learning (hrastinski, 2019). https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 27  flex model: content is mostly delivered through online platforms, and the instructor only provides face-to-face support on specific tutoring matters (staker & horn, 2013).  self-blend model: online courses that students voluntarily take part in to provide additional support for their face-to-face courses. (staker & horn, 2013).  enriched-virtual model: mixture of in-person and online lessons (hrastinski, 2019). 2.2.2. open and universal access to education one of the main objectives behind education digitalization is the promotion of universal access to education, as it enhances the development of a more inclusive and cultured society (fielding, 2016). within this framework, we encounter the term open education, which encompasses the set of practices that pursue the promotion of open access to high-quality education, with the main objective of making learning available for all (decuypere, 2019). from this idea, the concept of open educational resources (oer) is born. as reported by the unesco (2020), oers are “teaching and learning materials in any medium – digital or otherwise – that have been released under an open license with no-cost access”. oers facilitated an organic and quick transition from traditional to remote education during 2020 (tang, 2020). though traditional physical textbooks have always been considered the key learning tool for any student, it has been proven that oers can be a substitute for them, at zero cost, and in fact, in some cases oers have managed to improve students' academic results (hilton & laman, 2012). by introducing the concept of open education into the digital field, we encounter the term of massive open online courses (moocs). these are free online courses that offer open access through the internet, with an unlimited number of student participations (nobre et at., 2018). two types of moocs can be distinguished (downes & siemens, 2013): (i) cmoocs: seek to build the bases of knowledge among all members of the learning community. each student chooses what they want to learn as there is no syllabus to follow (downes & siemens, 2013). https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 28 (ii) xmoocs: online courses offered through commercial platforms. there is a syllabus, set duration and certain tasks must be completed. a certificate is issued after its completion (peco & mora, 2014). moocs have the potential to promote the universalization of education, since they offer access to those with the least resources. on top of this, they guarantee a reduction in education costs without loss of quality (peco & mora, 2014) 3. evidence from students’ perception 3.1. methodology as a way to comprehend students’ perception of the digitalization of learning environments, a self-elaborated online survey was conducted and subsequently a descriptive analysis and hypothesis tests on differences across study branches were carried out. the questionnaire was performed on microsoft forms, consisting of ten qualitative questions that tackle three main issues: the impact of education digitization, the adaptability to this new education reality and students’ perception over their professional future. six out of the ten total questions developed for this survey were adapted from existing literature, based on validated sevenpoint likert scales, as detailed in table 1. the questionnaire was shared through the main social networks and communication channels most frequently used by university students: instagram and whatsapp. a total of 305 valid responses were gathered. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 29 table 1 measurement scales author construct adapted question answers self-elaborated branch of study you are currently studying: 1. arts and humanities 2. sciences (mathematics, physics...) 3. health and veterinary sciences 4. social and legal sciences 5. engineering and architecture. self-elaborated what educational regime does your university use? 1. face-to-face learning in its entirety 2. online synchronous learning in its entirety 3. asynchronous learning online in its entirety 4. face-to-face and synchronous learning 5. presential and asynchronous learning 6. synchronous and asynchronous learning self-elaborated do you have the appropriate technical means (computers, internet connection, etc.) to participate in the online sessions? yes / no self-elaborated before the covid-19 pandemic, did you have any experience with distance education? yes / no crosby, l. a., & stephens, n. (1987) satisfaction with the recovery process. (p. 527 smh) i am satisfied with the methodology currently used by my university in terms of online learning. (1 – 7) not satisfied / satisfied chae b., & li, x., & zhu, r. (2013) product effectiveness (performance). (p. 467 smh) how much do you believe that your university workload has increased since the implementation of a hybrid learning system? (1 – 7) not at all / very much chae b., & li, x., & zhu, r. (2013) product effectiveness (performance). (p. 467 smh) how much do you believe that your academic results have improved since the implementation of the hybrid learning system? (1 – 7) not at all / very much bracken, c. c. (2005) telepresence (negative physiological response). (p. 603 smh) to what degree did you experience a worsening of your mental health since the pandemic started? (1 – 7) no worsening / a lot white, a., breazeale, m. and collier, j. e. (2012) word-of-mouth likelihood (negative). (p. 662 smh) based on my experience, i would not recommend keeping the hybrid education model in my degree. (1 – 7) disagree / agree crosby, l. a. & stephens, n. (1987) satisfaction with the recovery process. (p. 527 smh) i feel favorably about my professional future. (1 – 7) disagree / agree smh: bruner, g. c. (2015). marketing scales handbook. source: self-elaborated. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 30 table 2 descriptive analysis based on averages criteria/question total average average by study branch science health & veterinary engineering & architecture economics & law arts & humanities branch of study 100% 6% 39% 15% 33% 7% educational regime your university uses synchronous online 12% 26% 7% 17% 15% 0% asynchronous online 2% 5% 1% 2% 1% 10% face-to-face 8% 11% 8% 9% 7% 5% face-to-face & synchronous online 68% 53% 67% 65% 70% 80% face-to-face & asynchronous online 4% 0% 5% 2% 5% 5% synchronous & asynchronous online 6% 5% 12% 4% 2% 0% technological means to participate in the online sessions yes 88.20% 89% 88% 96% 83% 95% no 0.33% 0% 1% 0% 0% 0% provided 1.31% 11% 0% 0% 2% 0% shared 10.16% 0% 11% 4% 14% 5% remote learning experience before covid-19 yes 10% 26% 4% 17% 8% 15% no 90% 74% 96% 83% 91% 85% university online methodology satisfaction (1-7) 3.83 3.42 4.02 4.24 3.45 4.10 workload increase since hybrid learning (1-7) 4.92 3.76 4.83 4.63 5.42 4.75 academic results improvement since hybrid learning (17) 3.44 4.05 3.45 3.57 3.28 3.35 mental health worsening since the pandemic started (1-7) 5.07 4.53 5.06 4.87 5.23 5.30 would not recommend hybrid education model (1-7) 5.01 5.32 4.94 4.78 5.16 4.85 professional future (1-7) 3.88 4.00 4.01 4.46 3.47 3.70 source: self-elaborated. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 31 table 3 statistical two-sample t-test analysis average economics & law science health & veterinary engineering & architecture arts & humanities online methodology satisfaction (1-7) 3.45 3.42 4.02 4.24 4.10 p-value 0.9551 0.0127 0.008 0.1167 result insignificant significant difference significant difference insignificant workload increase since hybrid learning (1-7) 5.42 3.79 4.83 4.63 4.75 p-value 0.0002 0.0114 0.0067 0.1041 result significant difference significant difference significant difference insignificant academic results improvement since hybrid learning (1-7) 3.28 4.05 3.45 3.57 3.35 p-value 0.0586 0.4204 0.3148 0.8548 result insignificant insignificant insignificant insignificant mental health worsening (1-7) 5.23 4.53 5.06 4.87 5.30 p-value 0.1400 0.4799 0.2766 0.9587 result insignificant insignificant insignificant insignificant would not recommend hybrid education (1-7) 5.16 5.32 4.94 4.78 4.85 p-value 0.9017 0.4363 0.2864 0.5290 result insignificant insignificant insignificant insignificant professional future (1-7) 3.47 4.00 4.01 4.46 3.70 p-value 0.2612 0.0316 0.0034 0.6079 result insignificant significant difference significant difference insignificant source: self-elaborated. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 32 3.2. descriptive and statistical analysis table 2 provides a descriptive analysis of the average evaluations from the survey, differentiated by study branch. in order to perform an accurate statistical analysis of the results obtained in the survey performed between university students in spain, we have executed a two-sample t-test which allows us to comprehend whether there exists a significant or insignificant difference between the results of two variables. setting up pairwise comparisons, economics & law has been chosen as the reference category for the benchmark analysis. the results of the two-sample t-test are provided in table 3, reporting the two-sided p-value. for a significance level of 5%, if p-value returns an amount below 0.05, the hypothesis that states that both means are equal will be rejected, as this would indicate that there is a significant difference in perception between the students of each study branch. 3.3. results we identify a significant difference in perception for those students of health & veterinary and engineering & architecture majors (p-valuehv=0.0122, p-valueea=0.008), which suggests that students of these two study branches are much more satisfied with the educational methodology adopted by their universities. this could be due to these areas of study being less focused on social interaction. likewise, students of these degrees could be more familiar with the use of technology, making it easier for them to adapt to these new methodologies. furthermore, economics & law students are found to perceive a large increase in work volume since remote learning methodologies were implemented, while students that belong to science, health & veterinary and engineering & architecture do not perceive such change in their workload (p-values=0.0002, p-valuehv=0.0114, p-valueea=0.0067). our conjecture is that this may be due to the fact that many teachers, who suddenly had to adapt to remote learning during the pandemic, believed that the easiest way to rank students’ evolution was through projects and essays. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 33 regarding academic results, we cannot reject the hypothesis on equal perception across degrees (p-values > 0.05 for all), as the overall response from our survey points out that the majority of students feel their academic results have neither improved nor worsened since the implementation of hybrid methodologies of learning. the same holds for mental health worsening (p-value > 0.05 for all). the magnitude of the scores suggests that most students perceive a clear worsening in their mental health since the start of the pandemic. we believe that this could be a direct consequence of the large amount of hours that students have spent locked at home without being able to develop social relationships, which is of great importance for this particular age bracket. there were also no significant differences across study branches regarding the recommendation of implementing hybrid learning environments (p-value > 0.05 for all). the high score obtained across all degrees implies rejection of the online environment under normal circumstances. this represents how, regardless of their individual branch of study, most university students in spain seem not to recommend these hybrid methodologies of learning to be implemented long-term. considering students’ confidence regarding their professional future, the hypothesis on equal perception as the reference group is rejected for health & veterinary and engineering & architecture students. that is, economics & law, science and arts & humanities students do not feel confident regarding their professional future. this can be due to employment rates of these three study branches being deeply affected by the pandemic, as a large percentage of graduates will dedicate their professional future to the private sector, which is usually severely harmed after economic crises. on the contrary, health & veterinary, and engineering & architecture students remain positive about their professional future. in the case of health students, this could be due to the covid-19 pandemic leading to an increase in the demand for graduates in health-related degrees. 3.4. discussion of results results suggest that, on average, students tend not to recommend remote methodologies of learning to be implemented in the long-term. as previously mentioned, restrictions imposed by governments affected 90% of the world’s students through the closure of all educational institutions https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 34 (statista, 2020). for this, students might feel negative regarding hybrid learning due to the large amount of time spent isolated at home. they might not recommend these approaches to be implemented in the long-term, not because of the methodology as such, but because they would prefer to spend more time attending in-person classes in order to develop social relationships. though youth unemployment in spain has severely increased during 2020 (oecd, 2021), survey results indicate an overall positive feeling from students when asked about their professional future. based on the bank of spain forecast of spain’s future economic situation (bank of spain, n.d.), a slow but steady recovery is predicted by 2023. this could be the reason why current university students, who will graduate in the next few years, feel hopeful about their future job situation. lastly, the aforementioned study performed by the universidad del país vasco (2020) relates the increase in the use of digital devises to potential mental and psychological risks, which aligns with the survey’s results as most students feel their mental health has worsened since the start of the pandemic. 3.5. implications for the education framework we find that the most frequently implemented methodology is the mixture of both in-person and synchronous online lessons. based on the survey data, students seem to be content overall with the implemented methodology within their institutions. due to this, in order for educational centers to successfully transition to remote learning, it should be recommended to alter between in-person lessons and online live sessions. this would ensure that students remain satisfied with the implemented methodology, as it allows them to maintain active communication with their fellow partners and teachers (keengwe & kidd, 2010). moreover, we have identified significant differences in the workload perceived, which suggests that the digital transformation of teaching should be career specific, as opposed to a uniform solution. alternatively, it may be a result of differences in the initial digital endowment, both for professors and students. in any case, this confirms the existing challenge for teachers to find the best combination of learning possibilities (nuñez-canal & de obesso, 2021) and provides insights on which aspects the differentiation of the teaching approach across study branches should focus. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 35 one of the most striking results from our survey is the global affirmative response from students when asked about their perception of having experienced a worsening in their mental health since the start of the pandemic. educational institutions should be aware of the challenges students are facing nowadays regarding this matter in order to implement measures that will help them cope with their learning evolution. for this, it is of increasing importance for institutions to take proactive actions in order to support the mental well-being of their students, such as providing counselors to give professional help (sahu, 2020). as a response to the issues emerging from the rapid digitalization of education, unesco (2020) proposes a set of key recommendations that focus on protecting the social aspect of education. this assessment goes hand-in-hand with the statement above regarding university students’ mental health. for this, we believe that protecting the social environment of education is key for students to be able to maintain good self-esteem and mental stability. to this we add the fact that most students would prefer not to maintain hybrid methodologies of learning within their institutions. therefore, once the situation allows, we believe that returning to in-person lessons should be the long-term goal for education. 4. conclusion in order to comprehend the degree to which the spread of the covid-19 disease has impacted the field of education, a thorough research study was performed. after analyzing the general framework of education and the repercussions suffered during the pandemic, we came to the conclusion that this crisis has highlighted two key concepts: the digital divide and the social divide. these two notions have become the center of debate when it comes to handling the covid19 crisis and the digitalization of education. at the beginning of the pandemic, social distancing became the only prevention against the spread of the virus. with students attending remote lessons, social differences between children became greater than ever. based on the aforementioned literature, children belonging to household units with limited access to digital devices or an internet connection have suffered a greater loss in their educational outcome for the academic year. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 36 since the internet became a commodity service in most regions around the globe, the idea of digitalizing education gained awareness as it provides the perfect vehicle to universalize access to education. as the process of education digitalization requires a tremendous investment in time, money, and infrastructure, it was always perceived as a long-term objective. still, on account of the spread of the coronavirus, nations realized that remote learning was the safest solution to provide students with access to education. still, students’ perception regarding the new learning environment differs across branches of study. in particular, students of health & veterinary or engineering & architecture are significantly more satisfied with the hybrid teaching framework than economics & law students. at the same time, economics & law students perceive a higher increase in workload, and are found to be less optimistic regarding their professional future. however, academic results and mental health seem to be uniformly affected by the new teaching environment. overall, we conclude that the covid-19 pandemic has taken education digitalization from a question of “if…” to a question of “how…”, forcing countries to act without having time to plan a strategy. remote education, though a very good solution for an emergency situation such as the one imposed by the covid-19 pandemic, is not the long-term option desired by university students. digital education is here to stay, but in-person education provides a level of social satisfaction yet to be replicated by online learning. acknowledgements: this paper has been derived from the final degree project “digitalization of education: the impact of the covid-19 pandemic and student’s perception” by aurora estrada-presedo and maría de miguel-valdés, presented at esic business & marketing school. author contributions: all authors contributed equally to the conceptualization of the paper; data collection and writing original draft preparation, a.e.-p. and m.m.-v.; supervision and revision, a.e; all authors have read and agreed to the published version of the manuscript. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. data availability statement: data are available from the authors on reasonable request. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 37 references adedoyin, o. b. & soykan, e. (2020). covid-19 pandemic and online learning: the challenges and opportunities. interactive learning environments, 1-13. https://doi.org/10.1080/10494820.2020.1813180 ali, d. w. (2020, may 18). online and remote learning in higher education institutes: a necessity in light of covid-19 pandemic. retrieved from https://eric.ed.gov/?id=ej1259642 banco de españa (n.d.) macroeconomic projections for the spanish economy. banco de españa. https://www.bde.es/bde/en/areas/analisis-economi/analisis-economi/proyeccionesmac/proyecciones_macroeconomicas.html banco mundial. (2020, december 2). 72 millones de niños más podrían verse afectados por la pobreza de aprendizajes. retrieved from https://www.bancomundial.org/es/news/pressrelease/2020/12/02/pandemic-threatens-to-push-72-million-more-children-into-learning-povertyworld-bank-outlines-new-vision-to-ensure-that-every-child-learns-everywhere bernal, l. (2012, december 8). breve historia de la educación a distancia. retrieved from https://fisioeducacion.es/educacion/educaciononline/109-breve-historia-de-la-educaciona-distancia bicen, h., ozdamli, f., & uzunboylu, h. (2014). online and blended learning approach on instructional multimedia development courses in teacher education. interactive learning environments, 22(4), 529548. https://doi.org/10.1080/10494820.2012.682586 boyoun chae, xiuping li, rui zhu (2013). judging product effectiveness from perceived spatial proximity. journal of consumer research, 40(2), 317–335. https://doi.org/10.1086/670393 bracken, c. c. (2005). presence and image quality: the case of high-definition television. media psychology, 7, 191-205. https://doi.org/10.1207/s1532785xmep0702_4 bruner, g. c. (2015). marketing scales handbook: multi-item measures for consumer insight research (vol. 8). gcbii productions, llc. cambridge. (2020). the benefits and costs of using social distancing to flatten the curve for covid-19. retrieved from https://www.cambridge.org/core/services/aop-cambridgecore/content/view/204bd93c135ec727faefc62e3be72c3b/s2194588820000123a.pdf/benefits_a nd_costs_of_using_social_distancing_to_flatten_the_curve_for_covid19.pdf clifford, t. (2020, december 23). coronavirus sped up digital transformation by 6 years for companies, twilio ceo says. retrieved from https://www.cnbc.com/2020/12/22/coronavirus-sped-up-digitaltransformation-by-6-years-twilio-ceo-says.html crawford, b. (2021, march 4). covid-19 has accelerated digital transformation here's how companies can adapt. world economic forum. https://www.weforum.org/agenda/2021/03/covid-19-accelerateddigital-transformation-how-companies-can-adapt/ cronje, j. (2020). view of towards a new definition of blended learning. the electronic journal of elearning. https://doi.org/10.34190/ejel.20.18.2.001 https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1086/670393 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 38 crosby, l., & stephens, n. (1987). effects of relationship marketing on satisfaction, retention, and prices in the life insurance industry. journal of marketing research, 24(4), 404-411. https://doi.org/10.2307/3151388 decuypere, m. (2019). open education platforms: theoretical ideas, digital operations and the figure of the open learner. european educational research journal, 18(4), 439-460. https://doi.org/10.1177/1474904118814141 downes, s. (2005). e-learning 2.0. elearn magazine, 2005(10), 1. https://dl.acm.org/doi/fullhtml/10.1145/1104966.1104968 erdmann, a., & torres-marín, a. (2019). can we improve academic performance and student satisfaction without additional time cost for teachers? evidence from a blended methodology in microeconomics. multidisciplinary journal for education, social and technological science, 6(2), 54-91. https://doi.org/10.4995/muse.2019.11869 escuela andaluza de salud pública. (2020, july 31). la educación, determinante de la salud afectado por la covid-19. retrieved from https://www.easp.es/web/coronavirusysaludpublica/1259-2/ fielding, h. (2016). “any time, any place”: the myth of universal access and the semiprivate space of online education. computers and composition, 40, 103-114. https://doi.org/10.1016/j.compcom.2016.03.002 garcía aretio, p. d. l. (2002). historia de la educación a distancia. retrieved from http://espacio.uned.es/fez/eserv/bibliuned:20191/historia.pdf garcía, j. (2004). brecha digital, brecha social, brecha económica, brecha cultural. bibliotecas públicas a la vanguardia, 1(3). http://eprints.rclis.org/6440/1/garcia.pdf halili, s. h., & sulaiman, s. (2021). students’ perception to integrate education 4.0 in science program. multidisciplinary journal for education, social and technological sciences, 8(1), 45-57. https://doi.org/10.4995/muse.2021.14768 hilton, j. (2016). open educational resources and college textbook choices: a review of research on efficacy and perceptions. educational technology research and development, 64, 573–590. https://doi.org/10.1007/s11423-016-9434-9 hilton iii, j., & laman, c. (2012). one college’s use of an open psychology textbook. open learning: the journal of open, distance and e-learning, 27(3), 265-272. https://doi.org/10.1080/02680513.2012.716657 hrastinski, s. (2008). asynchronous and synchronous e-learning. educause quarterly, 31(4), 51-55. https://er.educause.edu/articles/2008/11/asynchronous-and-synchronous-elearning hrastinski, s. (2019). what do we mean by blended learning? techtrends, 63, 564–569. https://doi.org/10.1007/s11528-019-00375-5 hughes, a. (2014, november 26). comparing asynchronous and synchronous learning. learning solutions magazine. https://learningsolutionsmag.com/articles/1577/comparing-asynchronous-andsynchronous-learning ine, 2021. inedatabase, https://www.ine.es/ , accessed dates: 18-20 april, 1-15 may 2021, 1 june 2021. https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.easp.es/web/coronavirusysaludpublica/1259-2/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 39 james, g. (2002, july 15). advantages and disadvantages of online learning. the communication initiative network. https://www.comminit.com/ict-4-development/content/advantages-and-disadvantagesonline-learning keengwe, j., & kidd, t. t. (2010). towards best practices in online learning and teaching in higher education. merlot journal of online learning and teaching, 6(2), 533-541. https://jolt.merlot.org/vol6no2/keengwe_0610.pdf montenegro, s., raya, e., & navaridas, f. (2020). percepciones docentes sobre los efectos de la brecha digital en la educación básica durante el covid-19. revista internacional de educación para la justicia social, 9(3), 317-333. https://doi.org/10.15366/riejs2020.9.3.017 nobre, a., mallmann, e. m., nobre, v., & mazzardo, m. d. (2018). mooc and oer: identity management. in: emerging trends, techniques, and tools for massive open online course (mooc) management. igi global. https://doi.org/10.4018/978-1-5225-5011-2.ch001 nuñez-canal, m., & de obesso, m. (2021, january). el modelo educativo híbrido: una respuesta necesaria de la enseñanza universitaria a partir de la covid-19. preprint. https://doi.org/10.13140/rg.2.2.34706.89289 ocu. (2020a, september 4). volver al cole en el curso del covid. retrieved from https://www.ocu.org/consumo-familia/derechos-consumidor/noticias/gastos-vuelta-cole oecd. (2001). understanding the digital divide. oecd publications. https://www.oecd.org/digital/ieconomy/1888451.pdf oecd. (2020b, november 19). the impact of covid-19 on student equity and inclusion: supporting vulnerable students during school closures and school re-openings. retrieved from https://www.oecd.org/coronavirus/policy-responses/the-impact-of-covid-19-on-student-equity-andinclusion-supporting-vulnerable-students-during-school-closures-and-school-re-openings-d593b5c8/ oecd (2021). oecd economic outlook, interim report march 2021. oecd publishing paris. https://doi.org/10.1787/34bfd999-en oecd, 2021. oecd.stat. https://stats.oecd.org/ peco, p. p., & mora, s. l. (2014). los mooc: orígenes, historia y tipos. comunicación y pedagogía: nuevas tecnologías y recursos didácticos, 269-270, 41-48. plena inclusión. (2020). el derecho a la educación durante el covid-19. retrieved from https://www.plenainclusion.org/sites/default/files/el_derecho_a_la_educacion_durante_el_covid19.pd f rodicio-garcía, m. l., ríos-de-deus, m. p., mosquera-gonzález, m. j., & penado abilleira, m. (2020). la brecha digital en estudiantes españoles ante la crisis de la covid-19. revista internacional de educación para la justicia social, 9(3), 103–125. https://doi.org/10.15366/riejs2020.9.3.006 sahu p. (2020). closure of universities due to coronavirus disease 2019 (covid-19): impact on education and mental health of students and academic staff. cureus, 12(4). https://doi.org/10.7759/cureus.7541 save the children. (2015, march). iluminando el futuro. invertir en educación es luchar contra la pobreza infantil. retrieved from https://www.savethechildren.es/sites/default/files/imce/docs/pobreza-equidadeducativa-espana_iluminando-el-futuro.pdf https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 40 save the children. (2020). save our education. save the children. retrieved from https://www.savethechildren.net/save-our-education-report/ shafiq, m. n. (2010). the effect of an economic crisis on educational outcomes: an economic framework and review of the evidence. current issues in comparative education, 12(2). siemens, g., & downes, s. (2013). understanding massive open online courses: what is mooc? cemac edtech notes. https://www.cemca.org/ckfinder/userfiles/files/edtech%20notes%202_littlejohn_final_1june2013.p df staker, h., & horn, m. b. (2012). classifying k-12 blended learning. innosight institute. https://www.christenseninstitute.org/wp-content/uploads/2013/04/classifying-k-12-blendedlearning.pdf statista. (2020, october 22). matriculados en universidades de españa 2008/2009-2019/2020, por tipo universidad. https://es.statista.com/estadisticas/479407/alumnos-matriculados-en-universidades-enespana-por-tipo/ statista. (2020, march 31). casi el 90% de los estudiantes del mundo, sin clase por el covid-19. retrieved from https://es.statista.com/grafico/21284/paises-que-han-cerrado-sus-centros-educativos-por-lapandemia-del-covid-19/ tang, h. (2021). implementing open educational resources in digital education. educational technology research and development, 69, 389–392. https://doi.org/10.1007/s11423-020-09879-x unesco. (2020, august 10). startling digital divides in distance learning emerge. retrieved from https://en.unesco.org/news/startling-digital-divides-distance-learning-emerge unesco. (2020a). education in a post-covid world: nine ideas for public action international commission on the futures of education. retrieved from https://en.unesco.org/sites/default/files/education_in_a_post-covid_worldnine_ideas_for_public_action.pdf unesco. (2020b, september 10). ¿qué ha revelado la pandemia a nivel de brechas y deudas en la sociedad? retrieved from https://es.unesco.org/news/que-ha-revelado-pandemia-nivel-brechas-y-deudassociedad unesco. (2020, 13 mayo). open educational resources (oer). https://en.unesco.org/themes/buildingknowledge-societies/oer unicef. (2020). la educación frente al covid-19. retrieved from https://www.unicef.es/educa/biblioteca/laeducacion-frente-al-covid19#:%7e:text=propuestas%20para%20impulsar%20el%20derecho,con%20un%20enfoque%20de%2 0emergencia unicef. (2020b, december 1). education and covid-19. retrieved from https://data.unicef.org/topic/education/covid-19/ united nations. (n.d.). universal declaration of human rights. https://www.un.org/en/about-us/universaldeclaration-of-human-rights https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://es.unesco.org/news/que-ha-revelado-pandemia-nivel-brechas-y-deudas-sociedad https://es.unesco.org/news/que-ha-revelado-pandemia-nivel-brechas-y-deudas-sociedad multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 erdmann, estrada-presedo, and de miguel-valdés (2021) mult. j. edu. soc & tec. sci. (2021), 8(2), 19-41. https://doi.org/10.4995/muse.2021.16007 41 united nations. government and human rights. retrieved from https://www.ohchr.org/en/issues/localgovernment/pages/index.aspx#:%7e:text=although%20the% 20primary%20responsibility%20for,of%20human%20rights%20in%20society universidad del país vasco. (2021). estudio sobre confinamiento y salud en población infantil. retrieved from https://www.ehu.eus/documents/3638427/14308653/estudio-confinamiento-y-salud-opikencuesta.pdf/2b6a5600-af28-761e-7bbd-77544da65c6a?t=1586195983000 white, a., breazeale, m. & collier, j. e. (2012). the effects of perceived fairness on customer responses to retailer sst push policies. journal of retailing, 88(2), 250-261. http://dx.doi.org/10.1016/j.jretai.2012.01.005 who (n.d.). covid-19 events as they happen. retrieved from https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen world bank (2020). the economic and social impact of covid-19. western balkans regular economic report, no. 17, spring 2020. washington, dc. http://documents1.worldbank.org/curated/en/590751590682058272/pdf/the-economic-and-socialimpact-of-covid-19-education.pdf world bank. (2020, december 30). education and technology. retrieved from https://www.worldbank.org/en/topic/edutech world bank. (2020, march). covid-19: impacto en la educación y respuestas de política pública. https://openknowledge.worldbank.org/bitstream/handle/10986/33696/148198sp.pdf?sequence=6&is allowed=y world economic forum. (2018). 4 billion people still don’t have internet access. retrieved from https://www.weforum.org/agenda/2016/05/4-billion-people-still-don-t-have-internet-access-here-show-to-connect-them/ https://doi.org/10.4995/muse.2021.16007 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 39 an analysis of the link between metacognitive beliefs and academic health-oriented lifestyle preventive behaviours mediated by self-efficacy during the covid-19 pandemic fatemeh moradi a* , sara aghaei sabetb , sahar soufic adepartment of mathematics, faculty of basic sciences, yadegar-e-imam khomeini (rah) shahre rey branch, islamic azad university, tehran, iran bdepartment of psychology, faculty of humanistic sciences, yadegar-e-imam khomeini (rah) shahre rey branch, islamic azad university, tehran, iran cdepartment of educational and developmental psychology, faculty of education and psychology. shahid beheshti university, theran, iran * correspondence: dr.fateme.moradi@gmail.com received: 16 december 2021; accepted: 02 february 2022; published: april 2022 abstract the outbreak of the coronavirus and its ramifications, including lockdown and shutting down of many learning centers magnified the necessity of turning to virtual learning, has given rise to concerns about students’ learning and their reaction to the new teaching methods and the challenges they entail. the present study attempts to examine the link between metacognitive beliefs and behaviors that prevent health-oriented lifestyles through academic self-efficacy during the covid-19 pandemic. the statistical sample consisted of 239 randomly selected undergraduate students of engineering at azad university of shahre-e rey in the second semester of 1399-1400 (solar hijri calendar). the data were gathered using academic health-oriented lifestyle test salehzadeh et al. (2018) academic self-efficacy beliefs questionnaire (asebq), and wells’ metacognition questionnaire-30 (mcq-30). findings of the study indicated that metacognitive beliefs are related to behaviors preventing academic health-oriented lifestyle factors, such as learned helplessness and effort withdrawal, through self-efficacy. it is suggested that, according to the positive relationship that exists between metacognitive strategies and academic achievement. as well as these strategies are scalability learners need to be trained in skills such as self-regulation selfmonitoring planning and goal setting. until they dominate decided cognitive strategies and professors can also encourage students to use orientation by designing assignments, teaching study strategies, and using new teaching methods. keywords: metacognitive beliefs, self-efficacy, behaviors preventing academic health-oriented lifestyle, learned helplessness, effort withdrawal to cite this article: moradi, f., aghaei, s., soufi, s. (2022). an analysis of the link between metacognitive beliefs and academic health-oriented lifestyle preventive behaviours mediated by selfefficacy during the covid-19 pandemic. multidisciplinary journal for education, social and technological sciences, 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0001-9609-9057 https://orcid.org/0000-0002-9838-7770 https://orcid.org/0000-0001-5666-869x https://doi.org/10.4995/muse.2022.16879 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 40 1. introduction a review of empirical evidence derived from studies on prerequisites and implications of academic success indicates that cognitive processes have been largely overlooked in educational systems. metacognition is an important factor in formulating conceptual processes of academic success. metacognition can be defined one’s awareness of the way one learns, uses available facts to achieve an aim, evaluates cognitive processes useful for a particular task, chooses strategies to realize a goal, and assess one’s performance during and following an action. one’s awareness of tasks, strategies one adopts to fulfil those tasks, and one’s cognitive beliefs are the three metacognitive factors flavell identifies; furthermore, he asserts that metacognition can be useful in devising and adjusting plans (flavell & miller, 1998). metacognitive learning strategies are among the important components of self-regulated learning which enable the learners to monitor and regulate their cognition and emotions. students with more self-efficacy are more willing to adopt metacognitive strategies, and self-efficacy has a positive relationship with metacognitive strategies(ozgur, 2021; hayat & shateri, 2019). most experts distinguish metacognitive knowledge from metacognitive supervision. metacognitive knowledge consists of the information one acquires about one’s cognition and one’s learning strategies, and the way the latter influences the former. cognitive supervision refers to a range of executive functions, such as paying attention to, controlling, and identifying errors (biabangard,2003). experts have always recognized metacognition as a factor influencing learning and educational variables (osoli et al., 2016). metacognition refers to psychological structures, knowledge, and processes that control, modify, and interpret thoughts and perceptions. furthermore, metacognition can be defined as the awareness of the way one learns, uses available facts to achieve an aim, evaluates cognitive processes useful for a particular task, chooses strategies to realize a goal, and assess one’s performance during and following an action (salarifar et al., 2011). metacognitive beliefs are those parts of the cognitive knowledge that link a person’s beliefs about cognition to cognitive and emotional experiences (wells, 2009). metacognitive beliefs may function in positive and negative ways. negative metacognitive beliefs deal with the uncontrollability of cognitive thoughts and experiences, and also, the dangers they might pose. positive metacognitive beliefs, on the other hand, interpret concerns, anxieties, threats, and similar phenomena in a positive way, identifying the advantages and benefits of engaging in cognitive activities that comprise the https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 41 cognitive-attentional syndrome. cognitive-attentional syndrome is a reiterating thought process that manifests as anxiety and mind rumination fixed on threats and preventative behaviors such as repression and withdrawal (khamisabadi, mirmehdi & meraati, 2021; salarifar et al., 2011). negative metacognitive beliefs interpret cognitive phenomena such as thoughts and normal beliefs in a negative way, resulting in a disruption of thought control and negative interpretation of events (wells, 2009). the present study is an attempt to investigate metacognition as an important effect in learning and academic success. health-oriented lifestyle behaviors are among the important factors that may predict and formulate academic performance of students. in the context of health studies, health-oriented lifestyle behaviors are a series of behaviors under a person’s control, comprising that person’s daily behavioral patterns. developmental situations, such as learning, giving an examination, or partaking in classes may entail various demands, and positive or negative emotions (burić et al., 2016). facing positive or negative emotions is an inevitable consequence of teach (pekran et al., 2007). researchers have attempted to use process-based conceptual models of learning life effectiveness to determine the distinctions between values assigned to objective and subjective symptoms, emphasizing the crucial role played by meditating mechanisms, such as preferred methods of meeting motivational demands of learning as well as a wide range of conceptual factors facilitating cognitive r esources to face the inherent challenges of learning (amid et al., 2016). a factor that may be helpful in predicting and assessing students’ academic performance is their academic health-oriented lifestyle behaviors. the emerging idea of academic health-oriented lifestyle behaviors relies on the contemporary concepts of motivational development, such as self-efficacy theory, attribution theory (weiner, 1994), mastery goal orientation theory (anderman & anderman, 2009), self-esteem theory (covington, 1984), hope theory (snyder et al., 2002), self-determination theory (deci & ryan, 2000), expectancy theory (wigfield & eccles, 2000), and control-value theory (pekrun, 2006), and turns to positive learning to identify the most common behavioral indications of facing the challenges of the academic life. in educational studies, academic health-oriented behaviors are the preferred models of behaviors that may be chosen and expressed by learners. based on their nature and functions, they can improve learners’ academic wellbeing or threaten it. some of these behaviors are expressed as facilitators and promoters of academic wellbeing, such as mastery goal orientation, while others are inhibitors of academic wellbeing and disrupt learning, such as https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 42 learned helplessness and effort withdrawal (lajoie, pekrun, azevedo, & leighton, 2020; putwain, wood & pekrun, 2020; shaowei & chia-ching, 2019; salehzadeh et al., 2018). in the present study, two preventive factors of academic wellbeing, namely effort withdrawal and learned helplessness have been examined. effort withdrawal describes the behavior of learners that refuse to extend the necessary much effort for understanding and completing their academic assignments. researchers in the field of motivational development have presented evidence indicating that in addition to mastery and functional goal orientations, learners may also express effort withdrawal in learning contexts (tankari, zouhin toboula, ousseini, 2020; jarvis & seifert, 2002). many researchers have examined effort withdrawal as an orientation and have studied its associations with other variables. such a behavioral pattern in developmental contexts indicates a reduction in assigned values and loss of interest in academic activities. the reduction, which can be quite intense and serious in some cases, entails implications, such as indifference towards learning. the indifference will, in turn, lead to less value being assigned to learning and lack of interest in it (harackiewicz et al., 2008). an attitude of learned helpless suggests a person’s exposure to disrupting events and experiences of failure. such a person will gradually realize that the results and implications of these events are formed regardless of the choices he makes. in other words, the person registers the independence of results from his choices, and concludes that what he does has no bearing on the results. in these circumstances, the person experiences mental anxiety and reduced self-efficacy. the assumption is that when the implications of a behavior are independent of choices the person makes, there is no motivation to control the outcome. the lack of control will then entail its own set of outcomes: reduced motivation to respond, debilitation of cognitive capacity to perceive success, and finally, an increase in the number of negative emotional responses (putwain, wood & pekrun, 2020; shaowei & chia-ching, 2019; oyelekan, jolayemi & upahi, 2019; maier & seligmsn, 1979). research findings have demonstrated that after numerous failures, one’s efforts to respond to different situations slacken (maier & seligman, 1979). as one realizes behaviors and results are independent, one’s anticipation of change diminishes. in most cases, such a person’s voluntary responses to a situation grow weaker as well. people tend to extend their earned helplessness receptions to other circumstances as well; consequently, their learned helplessness turns into a helplessness as habit before long. (abramson et al. 1978) have proposed an attributive framework for such behaviors. according to their framework, based on the nature of the rationale one employs https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 43 to make sense of an event, and also based on the three criteria of stability (or instability), generality (or exclusivity) and internality (or externality), the resulting helplessness will be specified (peterson & steen, 2002) as a motivational factor, self-efficacy can boost academic performance and reduce anxiety; therefore, identifying factors influencing self-efficacy can play a major role in students’ academic success and the development of educational systems. self-efficacy is one’s capacity to face challenges to realize goals and achieve success. more than the students’ intelligence or their learning capabilities, personal characteristics such as trust in oneself (confidence), fighting spirit, ability to analyze reasons for failure (self-evaluation), ability to devise new arrangements and social methods to achieve goals (self-regulation), and the ability to control impacts (self-leadership) lead to academic development and success. self-efficacy influences behavior selection, engagement and effort extension, and pursuing goals. furthermore, it determines the way one faces obstacles and challenges. according to bandura’s theory, self-efficacy plays a critical role in a person’s psychological adaptability, resistance to mental problems, physical well-being, and also self-guided behavior-altering strategies (maddux, 2003). many researchers have studied the link between selfefficacy and academic success. (ozkal, 2019) studied the influence of self-efficacy and metacognition on mathematical performance of middle-school students in turkey, concluding that there is a relationship between self-efficacy, metacognition, and mathematical performance. göktepe yıldız (2020) demonstrated that teaching metacognitive concepts can improve geometrical performance of teachers who were undergoing education as part of their teaching careers. teaching metacognitive concepts can improve teachers’ awareness of learning. tossavainen et al. (2021) studied freshman engineering students in sweden to establish a link between self-efficacy and mathematical performance, showing that students with more self-efficacy perform better mathematically. nicoloff (2018) have studied self-efficacy, anxiety, and attitudes towards mathematics among students at a college in southwestern usa. their findings indicated that as self-efficacy increases, anxiety and negative attitudes towards mathematics can be alleviated. ozkal (2019) examined the influence of self-efficacy on learning and academic development of sixth, seventh, and eighth highschool forms in turkey. according to their findings, the students’ belief in their self-efficacy in mathematical learning and performance has a significant relationship with their educational development. https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 44 paying attention to the quality of education is crucial in promoting student acceptance and advancing science (ferrándiz, puentes, moreno & flores, 2016; hoveida & molavi, 2008; birnbum,1998) as centers of scientific, cultural and educational production, universities are the source of innovation necessary for solving society’s current issues (haji khajelu & hesampour, 2011). the ongoing and regular evaluation of students’ academic performance is necessary for devising better educational plans and improving the quality of education, especially in universities, making it an indispensable asset for improving the quality of education, and ultimately, for adjusting and enhancing administrative capabilities (hojjati et al., 2017; sajadi, 2003). the coronavirus pandemic has affected all human activities, and its impact on students’ academic life has been especially noticeable (papapicco, 2020). institutes of education were shut down in over 60 countries all around the world as a result of the critical conditions brought about by the pandemic, compelling educators to turn to virtual tools (iyer et al., 2020). while virtual education was already common before the coronavirus pandemic, its adoption by most governments as the prevalent educational system during in the critical situation caused by the pandemic was a notable change, and its quality and performance remain to be fully studied and evaluated (abasi et al., 2020). the present study is an attempt to examine the influence of metacognitive beliefs and behaviors preventing health-oriented academic lifestyle mediated by self-efficacy on students’ academic development during the coronavirus pandemic. based on these premises, the present study will try to answer the following question: is there a link between metacognitive beliefs through self-efficacy and health-oriented academic lifestyle preventive behaviors through learned helplessness? 1.1. methodology the present study employs a descriptive correlational research design. most variables were examined through the path analysis method. the participants were 239 randomly selected undergraduate engineering students at imam khomeini azad university. study goals were described and engaging participants’ attention, three questionnaires of metacognitive beliefs, self-efficacy, and healthoriented academic lifestyle were presented to them via an online platform. the collected data was analyzed using spss 26 and amos software programs. the inclusion criterion was the students’ involvement with education for at least one semester, and the exclusion criterion was their failing to answer at least 95 percent of questions. https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 45 wells’ metacognition questionnaire-30 (mcq-30). mcq-30 is a self-report scale developed in 1997 and revised in 2004 by wells and cartwright-hatton. in its current form, it consists of 30 questions using a 4-point likert scale (1 = do not agree, 2 = somehow agree, 3 = agree, 4 = completely agree). the scale comprises five sub-scales: 1. unruliness and danger, 2. positive beliefs about concerns, 3. confidence in memory, 4. necessity of thought management, 5. cognitive self-awareness. the psychometric attributes of the questionnaire were examined by (shirinzadeh et al., 2009). the questionnaire’s cronbach alpha was reported as 0.91 for the entire questionnaire, and 0.87, 0.86, 0.81, 0.80, and 0.71 for unruliness, positive beliefs, cognitive self-awareness, and the necessity of thought management in iranian samples, respectively. academic self-efficacy beliefs questionnaire (asebq). this questionnaire was developed by (zajacova et al., 2005). it takes account of four variables: confidence in academic performance in the classroom, confidence in academic performance outside the classroom, confidence in the ability to interact with the others in an academic context, and confidence in managing work, family, and academic lives. the questionnaire consists of 27 questions using a 10-point likert scale (0 = complete lack of confidence to 10 = full confidence), meaning the lowest score a participant can get is 0 and the highest score a participant can get is 270. according to zajacova, et al (2005) the reliability of the questionnaire can be between 0.72 to 0.90 (shokri et al. 2011) have standardized the questionnaire in iran. their confirmatory factor analysis confirmed these variables. furthermore, using the cronbach alpha they calculated the reliability of these variables as 0.88 for confidence in academic performance in the classroom, 0.85 for confidence in academic performance outside the classroom, 0.83 for confidence in the ability to interact with the other in an academic context, and 0.72 for confidence in managing work, family, and academic lives. the encouraging and preventive health-oriented academic lifestyle behaviors questionnaire. the questionnaire was developed by (salehzadeh et al., 2018). it consists of 124 items categorized into 13 behavioral patterns. it was designed as a self-report questionnaire that can be answered either individually or as a group. 48 sections deal with behaviors promoting academic health, and 76 sections are dedicated to behaviors preventing academic health. the participants’ answers are evaluated using a 5-point likert scale from completely agree (5) to completely disagree (1). in the present study, due to the high number of questions which might cause fatigue and, in turn, reduce accuracy in answering questions of different sections of health-oriented academic lifestyle preventive behaviors, two behavioral models of effort withdrawal and learned helplessness were chosen to be https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 46 evaluated. base structures and the number of items in each preventive aspect of academic health were learned helpless (10 items) and effort withdrawal (11 items). the internal consistency coefficients were calculated as 0.92 and 0.95 for learned helpless and effort withdrawal, respectively (salehzadeh et al., 2018). 1.2. results table 1 demonstrates the descriptive values of mean and standard deviation of the study. table 1. descriptive values of mean and standard deviation in the study (n = 239) base structures subscales mean standard deviation metacognitive beliefs unruliness and danger 20.33 6.816 positive beliefs about concerns 12.39 4.637 cognitive selfawareness 19.01 4.719 confidence in memory 9.15 4.011 necessity of thought management 9.36 2.951 total 68.26 18.223 self-efficacy total 186.40 61.257 learned helplessness total 24.21 12.575 effort withdrawal total 29.97 13.398 data distribution normality test: the results indicated that all variables are normal. correlation matrix of research variables: as demonstrated in table 2, the relationship between total metacognitive beliefs score, r = 0.587, is negative and significant (p ≥ 0.05). furthermore, the results indicate that the relationship between constructive communication pattern and academic healthoriented preventive behaviors (learned helplessness and effort withdrawal), r = 0.531 and r = 0.457, respectively, were positive and significant. https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 47 the results also point to a negative and signifcant relationship between the self-efficacy score and academic health-oriented preventive behaviors (learned helplessness and effort withdrawal), r = 0.485 and r = 0.394, respectively (p ≥ 0.05). table 2. correlation matrix of metacognitive beliefs, self-efficacy, and academic health-oriented preventive behaviors variable 1 2 3 4 5 6 7 8 9 unruliness and danger 1 positive beliefs about concerns 0.683** 1 cognitive selfawareness 0.579** 0.521** 1 confidence in memory 0.650** 0.554** 0.323** 1 necessity of thought management 0.693** 0.588** 0.566** 0.531** 1 metacognitive beliefs 0.915** 0.819** 0.748** 0.746** 0.806** 1 self-efficacy -0.332** -0.289** -0.040 -0.437** -0.285** -0.337** 1 learned helplessness 0.553** 0.418** 0.179** 0.616** 0.346** 0.531** -0.485** 1 effort withdrawal 0.426** 0.381** 0.151** 0.575** 0.330** 0.457** -0.394** 0.771** 1 * p<0.05, ** p<0.01 hypothesis under study: metacognition beliefs mediated by self-efficacy influence academic healthoriented lifestyle behaviors. in order to formulate a distribution model for academic health-oriented lifestyle preventive behaviors mediated by self-efficacy, the path analysis method was adopted (figures 1 & 2). as demostrated in table 3, chi-squared (x 2 ) was calculated as 857.89, chi-squared divided by degrees of freedom (x 2 /df) was calculated as 2.813, the comparitive fit index (cfi) was calculated as 0.91, the goodness of fit index (gfi) was calculated as 0.91, and the root mean square error of approximation (rmsea) was calculated as 0.077. according to hu and bentler (1999) values greater than 0.90 for cfi and gfi, and values less than 0.08 for rmsea indicate the proposed model’s goodness of fit with the data. table 3. goodness of fit indices for the proposed model after adjustment x 2 df x 2 /df gfi cfi rmsea before adjustment 1143/044 320 3.57 0.863 0.862 0.104 after adjustment 857.89 305 2.813 0.91 0.91 0.077 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 48 figure 1. proposed model’s distribution of scores for academic health-oriented lifestyle preventive behaviors through metacognitive beliefs mediated by self-efficacy, before adjustment. danger pb concerns cognitive selfawareness memory thought management metacognitive beliefs self-efficacy effort withdrawal learned helplessness https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 49 figure 2. proposed model’s disctribution of scores for academic health-oriented lifestyle preventive behaviors through metacognitive beliefs mediated by self-efficacy, after adjustment danger pb concerns cognitive selfawareness memory thought management metacognitive beliefs self-efficacy effort withdrawal learned helplessness https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 50 as demonstrated in table 4, before analyzing the results and presenting data derived from the base equations’ output model through amos software, research variables’ indices and markers, as well as path indicators were developed. table 4. the relationship between variables based on the proposed model. predictor criterion path type standard coefficient value of t significance metacognitive beliefs learned helplessness direct 0.503 7.268 0.001 metacognitive beliefs effort withdrawal direct 0.492 6.255 0.001 metacognitive beliefs self-efficacy direct -0.395 5.858 0.001 self-efficacy learned helplessness direct -0.288 -5.024 0.001 self-efficacy effort withdrawal direct -0.210 -3.369 0.001 metacognitive beliefs learned helplessness indirect 0.114 sobel test 2.908 <0.05 metacognitive beliefs effort withdrawal indirect 0.083 sobel test 2.31 <0.05 based on the findings, following a model adjustment, removing insignificant paths, and decreasing freedom degrees by establishing connections between predictor factors (figure 2 and table 4), metacognitive beliefs directly determined academic health-oriented lifestyle preventive behaviors through learned helpness, effort withdrawal, and self-efficacy with calculated distributions of  = 0.503,  = 0.492, and  = -0.395, respectively. furthermore, based on the findings (figure 2 and table 4), self-efficacy directly determined academic health-oriented lifestyle preventive behaviors through learned helplessness and effort withdrawal with calculated distributions of  = -0.288 and  = -0.210, respectively. the indirect influence of metacognitive beliefs through self-efficacy on determinig the variance of academic health-oriented lifestyle preventive behaviors in learned helplessness factor was calculated as 0.114, pointing to a significant mediatory effect. https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 51 the findings of the present study, therefore, establish a link between metacognitive beliefs mediated by self-efficacy and academic health-oriented lifestyle preventive behaviors of the learned helplessness factor. in other words, metacognitive beliefs mediated by self-efficacy influence academic health-oriented lifestyle preventive behaviors of learned helplessness. furthermore, the indirect influence of metacognitive beliefs mediated by self-efficacy in determining the variance of academic health-oriented lifestyle preventive behaviors of the effort withdrawal factor were calculated as 0.083, pointing to the significant influence of the mediator in this relationship. based on the these findings, it can be surmised that metacognitive beliefs mediated by self-efficacy are linked to academic health-oriented lifestyle preventive behaviors of the effort withdrawal factor. in other words, metacognitive beliefs mediated by self-efficacy influence academic health-oriented lifestyle preventive behaviors of the effort withdrawal factor. discussion and conclusion many researchers believe that the tools adopted for conducting research, especially in academic contexts, should be built on a strong theoretical foundation (nadi & sajjadian, 2011) among the crucial goals of an educational system are knowledge promotion, and identifying and removing the obstacles against learning to prevent academic failure among students (abasi et al., 2020; putwain, wood & pekrun, 2020; oyelekan, jolayemi & upahi, 2019). during the coronavirus pandemic, education inevitably moved towards virtual learning, and given the fact that each person’s potential capacity for learning is different, and people in similar circumstances learn at different rates and in different shapes, virtual learning entails advantages as well as challenges (khamisabadi, mirmehdi & meraati, 2021; papapicco, 2020). a review of literature indicates that self-efficacy and metacognitive beliefs are among the factors that may influence learning )salarifar et al., 2011, özcan, 2019; gümüs, 2019; nicoloff, 2018). the present study was an attempt to analyze the relationship between metacognitive beliefs and academic health-oriented lifestyle preventive behaviors through academic self-efficacy among students during the coronavirus pandemic. https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 52 to summarize the results of the present study, it may be asserted that students who take advantage of high-level metacognitive strategies learn in more meaningful ways. in other words, these students form connections between the information they acquire during their studies and their preexisting knowledge, providing them with an opportunity to learn coherently (as opposed to fragmented learning). therefore, they can achieve a more comprehensive understanding of what they learn, retain their acquired knowledge for more extended periods, perform better in their exams—especially in a subject like mathematics—, and form a more detailed image of their capabilities and self-efficacy . the findings indicated that metacognitive beliefs have a direct as well as an indirect link with learned helpless as one of academic health-oriented lifestyle preventive behaviors through self-efficacy. to expound on these findings, and following )nicoloff, 2018; özcan & eren gümüs, 2019). it should be noted that students with lower levels of learned helplessness are more self-efficient, as they can perform better academically. their better performance, in turn, influences their metacognitive beliefs and leads to a better awareness of learning strategies, and ultimately, their superior academic development. in other words, mental disruptions and lack of control are rarer in these students; therefore, their academic motivation is higher and their performance in subjects such as mathematics is better. experiantial evidence confirms these findings (ozkal, 2019). learners who achieve higher levels of metacognitive abilities pay attention to what they learn, and form connections between the new information and their preexisting knowledge. in other words, these learners are aware of their learning process. as such, they can decide on superior learning strategies fitting their subject of study. as the number of metacognitive strategies students employ increases, during their learning process—either in the classroom or when they are studying on their own—they try to make sense of the information as they learn, create a learning environment ripe for academic success, and improve their academic performance, and as a result of their efficacy, they leave their sense of learning inability and mismanagement behind (khamisabadi, mirmehdi & meraati, 2021; hayat & shateri, 2019; ozkal, 2019). similar to the study conducted by shoawei & chia-ching (2019), the present study's findings indicate a significant relationship between learned helplessness/effort withdrawal and academic performance/belief in self-efficacy. students suffering from learned helplessness believe that they have no control over the events influencing them. cognitive and metacognitive strategies can help these students build up academic self-efficacy and enthusiasm. furthermore, metacognitive beliefs can influence their self-confidence and improve their performance in carrying out their academic assignments. https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 53 furthermore, the results of the present study indicated that metacognitive beliefs can have a direct, as well as an indirect, relationship with effort withdrawal as an academic health-oriented lifestyle preventive behavior through academic self-efficacy viz. the higher the self-efficacy among the students, the lower the possiblity of effort withdrawal in them. in other words, it can be surmised that as (lajoie, pekrun, azevedo, & leighton, 2020 ;ozgur, 2021; hayat & shateri, 2019; salarifar et al. 2016; and flavell & miller, 1998) have proposed, metacognitive beliefs are influential factors in processes of repression, denial, and avoidance. as one’s metacognitive beliefs develop and evolve, one’s image of one’s abilities to face academic challenges improves. our findings demonstrated that mediated by self-efficacy, metacognitive beliefs can have both direct and indirect influences on effort withdrawal as a preventive behavior in health-oriented academic lifestyles. in other words, metacognitive beliefs can influence a student’s effort withdrawal via self-efficacy. additionally, in line with the studies mentioned in the introduction, regarding the relationship between self-efficacy and metacognitive beliefs and health-oriented academic lifestyle preventive behaviors, the present study surmised that an organized and coherent understanding of concepts, and refraining from disjointed learning can orient students towards better learning (göktepe yıldız, 2020; oyelekan, jolayemi & upahi, 2019). this approach alleviates a student’s learned helplessness and sense of inability. in addition, determined and purposeful learning tendencies lead to self-evaluation, academic behavior management, and ultimately, to academic enthusiasm, and effort withdrawal reduction and eventual cessation as results of self-efficacy and apt metacognitive beliefs (lajoie, pekrun, azevedo, & leighton, 2020; tankari, zouhin toboula, ousseini, 2020; shaowei & chiaching, 2019; nicoloff, 2018; salehzadeh et al., 2018). such a person will not try to avoid challenging situations, and will not withdraw his efforts in such situations. some researchers have demostrated that as academic health-oriented lifestyle promoting behaviors in students increase, their preventive behaviors diminish, leading to their academic development and success (putwain, wood & pekrun, 2020; shaowei & chia-ching, 2019; salehzadeh et al., 2018). several shortcomings afflicted the present study. the research sample was limited to the students studying in a specific field in a single university; therefore, the findings may not be applicable in other contexts. more extensive studies using samples from different universities and study fields are recommended. furthermore, researchers in the present study did not consider social favorability and participants’ biases. the studied factors should be analyzed in other environments to determine the https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 54 influence of culture and gender on the proposed model, and to gauge the impact of natural disasters and situations like covid-19. author contributions: authors contributed equally. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references abasi, f., hejazi, e. and hakimzadeh, r. (2020). lived experience of primary school teachers of teaching opportunities and challenges in the educational network students (shad): a phenomenological study. journal of research in teaching, 3(8), 1-24. https://journals.uok.ac.ir/article_61636.html abramson, l. y., seligman, m. e. p. and teasdale, j. (1978). learned helplessness in humans: critique and reformulation. journal of abnormal psychology, 87, 49-74. https://doi.org/10.4324/9780203884089 anderman, l. h., & anderman, e. m. (2009). oriented towards mastery, in r. gilman, e. scott huebner, m. j. furlong (eds.), handbook of positive psychology in schools (pp. 161-173). new york: oxford university press. https://doi.org/10.4324/9780203884089 amid, m., shokri, o. and zarani, f. (2016). casual relations model of antecedents and consequences of health behaviors in university students. rph, 10(3), 70-88. http://rph.khu.ac.ir/article-1-2521-en.html bandura, a. (1997). self-efficacy: the exercise of control. new york: w. h. freeman. birnbum, r. (1988). how colleges work: the cybernetics of academic organization and leadership. california: jossey-bass inc. biabangard, e. (2003). an analysis of metacognition and cognitive therapy. advances in cognitive sciences, 4(4), 32-39. https://icssjournal.ir/article-1-562-en.html burić, i., sorić, i. and penezić, z. (2016). emotion regulation in academic domain: development and validation of the academic emotion regulation questionnaire (aerq). personality and individual differences, 96, 138-147. https://doi.org/10.1016/j.paid.2016.02.074 covington, m.v. (1984). the self-worth theory of achievement motivation findings and implications. the elementary school journal, 85(1), 5-20. https://doi.org/10.1086/461388 deci, e. l. and ryan, r. m. (2000). the "what" and "why" of goal pursuits human needs and the selfdetermination of behavior. psychological inquiry, 11(4), 227-268. https://doi.org/10.1207/s15327965pli1104_01 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://journals.uok.ac.ir/article_61636.html https://doi.org/10.4324/9780203884089 https://doi.org/10.4324/9780203884089 file:///c:/users/admin/downloads/‎%20http:/doi.org/10.18869/acadpub.rph.10.3.70 file:///c:/users/admin/downloads/‎%20http:/doi.org/10.18869/acadpub.rph.10.3.70 https://icssjournal.ir/article-1-562-en.html https://psycnet.apa.org/doi/10.1016/j.paid.2016.02.074 https://doi.org/10.1086/461388 https://doi.org/10.1086/461388 https://doi.org/10.1207/s15327965pli1104_01 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 55 ferrándiz, e., puentes, c., moreno, p. j. and flores, e. (2016). engaging and assessing students through their electronic devices and real time quizzes. multidisciplinary journal for education, 3(2), 173-184. http://dx.doi.org/10.4995/muse.2016.6375 flavell, j. h., & miller, p. (1998). social cognition. in w. doman (series.ed) & d. kuhn & r. siegler (vol. eds.), handbook of child psychology: cognition. perception and language (5th ed., pp. 951-898). new york: wiley göktepe yıldız, s. (2020). geometrical problem-solving performance of preservice teachers: exploring the effectiveness of metacognitive strategies. malaysian online journal of educational sciences, 8(4), 34-47. https://eric.ed.gov/?id=ej1271740 haji khawjelu, s. r and hesampour, a. (2011). towards a new paradigm of science: some reflections on the trans-scientific nature of second model of production of science. journal of culture strategy, 15(14), 165-161. harackiewicz, j. m., durik, a. m., barron, k. e., linnenbrink-garcia, l. and tauer, j. m. (2008). the role of achievement goals in the development of interest: reciprocal relations between achievement goals, interest, and performance. journal of educational psychology, 100, 105–122. https://doi.org/10.1037/0022-0663.100.1.105 hayat, a. a. and shateri, k. (2019). the role of academic self-efficacy in improving students’ metacognitive learning strategies. j adv med educ prof, 7(4), 205-212. https://www.ncbi.nlm.nih.gov/pmc/articles/pmc6820011/ hojjati, a., yarmohammadian, m. and keshtiarai, n. (2017). design an appropriate model for studying development evaluation of technical and engineering based on the lived experiences of teachers and students. journal of new approaches in educational administration. 8(2), 131-151. http://jedu.miau.ac.ir/m/article_2410.html?lang=en hoveida. r. and molavi, h. (2008). academic quality improvement process from the viewpoints of faculty members of universities in isfahan province: a comparison based on academic quality improvement program (aqip). iranian journal of medical education, 8 (1), 132-141. http://ijme.mui.ac.ir/article-1784-en.html hu, l. and bentler, p. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling, 6 (1), 1-55. https://doi.org/10.1080/10705519909540118 iyer, p., aziz, k. and ojcius, d. m. (2020). impact of covid‐19 on dental education in the united states. j dent educ, 84, 718-722. https://doi.org/10.1002/jdd.12163 jarvis, s. and seifert, t. (2002). work avoidance as a manifestation of hostility, helplessness, and boredom. the alberta journal of educational research, 48(2), 174-187. https://psycnet.apa.org/record/200218284-002 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://dx.doi.org/10.4995/muse.2016.6375 https://eric.ed.gov/?id=ej1271740 https://psycnet.apa.org/doi/10.1037/0022-0663.100.1.105 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc6820011/ http://jedu.miau.ac.ir/m/article_2410.html?lang=en http://ijme.mui.ac.ir/article-1-784-en.html http://ijme.mui.ac.ir/article-1-784-en.html https://doi.org/10.1080/10705519909540118 https://doi.org/10.1002/jdd.12163 https://psycnet.apa.org/record/2002-18284-002 https://psycnet.apa.org/record/2002-18284-002 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 56 khamisabadi, m., mirmehdi, s. r. and meraati, a. r. (2021) relationship of academic vitality, academic self-efficacy, and metacognitive skills with lifelong learning concerning mediating role of study approaches. journal of health research. 11(1), 1-10. http://dx.doi.org/10.32598/jrh.11.1.1202.4 lajoie, p., pekrun, r., azevedo, r., and leighton, p. (2020). understanding and measuring emotions in technology-rich learning environments. learning and instruction. 70, 101272. https://doi.org/10.1016/j.learninstruc.2019.101272 maddux, j. e. (2003). the power of believing you can handbook of positive psychology. new york: oxford university press. maier, s. f. and seligman, m. e. p. (1976). learned helplessness: theory and evidence. journal of experimental psychology: general, 105(1), 3-46. https://doi.org/10.1037/0096-3445.105.1.3 nadi, m. a. and sajjadian, i. (2011). a study on factor construction, validity, reliability and normalization of organizational learning capability scale middle school teachers in the city of isfahan. quarterly journal of educational innovations, 40, 105-130. http://noavaryedu.oerp.ir/article_78962.html?lang=en nicoloff, a. e. (2018). mathematics anxiety and attitudes as predictors of mathematics self-efficacy in developmental mathematics courses. proquest llc, ph.d. dissertation, grand canyon university. osoli, a., shariarmadar, a. and kalantarhormozi, a. (2016). investigating the relationship between metacognitive beliefs and academic self-regulation and conflict between students and parents. counseling culture and psychotherapy, 22(7), 83-104. https://www.sid.ir/fa/journal/viewpaper.aspx?id=355298 özcan, z. c. and eren gümüs, a. (2019). a modeling study to explain mathematical problem-solving performance through metacognition, self-efficacy, motivation, and anxiety. australian journal of education, 63(1), 116-134. https://doi.org/10.1177/0004944119840073 ozkal, n. (2019). relationships between self-efficacy beliefs, engagement and: academic performance in math lessons. cypriot journal of educational sciences, 14(2), 190-200. https://doi.org/10.18844/cjes.v14i2.3766 ozgur, s. d. (2021). chemistry self-efficacy beliefs as predictors of students' metacognitive skills when solving chemistry problems. international online journal of education and teaching (iojet), 8(1). 132-147. oyelekan, o. s., jolayemi, s. s. and upahi, j. e. (2019). relationships among senior school students’ selfefficacy, metacognition and their achievement in chemistry. cypriot journal of educational sciences, 14(2), 208-221. https://doi.org/10.18844/cjes.v14i2.2564. papapicco, c. (2020). informative contagion: the coronavirus (covid-19) in italian journalism. online journal of communication and media technologies, 10(3), e202014. https://doi.org/10.29333/ojcmt/7938 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://dx.doi.org/10.32598/jrh.11.1.1202.4 https://doi.org/10.1016/j.learninstruc.2019.101272 https://psycnet.apa.org/doi/10.1037/0096-3445.105.1.3 http://noavaryedu.oerp.ir/article_78962.html?lang=en https://www.sid.ir/fa/journal/viewpaper.aspx?id=355298 https://doi.org/10.1177/0004944119840073 https://doi.org/10.18844/cjes.v14i2.3766 https://doi.org/10.18844/cjes.v14i2.2564 https://doi.org/10.29333/ojcmt/7938 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 57 pekrun, r. (2006). the control-value theory of achievement emotions: assumptions, corollaries, and implications for educational research and practice. educational psychology review, 18, 315–341. https://doi.org/10.1007/s10648-006-9029-9 pekrun, r., frenzel, a. goetz, t., & perry, r. (2007). the controlvalue of achievement emotions: an integrative approach to emotions in education. in p.a. shutz, & r. pekrun, emotion in education, (pp.13-36). san diego, ca: academic press. peterson, c. (2000). the future of optimism. american psychologist, 55, 44–55. https://doi.org/10.1037/0003066x.55.1.44 peterson, c., steen, & t. a. (2002). optimistic explanatory style, in c. r. snyder, s. j. lopez (eds.), handbook of positive psychology (pp. 244-256). new york: oxford university press. putwain, d. w., wood, p. and pekrun, r. (2020). achievement emotions and academic achievement: reciprocal relations and the moderating influence of academic buoyancy. journal of educational psychology. advance online publication. https://doi.org/10.1037/edu0000637 rakoczy, k., pingera, p., hochweberb, j., kliemea, e., schützec, b. and besserd, m. (2018). formative assessment in mathematics: mediated by feedback's perceived usefulness and students' self-efficacy. learning and instruction, 60,154-165. https://doi.org/10.1016/j.learninstruc.2018.01.004 salarifar, m., pouretemad, h., heydari, m. and asgharnejad farid ali, a. (2011). believes and meta-cognitive state, inhibitor or facilitator of academic self-regulation. counseling culture and psychotherapy, 7(2), 32-49. https://www.sid.ir/en/journal/viewpaper.aspx?id=251458 salehzadeh, p., shokri, o. and fathabadi, j. (2018). health-oriented academic lifestyle: the newly emerging idea in the field of academic health psychology. educational psychology, 13(46), 1-30. 10.22054/jep.2018.8474 salehzadeh, p., shokri, o. and fathabadi, j. (2018). development and validation of the promoting and preventing educational health academic lifestyle behaviors questionnaire: perspectives on contemporary theories of achievement motivation. quarterly of educational measurement, 8(30), 61105. https://www.sid.ir/en/journal/viewpaper.aspx?id=606423 shaowei, w. and chia-ching, t. (2019). the impact of learning self-efficacy on social support towards learned helplessness in china. eurasia journal of mathematics, science and technology education, 15(10). https://doi.org/10.29333/ejmste/115457 shirinzadeh, s., goodarzi, m., rahimi, c. and naziri, g. (2009). study of facture structures, validity and reliability of metacognition questionnare-30. journal of psychology. 12, 445-461. snyder, c. r., shorey, h. s., cheavens, j., pulvers, k. m., adams, v. h. and wiklund, c. (2002). hope and academic success in college. journal of educational psychology, 94, 820-826. https://doi.org/10.1037/0022-0663.94.4.820 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1007/s10648-006-9029-9 https://psycnet.apa.org/doi/10.1037/0003-066x.55.1.44 https://psycnet.apa.org/doi/10.1037/0003-066x.55.1.44 https://psycnet.apa.org/doi/10.1037/edu0000637 https://doi.org/10.1016/j.learninstruc.2018.01.004 https://www.sid.ir/en/journal/viewpaper.aspx?id=251458 https://dx.doi.org/10.22054/jep.2018.8474 https://www.sid.ir/en/journal/viewpaper.aspx?id=606423 https://doi.org/10.29333/ejmste/115457 https://psycnet.apa.org/doi/10.1037/0022-0663.94.4.820 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16879 social and technological sciences e-issn: 2341-2593 moradi et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 39-58. https://doi.org/10.4995/muse.2022.16879 58 snyder, c. r., rand, k. l., & sigmon, d. r. (2002). hope theory: a member of the positive psychology family. in c. r. snyder & s. j. lopez (eds.), handbook of positive psychology (pp. 257-276). new york, ny: oxford university press. snyder, c., lopez, s., shorey, h. r., rand, k. l. and feldman, d. (2003). hope theory, measurements, and applications to school psychology. school psychology quarterly, 18, 122–139. https://doi.org/10.1521/scpq.18.2.122.21854 tankari, m., zounhin toboula, m. and ousseini, h. (2020). implications of uncertainty avoidance and procrastination on some west african efl learners' academic behavior. international journal on studies in english language and literature (ijsell), 8(11), 1-19. https://doi.org/10.20431/23473134.0811001 tossavainen, t., rensaa, r. j. and johansson, m. (2021). swedish first-year engineering students' views of mathematics, self-efficacy and motivation and their effect on task performance. international journal of mathematical education in science and technology, 52(1), 23-38. https://doi.org/10.1080/0020739x.2019.1656827 weiner, b. (1994). integrating social and personal theories of achievement striving. review of educational research, 64(4), 557-573. https://doi.org/10.3102/00346543064004557 wells, a. (2009). emotional disorders and meta cognition: innovative cognitive therapy. wiley. wigfield, a. and eccles, j. s. (2000). expectancy–value theory of achievement motivation. contemporary educational psychology, 25(1), 68-81. doi:10.1006/ceps.1999.1015 zajacova, a., lynch, s. m. and espenshade, t. j. (2005). self-efficacy, stress, and academic success in college. research in higher education, 46, 677-706. https://doi.org/10.1007/s11162-004-4139-z zeidner, m. and matthews, g. (2017). emotional intelligence in gifted students. gifted education international, 33(2) 163–182. https://doi.org/10.1177/0261429417708879 https://doi.org/10.4995/muse.2022.16879 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://psycnet.apa.org/doi/10.1521/scpq.18.2.122.21854 https://doi.org/10.20431/2347-3134.0811001 https://doi.org/10.20431/2347-3134.0811001 https://doi.org/10.1080/0020739x.2019.1656827 https://doi.org/10.1080/0020739x.2019.1656827 https://doi.org/10.3102/00346543064004557 https://doi.org/10.1006/ceps.1999.1015 https://doi.org/10.1007/s11162-004-4139-z https://doi.org/10.1177/0261429417708879 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 71 english language proficiency of senior high school students jomel b. manuel cagayan state university, carig campus, tuguegarao city 3500, philippines * correspondence: jomelbmanuel@gmail.com received: 11 november 2021; accepted: 7 march 2022; published: april 2022 abstract this paper aimed to determine the english language proficiency of senior high school students as input for the language extension program at gammad national high school. specifically, it determined the students’ proficiency level on the eight (8) parts of speech; the reading comprehension level along literal, interpretative, evaluative, and integrative; and the reading comprehension level on different genres such as a poem, essay, and short story. a descriptive research design was employed in the study. the fifty-one (51) respondents of grade 11 senior high school students at gammad national high school were chosen using the total enumeration technique. an 82-item proficiency test was used as an instrument in gathering the data. weighted mean was used in analyzing the data. results of the study show that among the eight parts of speech, the senior high school students performed well in using adverbs, prepositions, and interjections but found it problematic of using verbs. on reading comprehension, the respondents excelled under literal level across all genres but not on integration and evaluation levels. the study concluded that the respondents are “proficient user of english” or “high” level in using grammar and reading comprehension. they have mastered the basics of using the english language; however, they need to be exposed more to meaningful activities and varied texts to attain a high level of proficiency. keywords: english language; level of proficiency; reading comprehension, eight parts of speech. (maximum one page: title + affiliations + abstract + keywords + to cite this article) to cite this article: manuel, jomel b. (2022). english language proficiency of senior high school students. multidisciplinary journal for education, social and technological sciences, 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ mailto:jomelbmanuel@gmail.com multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 72 1. introduction since english is the primary language of instruction and communication, you must demonstrate an adequate level of proficiency in english, regardless of your citizenship status or country of origin. this language requirement ensures that all students will have the opportunity for success in their chosen program. the philippines is recognized globally as one of the largest english-speaking nations. based on the business english index (bei) in 2012, only the philippines attained a score above 7.0, putting the country within the range of a high proficiency that indicates an ability to lead business discussions and perform complex tasks. the country’s proficiency in the english language is one of the strengths that has helped drive the economy and even made the philippines the top voice outsourcing destination in the world, surpassing india in 2012. according to mitra (2013), the philippines’ principal strength in the export service sector is the large educated workforce with strong englishlanguage capabilities. further, the influx of foreign learners of english is on the rise due to the relatively more affordable but quality english as a second language (esl) programs being offered locally. this is a manifestation that the country is doing fine in terms of english competency. however, gradual deterioration in the use of the english language has been observed not only in the academe but also in the business world. the level of english proficiency of college graduates from the philippines is lower than the target english proficiency of high school students in thailand, according to a study conducted by the hopkins international partners, which is the official philippine representative in the group called test of english for international communication (toeic). in a “24 oras” report by raffy tima, the hopkins international partners’ general manager rex wallen tan said the philippines lags behind most of its asean neighbors in english proficiency. in the speech of dr. prospero de vera during the seminar-workshop on the english roadmap of the philippines, recently in september 2017, there was a nursing migration program to the uk worth around php300 million in salaries under which around 90% of the philippines nurses recruited failed the english language test. further, senior high school graduates have two options whether to continue pursuing their tertiary education or to work in the industry immediately. with the two additional years, they are expected to be more competent and equipped with the competencies needed in the industries. however, language teachers and school heads both in private and public schools have the same observation as regards the low performance of the students in the english language. in 2013, research conducted by the business process association of the philippines (bpap) concluded that of the four basic skills they measured (english proficiency, cognitive ability, computer literary, and perceptual speed & accuracy), english proficiency development should be given priority when addressing the supply-demand gap of industry and academe. further, deped participated in the 2018 cycle of the programme for international student assessment (pisa), a triennial international assessment administered to 15-year old learners, who are near the end of their compulsory basic education. implemented by the organization for economic cooperation and https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 73 development (oecd), pisa specifically looks into the learners’ ability to apply the knowledge they have gained in formal education to everyday situations. reading literacy was assessed as the major domain. results showed that filipino students obtained an average score of 340 points in overall reading literacy, which was significantly lower than the oecd average of 487 points. in the k to 12 english curriculum guide (2016), senior high school focuses on the domains where language is integrated. grammar awareness and structure and reading comprehension are taught, for these allow students to progress from the foundational level to higher levels of language use. students would be able to read, speak and write correctly. they can communicate effectively, in oral and written forms, using the correct grammatical structure of english. further, they can discuss and analyze text to create new meanings and respond to literary text through the appreciation of literary devices. in this premise, the researcher is motivated to assess the english language proficiency of the senior high school students of gammad national high school, the adopted school of the college of arts and sciences, cagayan state university. results may serve as input for the enhancement of the language extension program. 2. literature review 2.1. proficiency the 2013 study, the third edition of ef’s english proficiency index, ranks 60 countries and territories around the world by adult english proficiency. as the report notes: “today, english proficiency can hardly be thought of as an economic advantage at all. it is certainly no longer a marker of the elite. instead, it is increasingly becoming a basic skill needed for the entire workforce, in the same way, that literacy has been transformed in the last two centuries from an elite privilege into a basic requirement for informed citizenship.” rao (2016) emphasized in her study that proficiency equates with fluency, but there is a distinction between both of them. it is possible to be fluent without being highly proficient, as it is also possible to be highly proficient without being fluent. she further mentioned that when it comes to second language learning, a few major aspects of fluency lay in the prosody of the language, pronunciation, and the timing of dialogue. fluency is impossible to attain without engaging in dialogue with a fluent speaker of the language. proficiency, on the other hand, is the primary focus of your ability to understand and communicate precisely in the language. proficiency is best practiced through reading and writing. proficiency is regarded as a continuum (james, 1985, p.2.) which means proficiency can be considered as a scale of related skills of language slightly and continuously changing at each level. as a continuum, proficiency is split into levels that include the gradually changing successive abilities in the use of language with the prior levels prerequisite accomplishing the following levels https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 74 (heilenman & kaplan, 1985, p. 61). for the use of language “function, context/content, and accuracy” are accepted as the three criteria. the function represents the individuals’ ability to complete a task linguistically, like asking questions, or describing; context/content refers to the setting of the functions; and accuracy means the level of correctness of an individual’s language use and each factor extends in breadth as the level of proficiency increases (bragger, 1985, p. 80). according to o’sullivan (2012), it is difficult to define the limits of the language that is used in a context, although identification of the specific aspects of language use, like vocabulary or syntax, can be described in a given context to attain information about the needs analysis of the language use given in a specific context (o’sullivan, 2006). the aspects of language used should be specified and what language is used in the given context needs to be determined to assess the language proficiency level. this study considered only the reading skills as well as the eight parts of speech. teachinglearning of the english language focusing on these components provides avenues to hone learners’ proficiency in the language well. 2.2. grammar generally speaking, students feel that english is a difficult language having complicated grammar rules to master. but is it true that for effective communication, there is a need for essential principles of grammar or syntax? language is supposed to be a great highway, the words are the cars and trucks, but the grammar is the road signs and markings that tell the cars and lorries where to go and how to drive. thus, having a good grasp of the rules of the english language would help us express our thoughts and communicate with others. in the study conducted by tuan (2017) entitled ‘communicative competence of the fourth year students: basis for proposed english language program in vietnam’, he found that the strengths and weaknesses of the respondents relative to linguistic competence are along with nouns, pronouns, and preposition. on the other hand, their weaknesses are along with verb, adverb, and interjection. according to him, the weakness of the respondents relative to verb indicates that they have low knowledge, ability, and skill in determining the function of verb that is to describe an action, state, or occurrence, and forming of the main part of the predicate of a sentence. they are also weak in determining the two categories of verbs and their function in the sentence. more importantly, the respondents are very weak in recognizing the functions of a verb as a system in which the form of the “verb reflects the person, number, and/or gender of one or more arguments.” the other weakness of the respondents relative to adverb implies that the respondents have low competence in understanding that an adverb in a word or phrase modifies or qualifies an adjective, verb, or adverb or a word group, expressing a relation of place, time, circumstance, manner, cause, and degree. meanwhile, the weakness of the respondents relative to interjection shows that they have difficulty understanding that interjections show the emotion or feeling of the author and that they can stand alone or be placed before or after an exclamation point. https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 75 in the study conducted by leyaley (2016) in kalinga-apayao, she found that the english language proficiency of freshmen students who are enrolled in the institute of teacher education is described as early intermediate regardless of the type of school they graduated from, the program they are enrolled in, and the honors they have received. this means that they can communicate basic and familiar topics containing previously taught vocabulary and sentences structures. on the other hand, they commonly make errors in word usage, pronunciations, and grammar. thus, they greatly benefit from explicit teaching and modeling of high-frequency words, grammatical structures, and sentence frames (baeta et al., 2012). 2.3. reading comprehension reading with comprehension is one of the primary goals of the senior high school curriculum. comprehension is what allows our senior high students to understand the information about the topic specifically to effectively perform other skills needed in various tasks. according to baier (2005), reading comprehension refers to the meaningful interface between your schema –what you already know and the meaning of the text –what the text is about. it is a cognitive process involving the conscious intellectual activity of formulating meaning from the text; more than just the reading skill itself. it goes beyond the ability to recognize every single word in the text that you are reading. it is more of interpreting and decoding meanings from a combination of unknown and familiar words. reading proficiency is essential for a wide variety of human activities-form following instructions in a manual; finding out the who, what, when, where, and why of an event; to communicating with others for a specific purpose or transaction. reading was the main subject assessed in program for international student assessment (pisa) 2018. the pisa 2018 assessment, which was delivered on a computer in most of the 79 countries and economies that participated, included new text and assessments formats made possible through digital delivery. the test aimed to assess reading literacy in the digital environment while retaining the ability to measure trends in reading literacy over the past two decades. pisa 2018 defined reading literacy as understanding, using, evaluating, reflecting on, and engaging with texts to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society. released on december 3, 2018, pisa results that the philippines scores 340 in reading, below the average of participating organization for economic cooperation and development (oecd). with the pisa results reflecting the learners’ performance in the national achievement test, deped recognizes the urgency of addressing issues and gaps in attaining the quality of basic education in the philippines. deped will lead this national effort for quality basic education through sulong edukalidad by implementing aggressive reforms in four key areas: (1) k to 12 reviews and updating, (2) improvement of learning facilities, (3) teachers and schools head’s upskilling and reskilling through a transformed professional development program; and (4) engagement of all https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 76 stakeholders for support and collaboration (https://www.deped.gov.ph/2019/12/04/sulongedukalidad/). giang (2017) in her study entitled ‘reading comprehension performance of efl first-year students: basis for reading enhancement program’ in vietnam found that the efl studentrespondents find english reading materials interesting with the highest interest in music reading materials while science is their least. they do not often indulge themselves in spending quality time in reading encyclopedia and reader’s digest, and they are only exposed sometimes to media learning facilities like tv radio; social media networks particularly facebook; and library holdings like books, newspapers, comics, and magazines. concerning reading comprehension, students are only evidently competent at the literal level from the select genres of literature which are a poem, essay, and short story. further, the level of students’ reading comprehension is relatively high along with poetry. however, it remains an undiscounted fact that they are yet weak along with an essay and short story. in a manila times on november 18, 2019, valderama mentioned that a couple of years ago, a study by the hopkins international partners showed that the level of english proficiency of college graduates from the philippines was lower than the target english proficiency of high school students in thailand. hopkins international partners was the official representative of the philippines to the test of english for international communication (toeic), an international standardized test of english skills of people working in an international environment. the same study showed that filipino graduates’ english skills were lower than the target for cab drivers in dubai. cab drivers in dubai, the largest city in the united arab emirates, are expected to have a toeic proficiency score of 650, compared to the 631.4 proficiency score of a filipino college graduate. in the study conducted by sijali (2016) in nepal which aimed to investigate the proficiency level of english language of higher secondary level students regarding gender, nature of institution, medium of instructional and stream, he found that the english language proficiency level of higher secondary level students in nepal was poor. regarding gender, the result showed that there was no statistically significant difference between female and male elt students in their english language proficiency level. however, elt students of government higher secondary level were found statistically significantly less proficient in the english language than that of private higher secondary level. navarro et al. (2015) in their study found that maritime students agreed that english is required in their chosen course and is useful in securing a job, especially in the metro manila area and abroad. on the other hand, maritime students are incompetent in vocabulary and reading comprehension and fairly incompetent in grammar. further, baeta et al. (2012) in central bicol state university revealed in their study that college freshmen were proficient in grammar and moderately proficient both in reading comprehension and phonology, but they are hardly proficient in vocabulary. generally, the proficiency level of college freshmen falls under moderate proficiency. https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.deped.gov.ph/2019/12/04/sulong-edukalidad/ https://www.deped.gov.ph/2019/12/04/sulong-edukalidad/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 77 2.4. genres learners have different strategies in learning. because of their unique exposure, they comprehend things differently. just like reading texts, others would prefer prose to poetry or just the opposite. exposing students to different types of texts early on builds their capacity to understand the wide variety of reading material they will encounter during their school years and beyond. in exposing students to a variety of texts, teachers should also ensure that a selected text (1) contains rich ideas and information, (2) is at an appropriate difficulty level for students’ word-reading and comprehension skills, and (3) aligns with the purpose of the lesson(https://files.eric.ed.gov/fulltext/ed597128.pdf). 2.4.1. poetry poetry can be a creative way of exposing english language learners to quality literature. students can use these short, fun, non-threatening pieces of text to begin to learn how to apply reading strategies to increase their comprehension (norris, 2011). because it is so accessible to students, it is a genre that can easily be used to help students begin to increase their awareness of reading comprehension strategies (hadaway, vardell, & young, 2001). 2.4.2. short story stories with strong literary value and informational texts that are accurate, well-written, and engaging are consistently good choices for teaching reading comprehension. pardede’s (2011) study at christian university of indonesia revealed that the majority of the students of the english teaching department found short stories interesting to use both as materials for self-enjoyment and as components of language skill classes. saka (2014), in her descriptive study supporting vural (2013)’s results, identified the ideas of 40 junior students of the elt department of akdeniz university about the contributions of studying short stories. the results suggested that reading short story increased their wish to read more and appreciation for literature. 2.4.3. essay expository (or informational) text is the primary source of reading material used to present academic content (e.g., science, social studies). as such, it is essential that students can comprehend expository text. this is recognized in the common core state standards for english language arts, which states that, as early as kindergarten, students should be able to engage with informational text in multifaceted ways, such as identifying the main topic, asking and answering questions about key details, and describing connections between pieces of information (national governors association center for best practices & council of chief state school officers, 2010). https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://files.eric.ed.gov/fulltext/ed597128.pdf https://ila.onlinelibrary.wiley.com/doi/full/10.1002/trtr.1590#trtr1590-bib-0022 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 78 3. methodology 3.1. research design this study made use of the descriptive-quantitative research design. as a research method, it represents a systematic and objective means of describing and quantifying phenomena (downewamboldt, 1992; schreier, 2012 cited in elo kaariaimen, kanste, polkki, utrianien & kyngars, 2014). thus, the present study described and quantified the english proficiency level of the senior high school students along with grammar and reading comprehension. 3.2. participants the participants of the study were the fifty-one (51) senior high school students of gammad national high school, iguig, cagayan for the school year 2017-2018. they were composed of 24(tvl track) and 27 (gass track). gammad national high school is an adopted school of the department of arts and humanities, college of arts and sciences, cagayan state university-carig campus. table 1. participants’ demographic profile. demographic profile frequency (n=51) percentage sex male 24 47.06% female 27 52.94% strand gas 27 52.94% tvl 24 47.06% language used at home ilocano 1 1.96% ibanag 2 3.92% english 0 0.00% itawes 37 72.54% tagalog 11 21.56% 3.3. research instrument this study used an english proficiency test as the main tool in gathering the data. part 1 included the profile of the participants, while part ii was the english proficiency test covering a 40item grammar test on the eight parts of speech (nouns, pronouns, verbs, adjectives, adverbs, prepositions, conjunctions, and interjections) and a 42-item reading comprehension test on the three literary genres: poem, essay, and short story. each text has four sets of questions of different https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 79 levels: literal, interpretation, evaluation, and integration. before the test was administered to the participants, it was piloted to another group of students who belonged to the same level. 3.4. analysis of data to determine the proficiency level of respondents on the eight parts of speech and reading comprehension tests, the mean average was used. the scale below was used to determine the english proficiency of respondents on the eight (8) parts of speech. proficiency level for the whole test in eight parts of speech score intervals proficiency level descriptive value 32.01-40.00 very proficient user of english very high 24.01-32.00 proficient user of english high 16.01-24.00 fair user of english moderate 0.01-16.00 modest user of english low 1.00-8.00 limited user of english very low overall level reading comprehension ability of respondents scores equivalent interpretative level 33.61-42.00 very high 25.21-33.60 high 16.81-25.20 moderate 8.41-16.80 low 1.00-8.40 very low 4. results and discussion proficiency level in the eight (8) parts of speech table 2 presents the language scores of the senior high school students in the eight (8) parts of speech. as gleaned from the table, most of the senior high school students got the highest scores on adverbs with a weighted mean average of 37, followed by conjunction with a weighted average https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 80 of 27.2, while the least was on the use of verbs with a weighted mean average of 16.8. generally, the respondents’ english language proficiency in the eight parts of speech has an overall mean average of 26. 7, which means that they are proficient users of english. this implies that despite showing good performance on function words, the participants could hardly use content words accurately, especially the copula and subject-verb concord. it means that the participants are still confused when using subject-verb agreement and do not know the rules of it. the result of the present study was strengthened by freeman and murcia (2010) in their grammar book in which research on l2 morpheme acquisition has shown that the third person singular present tense-s inflection causes persistent problems for learners even at more advanced stages of proficiency. the findings of the study corroborate also the study made by tuan (2017) in vietnam that the weakest point of students in grammar is the use of verbs. the weakness of the respondents relative to verb indicates that they have low knowledge, ability, and skill in determining the function of verb that is to describe an action, state, or occurrence. table 2. proficiency level in the eight (8) parts of speech. parts of speech mean descriptive value noun 23.8 moderate pronoun 23 moderate verb 16.8 moderate adjective 23.8 moderate adverb 37 very high preposition 31.4 high conjunction 27.2 high interjection 30.6 high overall mean average 26.7 high proficiency level in reading comprehension (poem) table 3 presents the proficiency level in reading comprehension using the poem. as gleaned from the table, most of the respondents almost get a perfect score in the literal level with a mean of 41.67 (very high), followed by an interpretation level with a mean of 23.4(high). on the other hand, it reveals that the respondents are weak in both evaluation and integration levels with a mean of 18.75 (low) and 18.5 (low) respectively. generally, the respondents’ proficiency level in reading comprehension using the poem has an overall mean average of 25.63(high), which means they are proficient users of english. this implies that the respondents excel in mastering facts or information questions but have the difficulty in understanding questions that require higher-order thinking skills (hots). poetry promotes oral fluency and lays a strong foundation for reading in a new language. because of the rhyming patterns, the repetition, the rhythm, and the manageable chunks of text, https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 81 poetry is an ideal genre for teaching ells how to read. and not only is it a powerful tool in the classroom, but it also can be used to introduce content in science, math, and social studies classrooms (vardell, hadaway & young, 2006). table 3. proficiency level in reading comprehension using the poem. levels of reading comprehension mean descriptive value literal 41.67 very high interpretation 23.6 moderate evaluation 18.75 moderate integration 18.5 moderate overall mean average 25.63 high proficiency level in reading comprehension (essay) table 4 presents the proficiency level of senior high school students in reading comprehension using an excerpt in essay text. as gleaned from the table, the respondents got the highest scores in the literal level with a mean of 31(very high), whereas in the three remaining levels: interpretation, evaluation, and integration, they encountered difficulty with mean scores of 15.8(low), 15.5(low), and 15(low). generally, the respondents’ reading comprehension level using essay text has an overall mean average of 19.32 (low), which means that they are fair users of english. the result is attributed to their attitude in reading this genre. nonfiction appeals to the intellect, unlike fiction which appeals to emotions. thus, reading nonfictions is like a workout for your brain. you need to think over the facts and analyze them. table 4. proficiency level in reading comprehension using an excerpt from essay text. levels of reading comprehension mean descriptive value literal 31 high interpretation 15.8 low evaluation 15.5 low integration 15 low overall mean average 19.32 moderate proficiency level in reading comprehension using a short story table 5 presents the respondents’ proficiency level in reading comprehension using a short story. as gleaned from the table, most of the respondents have the highest scores on the literal level with a mean of 32. 67(very high), followed by interpretation level with a mean of 27(high) and evaluation level with a mean of 26.5 (high). on the other hand, the respondents find difficulty in the integration level with a mean of 17.75 (low). generally, the respondents’ proficiency level in reading https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 82 comprehension using a short story has an overall mean of 25. 98 (high), which means that they are proficient users of english. this reveals that through short stories, students can be in contact with real language. this was supported by parede (2010) when he found in his research that short stories are interesting to use as material for personal enjoyment and help students achieve better mastery in the development of language skills. further, researchers show that among other literary genres, short stories are the most preferred ones in language classrooms (küçükoğlu and sariçoban, 2011). however, the study of giang (2017) in vietnam challenged the result of the present study when she found in her research that the respondents had poor comprehension when a literary piece is in prose like a short story, whereas in the present study this was even ranked 1 with high descriptive value. the result of the study is attributed to the language exposure of the respondents. they are fond of watching stories or movies which have a great contribution to easily understanding the plot of the story. however, it can be seen that the respondents do not appreciate and feel the importance or value extracted from the text. table 5. proficiency level in reading comprehension using a short story. levels of reading comprehension mean descriptive value literal 32.67 high interpretation 27 high evaluation 26.5 high integration 17.75 moderate overall mean average 25.98 high overall result of the reading comprehension level table 5 presents the overall result of the respondents’ reading comprehension level in the different literary genres. as gleaned from the table, the respondents performed well in the literal level with a mean of 35.2(very high), followed by interpretation level with a mean score of 21.78 (high). however, both evaluation and integration levels are displayed problems with a mean of 17.16 and 20.16. both got low descriptive value, which reveals that they are fair users of english. this reveals that the senior high school students have not developed yet their higher-order thinking skills (hots). they could hardly answer questions elicited by why and how. they lack skills in giving judgment and generating ideas that are implicitly stated in the selections. this means that the respondents are poor in reading between the line and reading beyond the line. this implies that the senior high school students are most likely capable only of comprehending literal meanings of facts and information constituted by simple written texts, sentences, or paragraphs through lower thinking skills (lots). the finding of the study supported the study conducted by tizon (2011) in misamis occidental, the philippines in which the majority of the respondents had average literal https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 83 ability; however, their higher level of thinking/comprehension was very low as reflected in their lowest creative ability, lower interpretative, and evaluative skills. in the study conducted by domingo (2018) on the english proficiency of the senior high school students of visayas state university, she corroborated the result of the present study. it was found in her study that the students are varied in their proficiency level in using the english language. that the overall results based on the mean scores show that students are competent in listening and good in reading, speaking, and writing. table 6. summary of the proficiency level in reading comprehension. levels of reading comprehension mean descriptive value literal 35.1 very high interpretation 21.78 moderate evaluation 17.16 moderate integration 20.16 moderate overall mean average 23.55 moderate 5. conclusions and recommendations generally, the senior high school students were categorized under “proficient user of english” or “high” level in using grammar and reading comprehension. although the english language is not the native language in the country, the students demonstrate commendable performance along with grammar and reading comprehension. this displays that they have mastered the basic linguistic units which serve as the foundation in learning the english language. however, it is still recommended that english language teachers may give more emphasis on the use of verbs. students should be exposed to more meaningful activities or drill to appreciate and eventually master their use. further, teachers may not only consider convergent questions during the class discussion but also divergent questions or analytical questions to help students develop their higher order thinking skills (hots) skills. finally, esl teachers may continue to look for creative ways to teach reading comprehension to help english learners. acknowledgments: the research would like to acknowledge the people who in one way or another have contributed to the success of this extension program “communication enhancement training (cet)”. the cagayan state university for approving and funding this program to enhance the english language proficiency of the senior high school students at gammad national high school, the adopted school of the department of arts and humanities, college of arts and sciences, cagayan state university-carig campus. the learning facilitators of the department of arts and humanities, college of arts and sciences https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 84 for the service and commitment they have shown during the conduct of the extension program. the teachers and the senior high school students at gammad national high school for embracing the extension program. references baclig, c. e. (2020). philippine drops further in global english proficiency ranking. inquirer.net. https://newsinfo.inquirer.net/1362951/philippines-drops-further-in-global english-proficiency-rankings baeta,e.o, galvan, m.l., solomo, o.c., zamudio, j.r.d., haber, c.c.,& osea, g.b. (2012). english language proficiency of college freshmen at central bicol the state university of agriculture. https://garph.co.uk/ijarmss/feb2016/20.pdf. accessed 2 january 2019. baier, j. r. (2005). reading comprehension and reading strategies. published dissertation. master of education degree in education. the graduate school: university of wisconsin-stout. blas, f. a., meraña, m., averion, r. f., badanoy, m. a., leon, a. l., ragudo, y. c., seguerra, r., vargas, m. g. (2018). articulated difficulties of grade 11 students at taytay seniorhigh school (tshs) in english proficiency. people: international journal of social sciences, 3(3),1186-1202. https://dx.doi.org/10.20319/pijss.2018.33.11861202 bragger, j.d. (1985). materials development for the proficiency-oriented classroom. c.j. james (ed.), foreign language proficiency in the classroom and beyond (79-116). lincolnwood: national textbook company. https://www.jstor.org/stable/1476842. accessed 20 december 2019. domingo, p. (2018). english language proficiency of students in visayas state university. journal of literature and language teaching, 11(2). https://doi.org/10.2139/ssrn.3355148 elo, s., kaariainen, m., kanste, o., polkki, t., utrianien, k., kyngars. h., (2014). qualitative content analysis; a focus on trustworthiness. sage open, 1-10. https://doi.org/10.1177/2158244014522633 elo, s., kääriäinen, m., kanste, o., pölkki, t., utriainen, k. and kyngäs, h. (2014) qualitative content analysis: a focus on trustworthiness. sage open, 4, 1-10. https://doi.org/10.1177/2158244014522633 global english corporation (2014). philippines: world's best country in business english. business english index (bei), 23(4), 35-40, september 2014. giang, b.t.k. (2017). reading comprehension performance of the efl first-year students: basis for reading enhancement program. english language teaching, 10(7),89. https://doi:10.5539/elt.v10n7p89 hadaway, n.l., vardell, s.m., & young, t. a. (2001). scaffolding oral language development through poetry for students learning english. the reading teacher, 54(8), 796-806. https://www.jstor.org/stable/20204994. accessed 19 february 2022. hadaway, n.l., vardell, s.m., & young, t. a. (2006). matching books and readers: selecting literature for english learners. international reading association, 59(8), 734-741. https://doi:10.1598/rt.59.8.1 https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://newsinfo.inquirer.net/1362951/philippines-drops-further-in-globalhttps://garph.co.uk/ijarmss/feb2016/20.pdf https://dx.doi.org/10.20319/pijss.2018.33.11861202 https://www.jstor.org/stable/1476842 https://doi.org/10.2139/ssrn.3355148 https://doi.org/10.1177/2158244014522633 https://doi.org/10.1177/2158244014522633 http://dx.doi.org/10.5539/elt.v10n7p89 https://www.jstor.org/stable/20204994 https://doi:10.1598/rt.59.8.1 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 85 heilenman, l. k., kaplan. i. (1985). proficiency in practice; the foreign language curriculum. in c. j. james (ed.), foreign language proficiency in the classroom and beyond (pp. 55-78). lincolnwood: national textbook company. james, c. j. (1985). learning from proficiency: the unifying principle. in c.j. james (ed.), foreign language proficiency in the classroom and beyond (pp.1-8). lincolnwood: national textbook company. küçükoğlu, h. and sariçoban, a. (2011). using literature in efl classes: short story. in 1st international conference on foreign language teaching and applied linguistics fltal 2011 proceedings book (160-170). sarajevo: fltal. leyaley, r.g. (2016). the english language proficiency of freshmen students in the institute of teacher education, kalinga apayao state college. international journal of advanced research in management and social sciences 5(2), 1-22. mitra, r. m. (2013). leveraging service sector growth in the philippines. asian development bank. economics working paper series, 366. murcia, m.c. & larsen-freeman, d. (2010). the grammar book. cenage learning, inc. navarro, j. d., garbin, z.z., & sagera, e.m. (2015). maritime students’ english proficiency and their feedback on instructional materials. asia pacific journal of maritime education, 1(1), 63-83. norris, j.m. (2011). using poetry to teach reading comprehension strategies to english language learners. school of education and leadership student capstone theses and dissertations. 451. https://digitalcommons.hamline.edu/hse_all/451 o’sullivan, b. (ed.) (2006) issues in testing business english: the revision of the cambridge business english certificates (studies in language testing, volume 17), cambridge: cambridge university press. o’sullivan, b. (2012). assessment issues in languages for specific purposes. modern language journal, 96(11), 71-88. https://doi.org/10.1111/j.1540-4781.2012.01298.x pardede, p. (2011). short stories use in language skills classes: students’ interest and perception. [online]. available at: http://www.researchgate.net/publication /256453850_short_stories_use_in_language_skills_ racca, r.b. & lasaten, r.c.s. (2016). english language proficiency and academic performance of philippines science high students. international journal of languages, literature, and linguistics, 2(2), 44-49. https://doi.org./10.18178/ijll.2016.2.2.65 rao, c.s.v. (2016). a brief study of english language proficiency: employability. english for specific purposes world, 17(49),1-9. saka, ö. (2014). short stories in english language teaching. international online journal of education and teaching (iojet), 1 (4), 278288. http://iojet.org/index.php/iojet/article/view/59/77 sijali, k.k. (2016). english language proficiency level of higher secondary level students in nepal. journal of advanced academic research, 3(1), 59–75. https://doi.org/10.3126/jaar.v3i1.16618 https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1111/j.1540-4781.2012.01298.x https://doi.org./10.18178/ijll.2016.2.2.65 https://doi.org/10.3126/jaar.v3i1.16618 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.16638 social and technological sciences e-issn: 2341-2593 manuel (2022) mult. j. edu. soc & tec. sci. (2022), 9(1), 71-86. https://doi.org/10.4995/muse.2022.16638 86 tan, r. w. (2016). english language proficiency of college graduates in the philippines. retrieved from https://simplyeducate.me/2019/09/13/why-are-english-language-proficiency-exams-important/. accessed november 11, 2019. tima, r. (2018, feb. 21). english proficiency of pinoy students, teacher lagging-survey. gma news online. https://www.gmanetwork.com/news/topstories/nation/644114/english-proficiency-of-pinoy-students teachers-lagging-survey/story/ tizon, m. n. (2011). reading comprehension ability of grade vi pupils of kinangay sur elementary school, proceeding of the global summit on education 2013, 11-12 march 2013, kuala lumpur. http://local.isu.edu.ph/institutional_research_office/publications/vol.16no.1/3.html. accessed 16 july 2018. tuan, v.v. (2017). communicative competence of fourth year students: basis for proposed english language program. canadian center of science and education,10(7), 104-122. http://doi.org/10.5539/elt.v10n7p104 valderama, t. c. (2019). pinoy’s english proficiency declines sharply. the manila times. retrieved from manilatimes.net/2019/11/18/opinion/columnists/topanalysis/pinoys english-proficiency-declines-sharply/65674/. accessed 23 november 2020. vural, h. (2013). use of literature to enhance motivation in elt classes. mevlana international journal of education (mije), 3(4), 5-23. http://dx.doi.org/10.13054/mije.13.44.3.4 https://doi.org/10.4995/muse.2022.16638 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://simplyeducate.me/2019/09/13/why-are-english-language-proficiency-exams-important/ https://www.gmanetwork.com/news/topstories/nation/644114/english-proficiency-of-pinoy-studentshttp://local.isu.edu.ph/institutional_research_office/publications/vol.16no.1/3.html http://doi.org/10.5539/elt.v10n7p104 http://dx.doi.org/10.13054/mije.13.44.3.4 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 58 a survey analysis of art teachers’ use of transmedia technology liudmyla havrilovaa* , valentyna oriekhovaa , olena beskorsaa , olha churikovakushnirb , zoya sofroniib ashei “donbas state pedagogical university”, sloviansk, 84112, ukraine byuriy fedkovych chernivtsi national university, chernivtsi, 58012, ukraine *correspondence: havrilovalg@gmail.com received: 4 january 2021; accepted: 10 march 2021; published: april 2021 abstract transmedia technology is a modern innovative system which scientists consider to be real communication revolution. transmedia technology is investigated and actively implemented in different life spheres and social activities, such as mass media, journalism, marketing, culture, education etc. there is an urgent need to study the opportunities of using the transmedia technologies in art education, in particular in music teachers’ professional training. the implementing of transmedia technology in music teachers’ professional training contributes to the qualitative development of their professional competence, broadening of their outlooks, the improvement of artistic abilities, and professional skills. to study in detail both teachers’ and hei students’ mastering of transmedia technology, the online survey “transmedia technology in art education” was held. hei lecturers, secondary school art teachers, teachers of art school in sloviansk, students of shei “donbas state pedagogical university” and communal institution “pokrovsk pedagogical college” took part in this questionnaire. the analysis of tests has proved the low level of acknowledgement about the essence of transmedia technology. however, it has demonstrated a high level of general technology knowledge and skills that makes it possible to implement the transmedia technology in educational process. we have revealed a number of issues which require the attention and further development. keywords: transmedia; transmedia technology; digital technologies; music teachers; professional training. to cite this article (example): havrilova et al. (2021). a survey analysis of arts teachers’ use of transmedia technology. multidisciplinary journal for education, social and technological sciences, 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0003-1814-5323 http://orcid.org/0000-0002-7909-1077 https://orcid.org/0000-0001-8018-8549 https://orcid.org/0000-0002-9166-7008 https://orcid.org/0000-0002-3163-460x https://doi.org/10.4995/muse.2021.14875 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 59 1. introduction the concept of transmedia technology is rather new in modern science. it has become popular in the previous decade and it is considered in the sphere of journalism and mass media. the emergence of this phenomenon is greatly connected with the constant development of digital technologies and their active implementation in all spheres of human activities. the founder of the concept “transmedia” is considered to be american philosopher and culturologist jenkins (2003) who explains the concept of “transmedia” as “across media”. the researcher uses the modern concept “transmedia” in the chapter “storytelling”, paying special attention to the multiplatform, the extend, and the interactivity of storytelling. in literary sources, the concept of “transmediality” is also used in the context of multiplatform that is characterized by a diversity of platforms, channels, and media forms (jenkins, 2003). transmediality is one of the signs of the modern phenomenon of technological media culture. the main advantage of new technology is an opportunity to create a polyphonic (syncretic) communication platform, which has various elements combined by one theme (al-khanaki, 2016). in ukrainian literature, the conceptual framework of transmedia is developing. based on the analysis of foreign researches, ahrikova (2016), vyhovska (2014), zenchenko (2019), zhurba (2018), and tormakhova (2018) study conceptual variability of transmedia in the scientific discourse of the digital age. some researchers (al-khanaki, 2016; zhenchenko, 2019) think that the concepts of “transmedia” and “transmedia storytelling” are equal, implying a way of storytelling that disseminates information using different media platforms. whereas some scientists (ahrikova, 2016; vyhovska, 2014; jenkins, 2003) distinguish them as two different concepts. thus, ahrikova (2016) defines transmedia as a set of principles that identify the development of a media product: dissemination, distribution, engagement, seriality (multivectority), length (form openness), subjectivity, immersiveness, and co-creation; however, transmedia storytelling is a product of the professional activity, developed according to the principles of transmedia. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 60 so, transmedia is an important issue in journalism, which is full of news and reports. in marketing, it is a brand for wide dissemination among users; in art, as it combines a variety of genres, types, manifestations, and it is represented by several art lines and stories in mass media platforms (media), which include both seven traditional technological aspects (printed publication, audio and video records, radio, cinema, television, computer networks, and mobile technologies) and means of non-verbal communication (body language, gestures, facial expressions, etc.) (scolari, 2016). we have defined the concept of transmedia technology as a complex system of combining various languages (digital and non-digital; verbal and non-verbal) and formats (video, comics, games, books, etc.), providing integration of modern media formats with digital technologies. revealing the concepts of a narrative occurs at several platforms: a book, a film, internet resources, social and interactive media environments. this makes an effect of kaleidoscopic perception that has a positive influence on attention, active position, a level, and the amount of material to master (jenkins, 2003). after disclosing the opportunities of transmedia technology in different spheres of professional activities, it’s essential to emphasize that there isn’t much experience of using transmedia technology in future music teachers’ training. however, a wide range of documented cases of implementing transmedia strategy in educational activities suggests the benefits of using transmedia technology while studying the artistic and aesthetic academic disciplines. so, the objective of the study is to analyze music teachers’ mastery of transmedia technology that contributes to improving their professional competence. 2. literature review the experience has proved the benefits of using transmedia technology in different social spheres: marketing, journalism, design, culture education, etc. so, one of the founders and active supporters of transmedia technology scolari (2018b) pays special attention to the benefits of transmedia strategy in branding in his article “transmedia branding: brands, narrative worlds, and the mcwhopper peace agreement”. according to his point of view, transmedia narratives influence https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 61 the recombination and flexible change of branding that makes it possible to follow the transmedia universe all over the world. also, scolari, masanet, guerrero-pico, & estables (2018) emphasize the feasibility of using the transmedia strategy for the development of teenagers’ transmedia skills and informal learning strategies, giving an example of creating the transmedia universe of archaeological site atapuerca (burgos, spain), announced to be an object of the world heritage by unesco. the scientists point to the problem of bifurcation of communication policy while publishing the results of the research of the real existence of atapuerca (information for both the scientists and society members). the purpose of the scientific experiment is to create a transmedia model of the atapuerca universe, which, on the one hand, is controlled by producers-scientists and, on the other hand, it is shared by prosumers (disseminators, users) who complement, exchange, change and generate new content by using social networking sites (facebook, twitter, instagram, youtube, etc.). modern scientists consider transmedia as one of the communication types that is influenced by contextual factors (culture, history, politics, ideology, education, and interpretive communities). transmedia storytelling is of special importance as it is widely used by mass media as narrative transmediation (ryan, 2020; chinita, 2020). transmedia is also used in film production and tv for making special effects. making a review of b. rehak’s more than meets the eye: special effects and the fantastic transmedia franchise, wood (2020) discusses the transmedial approach to special effects. he says that special effects stay behind the screen images which are created in the context of a negotiated dialogue with fan culture. the researchers show and prove that the strategies of transferring information by the use of transmedia technology can be implemented in scientific projects. chenu et al. (2014) describe the project culte (cultural urban learning transmedia experience) that is aimed at studying the role of museums in suggesting new ways of interacting with the cultural heritage, as well as using the transmedia storytelling for interpreting cultural heritage. the project authors analyze how transmedia technology can help designers develop narratives and museum workers make media texts and develop games. it can integrate the industry of entertainment and museum context. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 62 brusk & engstrom (2020) present a unique inclusive transmedia project, called marvinter, that focuses on making radio series and mobile games with similar episodes. the project is designed for people who have hearing or vision impairment that is why transmedia technology is needed due to its multi environmental nature. yeates (2020) reveals the successful use of transmedia technology for making art series, in particular fiction podcasts. the adventure zone is given as an example that shows the changes in modern culture through transmedia, fan-focused form of storytelling, and other forms of new media technologies. garcia-roca (2020) presents an original view of the interpretation of literary works by the use of digital tools. digital reading is determined not only by the nature of the text (structure, textuality, or support) but also by the behavior of digital readers who can create communities, discuss, analyze and develop fan-works. as a result, multilinear, hypertextual, and transmedial reading emerges. the scientific development of dalelio (2020), who works on the collaborative identity project, is of special interest in the context of our study. this project focuses on media literacy, identity, technology, digital media, social media, mediated communication, and interpersonal communication. the main objective of the project is to disseminate the transmedia experience in the digital environment. the scientists garcía-marín & aparici (2018) substantiate the importance of using transmedia strategy in the information sphere. the authors establish a parallel between the transmedia technology and podcasts, calling them a new sound communication. we consider podcasting to be a radio on the internet that can broadcast audio and more rarely video offline (what are podcasts, 2019), to be the dissemination of the audio and video content according to the chosen theme through the internet (myatin, 2006). having analyzed the spanish podcasts and using in-depth interviews, the authors substantiate the significance of a multiplatform strategy where messages are presented in a wide range of platforms and complement one another. the researchers show the positive results of broadcasting information materials in different media languages that contribute to the complete disclosure of the content. transmedia is also regarded as a type of digital technology. shim, yecies, ren & wang (2020), analyzing some well-known digital webtoons (digital comics) and online platforms for https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 63 webnoveling (a novel, published on the web platforms), confirm the transformation of creative work and the emergence of transmedia intellectual property (ip). transmedia technology contributes to developing digital literacy. the scientific findings of alves, ricardo, flexor & ochi (2020) and girón garcía & navarro ferrando (2014) prove the urgent need to develop media and transmedia literacy of all the people as the net promotes sharing of multimodal content at different platforms and requires users to have new skills to work with various hardware, software and their protocols. sime & themelis (2020) have introduced the concept of transmedia identity that is determined as the ability to make and manage multiple identities on different platforms. the scientists assert that transmedia identity is the essential competence for online educators. sixto-garcia, melo & guilhermina (2020), focusing on the necessity to improve the digital competence of university teachers, think that it is crucial to integrate transmedia storytelling with other technological innovations. the forms of transmedia storytelling are studied by javanshir, carroll & millard (2020), who present their model for disclosing the fundamental structural features of transmedia storytelling. they also distinguish three groups of patterns that are identified in all transmedia stories. moya & moya (2018) explore the sphere of implementing transmedia technology in management. the scientists call “transmedia storytelling” the real communication revolution or a new business model. in the world of the constant digital revolution, they pay attention to the new paradigm of content generation and improvement of digital networking technology connection. these factors have a positive impact on the development of the collaborative environment by working together. transmedia strategy provides strong links between producers of goods and services, advertising agencies, mass media, and society. the internet and social networking sites allow citizens to defend their interests and values of branding. the authors consider prosumers (citizens) to be “a smart swarm” that acts as not only an observer with limited opportunities of the expression, they (prosumers) become the real investors. they mean emotional, social, and intelligent investments. the authors substantiate the importance of implementing the transmedia strategy in the marketing that contributes to generating the importance, degree, and dependence of a continuous process of participation in common activities. the key factors of the success and the efficiency of https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 64 advertisement is the level of satisfaction which makes the members of the advertisement take part in transmedia designing and communication strategy of brands. the conclusions, made by gonzalez-diez, puebla-martinez & perez-cuadrado (2018), about the importance of implementing the transmedia technology in the sphere of design (“from layout to transmedia narrative: a review of the concept of “news design”), are important for our research. the scientists point to the significant changes that take place in the area of journalism. these changes concern not only the style and form of presenting the news but also their external arrangement and design. the design in mass media is connected with several spheres: paper design, web-design, news design, etc. it is changing all the time and includes new elements. previously, the design layout was based on images, colors, and compositions. nowadays it is supplemented by sound, video, and active links, which connect them (hyperlinks, qr codes, etc.). that means the use of transmedia technology contributes to bringing users closer to their usual communication style while developing the design layout. mateos-rusillo & gifreu-castells (2018) disclose the area of using transmedia technology in museums, giving the national museum del prado as an example. the scientists analyze traditional ways of the functioning of museums and define the level of an approximation of the bosch project to the didactics of transmedia storytelling. the scientists assert that museums naturally use transmedia technology in their practice for describing historic events over the past decades, without recognizing its implementation. in education, transmedia technology plays an important part. beskorsa (2017) studies the ways of implementing transmedia technology for developing the virtual learning environment for future teachers’ training. mccarthy, tiu & li (2018) disclose the feasibility of implementing the transmedia technology while learning mathematics in preschool and primary school period (“learning math with curious george and the odd squad: transmedia in the classroom”). they have conducted two pieces of research. during the first one, preschool teachers had been doing mathematical problems with the animation character “curious george” for four weeks. the games involved the materials and video-episodes, downloaded from the free web site curious george. they were also supplemented with the materials from books and other general learning activities. the research included the observation, interview with teachers-participants, and control questionnaires. according to the mathematical results of questionnaires, the level of knowledge had https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 65 increased considerably that proved the positive influence of transmedia technology on preschool children’s mathematical knowledge. in the second research, the learning process included digital and non-digital learning resources from the popular social series odd squad (video, online games, commercials, practical tasks from transmedia package odd squad). the research contained an oral and written assessment of the firstyear students’ mathematical skills, interviews with students and parents. after experimenting, the considerable improvement was noted in the level of general mathematical knowledge in numbers and operations with them, as well as in algebraic thinking and vocabulary. the scientists claimed that the implementation of transmedia technology in the learning process contributed to the students’ motivation for learning activities. moreover, the authors said that the game was the main activity in the preschool and primary school period. thus, we are sure that transmedia technology has an influence on different spheres of social life, including education. we suppose that the use of transmedia technology in music teachers’ professional training contributes to the qualitative development of professional competence, broadening of the outlook, the development of artistic abilities and professional qualities. to predict the implementation of transmedia technology in music teachers’ professional training, it is necessary to study the present state of information and communication awareness of future and practicing educators. it is important to understand which knowledge and skills students have, which communication and digital skills they use in their activities, which level of motivation for the use of innovative technologies in their practical and everyday activities, in particular the transmedia technology, they have. 3. methodology of research the research is based on the quantitative method used for measuring university students’ and lecturers’ level of the knowledge and mastering transmedia technology. the research sample were gained from 40 students of state higher education institution “donbas state pedagogical university” and communal institution “pokrovsk pedagogical college” and 31 university lecturers and secondary school music teachers. the students were selected based on the study year, they https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 66 were three-year and four-year students who had already studied the courses of teaching methods of music and might be aware of the essence of transmedia technology. the lecturers were selected based on their engagement in the process of music teachers’ training, as well as school teachers were engaged in the process of teaching music. the lecturers worked at the department of primary education theory and practice, the department of music and choreography, music teachers were from sloviansk secondary schools and sloviansk art school. so, we organized online survey “transmedia technology in art education” for students, university lecturers and school teachers. each survey participant had to answer 12 questions. online questionnaires were developed on the google-platform. the survey reference for students (https://forms.gle/1jkyc7brpaa4xacc6), lecturers and school teachers (https://forms.gle/e4s8evbyrqcdppoy6) were provided by the department administrative staff. the participation in the survey for all the respondents was voluntary. the survey questions for defining the general level of students’ and teachers’ (lecturers’) knowledge (awareness) in the field of transmedia technology were divided into three blocks by their content and research area (table 1): block 1 includes questions about the level of knowledge and understanding of transmedia technology definitions, the awareness of the concepts that concern this technology; block 2 contains the questions for defining the level of knowledge and using cloud services and ic technologies whose knowledge contributes to the implementation of the transmedia technology. block 3 is aimed at disclosing the level of motivation and readiness to use transmedia technology while doing learning and professional activities. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 67 table 1 list of survey questions. 1. are you familiar with the concept of “transmedia technology”? 2. what is transmedia? 3. which of the proposed concepts don’t you understand: media, convergence, content, multimedia, media platform, media format, hyperlinks, interactive poster, blog, chat, qr code? 4. what modern ict technologies do you use in your learning/teaching activities? 5. what elements of ict technologies do you use in your work? 6. are you an active user of multimedia software? 7. what online services do you know? 8. are you aware of the concept “cloud services”? 9. what cloud services do you know? 10. are you an active user of social networking sites? 11. are you interested in the issue of transmedia technology? 12. do you want to use transmedia technology in learning/teaching activities? 4. research results we proceed to analyzing the results of the students’ survey. 17 master students, 21 bachelor students, and 2 junior bachelors participated in the survey (table 1). table 2 analysis of the level of students’ awareness of the definition “transmedia technology” hear for the first time see in the papers and conference proceedings know and study use in their practical activities master students 3 (7,5%) 11 (27,5%) 2 (5%) 1 (2,5%) bachelor students 12 (30%) 4 (10%) 1 (2,5%) junior bachelors 1 (2,5%) 1 (2,5%) answering the question “do you know anything about “transmedia technology”?”, 20 respondents (50%) answered that they had never heard about it before; 16 students (40%) said that they had seen and heard about it in scientific papers and conferences; 3 participants (7,5%) said that they knew about this technology and they were studying this issue; only 1 respondent (2,5%), who was a master student, was a member of the transmedia project. also, master students presented the highest number of participants, who saw the concept “transmedia technology” in papers and proceedings of conferences – 11 persons (27,5%); two students (5%) were familiar with the definition and were studying it. only three students (7,5%) claimed that they heard this definition for the first time. the bachelor students had lower indicators: 12 of them (30%) heard https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 68 for the first time: 4 students (10%) had seen in papers and proceedings of conferences; 1 respondent (2,5%) was familiar with the issue and was studying it. taking into consideration a small percentage of students, who knew the definition “transmedia technology”, the answers on the second questions were unexpected for us. answering the question about what the transmedia technology was, 19 respondents (47%) defined this concept correctly. it was an interesting fact that 11 of them heard the concept for the first time. this leads us to conclusions about an intuitive understanding of the proposed concept or a sufficiently high level of general digital awareness. to answer the third question “which of the concepts given can’t you understand?”, the students were proposed the concepts that they could often see in the context of the transmedia technology: media, convergence, content, multimedia, media platform, media format, hyperlinks, interactive poster, blog, chat, qr code. the results of this survey showed that the largest number of respondents (32 – 86,5%) were not familiar with the concept “convergence”, the concept “content” (8 participants – 21,6%), and “interactive poster” were significantly less problematic. the unknown concepts were the following: qr code – 5 (13,5%), media platform – 4 (10,8%), media format – 4 (10,8%), hyperlink – 2 (5,4%), media – 1 (2,7%) and multimedia – 1 (2,7%) (fig. 1). thus, analyzing the first block of questions, we could see that the issue of transmedia technology was quite new for students. the number of students, who had never heard about this concept before and were familiar with it only superficially, was approximately the same. the students who studied to obtain master’s degree had better awareness of the issue. this leads us to the conclusion that master students were more interested in scientific and research work than bachelor students. taking into consideration a high percentage of participants who gave correct answers, defining “transmedia” (in particular, students who had heard this concept for the first time), we could conclude that the understanding of this concept was intuitive. the analysis of unknown concepts let us define the frameworks of the issues that need attention and further study. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 69 figure 1. unfamiliar concepts that concern the transmedia technology the analysis of the answers of the second block proved that about half of the respondents (18 persons – 45%) always used cloud services in learning and everyday life, 8 persons (20%) studied the opportunities to use the cloud technologies; and the certain number of participants (7 persons (17,5%) of the whole number of respondents) were unfamiliar with the concept “cloud services” or were uninterested in this issue (fig. 2). figure 2. level of knowledge and using cloud services https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 70 we should note that google disk was the most popular cloud service, 32 respondents (80%) were familiar with it; 17 persons (42,5%) knew microsoft sky drive; 11 respondents (27,5%) were familiar with dropbox; 7 respondents (17,5%) – edisk; 3 persons (7,5%) – mega service; 1 participant (2,5%) proposed his/her variant of sound service – icloud. the survey also proved that the students actively used modern ic-technologies in their learning activities: mobile learning – 26 respondents (65%); multimedia technology – 25 (62.5%); distance learning – 19 (47%); blended learning – 16 (40%); open online courses – 15 (37,5%). and only 1 (2,5%) respondent didn’t use ic-technologies in the learning (fig.3). figure 3. ic-technologies that are used by university students in their learning activities we’ve found out that most students (34 participants (85%)) used multimedia presentations in their learning practice. also, a great number (28 participants (70%)) actively implemented e-mail resources and looked for information on the internet. 26 (65%) participants of the survey used electronic multimedia guides. much fewer students (13 persons (32.5%)) took part in the online stream of webinars, conferences, etc., 12 persons (30%) made interactive posters, 12 persons (30%) used qr-codes, 8 persons (20%) – electronic journals and diaries. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 71 that’s expected that among software word – 38 participants (95%) and powerpoint – 26 participants (90%) were the most popular. also, students almost equally used excel (22 respondents (55%)) and photoshop (20 students (50%)). the level of the students’ knowledge about learning online services has also drawn our attention: about half of the participants (17 persons (43,6%) were familiar with learning apps, the same number of respondents were not familiar with online learning services at all. fewer students possessed information about thing link (9 students (23,1%), glogster (8 persons (20,5%), and other programs (less than 10%). thus, analyzing the second block of questions, we have made conclusions that most of the students were knowledgeable about cloud services and half of the respondents used them in their everyday lives; modern ic-technologies, in particular multimedia and mobile learning, were an integral part of modern education; the students could freely use necessary applications, which they applied in their everyday life and in their learning activities to search for information, reproduce and share it. the low level of knowledge about learning online services has drawn our attention. this suggests that students more often used traditional computer programs than online services and online resources which they used for finding and sharing information. the third block of the questions makes it possible to define social networking activity and the sphere of using electronic resources in everyday life. so, we found out that most of the respondents were active users of social networking sites: instagram – 29 persons (72,5%); youtube – 28 persons (70%), facebook – 25 persons (62,5%). to define the purposes of using the internet resources, we found out the following: 35 participants (87,5%) used the resources of the internet to communicate, comment and share information; 29 respondents (72,5%) – to search for information; 27 respondents (67,5%) – to support self-development and self-education; 24 respondents (60%) – to fulfill the learning tasks; 20 participants (50%) – to surf social networking sites; 15 participants (37,5%) – to play online games and to entertain. considerably a smaller number of respondents use the internet for designing their site, blog, or other internet products (less than 30%) (fig. 4). thus, according to the results of the survey, we can assert that the concept of transmedia is new for students, but their general level of knowledge makes it possible to implement this technology in the learning process. we believe that the students surveyed more likely to use passively already existing online products on the internet than create new content themselves. in practical activities, they prefer traditional well-known technologies and software, which is installed https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 72 on your computer (devices), to online services. the respondents are knowledgeable about where and how to search for information, as well as what to do with it. they also use their knowledge and skills for communicating, learning, and entertaining. figure 4. purposes of using internet resources in the survey for university lecturers and secondary school music teachers, the respondents of different age and scientific levels took part. they were: 1 respondent (3,2%) – doctor of sciences, 8 persons (25,8%) – doctors of philosophy and 22 persons (70,1%) didn’t have any science degree. according to the age indicators, 12 respondents (38,7%) were older than 50, 10 participants (32,25%) were between 36 and 50, 9 persons (29%) were between 20 and 35. analysis of the question about the knowledge of the transmedia technology gave such results: 18 respondents (58%) hadn’t heard about “transmedia technology” before; 11 participants (35,5%) had seen this concept in the papers and conference proceedings; 2 respondents (6,4%) knew about this technology and studied this issue, and no one implemented the technology in their practical activities. 11 respondents (35,5%) correctly defined the concept “transmedia”, however, in comparison with the students, quite a large number of teacher respondents (8 persons (25,8%) choose the definition that corresponded to the characteristics of multimedia technology. this makes https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 73 us think that participants of the survey were to a greater extend practicing lecturers who worked more often with multimedia while using digital tools. several unclear questions, which concerned transmedia technology, was quite large. so, the most difficult question was about “convergence” – 22 participants (70,9%); the concept of media platform was less incomprehensible (11 persons (35%), qr code (10 persons (32,2%), hyperlinks (9 persons (29%), content (8 persons (25,8%). few respondents hesitated to understand the concepts of media format (6 persons (19,3%), interactive poster (5 persons (16,1%), etc. (fig. 5). figure 5. unfamiliar concepts for lecturers (teachers) that concern transmedia technology if we compare indicators of lecturers’ (teachers’) answers with the students’ diagram with unfamiliar concepts (fig. 1), we can see that the number of concepts, which require explaining and paying attention to, is common for both categories of respondents. thus, according to the survey results about the theoretical knowledge of transmedia technology, we can conclude that the issue of transmedia technology is quite new and unclear for a great number of university lecturers and secondary school music teachers. a lot of concepts require explaining or correlating with practical activities. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 74 the next range of questions gave us a clear understanding of the learning and practical use of ic-technologies in the learning process. as the survey has shown, quite a large number of respondents (17 persons (54,8%) used multimedia and blended learning technology in their practical activities; a bit fewer respondents used distance learning (14 persons (45,2%)). few survey participants also implemented mobile learning (7 persons (22,6%) and open online courses (4 persons (12,9%). 7 respondents (22,6%) didn’t use ic-technologies in their practical activities at all that was unclear, taking into account the intensive informatization of modern education (fig.6). figure 6. ic-technologies which lecturers (teachers) use in their practical activities the most often (28 cases – 90,3%) respondents used ic-technologies to apply the e-mail resources and to search for information on the internet. also, a great number of survey participants (23 persons (74,2%) involved electronic multimedia textbooks into the learning process; 15 survey participants (48,4%) developed multimedia presentations; 9 respondents (29%) created interactive comics and posters. conducting online streams and seminars (5 persons (16,1%), the use of qrcodes (6 persons (19,4%), keeping electronic journals, diaries, etc. (4 persons (12.9%) and https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 75 implementation of social network resources in scientific research (1 person (3,2%) were less popular (fig. 7). figure 7. ways of using internet resources by lecturers (teachers) the respondents actively used such software as word (29 persons (93,5%); power point (20 persons (64,5 %)); excel (15 persons (48,6%)) and photo shop (10 persons (32,3%)). the survey of the use of learning on-line services showed that more than half of all respondents (17 persons (56,7%) were not interested in them at all. learning apps (12 participants (40%), thing link (10 participants (33,3%), glogster (7 participants (23,3%) were the most used among online programs. also, the participants suggested using mind map, padlet, and online resources, developed by google, such as google disk, classroom, google presentations, etc. thus, we have confirmation that the respondents are active users of ic technologies, in particular multimedia, blended, and distance learning. however, most often the respondents use ict to find and share information on the internet, use electronic guides and textbooks, and create their multimedia products. fewer respondents prefer active methods of using ict, such as an online stream, seminars, e-journals, learning materials with qr codes. we pay special attention to the fact that supporters of such types of learning activities are to a greater extent, secondary school https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 76 teachers. this may be explained by the implementation of the concept of the new ukrainian school. we should also note that participants actively used traditional system programs word, powerpoint, excel. however, learning online programs and cloud services were less popular. so, only 9 respondents (29%) answered that they used cloud services in their practical activities and 4 persons (12,9%) were studying this issue. at the same time, 10 respondents were unfamiliar with this issue and didn’t know any cloud services; 8 respondents (25,8%) saw them in methodological materials, but they didn’t pay any attention to them. google disk (21 survey participants – 67,7%) was the most popular among the respondents, who used cloud services. edisk – 7 persons (22,6%), dropbox – 6 persons (19,4%), microsoft ski drive – 3 persons (9,7%) were less popular. so, the survey participants, who were the active users of ic-technologies, preferred traditional programs and system programs. that might be caused by the characteristics of their professional activities. the majority of survey participants were lecturers and secondary school music teachers, in particular music teachers, who used the training of performing and creative activities in their work that did not require the direct use of internet services, including cloud services. however, it is necessary to focus on the attempts to implement new technologies in the professional activities: the use of qr codes, blended learning, cloud services, the implementation of social networking resources in scientific research, etc. such results allow us to assert that the ict competence of educators is improving that corresponds to the requirements of modern secondary and higher education. according to the results, we can see that the majority of respondents were active users of social networking sites: facebook – 23 persons (74,2%), youtube – 19 persons (61,3%), instagram – 11 persons (35,5%). and only 4 persons (12,9%) were not registered in any social networks. that makes us conclude that social networking sites are integrated into the everyday life of the representatives of the population that can become a significant reason for solving not only communication problems but also educational problems. so, according to the results of the online survey “transmedia technology in art education”, we can conclude that lecturers and music teachers have a certain level of knowledge about the tools https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 77 of ic-technology (they know about transmedia a little). this can be the basis for implementing transmedia technology in music teachers’ professional training. 5. discussion the phenomenon of using transmedia technology in music teachers’ training is poorly represented in researches of ukrainian educators and it has been started implemented in foreign teaching practice by jenkins (2003) and scolarli (2016) over the last decade. however, some scientists confirm the possibility and significance of implementing transmedia technology in different spheres of human activities while conducting their research. so, fleming (2013), a library media specialist, discloses the concept transmedia learning world (tlw). the author emphasizes that transmedia technology makes it possible to combine the benefits of digital technology and traditional teaching techniques in education that contributes to getting a deep and powerful learning experience. modern trends in education make schools study the opportunities of digital learning more carefully, encouraging significant changes in pedagogical thinking and practice. the use of transmedia methods, in particular the recognition of the transmedia learning world concept, gives a teacher tools that allow paying attention to each child. transmedia methods provide the use of the collective intelligence of students who focus on their own pace and learning type. transmedia technology immerses teachers and students in the rich virtual and physical environment that facilitates students’ real emotional interaction with the learning process. fleming (2013) concludes that learning technologies have gained new results. we agree with the statement that teachers have broad access to high-quality digital content which allows students to demonstrate their knowledge and skills in exciting and interesting ways. students’ successful professional training requires the transition from teacher-centered learning to student-centered practice. students learn better if education reproduces their own choice, selecting tools and information from different sources. the transmedia learning model creates new tasks and requires a broad and qualitative mastering of media literacy skills to correspond to competency-based requirements of the digital era. the use of transmedia methodology, according to the scientists’’ opinion, can contribute to the development of professional education. the content, which is located https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 78 on different media platforms, enables the achievement of high results of every student regardless of their initial level. martinez, atienza & zamora (2018) pay special attention to the use of transmedia technology in “hyperlinked society” in university students’ professional education. the paper authors have conducted research among the university students in the sphere of social education for defining the role that transmedia content plays in cultural and social development. they reasonably note that the need to develop a culture of convergence and transmedia literacy is becoming a key challenge in education institutions. the scientists say that learning results at universities have improved after the implementation of transmedia learning content which has been created by students. it is important to conclude that the correlation of transmedia literacy with developing public position and learning through creative experiments plays an important role in professional education and development of students’ social position. castells & illera (2018) also emphasize the positive learning results that are gained due to the implementation of the transmedia approach to education, compared with other approaches, in particular with the context-based approach. the scientists support the idea of the transmedia organization of the learning process, based on jenkins’s definition of the benefit of combining different media at different times on different platforms to immerse the user in a new situation and motivate them to understand the learning topic. after implementing the project “ancestors’ letter”, developed for 15-year-old students, the scientists concluded that the transmedia-based approach contributes to the deeper awareness of effective coordination between the curricular and extracurricular settings, between digital institutions and everyday life, both in terms of content location and dissemination. perez-manzano & almela-baeza (2018) promote a hypothesis about the usefulness of combining transmedia with gamification for encouraging teenagers to a scientific career. the scientists disclose the issue of using digital programs in learning and teaching activities. they say that digital games (dgbl) have powerful educational potential and reveal new prospects in the sphere of education. the scientists prove the credibility of this statement based on designing the gaming web-site, developed especially on the gaming basis, and supplemented with transmedia resources. the web-site operations are aimed at the development and improvement of both https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 79 scientific and technological professional achievements of teenagers in the period of forming their career preferences. the research results allow us to assume a high level of implementation efficiency of such projects. the authors record the significant increase of the interest in scientific professions and research activities, as well as an understanding of their social advantages. this confirms the students’ positive attitude toward scientific knowledge and skills. it is possible to gain high results of students’ motivation for scientific career preferences together with technological and the specific components of the project due to getting acquainted with real cases of scientific and research activity and covering successful scientific achievements and attitudes. as the scientific findings prove, the elements of transmedia technology are used in different directions of scientific activities, including the development of ethical skills, professional skills, students’ language skills, etc. within the new european higher educational area (ehea), sotelo (2018) studies the issue of opportunities that digital technologies give to improve the development of critical thinking skills, ethics and to actively involve university students into the contemporary debate. the scientist draws attention to the digital posters (infographics) and transmedia storytelling (the scientist defines it as a 3-minute video intermediation format) as new forms of communication and self-expression among teenagers. to develop university students’ critical thinking and ethical skills, in 2016 the interdisciplinary innovative project “new narrative for europe project from the humanities” (https://www.ucm.es/siim/new-narratives-for-europe) was launched (the project is currently closed). in this project the current crisis state of ethical values is highlighted through media-digital content: 1) the development of opposing thoughts and use of reliable resources to substantiate their thoughts (result – infographics); 2) the development of problem-solving skills with the results of creating interartistic video with elements of transmedia technology (3-5 minutes) with the proposition of social activity for the common good in the european context. scolari (2016) succeeds in his research and pays attention to the fact that it’s not easy to implement transmedia technology in education. we should emphasize the development of teenagers’ transmedia literacy, which plays an important role in the implementation of the transmedia technology in education. internet skills and transmedia literacy, which is called an https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 80 absolutely new approach that regards a subject as a user (producer + user), but not as a passive and estranged user. while traditional media literacy focuses on books and television, transmedia literacy puts digital networks and interactive media experiences in the center of its analytical and practical experience. scolari (2018) together with his team of scientists has researched for gaining a better understanding and analyzing of how teenagers use transmedia skills in informal education. the research team has presented a comprehensive description of the main transmedia skills that young people should possess. the result of the research is represented as a map of transmedia skills, which include playback, judgments (analysis), transmedia navigation, networking, and communication components. the research covers a significant result list: from specifically technological skills to skills that concern ideological and aesthetic manifestations. transmedia skills range from video game problem solving to producing and sharing content in the context of web platforms and social networking sites; developing, producing, sharing, and critically consuming the narrative content are also parts of transmedia competence. the scientist makes an important conclusion that as a result of the accelerated pace of technological evolution and rapid changes in the media environment transmedia skills and learning strategies need periodic updating. in any case, the most relevant issue is the reduction of the distance between the formal learning environment and the extremely active “digital life” of teenagers on social networking sites and the internet (scolari, 2018a). however, the search for scientific publications that highlight the experience of using transmedia technology in music teachers’ professional training did not produce successful results, we can find confirmation of the continued influence of transmedia on the development of contemporary artistic trends, including music. haamer (2020) asserts that the increase in popularity of the internet and mobile devices causes the prevailing interest in digital music (ifpi digital music report). havrilova, ishutina, zamorotska, & kassim (2019) substantiate the use of electronic internet resources and multimedia tools for developing the distance courses of music teachers’ professional training. the scientists consider these resources as new forms and methods that allow combining different art types that are a crucial issue for contemporary artistic pedagogy. music is widely used and no longer plays the role of sound history: it is used in movies (flaming lips, kanye west), becomes a game (jay-z), it is released as a series of applications https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 81 (björk), it becomes virtual reality (gorillaz), etc. the phenomenon of digital music culture, used in all mass media, is looking for its audience. and there is only one question if a person has time to perceive this content. the entertainment industries compete with one another, so it needs to be unique and admirable (haamer, 2020). 6. conclusions thus, based on the results of the research and analysis of scientific theoretical findings, we can conclude that the transmedia technology is at its initial stage, however, it is steadily gaining ground in national science. after analyzing the opportunities, given by transmedia technologies for professional activities in the modern world, we can say that currently there is no experience of using transmedia technology in the sphere of music teachers’ professional training. however, there is a wide range of documented cases of transmedia strategy in educational activities that make it possible to suppose that the implementation of the transmedia technology in learning the art subjects, in the integrated course “art”, is very useful. we assume that the implementation of the transmedia strategy in music teachers’ professional training contributes to the qualitative development of their professional competence, the broadening of outlook, the development of artistic skills, and professional qualities. the research has shown that the levels of students’, lecturers’, and secondary school music teachers’ digital knowledge and ic skills are rather high. the teachers master digital technologies and internet services and use them in their learning, professional and everyday activities. the respondents are knowledgeable about the issues of ic technologies and cloud services, they master computer programs and know about online services and programs (but due to the low level of knowledge this issue requires more attention). also, the respondents are active users of social networking sites, they use ict-technologies in their practical activities and everyday life to obtain new knowledge, to fulfill and solve the tasks, to get and share the information, and to communicate. this enables and makes meaningful the introduction of transmedia technology into the music teachers’ professional training. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 82 as the concept of transmedia technology hasn’t applied to the art education yet and the opportunities of its implementation in practical and creative activities haven’t been realized, we think it is necessary to implement a special course “transmedia technology in music education” for music teachers’ professional training or to introduce a separate module “using transmedia technology in music and educational activities” into the syllabus of the course “methodology of teaching music”. it is feasible to divide the course transmedia technology in music education” into three content blocks that disclose the general definition of transmedia technology, technic, and technological features of using transmedia technology in music art and the possibilities of implementing the transmedia technology in the course “music art” in primary school. the purpose of studying the special course is to reveal the content of the concept “transmedia technology”, to gain the deeper understanding of the transmedia world in the context of musical art, to facilitate the understanding of the identification of patterns and contexts of using transmedia technology, to disclose the contextual links when using media platforms, to involve students into transmedia projects. funding: this research received no external funding. conflicts of interest: the authors declare no conflict of interest. references ahrikova, e. v. (2016). formirovanie navykov transmediynogo povestvovaniya u studentov – budushchikh zhurnalistov sredstvami inostrannogo yazyka [development of transmedia storytelling skills of students – future journalists using foreign language]. obshchestvo: sotsiologiya, psikhologiya, pedagogika, 8, 91–94. https://cyberleninka.ru/article/n/formirovanie-navykov-transmediynogopovestvovaniya-u-studentov-buduschih-zhurnalistov-sredstvami-inostrannogo-yazyka al-khanaki, j. (2016). printsipy transmediynogo povestvovaniya [principles of transmedia storytelling]. uspekhi sovremmenoy nauki i obrazovaniya, 11 (5), 108–113. alves, o., ricardo, g., flexor & ochi, c. (2020). the impact of smart cities on the emergency of transmedia literacy. humanidades & inovacao, 7(6), 221–229. beskorsa, o. (2017). virtual learning environment moodle as a means to organize blended learning in teaching practical phonetics of german language. information technologies and learning tools, 62, 86–97. https://doi.org/10.33407/itlt.v62i6.1832 brusk, j. & engstrom, h. (2020). marvinter: a case study of an inclusive transmedia storytelling production. convergence: the international journal of research into new media technologies. https://doi.org/10.1177/1354856520923972 castells, n. m.& illera, j. l. (2018). activity and learning contexts in educational transmedia. digital education review, 33, 77–86. https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://cyberleninka.ru/article/n/formirovanie-navykov-transmediynogo-povestvovaniya-u-studentov-buduschih-zhurnalistov-sredstvami-inostrannogo-yazyka https://cyberleninka.ru/article/n/formirovanie-navykov-transmediynogo-povestvovaniya-u-studentov-buduschih-zhurnalistov-sredstvami-inostrannogo-yazyka https://doi.org/10.33407/itlt.v62i6.1832 https://doi.org/10.1177/1354856520923972 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 83 chenu, c., german, r., gressier-soudan, e., levillain, f., astic, i. & roirand, v. (2014). transmedia storytelling and cultural heritage interpretation: the culte project. mwf2014: museum and the web florence, mar 2014, florence, italy. http://mwf2014.museumsandtheweb.com/paper/transmedia-storytelling-and-cultural-heritageinterpretation-the-culte-project/ chinita, f. (2020). peter greenaway’s the tulse luper suitcases project (20032005) transmedia storytelling as self-reference multimediality. transmediations: communication across media borders / ed. salmose, n. & ellestrom, l. routledge, oxford, england. pp. 31–51. https://doi.org/10.4324/9780429282775-3 dalelio, c. m. (2020). the collaborative identity project: a shared transmedia experience in digital representation. communication teacher, 34(2), 153–160. https://doi.org/10.1080/17404622.2019.1653486 . fleming, l. (2013). expanding learning opportunities with transmedia practices: inanimate alice as an exemplar. journal of media literacy education, 5(2), 370–377. https://digitalcommons.uri.edu/cgi/viewcontent.cgi?article=1124&context=jmle garcía-marín, d., aparici, r. (2018). new sound communication. cartography, grammar and transmedia narrative of podcasting. el profesional de la informacion, 27, 964–974. https://doi.org/10.3145/epi.2018.sep.11 garcia-roca, a. (2020). virtually digital reading: the collective challenge of textual interpretation. cinta de moebio, 67, 65–74. https://doi.org/10.4067/s0717-554x2020000100065 girón garcía, c. & navarro ferrando, i. (2014). digital literacy and metaphorical models. multidisciplinary journal for education, social and technological sciences, 1(2), 160–180. https://doi.org/10.4995/muse.2014.2991 gonzalez-diez, l., puebla-martinez, b., & perez-cuadrado p. (2018). de la maquetación a la narrativa transmedia: una revisión del concepto de 'diseño de la información periodística. palabra clave 21, 445–468. https://doi.org/10.5294/pacla.2018.21.2.8 haamer, k. (2020). transmedia music. perspectives on the future of music. http://krishaamer.com/transmedia-music/ havrilova, l., ishutina, о., zamorotska, v., & kassim, d. (2019). distance learning courses in developing future music teachers’ instrumental performance competence. cloud technologies in education. proceedings of the 6th workshop cte 2018, kryvyi rih, ukraine. eur-ws.org, online, 429–442. http://ceur-ws.org/vol-2433/ javanshir, r., carroll, b. & millard, d. (2020). structural patterns for transmedia storytelling. plos one, 15(1). https://doi.org/10.1371/journal.pone.0225910 jenkins, h. (2003). transmedia storytelling. mit technology review. https://www.technologyreview.com/s/401760/transmedia-storytelling/ martinez, r. a., atienza, l. d., & zamora, e. g. (2018). creation of transmedia contents in the hyperconnected society. revista fuentes, 20(1), 29–41. https://doi.org/10.12795/revistafuentes.2018.v20.i1.02 mateos-rusillo, s. m. & gifreu-castells, a. (2018). transmedia storytelling and its natural application in museums. the case of the bosch project at the museo nacional del prado. curator, 61, 301–313. https://doi.org/10.1111/cura.12252 mccarthy, e., tiu, m., & li, l. (2018). learning math with curious george and the odd squad: transmedia in the classroom. technology, knowledge and learning, 23, 223–246. https://doi.org/10.1007/s10758-018-9361-4 more than meets the eye: special effects and the fantastic transmedia franchise by bob rehak. new york university press, 2018. moya, e. (2020). transmedia and new politics. isegoría. revista de filosofía moral y política, 62, 55–74. https://doi.org/10.3989/isegoria.2020.062.03 https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://mwf2014.museumsandtheweb.com/paper/transmedia-storytelling-and-cultural-heritage-interpretation-the-culte-project/ http://mwf2014.museumsandtheweb.com/paper/transmedia-storytelling-and-cultural-heritage-interpretation-the-culte-project/ https://doi.org/10.4324/9780429282775-3 https://doi.org/10.1080/17404622.2019.1653486 https://digitalcommons.uri.edu/cgi/viewcontent.cgi?article=1124&context=jmle https://doi.org/10.3145/epi.2018.sep.11 https://doi.org/10.4067/s0717-554x2020000100065 https://doi.org/10.4995/muse.2014.2991 https://doi.org/10.5294/pacla.2018.21.2.8 http://krishaamer.com/transmedia-music/ http://ceur-ws.org/vol-2433/ https://doi.org/10.1371/journal.pone.0225910 https://www.technologyreview.com/s/401760/transmedia-storytelling/ https://doi.org/10.12795/revistafuentes.2018.v20.i1.02 https://doi.org/10.1111/cura.12252 https://doi.org/10.1007/s10758-018-9361-4 https://doi.org/10.3989/isegoria.2020.062.03 multidisciplinary journal for education http://polipapers.upv.es/index.php/muse/ social and technological sciences e-issn: 2341-2593 havrilova et al. (2021) mult. j. edu. soc & tec. sci. (2021), 8(1), 58-84. https://doi.org/10.4995/muse.2021.14875 84 moya, e., & moya, j. (2018). prosum, swarming and transmedia. towards a new concept of stakeholder. revista icono 14-revista cientifica de comunicacion y tecnologias, 16(2), 25–50. https://doi.org/10.7195/ri14.v16i2.1213 myatin, е. (2006). podcasting – synthesis of the internet and radio. https://www.ixbt.com/td/podcasting.shtml perez-manzano, a. & almela-baeza, j. (2018). gamification and transmedia for scientific promotion and for encouraging scientific careers in adolescents. comunicar, 55, 93–103. https://doi.org/10.3916/c552018-09 ryan, m.-l. (2020). transmedia storytelling and its discourses. transmediations: communication across media borders / ed. salmose, n. & ellestrom, l. routledge, oxford, england. pp. 17–30. https://doi.org/10.4324/9780429282775-2 scolari, a. c. (2016). alfabetismo transmedia. estrategias de aprendizaje informal y competencias mediáticas en la nueva ecología de la comunicación, 103, 12–23. scolari, a. c. (ed.). (2018a). teens, media and collaborative cultures: exploiting teens’ transmedia skills in the classroom. barcelona: transmedia literacy. scolari, a., masanet, m.-j., guerrero-pico, m., & establés, m.-j. (2018). transmedia literacy in then we media ecology: teens’ transmedia skills and informal learning strategies. el profesional de la información, 27(4), 801–812. https://doi.org/10.3145/epi.2018.jul.09 scolari, c. a. (2018b). transmedia branding: brands, narrative worlds, and the mcwhopper peace agreement. semiotika, 224, 1–17. https://doi.org/10.1515/sem-2016-0216 shim, a., yecies, b., ren, x. & wang, d. (2020). cultural intermediation and the basis of trust among webtoon and webnovel communities. information communication & society, 23(6), 833–848. https://doi.org/10.1080/1369118x.2020.1751865 sime, j.-a., themelis, c. (2020). educators’ perspectives on transmedia identity management: redefining tele-teacher presence. distance education, 41(1), 70–85. https://doi.org/10.1080/01587919.2020.1727292 sixto-garcia, j., melo, s. d. & guilhermina, a. (2020). self-destructive content in university teaching: new challenge in the digital competence of educators. communication & society-spain, 33(3), 187– 199. https://doi.org/10.15581/003.33.3.187-199 sotelo, x. “critical thinking competence, problem solving skills and social responsibility through transmedia stories of social intervention” in edulearn18 conference proceedings 10th annual international conference on education and new learning technologies. (pp. 8033–8040). tormakhova, a. m. (2018). transfeminizm ta yoho vplyv na vizualni praktyky transmedia [transfeminism and its influence on visual practice of transmedia]. ukrainska kultura: mynule, suchasne, shliakhy rozvytku, 29, 131–135. vyhovska, n. a. (2014). transmediynyy rasskaz / multimediynaya istoriya kak novye sposoby dostavki kontenta sovremennoy zhurnalistiki: teoreticheskie aspekty [transmedia storytelling / multimedia story as a new means of delivering the content of modern journalism: theoretical aspects]. scientific lettersof academic societyof michal baludansky, 1 (1), 168–172. what are podcasts, how are they listened to and why are the useful? (2019). https://blog.comfy.ua/chto-takoepodkasty-kak-ikh-slushat-i-pochemu-ehto-polezno/ wood, а. (2020). more than meets the eye: special effects and the fantastic transmedia franchise by bob rehak (review). jcms: journal of cinema and media studies, 59(2), 192–196. https://doi.org/10.1353/cj.2020.0016 yeates, r. (2020). serial fiction podcasting and participatory culture: fan influence and representation in the adventure zone. european journal of cultural studies, 23(2), 223–243. https://doi.org/10.1177/1367549418786420 zhenchenko m. (2019). tsyfrovi transformatsii vydavnychoi haluzi [digital transformation of the publishing industry]. kyiv: zhnets. zhurba, m. (2018). the transmedia universe of digitized culture. national academy of managerial staff of culture and arts herald, 3. https://doi.org/10.32461/2226-3209.3.2018.147088 https://doi.org/10.4995/muse.2021.14875 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.7195/ri14.v16i2.1213 https://www.ixbt.com/td/podcasting.shtml https://doi.org/10.3916/c55-2018-09 https://doi.org/10.3916/c55-2018-09 https://doi.org/10.4324/9780429282775-2 https://doi.org/10.3145/epi.2018.jul.09 https://doi.org/10.1515/sem-2016-0216 https://doi.org/10.1080/1369118x.2020.1751865 https://doi.org/10.1080/01587919.2020.1727292 https://doi.org/10.15581/003.33.3.187-199 https://blog.comfy.ua/chto-takoe-podkasty-kak-ikh-slushat-i-pochemu-ehto-polezno/ https://blog.comfy.ua/chto-takoe-podkasty-kak-ikh-slushat-i-pochemu-ehto-polezno/ https://doi.org/10.1353/cj.2020.0016 https://doi.org/10.1177/1367549418786420 https://doi.org/10.32461/2226-3209.3.2018.147088 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 1 undergraduate research efl students’ experiences in building eportfolios sami mohammed alanazia , elias bensalema* department of languages and translation, northern border university, arar, saudi arabia * correspondence: bensalemelias@gmail.com received: 15 february 2022; accepted: 26 august 2022; published: october 2022 abstract high-impact practices (hips) have been adopted by many universities around the world to enhance students learning. the aim of this qualitative study was to analyze how building eportfolios may impact students learning experience via weekly reflections. a total of 45 senior undergraduate english as a foreign language (efl) students from a public university in saudi arabia enrolled in graduation research projects shared reflections about their research experiences via the development of eportfolios. data analysis was carried out through deductive coding of the transcribed focus groups and weekly reflections. findings suggest that the participants used their eportfolios to improve their writing skills and increase their knowledge. students were able to track their progress and felt more engaged in learning. they also showed appreciation to the feedback provided by their instructors on their reflections. however, the students shared some challenges they faced while developing their eportfolios. despite these challenges, there was a positive impact of eportfolios on students’ learning experiences. implications and further directions for research are discussed. keywords: eportfolios; english as a foreign language; high impact educational practices; learning experience; undergraduate research to cite this article: alanazi, s.m.; bensalem, e. (2022). undergraduate research efl students’ experiences in building eportfolios. multidisciplinary journal for education, social and technological sciences, 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0003-0912-6980 https://orcid.org/0000-0002-6018-0897 https://doi.org/10.4995/muse.2022.17167 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 2 1. introduction the use of electronic portfolios (eportfolios) is one of the high-impact practices (hips) in higher education (watson et al., 2016). eportfolios positively impact learning (kuh, 2016; mueller & bair, 2018). as a relatively recent innovation, eportfolios are currently employed by colleges and universities around the world (kuh, 2017; watty & mckay, 2016) to assess and document students learning. collection of data from eportfolios also aids with quality assurance and enhancement pursuits (wong et al., 2017). eportfolios lack a specific definition in the literature, probably because of their “surprising variety of uses”. definitions seem to vary among academics depending on how eportfolios are used and the educational setting in which they are used (batson, 2010). nonetheless, some have attempted to provide the needed clarity, such as klenowski (2000) who described eportfolios as “a digital collection of diverse evidence of an individual’s achievements over time involving selection, design, and reflection for a particular purpose and presentation to one or more audiences” (p. 236). characteristics of eportfolios, when embedded in the curriculum, are that they are a repository of students’ work that will be subsequently evaluated. eportfolios help students to be engaged in reflective practice (andrade & ziegner, 2021; song, 2021; weber & myrick, 2018), receive timely feedback (egan et al., 2018; hager, 2013; light et al., 2012), own their learning (jenson & treuer, 2014), and enhance personal growth (buyarski, 2014). eportfolios can be used as a tool to track students’ learning while they are involved in other hips, such as faculty-mentored undergraduate research and study abroad (hubert et al., 2015). however, studies that examine the impact of eportfolios on undergraduate research are lacking (webber & myrick, 2018), particularly in the second-language acquisition field and more precisely in the english as a foreign language (efl) context. this pilot study which was carried out at public university in saudi arabia aims to bridge the gap by examining how using eportfolios promote students’ reflections and how these reflections, in turn, may affect their learning experience as they conduct their graduation research projects. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 3 2. literature review 2.1. theoretical framework the use of eportfolios as a learning tool is grounded in constructivism. it is aligned with social and active learning (thibodeaux et al., 2017). as an approach to learning, constructivism holds that “people actively construct or make their own knowledge and that reality is determined by the experiences of the learner” (elliott et al., 2000, p. 256). vygotsky (1978) asserted that community plays a key role in the process of "making meaning". in the same vein, schrader (2015) argues that active engagement aids learners in shaping their thinking and offers ample learning opportunities. based on the principles of constructivism, integrating eportfolios in the learning environment provides students with opportunities for engagement (pitts & lehner-quam, 2019). eportfolios should be linked to reflective practices so that students can be engaged in deep learning, (pitts & lenher-quam, 2019). dewey (1938) asserts that “we do not learn from experiences; we learn from reflecting on our experiences” (p.78). therefore, reflection becomes a valuable tool (ramirez et al, 2015). reflection and education go hand-in-hand in that reflection refers to “the process of turning experience into learning” (boud, 2001, p. 10). thus, engaging in reflective practice helps students to take ownership of their learning and become self-directed and lifelong learners (wong et al, 2017). engaging in the reflective process may result in increased self-awareness, efficacy, and better communication skills (eynon & gambino, 2017). as an hip, eportfolios are an effective means for boosting reflection (landis et al., 2015). the potential role played by eportfolios in enhancing students’ learning justifies the adoption of eportfolios as a learning tool (song, 2021). 1.3. previous research on eportfolios for self-reflection the increasing use of eportfolios as a tool in promoting reflection has not only been the result of technological advancements (payne et al., 2020) but has also been part of the shift of focus to a learner-centered rather than the instructor-centered approach (wickersham & chambers, 2006). sufficient empirical evidence propelled the recognition of eportfolios as “the eleventh high impact practice” (hip) by the association of american colleges and universities (aac&u, 2013; kuh, 2017; watson et al., 2016) because of the value they bring to the learning environment. the concept https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 4 of hips started to draw the attention of educators with the seminal work of kuh (2008), who identified a number of hips believed to enhance deep learning and maximize student success. kuh (2008) contended that hips help students to develop their critical thinking as well as new skills and knowledge. the implementation of hips, which involves active learning, also results in increased retention and graduation rates among college students (aac&u, 2013). subsequent work by kahn and scott (2013) identified eportfolios as an additional hip. when paired with the other hips, such as the project capstone and service learning, deeper and more effective learning occur. in their attempt to summarize the uses of eportfolios, yang et al. (2016) referred to them as “a form of authentic assessment with formative functions that include showcasing and sharing artifacts” as well as self-reflection and other forms of engaged learning (p. 1276). eportfolios offer instructors a way to evaluate students. proof of students’ learning is no longer determined by grades only (terheggen et al., 2000). instructors can evaluate students’ artifacts and engagement to determine the extent of their learning using eportfolios (mcallister, 2015; wickersham & chambers, 2006; yang et al., 2015). they provide students with opportunities to showcase their achievements (cambridge, 2010) and foster their career-development readiness (barrett, 2007; chen & penny light, 2010; fallowfield et al., 2019; hallam & creagh, 2010; penny light et al., 2012). further, eportfolios are particularly important for institutions seeking professional accreditation because they provide transparent demonstrations of students’ learning. in fact, student artifacts can be used as evidence of academic achievement (aac&u, 2013). eportfolios facilitate learning through self-reflection (payne, et al., 2020). eportfolios provide opportunities for students to respond to prompts that guide the learner through different stages of high-level thinking. reflective prompts can facilitate learning about coursework, various experiences, and skills through reflection incorporated into the eportfolios. the use of reflection encourages students to make sense of what they are learning (franco vázquez & gillanders, 2014; nicholl & higgins, 2004) and transfer that knowledge to new contexts. eportfolios help students identify and communicate what they know, reinforcing the higher-level critical thinking skills (landis et al., 2015) that students should possess and improve professionalism (nicholl & higgins, 2004), a desirable attribute of graduates. in addition, engaging in reflection boosts students’ confidence and results in deep learning (penny light et al., 2012; reynolds & patton, 2014). because of their value to integrative and lifelong learning, some scholars advocate adoption of eportfolios https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 5 throughout higher education (wong et al., 2017), and an increasing number of colleges and universities around the world are implementing eportfolios (watty & mckay, 2016). most studies about eportfolios have been conducted at higher education institutions in the united states (e.g., cleveland, 2018; fallowfield et al.,2019; payne et al., 2021; thibodeaux et al., 2020). evidence in the literature is replete with indications of positive outcomes from using eportfolios as a reflection tool to promote deep learning. in a study involving pre-service teachers, denton (2012) instructed participants to create eportfolios and write entries related to their program, such as course content, assignments, and field experiences. results suggest that the participants’ writing entries fostered reflection leading to deeper learning. similar results were reported by cleveland (2018), who implemented eportfolios in a counselor education program to help prepare graduates for the job market. cleveland also reported fostering students’ reflective writing. participants demonstrated consistently higher reflection mean scores. the assessment was rubric focused and was based on faculty observations. a recent pilot study conducted by webber and myrick (2018) assessed the impact of using eportfolios on student learning through reflective practice. eleven undergraduates from diverse majors and different years of study (sophomore, junior, and senior) participated in a summer research program. the participants were instructed to reflect and write weekly about their research experiences by developing their eportfolios. the researchers analyzed the transcripts from the focus groups and the interviewees’ responses to questions pertaining to eportfolios to identify themes. results showed that eportfolios enhanced the students’ learning experiences because they communicated their engagement in their research projects. students reported a considerable increase in their knowledge and acquired skills. they appreciated their ability to track their achievements, which was a source of motivation. the researchers emphasized the importance of giving students opportunities for self-reflection to guide their educational path. promoting self-reflection through eportfolios allows faculty and staff to analyze the learning outcomes and experiences of their students. other educational institutions around the world, especially in europe and asia, have started to adopt eportfolios as reported in recent studies (alexiou & paraskeva, 2020; ciesielkiewicz et al., 2019; gonzález-mujico, 2020; poole et al, 2018). wong et al. (2017) reported on a pilot study using https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 6 eportfolios to engage students in reflective practice on their learning experiences at a public higher education institution in hong kong. the researchers found that eportfolios were successfully implemented to improve the learning experience of participants who were engaged in research projects. students had the opportunity to showcase their projects at the university library and at a conference to obtain feedback from their audiences. the researchers argued that, to use eportfolios effectively, learners needed to understand their applicability and utility. implementing eportfolios is not free of challenges. building eportfolios can be a daunting task for students who may not be proficient with the technical aspects of various platforms used in creating eportfolios (fallowfield et al., 2019). students’ main concerns were the time involved in developing the eportfolios and their ability to create a website that is “aesthetically pleasing” (weber & myrick, 2018). mapundu and musara (2019) reported that students felt it took excessive time to build their eportfolios. there were also privacy concerns, as observed by cheng (2008). he found that more than 90% of the students who were interviewed indicated that they were reluctant to share their own work with their classmates. a number of higher achieving students argued that sharing their work with their peers might lead to plagiarism, while some of the low-achieving peers were reluctant to share their teachers’ feedback and portfolio assessment results. another challenge is that improper eportfolio implementation causes students to react negatively. in their study, payne et al. (2021) reported that more than one-third of their students felt that building eportfolios wasted their time, and less than half complained that developing eportfolios did not establish any link between courses. payne et al. (2021) argued that the results might be attributable to the failure of instructors to explain to students the benefits of using eportfolios. for teachers, challenges include the need to spend additional time to learn how to effectively use an electronic system and dealing with student plagiarism (cheng, 2008). keeping track of students’ reflections can be time-consuming (ciesielkiewicz et al., 2019). although a number of scholars examined the advantages of implementing eportfolios, studies that explore the impact of implementation eportfolios on students’ learning experience while conducting undergraduate research are still lacking (webber & myrick, 2018). furthermore, no single study on eportfolios was carried out in the under researched context of the arabian gulf region. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 7 3. rationale this study aims to contribute to the line of research on the impact of using eportfolios on students’ undergraduate research experience through reflective practice. specifically, this qualitative study aims to examine how might the adoption and implementation of eportfolios foster efl students’ reflection and impact their undergraduate research experience. 4. method 4.1. context of the study this research was conducted at a public university in saudi arabia. the educational system in the kingdom is similar to the one in the united states. students can earn a bachelor’s degree when a student successfully completes a minimum of 120 semester units. students typically graduate within four years. the university’s strategic plan for 2020–2025 called for the implementation of hips following the recommendation of george kuh and other scholars from the aac&u who argued that hips, including undergraduate research and eportfolios, promote deep learning and student success (aac&u, 2013). eportfolios have been proved to play a vital role in enhancing student engagement (barrett, 2007). some scholars argue that eportfolios help develop real-world job skills such as digital competence and editing skills which are enhanced when students compile and manage content for eportfolios. other real-world job skills include reflection, communication, organization and visualization (laurikainen & kunnari, 2018). hence the need for this pilot study to demonstrate how using eportfolios may help provide evidence of student learning. this can be achieved by engaging students in reflecting on their learning experiences as they are conducting their graduation research projects using eportfolios. the current study reports the implantation of the eportfolios for the first time at northern border university and probably in the whole kingdom. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 8 4.2. participants in this study, a total of 45 efl university students were recruited (38 females, 7 males). the sample was selected from senior graduating english-major students who were enrolled in the graduation project in the department of languages and translation at northern border university in saudi arabia. the sample comprised 38 females and 7 males aged between 22 and 24 (m = 22.70, sd = .82). this gender distribution was almost identical to the overall gender distribution of all students in the program, so it was determined that this sample was representative from the perspective of gender. the participants’ gpas ranged between 3.22 and 4.90 out of 5.0 (m = 4.05, sd = .62). all students are required to complete a capstone graduation project during their final semester as a graduation requirement. the institutional review board at the northern border university approved this study. all participants provided signed informed consent. because the educational system is gender segregated with females being educated at dedicated females’ campuses, a female collaborator worked closely with the female participants in coordination with the rest of the research team. two members of the research team worked with the male students. no differences in patterns of interaction were witnessed between male and female students. the research team created a whatsapp group to communicate with the participants. researchers sent regular reminders to students to submit their weekly entries and reflections. the whatsapp group was the communication platform used by the students to post inquiries and to collaborate, especially regarding creating and posting their eportfolios online. none of the participants had prior knowledge about eportfolios and their use. 4.3. procedures due to the covid-19 pandemic the interaction between the research team and students was online. during the first week of classes, members of the research team introduced the project of eportfolios and briefed the participants about its purpose and the requirements for a successful implementation of eportfolios. one of the researchers provided students with a tutorial on how to build an individual eportfolio and showed samples of complete eportfolios. then participants received guidelines regarding the weekly reflective journal entries. students were reassured that their reflections will be kept confidential and that they will not be graded. however, participants were https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 9 encouraged to submit well-thought reflections. the emphasis was on content rather than on form. in other words, students were told that the instructors will not focus on grammatical issues but rather on coherence. students’ journal entries were posted online and were considered part of the eportfolio. most students had no previous experience with reflective learning. 4.4. eportfolio structure the researchers chose google sites to host students’ eportfolios because it is easy to create an eportfolio site. student eportfolios were made up of three web pages: about me, graduation project, and eportfolio reflections. the about me page included a brief autobiography of the students. graduation project served as a page where students posted different parts of their projects (title, introduction, research questions, literature review, method, discussion, conclusion, and references). in addition, the students’ powerpoint presentations were posted and were available for downloading/viewing. finally, the “eportfolio reflections” page comprised students’ individual weekly reflections based on prompts. site viewers could browse the list of reflections and read each reflection by clicking on individual post titles. students received training on how to develop their eportfolios at the beginning of the semester. 4.5. focus groups following webber and myrick (2018), the 45 eportfolio students were sent an invitation to participate in focus group interviews. students who decided to join the discussion joined the focus group meeting on blackboard. a total of three focus group discussions were carried out by the researchers and one collaborator. an average number of 16 students participated in each focus group which lasted around 50 minutes. the focus group discussion questions were all in english. however, students were told that if they could not express a specific idea because they could not find the word in english they could say it in arabic. questions pertaining to students’ experience in building their eportfolios and how they benefited from using eportfolios guided the discussion. specifically, the researchers asked participants about the benefits they may have gained, their motivation and https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 10 engagement while building their eportfolios, and any challenges they encountered. sample focus group questions are included in the appendix. 4.6. students’ weekly reflections in their eportfolios, students reflected on their learning as they were engaged in their research project. in these reflections, student teams were asked to reflect on what worked and what did not work in their projects, what they needed to do differently to address the challenges they faced, how they had responded to feedback, and so forth. the reflections were designed to aid students to examine and improve their work. students were encouraged to be frank and honest so that their reflective journals to capture both positive and negative lived experiences. the reflections were written entirely in english. 4.7. research design the focus groups were audio recorded and transcribed. transcripts from the focus groups were analyzed for common themes derived from students’ responses to questions related to eportfolios using open coding as laid out in the grounded theory approach (strauss & corbin, 1990). then additional content analysis was carried out to ensure that the common themes were consistent with the content generated from the students’ reflections about eportfolios. the researchers used triangulation to ensure an accurate interpretation of the collected data (creswell & guetterman, 2019). 5. results and discussion five dominant themes emerged from the analysis of the participants’ reflections and the focus groups: improved skills and gained knowledge, tracked progress, increased engagement, valuable feedback, and challenges. table 1 reports the number and percentages of coded responses per theme. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 11 table 1. number and percentages of coded responses per theme. themes number of coded responses percentage of coded responses improved skills and gained knowledge 120 28.92% tracked progress 81 19.52% increased engagement 92 22.17% valuable feedback 73 17.59% challenges 49 11.81% 5.1. improved skills and gained knowledge this recurring theme emerged with the biggest number of coded responses. students reflected on how building eportfolios helped them not only to acquire a new set of skills regarding creating and posting materials online, but also to enhance their communication and writing skills. writing reflections as part of an eportfolio proved to be a useful tool to improve students’ writing skills. the quality the eportfolios provides evidence of students’ acquisition of digital skills. in addition, the content of reflections demonstrates students’ steady improvement of their writing skills. in fact, instructors noticed that reflections were well-thought and written in better english. one of the students stated: “this is probably the biggest number of writing task that i have done since my first year. i think that the quality of my writing improved a lot. in addition, i started more attention to details since i did a lot of editing. the same opinion was shared by a number of students during the focus group discussions. one student observed: “writing reflections was an opportunity to practice writing in english. even though my instructor mentioned that journal entries should be well thought, i made a lot of efforts to submit reflections that are mistake free. i had to write multiple drafts and treated each reflection as a final exam. writing is very difficult for me because i make a lot of spelling and grammatical mistakes”. (student #7, focus group 1). another student shared the same sentiment: “i have to confess that i did not like the idea of writing a weekly reflection. i always struggled with writing. it is my weakest skill. i was scared of getting a bad grade because of the quality of my writing. however, i noticed an improvement in my writing skills. i gained more confidence in my ability to express my ideas in writing. i learned from my instructor’s corrections. i appreciate the fact that my https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 12 instructor took the time to highlight the areas that needed improvement. i even felt that my speaking skills have improved as a result of learning new words. this is the best preparation for the job market as i’m graduating in a few weeks and i need to be fluent in english in order to get a job”. (student #18, focus group 3). other students stressed how the knowledge gained while working on their eportfolios bolstered their confidence: “i’m not a good student even though i try to work hard. i’m always afraid of making mistakes and of my classmates laughing at me. as i worked on my eportfolio and compared it with the eportfolios of other students i felt that i could be a good student too. my instructor once told me that my reflections were getting better and that my project was good”. (student #22, focus group 3). 5.2. tracked progress the participants valued the opportunity to keep track of the progress they made and the amount of knowledge they acquired through weekly reflections. the eportfolios helped students regulate their own time and focus on the tasks they need to complete. one student said: “i feel that i’m more organized. because i know that i have to write about the progress that i made each week then i had to accomplish at least one task to show that i’m a hard worker” (student #7, focus group 1). another student noted: “there is a self-imposed stress that pushes me to work on my project and update my eportfolio. i hate to see myself lagging behind while my peers are making progress. without completing any task i can’t submit a decent reflection, which will result in losing points”. (student #3, focus group 2). these comments document the role played by eportfolios in promoting self-regulated learning because students were able to monitor their own knowledge, acquired skills, and overall progress (andrade & ziegner, 2021) throughout the semester. in this regard, parker et al. (2012) argued that eportfolios help students monitor their work more efficiently than supervised learning. eportfolios promote student-centered learning where learners can track their own progress and contribute to lifelong learning (mason et al, 2004; meyer et al., 2010; yusuf & tuisawau, 2011). https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 13 5.3. increased engagement the use of eportfolios increased students’ engagement and boosted their self-regulated learning. previous studies (e.g., rogers & swan, 2004) have documented that self-regulated learners are active participants in the learning process and are mostly like to achieve academic success. one student wrote in her reflective journal entry: “i spent a lot of time checking my eportfolio and making sure that it is good enough for an a. it became a habit to log in daily and polish my work even though i still have time.” another student argued that “building an eportfolio to display my work was a cool idea. i always wanted to have my own web space that includes the “fruits of my school work”. i had to spend long hours learning how to present the different parts of my graduation project in a nice way. i had to seek the help of my classmates but i managed to complete all the requirements. because i knew that my instructor and peers will be checking my eportfolio i had to make sure it was perfect”. (student #10, focus group 1) this proves that eportfolios aided students in evaluating their success and deciding on ways to improve their work. another student emphasized her learning curve: “i have to admit that i’m not a very techy person. being able to build an eportfolio from scratch and most importantly posting my graduation project in a neat way is a huge achievement. i feel that this interesting experience helped me gain more knowledge than during my call course. i gained confidence in my ability to build an eportfolio because of the amount of time i dedicated for improving my project and finish the assigned tasks as scheduled so that i could get an a. i had a slow start because i was overwhelmed by the number of tasks but i was able to set specific goals that i needed to achieve. i was not sure i could do that at the beginning of the semester”. (student #11, focus group 3) similar experiences were reported by mapundu and musara (2019), who found that students regarded eportfolios to be fun and interesting, and therefore they were engaged in learning. student engagement through the implementation of eportfolios can help overcome academic shortcomings (meyer et al., 2010). https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 14 5.4. valuable feedback the participants enthusiastically appreciated the feedback they received from their instructors because it helped them to consolidate their learning and increase their confidence with knowledge and skills. they thought constant feedback made them feel that they had the support they needed in the process. formative feedback encouraged reflection and revision of the students’ work. this view is shared by many students. for example, one student asserted in her reflective journal entry: “i appreciate the feedback from my teacher. her comments were precise and very clear. they guided me towards correcting my mistakes”. another participant mentioned that “the constant appropriate and relevant feedback that i received from my instructor made me feel that i’m not alone in this journey. knowing that my instructor will read and react to my reflection which includes my need for help (student #2, focus group 1). the same opinion was echoed by another student who indicated that “the comments offered by my instructor regarding my work helped identify the areas that require improvements” (student #4, focus group 1). another comment that further highlights the importance of feedback provided by instructors was the following: “at first i felt that building an eportfolio and sharing weekly reflections are additional unnecessary tasks that were time-consuming. then i realized that without the constant feedback i received from my instructor i would not be able to finish my project on time”. (student #3, focus group 2) the value of feedback corroborates the findings of previous studies (e.g., mapundu & musara, 2019; mcdermott & gallagher, 2011; o’keeffe & donnelly, 2013; webber & myrick, 2018). 5.5. challenges students reported experiencing a number of challenges through their reflection journal entries and during focus group discussions. this outcome was not surprising given the fact that students had no prior experience in building eportfolios. furthermore, implementation of new tools does not go smoothly. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 15 a number of students complained about the amount of time needed to build the eportfolio. in one reflection piece, a student suggested reducing the number of reflections: “i think i’m running of out of ideas. i shared everything about my experience. i think that five reflections should be enough”. the same sentiment was shared by another student: “having to write a weekly reflection is a huge burden. i think that writing a biweekly reflection would have been more reasonable. understand the benefits of reflection but i have other assignments from other courses.” (student #17, focus group 1) another student indicated that developing an eportfolio was time consuming and that caused her a lot of stress: “during my mid-term exams i had no time to write a reflection note because i’m a slow writer who needs a lot of time to write an acceptable journal entry. i wish that our reflections were not worth points.” (student #11, focus group 2). other students thought that having to update their eportfolio each week was too demanding because it involved logging to google sites and make the necessary changes. one student pointed out: “having an eportfolio is cool but sometimes i don’t feel like i don’t have enough new content to post on the site. when i’m busy with other courses i tend to delay my work on the graduation project. i need to take care of other courses”. (student #23, focus group 1) the same concerns with regard to time consumption were voiced by students in other contexts, as reported by mapundu and musara (2019) who found that students in south africa complained about the excessive amount of time needed to build eportfolios. similar views were expressed by some american students who were not willing to put the time and effort into developing an eportfolio (webber & myrick, 2018). instructors were prompt in helping their students overcome their challenges. some students needed more time. other students needed encouragement and confidence in their abilities to complete the assigned task. one student stated in a reflective journal entry that her teacher was very accommodating: “i appreciate the fact that i got more time to finish up my tasks since i was busy with mid-term exams. i did not think it was smart to spend a lot of time writing a journal entry and lose focus on what matters most. i know that writing such journal entries help me practice my writing more but i need to maintain my high gpa”. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 16 another recognition to the support provided by the teachers comes from a male student who reflected in his journal entry on the role played by his teacher in helping him overcome some of the challenges he faced: “without the help and support from dr. b. i would not be able to finish the weekly tasks. he understands my fear and anxiety. i’m someone who panics fast and hate deadlines. dr. b was very lenient and always believed in me and my classmates. his comments on the class whatsapp group were very encouraging. i felt bad that i complained a lot”. 6. implications the findings from this study suggest that the implementation of eportfolios can help students to acquire knowledge and a set of skills that result in enhancing their learning experience. therefore, the decision of northern border university to adopt the implementation of eportfolios along with other hips can boost the academic performance of students and equip them with the necessary skills that prepare them for the job market. 7. conclusion and potential future research this study provided evidence that building eportfolios experience provided students with opportunities to review their work in progress, make improvements, and recognize their achievements. in addition, the implementation of eportfolios increased students’ engagement and honed their writing skills. the act of reflecting aided them to see what they had accomplished and what areas they needed to address for improvement. students’ overall positive responses during focus group discussions and their weekly reflections suggest the fact that eportfolios may enhance students’ learning experience. despite the positive outcomes, the current study has a number of limitations. the sample included only english major students from one single institution. a future study might explore two cohorts—one group majoring in english and another majoring in a scientific field such as computer science. in addition, the study involved participants who worked on individual projects. setting up group projects could have given the researchers the opportunity to measure collaboration and peer interaction, which are important elements that enhance students’ learning experience. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 17 students in group projects may experience a sense of community when they interact with each other through a forum or group discussions (bowman et al., 2016; hadley, 2007). such a setup could have added depth to the study analysis. an additional limitation was that the study occurred at a single newly established university. it would be interesting to learn whether the researchers would obtain similar results if the participants were students from a top-tier university where motivation to conduct research might be higher. funding: this research was funded by the deputyship for research & innovation, ministry of education in saudi arabia for funding this research work through project number "if_2020_1661". conflicts of interest: the authors declare no conflict of interest.” acknowledgements: the authors extend their appreciation to the deputyship for research & innovation, ministry of education in saudi arabia for funding this research work through project number "if_2020_1661". special thanks for professor gary musa for his valuable feedback and comments. author contributions: authors contributed equally. references alexiou, a., & paraskeva, f. (2010). enhancing self-regulated learning skills through the implementation of an e-portfolio tool. procedia – social and behavioral sciences, 2(2), 3048-3054. https://doi.org/10.1016/j.sbspro.2010.03.463 andrade, m., & zeigner, s. (2021). team eportfolios in management education: insights into students’ skill development. e-journal of and scholarship of teaching, 15(1), 40-54. association of american colleges & universities (aac&u). (2013). high impact practices. retrieved from https://www.aacu.org/resources/high-impactpractices barrett, h. c. (2007). researching electronic portfolios and learner engagement: the reflect initiative. journal of adolescent & adult literacy, 50(6), 436-449. https://doi.org/10.1598/jaal.50.6.2 batson, t. (2010). eportfolios: let me count the ways. campus technology, 1, 1–3. boud, d. (2001). using journal writing to enhance reflective practice. new directions for adult and continuing education, 9-18. https://doi.org/10.1002/ace.16 bowman, j., lowe, b. j., sabourin, k., & sweet, c. s. (2016). the use of eportfolios to support metacognitive practice in a first-year writing program. international journal of eportfolio, 6(1), 1-22. buyarski, c. (2014). reflection in the first year: a foundation for identity and meaning making. https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://www.sciencedirect.com/science/article/pii/s1877042810005033 https://www.sciencedirect.com/science/article/pii/s1877042810005033 https://www.aacu.org/resources/high-impactpractices multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 18 retrieved from http://iupui.mcnrc.org/ref-practice/ cambridge, d. (2010). eportfolios for lifelong learning and assessment. jossey-bass. chen, h., & penny light, t. (2010). electronic portfolios and student success: effectiveness, efficiency, and learning. washington, d.c.: association of american colleges and universities. cheng, g. (2008). implementation challenges of the english language eportfolio system from various stakeholder perspectives. journal of educational technology systems, 37, 118 97. https://doi.org/10.2190/et.37.1.h ciesielkiewicz, m., wisser, w., & rozells, d. (2019). international perspectives on eportfolios in higher education: case studies from asia, north america, and europe. international journal of emerging technologies in learning, 14(21), 96–109. https://doi.org/10.3991/ijet.v14i21.11048 cleveland, r. e. (2018). using digital portfolios: reflection, assessment & employment. techtrends, 62(3), 276– 285. https://doi.org/10.1007/s11528-018-0262-0 creswell, j.w., & guetterman, t. c. (2019). educational research: planning, conducting, and evaluating quantitative and qualitative research (6th ed.). pearson. denton, d. d. (2012). improving the quality of evidence-based writing entries in electronic portfolios. international journal of eportfolio, 2(2), 187-197. dewey, j. (1938) experience and education. collier books. egan, j. p., cooper-ioelu, p, spence, f. & peterson, m. l. (2018). the curricular and technological nexus: findings from a study of eportfolio implementation. international journal of eportfolio, 8(2), 127-138. https://doi.org/10.21125/inted.2017.0333 elliott, s.n., kratochwill, t.r., littlefield cook, j. & travers, j. (2000). educational psychology: effective teaching, effective learning (3rd ed.). mcgraw-hill college. eynon, b., & gambino, l. (2017). high-impact eportfolio practice: a catalyst for student, faculty, and institutional learning. stylus. fallowfield, s. m., urtel, m., swinford, r., angermeier, l., & plopper, a. s. (2019). a case study in eportfolio implementation: a department-wide perspective. international journal of eportfolio, 9(2), 111–118. franco vázquez, c., & gillanders, c. (2014). exploring innovative and creative ways of teaching. multidisciplinary journal for education, social and technological sciences, 1(2), 53. doi:10.4995/muse.2014.2228 gonzález-mujico, f. (2020). the impact of eportfolio implementation on motivation, selfregulation and academic language development: the learners’ and the teachers’ perspectives. the asian esp journal, 16(3), 209–242. hadley, n. j. (2007). the portfolio forum: power in reflection. journal of educational technology https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://iupui.mcnrc.org/ref-practice/ https://doi.org/10.2190/et.37.1.h https://doi.org/10.3991/ijet.v14i21.11048 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 19 systems, 35(4), 449-455. https://doi.org/10.2190/c521-u100-535u-7626 hallam, g., & creagh, t. (2010). eportfolio use by university students in australia: a review of the australian eportfolio project. higher education research & development, 29(2), 179–193. https://doi.org/10.1080/07294360903510582 hubert, d., pickavance, j., & hyberger, a. (2015). reflective e-portfolios: one hip to rule them all? peer review, 17(4), 15–18. jenson, j. d., & treuer, p. (2014). defining the e-portfolio: what it is and why it matters. change, 46(2), 50-57. https://doi.org/10.1080/00091383.2014.897192 kahn, s., & scott, s. (2013). scaling up eportfolios at a complex urban research university: the iupui story. retrieved from http://iupui.mcnrc.org/scalingstory/#catalyst-and-connector klenowski, v. (2000). portfolios: promoting teaching. assessment in education: principles, policy & practice, 7(2), 215-236. https://doi.org/10.1080/713613329 kuh, g. d. (2017). foreword: and now there are 11. in b .eynon, l. m. gambino, g. d. kuh (eds.), high-impact eportfolio practice: a catalyst for student, faculty, and institutional learning (1st ed., pp. viixi). stylus. kuh, g. d. (2016). making learning meaningful: engaging students in ways that matter to them. new directions for teaching and learning, 2016(145), 49–56. https://doi.org/10.1002/tl.20174 kuh, g. d. (2008). high-impact educational practices: what they are, who has access to them, and why they matter. washington, dc: association of american colleges and universities. landis, c. m., scott, s. b., & kahn, s. (2015). examining the role of reflection in eportfolios: a case study. international journal of eportfolio, 5(2), 107-121. laurikainen, m. & kunnari. i. (2018). eportfolios as a way to empower students and bridge them to future work places. journal of finnish universities of applied sciences, eapril 2018 edition. https://uasjournal.fi/in-english/eportfolios-way-to-empower-students/ mapundu, m., & musara, m. (2019). e-portfolios as a tool to enhance student learning experience and entrepreneurial skills. south african journal of higher education, 33(6), 191-214. https://doi.org/10.20853/33-6-2990 mason, r., pegler, c., & weller, m. (2004). e-portfolios: an assessment tool for online courses. british journal of educational technology, 35(6), 717– 727. https://doi.org/10.1111/j.1467-8535.2004.00429.x mcallister, l. (2015). an eportfolio approach: supporting critical reflection for pedagogic innovation. in m. e. ryan (ed.), teaching reflective learning in higher education (pp. 173– 187). springer international publishing. https://doi.org/10.1007/9783-319-09271-3 mcdermott, k. & s. gallagher. (2011). integration of e-portfolios into cooperative education: https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1111/j.1467-8535.2004.00429.x multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 20 lessons learnt. asia pacific journal of cooperative education 12(2): 95‒101. meyer, e., abrami, p. c., wade, c. a., aslan, o., & deault, l. (2010). improving literacy and metacognition with electronic portfolios: teaching and learning with epearl. computers & education, 55(1), 84–91. https://doi.org/10.1016/j.compedu.2009.12.005 mueller, r. a., & bair, h. (2018). deconstructing the notion of eportfolio as a ‘high impact practice’: a self-study and comparative analysis. the canadian journal for the scholarship of teaching and learning, 9(3). https://doi.org/10.5206/cjsotlrcacea.2018.3.6 nicholl, h., & higgins, a. (2004). reflection in preregistration nursing curricula. journal of advanced nursing, 46, 578-585. https://doi.org/10.1111/j.1365-2648.2004.03048.x o’keeffe, m. & donnelly, r. (2013). exploration of e-portfolios for adding value and deepening learning in contemporary higher education. international journal of eportfolio 3(1): 1‒ 11. parker, m., a. ndoye and a. d. ritzhaupt. (2012). qualitative analysis of scholar perceptions of eportfolios in a teacher education program. journal of digital learning in teacher education 28(3): 99‒107. https://doi.org/10.1080/21532974.2012.10784687 payne, b. k., paredes, t., cross, b. (2020). student perceptions about the production of electronic portfolios: technology, process, and showcase insights. education, 141(2), 67-78. penny light, t., chen, h. l., & ittelson, j. c. (2012). documenting learning with eportfolios: a guide for college instructors. jossey-bass. pitts, w., & lehner-quam, a. (2019). engaging the framework for information literacy for higher education as a lens for assessment in an eportfolio social pedagogy ecosystem for science teacher education. international journal of eportfolio, 9(1), 29-44. poole, p., brown, m., mcnamara, g., o’hara, j., o’brien, s, & burns, d. (2018). challenges and supports towards the integration of eportfolios in education: lessons to be learned from ireland. heliyon, 4(11), 1-23. https://doi.org/10.1016/j.heliyon.2018.e00899 ramirez, c., lorenzo, j., jiménez, a. c., & rivilla-garcía, j. (2015). the effect of the reutilization of an open educational experience on the improvement of the reflexive process of education learning in university students. multidisciplinary journal for education, social and technological sciences, 2(1), 141. https://doi.org/10.4995/muse.2015.2253 reynolds, c., & patton, j. (2014). leveraging the eportfolio for integrative learning. stylus. rogers, r. r. (2001). reflection in higher education: a concept analysis. innovative higher education,26(1), 37-57. https://doi.org/10.1023/a:1010986404527 rogers, d., & swan, k. (2004). self-regulated learning and internet searching. teachers college record: the voice of scholarship in education, 106, 1804-1824. https://doi.org/10.1177/016146810410600907 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 21 schrader, d.e. (2015), constructivism and learning in the age of social media: changing minds and learning communities. new directions for teaching and learning, 2335. https://doi.org/10.1002/tl.20160 song, b. k. (2021). e-portfolio implementation: examining learners’ perception of usefulness, self-directed learning process and value of learning. australasian journal of educational technology, 37(1), 68–81. https://doi.org/10.14742/ajet.6126 strauss, a., & corbin, j. m. (1990). basics of qualitative research: grounded theory procedures and techniques. sage. terheggen, s. l., prabhu, r. p., & lubinescu, e. s. (2000). from product to process: enhancing learning through the use of student electronic portfolios. u. s. department of education, educational resources information center (eric). he033685. retrieved from https://eric.ed.gov/?id=ed448674 thibodeaux, t. n., harapnuik, d. k., & cummings, c. d. (2020). graduate students’ perceptions of feedback for learning in an online program. international journal of eportfolio, 10(1), 19-32 thibodeaux, t. n., harapnuik, d. k, & cummings, c. d. (2017). factors that contribute to eportfolio persistence. international journal of eportfolio, 7(1), 1-12. vygotsky, l. s. (1978). mind in society: the development of higher psychological processes. harvard university press. watty, k., & mckay, j. (2016). pedagogy and eportfolios: purpose aligned to design (or the why and how). international journal of pedagogies and learning, 3(10), 194-207. https://doi.org/10.1080/22040552.2015.1135498. watson, c. e., kuh, g. d., rhodes, t., light, t. p., & chen, h. l. (2016). eportfolios: the eleventh high impact practice. international journal, 6(2), 65-69. weber, k. & myrick, k. (2018). reflecting on reflecting: summer undergraduate research students’ experiences in developing electronic portfolios, a meta-high impact practice. international journal of eportfolio, 8(1). 13-25. wickersham, l. e., & chambers, s. m. (2006). eportfolios: using technology to enhance and assess student learning. education, 126(4), 738746. wong, e.y. w., kwong, t. f. n., & lau, p. f. m. (2017). using student eportfolios to showcase students’ learning: experience from hong kong baptist university. in t. chaudhuri & b. cabau (eds.), e-portfolios in higher education: a multidisciplinary approach (pp. 157-170). springer nature. https://doi.org/10.1007/978-981-10-3803-7_11 yang, j. l., coleman, k., das, m., & hawkins, n. (2015). integrated career development learning and eportfolios: improving student self-efficacy in employability skills in an undergraduate science course. international journal of adult, community, and professional learning, 22(1), 1–17. yang, m., tai, m., & lim, c. p. (2016). the role of e-portfolios in supporting productive https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://eric.ed.gov/?id=ed448674 http://ouleft.org/wp-content/uploads/vygotsky-mind-in-society.pdf multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17167 social and technological sciences e-issn: 2341-2593 alanazi & bensalem (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 1-22. https://doi.org/10.4995/muse.2022.17167 22 learning. british journal of educational technology, 47(6), 1276–1286. https://doi.org/10.1111/bjet.12316 yusuf, j. & tuisawau, p. (2011). student attitudes towards the use of e-portfolios: experiences from the university of the south pacific. malaysian journal of educational technology, 11(4), 31-41. appendix sample focus group questions 1. how would you describe your experience in building your eportfolio? 2. what have you learned from using your eportfolio? 3. what are the benefits/advantages of building an eportfolio? 4. what are the challenges of building an eportfolio? 5. was there anything you gained from uploading your project to the website that you would not have learned otherwise? 6. were the reflective journal entries valuable? https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022.17167 https://doi.org/10.4995/muse.2022. http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 1 higher education quality assessment in ethiopia: a comparative study l. geleto wariyo* department of curriculum and instruction. addis ababa university, college of education, weatherall st, addis ababa, ethiopia. * corresponding author: email: lemechageleto@yahoo.com; phone: + 251913695748 received: 17 july 2019; accepted: 05 february 2020; published: april 2020 abstract the major objective of this study was analyzing the ethiopian higher education quality assessment model in line with another world. the total 46 key informants were purposively selected from the data sources of this study (ethiopian public heis and the ministry of education) and interviewed. document analysis was another instrument. using descriptive qualitative research design, data were analyzed in themes qualitatively. literatures describing the higher education quality assessment models were reviewed. the literature on the models of higher education quality assessment generally tends to converge to the general model of higher education quality assessment and tends to diverge from it while it adds many approaches to the dimensions. it is recommended that the ethiopian system better to be governed by an independent agency that has strong international linkage, and the system should emphasize the need of stakeholders in quality assurance and assessment. use of diversified methodologies and existence of explicit standards for resource utilization were recommended. rigorous interdisciplinary and cross-disciplinary peer reviews are strongly recommended in this study. keywords quality assessment; quality assurance; quality control https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 2 1. introduction 1.1. background of the study universities over the world face challenges to meet the increasing number of students, providing life-long learning for larger parts of the population, and of dealing with growing student heterogeneity. for this reason, higher education institutions (heis) are required to provide and maintain quality for higher education learning circumstances based on a standard high education quality criteria (noaman, ragab, madbouly, khedra and fayoumi, 2015). according to sarrico, et. al. (2010) heis live today in a society where their once privileged situation and the financial commitment to their academic values can no longer be unquestioned. sarrico, rosa, teixeira, and cardoso (2010) stated this situation has been called ‘the erosion of trust’, meaning that the public’s trust in higher education is being lost, which can finally put heis at risk. the main importance of maintaining the quality and privilege of a hei is also seen in its centrality for economic, political, and social development; its importance to competitiveness in a continuously globalizing knowledge society; and its vitality as an instrument for technological catch-up (el-khawas, elaine, 1998; materu, 2007). believing this, ethiopia has been expanding its heis, struggling with the question of quality in line with quantity, however. in sub-saharan africa, fast growth in enrollments amid declining budgets during the 1980s and 1990s, the expansion of private provision of higher education and pressure from a rapidly changing labor market have combined to raise new concerns about quality. as a result, african countries, including ethiopia, became more conscious of the need for quality improvement (materu, 2007). as a result of these changes and increased need for accountability, higher education is facing the challenge of re-conceptualizing methods and procedures used to show quality and excellence, including those used for assessing and evaluating the quality of education programmes (noaman, ragab, madbouly, khedra and fayoumi, 2015). according to materu (2007), in africa, per unit costs amidst rapidly https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 3 rising enrollments; brain drain, retirements, and hiv/aids; low internal and external efficiency; and poor governance contributed for the decline of higher education quality. higher education in ethiopia has experienced numerous challenges throughout its short history, such as its inability to produce sufficient manpower to meet the country’s needs, unresponsiveness of the programmes and the curriculum to the practical needs of the country, the extremely conservative orientation of the institutions, a lack of genuine commitment to academic freedom and institutional autonomy, and scarcity of resources. in addition, maintaining student retention and building academic competencies thereby assuring educational quality has a great pitfall since the universities operate with overcrowded and deteriorating physical facilities, limited and obsolete library resources, insufficient equipment and instructional materials, poorly prepared secondary students, and an absence of academic rigor among students. as a result, the education and training programs provided have not adequately contributed to the alleviation of poverty and other related social problems of ethiopia (teshome, 2008; daniel, 2010; tesfaye, 2011). higher education proclamation number 351 (ethiopian federal ministry of education, 2003: 21) made provision for the creation of the higher education relevance and quality agency (herqa) with the aim of enhancing the quality and relevance of higher education in the country (kebede, 2014). since its establishment, herqa has developed quality assurance systems and introduced the systems for achieving quality education provision. tesfaye (2011) reported that heis assure the quality of educational programs they offer through three mechanisms: internal selfassessment, external review based on the self-assessment and monitoring and follow-up. 1.2. statement of the problem according to materu (2007), though some attempts to record the developments have been made by different individuals in africa, no comprehensive mapping and analysis of https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 4 quality assurance systems in the region has yet been undertaken. despite the advances in quality assessment system in ethiopian heis, still, there is a shortage of diverse research findings that use a variety of models to assess the quality of higher education. studies conducted on assessing the quality of education largely emphasized the description of existing problems and factors that impact the quality of education. for instance, kebede’s (2014) study described the internal and external quality assurance mechanisms in sample universities. the other study, kahsay’s (2012) study, described the general practices of quality assurance in ethiopian heis as related to student learning and achievement. misgana’s (2013) study also assessed the implementation of the quality assurance guidelines of the herqa in public universities in ethiopia. this study analyzes the major internal and external quality assessment mechanisms in ethiopian heis in order to firstly, understand the general trends, similarities, differences, weaknesses, and strengths existing in line with the systems and models of quality assessment discussed in the literature. the gap in the previous studies on quality of ethiopian heis is that they did not bring the examination of different higher education quality assessment models in the world while assessing and investigating the quality assessment practices in ethiopian heis. a review of different higher education quality assessment models in the world helps to identify the gaps that exist in quality assurance practices both nationally and internationally. the rationale behind doing this is to observe how different models view and approach the multifaceted concept of quality in heis. the researcher believes that the advantage will be coming up with understanding multiple ways of coping with quality problems in heis. 1.3. objectives of the study the objectives of this study are to: 1) discuss how the internal and external higher education quality assessments are conducted in ethiopian heis, and 2) analyze the https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 5 internal and external quality assessment mechanisms in ethiopian heis in line with the general model of quality assessment and quality assessment systems in higher education. 1.4. the research questions the major research questions raised in this study are: 1) how are the internal and external quality assessments conducted in ethiopian heis? 2) how are the elements of the general higher education quality assessment processes being handled in ethiopia? 3) what are the similarities and differences between higher education quality assessment systems in ethiopia and another world? 1.5. significance of the study this study is believed to give inputs for higher education quality assessment of the country by firstly, bringing different views of higher education quality assessment perspectives and their approach to define and deal with quality issues thereby helping to observe gaps, strengths, and weaknesses. by comparing the ethiopian higher education quality assessment system with general quality assessment, this study may show how the ethiopian higher education quality assessment system deviates from an international standard. this study informs the importance of being conscious of the international higher education quality standards and models and utilizing and keeping those standards in order to maintain the real higher education quality; taking into consideration the local and contextual situations to fit one's own conditions and mechanisms. it is believed that the issue of quality higher education is the issue of globalization that binds and networks the quality assurance agencies to bring about the comparable quality higher education in the world. https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 6 1.6. delimitation of the study the study is delimited to the ethiopian public heis excluding private heis because of the time and financial constraints. the variables are quality assurance variables (internal and external quality assurance mechanisms) that are described by university quality assurance officers and herqa’s officers; as well as those mechanisms discussed by different quality assessment systems and the conceptions of the term quality in literature. these variables were revealed through qualitative inquiry and document analysis. 1.7. theoretical framework this study used the general model of quality assessment as a framework and used standards of higher education quality assessment listed by higher education quality assessment networks in the world. van vught & westerheijden (1994) discussed the common general elements of the general model of higher education quality assessment: a) the managing agency. they stated that the agency should be free from external influences such as government politics and policies, and not having a mission to impose upon the institutions and should own the sole responsibility to manage the quality assessment system; b) self-evaluation. the academics should be able to accept and put the changes into practice; they must acknowledge the ownership of the process in which problems are defined and solutions are designed; c) the third element of the general quality assessment is peer review, a site visits by external experts; d) reporting of the results of the quality assessment along with the methods applied. these authors asserted that reports should not have the purpose of judging or ranking the heis. rather, their objective should be to help heis to enhance their quality. however, approaches in different countries vary in this element. in usa and canada, the reports are often kept confidential. in france, institutional self-evaluations are kept confidential while the report by external experts is public, and e) the relationship between the outcomes of a quality review and the governmental decisions about the funding of heis. the authors argued that a direct and rigid relationship between quality assessment reports and funding https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 7 decisions damage the operation of the quality assessment system. in general, the main point the authors emphasized is that there should be sound bases for heis to keep their important place in community that the model should offer. the perception of quality assurance is very multidimensional, contextual and a gap exists in the view of professionals in quality assurance, academic staff, and students (ryan, 2015). based on a thorough literature review, lagrosen, hashemi, and leitner (2004) have classified the definitions of quality into five major groups: (1) transcendent definitions, subjective and personal definitions, e.g., beauty and love. (2) product-based definitions; viewed it as a measurable variable; (3) user-based definitions; a means for customer satisfaction. (4) manufacturing-based definitions; conformance to requirements and specifications. (5) value-based definitions; viewed in relation to costs. in ethiopian context, quality is defined by (moe/herqa) as the totality of the university's effectiveness in its core processes and functions to satisfy stakeholder's needs, priorities, and requirements (fitness for purpose); to fulfill requirements of relevance in transforming learners, and to be responsive for accountability purposes. 2. literature review 2.1. quality assurance methodologies defined throughout quality assurance and assessment systems, different methodologies are used even though the methodologies used in various quality assurance reviews vary considerably, most quality reviews depend on one or a combination of a limited number of key methodologies. self-evaluation is the study of institutional activities and practices by members of the respective institution (kebede, 2014). craft (2005) defined it as seeing one-self using external support by quality assurance agents; through self-training or self-evaluation staff https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 8 development; and using both local and global information regarding performance indicators, descriptions of best practices and new developments in teaching, learning, and assessment. accreditation, according to (kebede, 2014), is the process by which a government or private agent assesses the quality of an institution as a whole or a program in order to formally recognize it as having met certain pre-determined minimum standards or thresholds. craft (2005) asserted that accreditation assures the academics, the community, and other agents that an institution/programme (1) has clearly stated and educationally relevant objectives, (b) indicates situations under which their achievement can be expected, (c) achieving them substantially, and (d) can be feasible. quality audit, according to moe/herqa (2006), is a process of review of the university’s core process by herqa or other agency to check that quality and relevance of the programs, curricula, staff infrastructure, and other elements meet the stated objectives and aims of the university and to determine the level of the university’s system of quality care and accountability. craft (2005) defined quality audit as an assessment by a group external to a university to verify that the quality assurance and quality control processes are appropriate and working properly. peer review generally involves a visit by a group of well-regarded academics in a particular field to undertake an assessment (kebede, 2014). craft (2005) defined peer review as an involvement of people such as active university teachers, researchers, and practicing professionals to offer advice and to make judgments and decisions about proposals for new programmes, the continuation and modification of existing programmes, the quality of research programmes or the quality of institutions. in defining quality assurance, different definitions are used by different countries. moe/herqa (2006) defined quality assurance as planned, systematic, structured, https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 9 continuous review and evaluation of all programs, courses, instructional materials, teaching, learning, assessment at the university with the purpose of maintaining and improving these through continuous quality care efforts; and confirming the conditions are in place for students to achieve the standards set by the university and relevant national bodies. world bank (2013) stated that ‘quality assurance’ refers to planned and systematic processes that provide confidence in educational services provided by training providers under the remit of relevant authorities. quality control is defined as a system by which an enterprise checks whether the raw materials it uses, the product it makes, or the service it provides reach minimum predefined (threshold) standards so that the sub-standard can be rejected. mostly, this is done on a sampling basis by a group of controllers or inspectors, who are independent of the main workforce, and who have powers to reject sub-standard products or services. quality control is not sufficed. the overall quality of a university must be the concern of everyone who works there. this leads us to quality assurance (craft, 2005). van vught & westerheijden (1994) defined quality assessment as external reviews of and judgments about the quality of teaching and learning in institutions. moe/herqa (2006) also defined quality assessment as a periodic review and evaluation of programs, courses, instructional materials, teaching-learning, and outcomes based on the bpr (business process re-engineering) requirements. 2.2. historical roots of quality assessment in heis: the medieval higher education van vught & westerheijden (1994) discussed the roots of quality assessment as it is summarized as follows: https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 10 french model. the authors stated that, in medieval france, universities were seen as ecclesiastical colonies. for this reason, the delegate of the bishop of paris, a chancellor above the masters' guild, had the power to grant or withhold the teaching license and to decide contents of the study. this model is considered to be the archetype of quality assessment in terms accountability. english model of self-governance. the authors also stated that, in medieval england, the masters were independent of external jurisdiction. these fellows had the right to judge the quality of their colleagues. the masters decided what to teach and how to teach. today’s expression of what we call peer review was applied. the authors asserted that these two systems can be considered to be important dimensions of any present-day system of quality assessment in higher education. these two systems refer to the two general concepts of intrinsic and extrinsic qualities. intrinsic qualities refer to the ideals of the search for truth and the pursuit of knowledge while extrinsic qualities refer to the services higher education gives to outside community. by combining both intrinsic and extrinsic qualities, heis have been able to show a remarkable historical persistence. 2.3. the recent developments in higher education quality assessment according to van vught & westerheijden (1994), since the early 1980’s, quality assessment in higher education has become a central concept in usa and canada, uk, france, netherlands, denmark, finland, spain and other european countries. they stated that the factors that explained this recent increase of the attention for quality in higher education are: 1) expansion of higher education systems, a rapid increase of student body, and fields of study and whole new institutions triggered questions about the amount and direction of public expenditure for higher education. 2) the public https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 11 expenditures reached in the countries and the budget cuts and retrenchments 3) shifting process to technology-based economies leading students to the fields important for economic development. the authors stated that in previous decades especially extrinsic values of higher education have driven countries to policies of quality control in higher education. as a result, new systems and mechanisms of quality assessment control have been being developed in several countries. experiences in usa and canada. in the usa, the system is market-oriented and competition between heis was generally accepted. government control was limited compared with continental europe. the diversity in institutional forms and initial lack of centrally defined standards in the 19thc created controversies in the us higher education. as a result, the institutions took the initiative to develop two processes of quality assessment: accreditation and the intra-institutional process of systematic review of study programmes (van vught & westerheijden, 1994). developments in western europe. the authors also stated that except britain, in many countries in western europe, heis were state-controlled and government funded. however, during 1970's and 1980's, the western european higher education systems encountered far-reaching changes. due to shifts in governmental strategies, the value for money approach was emphasized. as a result, funding linked to the performance of heis and quality of higher education became one of the central issues. the second development in higher education policymaking in western europe according to the authors was the rise of the government strategy of ‘self-regulation', resulting in increased autonomy and competitiveness among heis. from these developments described above, new attempts to set up quality assessment systems arose in some western european countries. https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 12 france. the authors stated that in france, the traditional, centralized system of quality control ended and the comite national d’evaluation (cne) established. the cne has two parts: institution-wide evaluation and horizontal disciplinary reviews. the united kingdom. the authors stated that since the first half of the 1960’s nonuniversity he quality was evaluated by council of national academic awards (cnaa) and controlled by her majesty’s inspectorate (hmi). cnaa is government initiated, and independent which evaluated and validated polytechnic courses. it was ceased to exist in 1992. in mid-1980's, reynold's report laid down criteria for internal quality management systems for universities while the jarratt report announced the discussion of performance indicators and their role in quality-based learning. in the years 1990-1991, the academic audit unit (aau) was introduced by the organization committee of vice-chancellors and principals (cvcp) to counter the hmi. after its introduction, cvcp consisted of external examiners. aau evaluated the quality of institutions' evaluation methods. according to the authors, changes following the 1991 white paper brought about changes in organizational structures and have led to the introduction of the specific meanings to the following terms in britain context: 1) quality control mechanisms within institutions for maintaining and enhancing the quality of their provision; 2) quality audit external scrutiny aimed at providing guarantees that institutions have suitable quality control mechanisms in place, and 3) quality assessment external reviews of and judgments about the quality of teaching and learning in institutions (responsibility of funding councils). https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 13 3. methodology 3.1. study design the study design used in this study was descriptive qualitative research design. the interview responses were interpreted and qualitatively discussed. all public universities in ethiopia were the population of the study. the sample design used in this study was non-probability sampling in which the key informants were purposively selected and interviewed. the total 46 key participants from herqa and heis: 6 respondents from herqa and the other 40 respondents from the six universities those purposively selected for the study. 8 participants from each university were participated in the study. 3.2. data collection tools the semi-structured interviews and document analysis were the major tools used for data collection. the 11 interview questionnaires were developed by the researcher and administered to the key informants. the questionnaires were organized in themes and the responses were collected and organized in these themes. during the interview, the responses of the participants were recorded by writing in the notebook. regarding the document analysis, herqa documents were downloaded from their website (http://www.higher.edu.et) and the other university documents were downloaded from their websites. the respondents show and gave the available documents at their office and the researcher observed and recorded the needed data. 4. data presentation, discussion and interpretation 4.1. quality assessment in ethiopian heis until 2003, the issue of quality was missing both in the ministry of education (moe) and heis (abebaw and aster, 2012). it is after 2003 that higher education quality issue has got due attention by the ethiopian government. supporting this, abebaw and aster (2012) https://doi.org/10.4995/muse.2020.10512 http://www.higher.edu.et/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 14 stated that the government of ethiopia gave a due acknowledgment for higher education quality in 2003 by ratifying the higher education proclamation and establishing the agency in charge of higher education relevance and quality. herqa was established through the hep (351/2003, article 78) as an autonomous legal body to supervise the relevance and quality of higher education offered by heis. in addition, following this establishment of herqa, a couple of pilot external quality assessments were conducted in one private college and one governmental university in the 2005 academic year. later on, a large-scale quality audit was conducted in the relatively older nine public universities. as well, the hep number 650/2009 has given directions to the higher education sector in the country by formulating improved policy and mandating structural changes (federal democratic republic of ethiopia 2009, as cited in tefera, 2014). internal quality assessment. according to the interview with the herqa expert and the sample universities’ quality assurance officers, the major internal quality assessment mechanism used is self-assessment. the officers stated that the selfassessment is done in two forms. the first form of self-assessment is done in the form of institutional continuous self-assessments of the functions of the institutions, such as timely teacher evaluations, yearly staff evaluations done at the department level, faculty level and university level. in this type of self-evaluation, the staff evaluates each other in aspects of the teaching-learning process, management, community service and security of the campus. the second form of self-assessment is the overall self-assessment process aimed at assessing the institution’s strength, weakness, opportunities, and threats. this selfassessment is guided by the ten focal areas of the quality assessment in heis. as it is stated in herqa (qa05/06/v1, 2006) self-evaluation takes the following procedures: a) establishing a team for the self-evaluation; b) compiling and communicating a timetable for the self-evaluation; c) gathering and analyzing information for self-evaluation; d) reporting the self-evaluation, and e) making use of the self-evaluation. https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 15 external quality assessment. generally, ethiopia uses two external quality assessment mechanisms in heis: quality audit and institutional/program accreditation. there is a tendency towards “accreditation of program and institution” in the private and “institutional audit” in both public and private heis ( herqa experts; heis quality assurance officers; and abebaw and aster, 2012). quality audit. a herqa institutional quality audit assesses the appropriateness and effectiveness of heis’s systems of accountability and quality assurance and of its internal review mechanisms (herqa qa05/06/v1, 2006). craft (1992) also stated that external quality assessment is never an end in itself, but rather an extension of internal quality control. abebayehu said that until now the auditing is done for first, second and third generation universities and as evidence, he showed the researcher the published materials of quality audits of the universities. in addition, addis ababa university quality assurance officer also told the researcher that quality auditing had been accomplished and the reports have been published after discussing with the concerned officials of the university. he said that depending on the recommendations, the action plan is prepared and weaknesses are improved. addis ababa university science and technology university and wachemo university quality assurance directorates also said that they have already finished the self-evaluation process as they had been instructed by herqa and they were going to submit the self-evaluation document to herqa. as it is stated in herqa qa05/06/v1 (2006) and the interview with the expert at herqa, institutional quality audit proceeds through the following steps: a) hei carries out an institutional selfevaluation and prepares a self evaluation document. b) hei sends herqa its self evaluation document and informs herqa of their wish to have an institutional quality audit; c) herqa and hei agree a date for the institutional quality audit; d) herqa establishes an external institutional quality audit team in consultation with the hei; e) herqa institutional quality auditors make a one-day briefing visit to the hei; f) herqa institutional quality audit team makes a four-day institutional quality audit visit https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 16 to the hei; g) herqa issues a quality audit report; and h) hei prepares an action plan to enhance quality and relevance. according to the interview response, herqa institutional quality audit covers the following ten focus areas: 1) vision, mission, and educational goals; 2) governance and management system; 3) infrastructure and learning resources; 4) academic and support staff; 5) student admission and support services; 6) program relevance and curriculum; 7) teaching, learning, and assessment; 8) student progression and graduate outcomes; 9) research and outreach activities, and 10) internal quality assurance. accreditation. according to the herqa expert, herqa is authorized to accredit the private heis and it is also recently given a mandate of accrediting public distance programmes. regarding accreditation in ethiopian public heis, the type of accreditation used is internal accreditation. internal accreditation is defined by abebaw and aster (2012) as the establishment or of the status, legitimacy or appropriateness of a study program where departments and faculties seek the establishment or legitimacy of a new study program from the responsible body of the institution. since the study programs are not accredited by herqa, it is the mandate of the heis to internally accredit their study programmes with joint consultation from external study program experts that come to make a programme review. generally, the internal accreditation follows this procedure: firstly, the department level curriculum committee writes a draft study program in line with the legislation and the higher education proclamation. after that, an application that consists of a brief overview of the program is prepared by the department. secondly, the application is submitted to the faculty of the department for evaluation. if comments come from the faculty, the department examines the comments and resends them to the faculty. if the faculty understands the program significantly relevant, it transfers the document and its comments to the institution’s academic program office. thirdly, after discussing with the faculty and the department, the academic program office invites other institutions for https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 17 external review. the academic program office arranges a conference where external reviewers present and discuss the result review results. next, the academic program office sends feedbacks given by the external reviewer to the department for reconsideration. finally, if the academic program office finds the suggestions given by the external reviewers are properly defended by the department, it presents the documents to academic standards and curriculum review committee of the senate for the final decision. in ethiopia, internal accreditation is mandatory for starting of every new degree program. on the other hand, programs can be adopted from other institutions and can be started without passing through "internal accreditation" process and can equally function with the internally accredited ones (abebaw and aster, 2012). peer reviews. according to the interview with the respondents, external peer reviewers come from different organizations for the purpose of sharing experiences and evaluating the programmes. in addition to universities, they come from ngos such as sida, unesco, world bank, etc. these peers share their experience in their special areas they find in the universities being and cooperating with the specialists in the programmes. this is facilitated by the quality assurance office of the university. 4.2. comparison of the sets of standards of quality assurance of herqa with world quality assurance networks’ standards a bulk of literature in the world reveals that agencies in the world have their own sets of standards of both internal and external quality assurance. these standards help them as a guideline in all functioning of quality assurance, assessment, and control. these sets of standards are published and publicly known so that every stakeholder knows its constituents. therefore, in this part, the ethiopian quality assurance sets of both internal and external quality assurance standards were compared depending on the comparative study of aelterman (2006). this author compared these quality assurance networks https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 18 depending on the specific standards they listed. these networks are european association for quality assurance in higher education (enqa, 2005), european consortium for accreditation in higher education (eca, 2004), international network for quality assurance agencies in higher education (inqaahe, 2005), asia-pacific quality network (apqn, 2006), the oecd–unesco guidelines for quality provision in cross-border higher education (oecd, 2005) and the member code of good practice of the association of specialized and professional accreditors (aspa, 1995). the comparison was made under the following themes and this study also used these themes to analyze herqa’s quality assurance standards: mission and statement. when the ethiopian quality assurance document is compared with these quality assurance network standards, we observe that the mission statement is stated in the quality assurance document (i.e. in the focal areas and in the policy documents). the relationship between agency and heis. regarding this, the relationship between agency and heis in the process of quality assurance is stated in the policy documents as a democratic relationship in which the institutions can take the responsibility of assessing and managing their own quality assurance system and non-domination of the external evaluators is encouraged and stated. therefore, the relationship in practice, according to the respondents from heis, is not somewhat the agency dominating the heis. however, some respondents stated that herqa instructs the heis to do things hurriedly during auditing within short time resulting in shallow assessment results. decision making. it is explicitly discussed in the quality audit report document that, after the quality audit is accomplished depending on the self-evaluation of the institutions, the final checking and discussions with the heis are done before publishing the report. however, regarding the share of decisions in the quality assessment outcome, decision making between government (moe) and herqa, an explicit guideline is not written as https://doi.org/10.4995/muse.2020.10512 http://www.enqa.eu/ http://www.enqa.eu/ http://www.ecaconsortium.net/ http://www.ecaconsortium.net/ http://www.inqaahe.org/ http://www.inqaahe.org/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 19 far as my knowledge is concerned. some of the respondents said that the decision making of herqa is largely affected by moe. public eye. it is explicitly stated in the quality audit report and quality assurance policy documents that the report of the outcomes of quality assessments is done to the public using different medias. the quality audit report is published and the documents are sent to the heis, moe, house of people’s representatives and other concerned offices. documentation. it is stated in the qa policy document (no. 6) that there is appropriate archiving used and the intranet system is used to appropriately document the quality assessment processes. resources. no standard is set externally although the resource utilization of a university is assumed to be assessed as it is indicated in the focal area document (no.3). explicit standards for the resource should be there at the policy level, external quality assurance agency (herqa) and heis level. appeal. although the process of appealing comments by the stakeholders is not discussed in detail, some general statement is written in the equal opportunity document of herqa. there should be explicit and detailed procedures for appealing the comments in order to cultivate the culture of ownership of the heis in the stakeholders. the voices of the stakeholders should be heard and their problems should be solved democratically if the real quality is to be expected. external review of the agency. the process in which the herqa conducts the quality audit of heis is explicitly discussed in the quality audit process document of herqa and preparations and things may be made by heis before and during the external review is clearly stated in the documents. https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 20 collaboration with other agencies. the quality policy document of herqa (no. 4) stated that herqa makes a network with local and international quality assurance agencies, especially with inqaahe. however, according to the response from herqa's expert (ato abebayehu), there is less linkage with the external world as only one uk national is working in the herqa office by his own will. this may be one of the weaknesses of ethiopian quality assurance system. networks listed above have their own strong linkages and many agencies are included in them. in africa, this linkage seems at an infant stage as the african standards and guidelines for quality assurance in higher education (asg-qa, 2017) is currently under draft version for consultation. official status. the official statue which delineates the power of herqa as an independent agency from third parties is absent. this makes the herqa somewhat powerless as a quality assurance agent in heis. what is written in the higher education proclamation is herqa duties and responsibilities in the accreditation and quality assurance process of private heis. 4.3. analyzing ethiopian quality assessment system in line with the general model in higher education institutions managing agent of the quality assessment of quality assessment system. this dimension of the general quality assessment model deals with the independence of quality assessment system from external influences, the agent’s level of legality and accreditation and the adequacy and formality of its information of procedures and formats that can be used by the institution. according to the interview with mr. ababayehu terefe, quality audit and enhancement expert at herqa, the agency’s stated mission is ‘to ensure a high quality and relevant higher education system in ethiopia.’ its operational objectives include: a) assessing the https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 21 relevance and quality of higher education; b) ensuring that the curriculum supports the country’s development needs; c) providing an efficient and transparent accreditation system, and d) disseminating information regarding standards and programmes. to compare with this, billing (2004) summarized the surveys from 38 countries, stated that the purposes of external quality assurance appear to be variants of a mix of the same functions, which can be boiled down to a) improvement of quality, b) publicly available information on quality and standards, c) accreditation, (i.e. legitimization of certification of students), d) public accountability: for standards achieved, and for use of money, and e) to contribute to the higher education sector planning process. herqa’s expert stated that since the agency functions under the ministry of education, it is semi-autonomous. for instance, he said, the authority to accredit the public heis is not given for heqra (herqa accredits mainly private heis), although herqa is recently authorized to accredit the public distance higher education programmes. in addition, there is no documentation such as legislative acts or statutes of the organization that stipulates the independence of the agency's work from third parties such as heis, government or other stakeholder organizations. for this reason, the decisions of herqa over higher education quality issues are shared by moe. selfassessment in the quality assessment system. it is argued in the quality assessment literature in heis that in order for the academics to accept and implement changes they must trust and own the process in which problems are defined and solutions are designed (van vught, &westerheijden, 1994). regarding this, mr. abebayehu and others officers stated that there are explicit and formal procedures by which the institutions make self-evaluations. at the first place, every institution has its own quality assurance officers who are responsible for all quality assurance activities. self-assessment of the institutions is accomplished before quality https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 22 auditing conducted by the agency, and the agency gives the appropriate training on the self-evaluation procedures. then, using their own ways, the heis are given freedom to conduct self-evaluation following the written system and guidelines by the agency. however, some respondents said that the self-evaluation is done hurriedly and gives less information on the real quality of the institutions. in the senate legislation of the sample universities, the quality assurance committee duties and responsibilities were observed in this study. the duties and responsibilities of quality assurance bodies are clearly stated in the legislation of the sample universities. generally, what is listed in the legislation documents of the universities regarding the quality assurance is the general description of the processes of program review, curriculum review, course review, issues on class size, quality assurance, assessment, and auditing issues. the inclusion of different committees from different offices of the university as committee members in the quality assurance committee is an evidence that the universities gave due attention to the quality assurance of their institutions. mechanisms of peer review and site visits by external experts. the quality assurance officers said that external experts come from other universities to their universities for different purposes such as experience sharing and for program reviews. experts also come from different ngos to get information and make a review of the area they need and to give different assistance, training, etc., depending on their evaluations. however, the researcher understands from the responses and his readings that peer review methodology has still a shortage of effective guidelines and mechanisms and its application is not to the required standards in ethiopian heis. for instance, in other world universities, according to my readings, peer reviewers do a lot of jobs in quality assessment of the study programmes and others. these peer reviewers are free from any biases and influences compared with the experts from the agency and the validity of their assessment results are high. from experience of usa, israel, and uk we also learn that https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 23 peer review dominates the quality assessment process and its value is high in terms of validity and cost. peer reviewers are experts in the specific fields compared with agency officers who come to conduct the quality audit. the number of experts at herqa is small and to compensate this shortage of manpower, the use of external experts from other countries and in the country to assess quality is the best mechanism. reporting of the quality assessment results. a report should not have the function of judging or ranking the institutions or programs that have been visited. it rather should target at helping heis and study programs to improve their levels of quality. a crucial phase in the reporting process, therefore, concerns providing the opportunities to the heis to comment on the draft version of the report and to formulate counter-arguments, if necessary (van vught, & westerheijden, 1994). in this element, approaches in different countries differ. according to the interview responses from herqa expert and sample universities officers the main objectives of reporting are: 1) supporting the heis improve their mechanism of self-evaluation 2) showing the achievements and strengths to concerned and forwarding recommendations and implications as well as helping the heis to be satisfied and proud of their achievements and open the way of looking at their own weaknesses and improving it. however, some respondents said that reports are being used to rank the universities in ethiopia and the criteria used to rank the universities lack clarity and validity. for instance, tefera (2014) illustrated that the institutions are distracted from the real work of quality improvement by the emerging domestic annual ranking of universities, which is the quality assurance showcase of the ethiopian higher education system, positively deceiving institutions into thinking that they are performing well. the relationship between quality assessment results and funding. van vught, & westerheijden (1994) argued that a direct, rigid relationship between quality review reports and funding decisions should not be established because such a relationship harms the operation of the quality assessment system. the respondents also said that as a result https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 24 of the results of the quality assessment, there is no funding difference done for universities by the government. results of quality assessment are absolutely aimed at helping universities improve themselves. however, as a result of the evaluation result reports to the public, said the respondents, the universities' internal income, and privilege can negatively or positively be affected. resources. according to the respondents and herqa expert, resource constraint is the major problem in ethiopian heis quality assurance at this time. the shortage of human, material, and financial resources is the setbacks of ethiopian heis quality assurance, said the respondents. for instance, herqa expert responded that “at this time, at the agency level, there are only eight experts. when we compare this number of experts with the number of heis to be evaluated, it is mismatching. it is with this little number of human resources that we are trying to work with many private and public heis. not only is the shortage of number, but also there is a shortage of experience and qualification". there is also a shortage of international experts in the agency; only one uk citizen who is currently working in the agency in his own willingness. regarding this issue, el-khawas, elaine h. (1998), in describing the world bank’s report, stated that the report found that developing countries were particularly hard hit by the crisis in higher education. these authors also asserted that the fiscal constraints faced by many countries, coupled with increasing demand, has led to overcrowding, deteriorating infrastructure, lack of resources for non-salary expenditures, such as textbooks and laboratory equipment, and a decline in the quality of teaching and research activities. based on a review of countries’ experience, the world bank report, said the authors, suggested four key directions for reform: a) encouraging greater differentiation of institutions, b) providing incentives for heis to diversify sources of funding, c) redefining the role of government in higher education, and d) introducing policies explicitly designed to give priority to quality and equity objectives. https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 25 cyclical external review of agency. the expert at the agency said that they planned to conduct the quality auditing of the heis every five years, but they practically do not do it right at five yearsit may take longer due to the shortage of manpower and material resources. 5. discussion, conclusion and recommendation 5.1. discussion in this section using different kinds of literature the results of the analysis are discussed especially different approaches to quality assessment in different countries are compared with ethiopia’s higher education quality assessment system. in addition, the major findings are forwarded and recommendations are given. weber, mahfooz, & kate (2010) used four criteria listed below to systematically analyze a range of country quality assurance systems. they were: 1) the object and nature (formative or summative) of evaluation; 2) the relative role of heis, agencies, and governments; 3) the consequences and impact of decisions and/or recommendations, and 4) the costs of hei quality assurance systems in relation to the expected benefits. the quality assurance system in public heis of ethiopia follows formative (that is, encourage institutions to identify their own strengths and deficiencies and develop plans to address the problems and to improve their weaknesses). here, when we interpret ethiopia's system depending on the data, the role of the agency over public heis can be interpreted as a medium because the role of agency over public heis is shared by the moe. those systems which use summative approaches of quality assurance have the higher role of agencies over themselves. concerning the next criteria used to compare the heis, which is the relative role of the government over agencies, in ethiopia’s case, the role of government over heis can be interpreted as equal to agency's role because especially the final decisions of the quality evaluation results made by agencies are affected by the moe. the next criteria by which https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 26 the systems were compared are the consequences and impact of decisions and/or recommendations. in ethiopia, it can be understood that the consequences and impact of decisions and/or recommendations are medium for public heis. regarding this a quality assurance expert at herqa stated that after auditing, the strengths, weaknesses, and recommendations are reported to the universities and the time interval in which they can improve the weaknesses are made known by the universities. if the universities repeatedly fail to improve the weaknesses, they are reported to the moe and moe takes its measures. the cost of hei quality assurance systems in relation to the expected benefits is next criterion. the analysis indicated that eua system needs the lowest cost to achieve its quality assessment goals than others. the more the heis are responsible for their own quality assurance processes being free from the influences of others, the less energy they lose to achieve the expected objectives because quality assurance in higher education is a matter of making each and every member of the university responsible and owner of the university. then, everybody works to assure the quality at a lower cost. in the case of ethiopian system, according to the responses from the respondents, quality assurance issue is costly in terms of the readiness and responsibility of the university community and stakeholders to take care of the quality of higher education. regarding this, jeliazkova and westerheijden (1998) stated that the ability to relate the quality with the cost it expends is determined by the maturity level of the university; the highly qualified heis tend to assess themselves in relation to the cost they need to achieve the stated objectives. there are international similarities and differences in quality assurance models, in the following dimensions: 1) the concept of quality 2) objectives of the quality assurance system; 3) the methodologies; 4) the responsible agent; 5) type of participation, i.e., voluntary or compulsory; 6) emphasis on research or teaching, or both of them; 7) emphasis on programme reviews, disciplines, or the whole university; 8) confidentially or https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 27 publicity of reporting (with/without grading); 9) type of follow-up activities and 10) the use of or not use of quality assurance results for decisions such as funding, accreditation etc. (billing, 2004). billing (2004) added some points to the van vught & westerheijden’s (1994) general model of higher education quality assurance framework used in this study. these points were the importance of transparency of external processes, of internal quality care in the institutions, and of a follow-up process after the report. depending on several studies, billing (2004) considered that national external quality assurance frameworks were converging internationally. the features converged upon are discussed above ( i.e., the model used in this study including those points added by vroeijenstijn, 1995), plus: a) effective quality assurance processes internal to the hei; b) support of self-evaluation by standard quantitative data on effectiveness of performance; c) distinctions between the level of aggregation evaluated, which may be programme, subject, department/faculty or institution. in addition, el-khawas, elaine h. (1998), stated that despite the continuity of policy debates, some commonalities have emerged around an approach to quality assurance for higher education. this convergence indicates a broad cultural "borrowing" among countries. these common features converged were, according to these authors, are: a) semi-autonomous agents/agencies; b) explicit standards; c) self-study by the academic institution; d) external peer-reviews; e) written recommendations; f) public reporting and g) attention to process or capacity and results. 5.2. conclusion both internal and external quality assessment mechanisms are being implemented in ethiopian heis. in addition, formative and summative approaches to quality assurance https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 28 mechanisms are being used complementarily in the system. the major external quality assessment currently used to assess the quality of heis are quality audit (for both public and private heis) accreditation (especially for private heis). other quality assessment methodologies such as self-evaluation, peer reviews, and site visits are also used. the analysis of the quality assessment framework in ethiopian heis against the general quality assessment model in this study revealed the ethiopian quality assurance agency is semi-autonomous. in addition, the role of the agency on heis is a medium since this role is shared with the moe. self-assessment is a widely used quality assessment method – in this case, it conforms to the general quality assessment model. despite the fact that peer review is used in ethiopian system, this study reveals that it is not used to the standard. the literature shows that peer review is used for multi-purposes in different ways. the outcomes of quality assessment in ethiopian system are published and reported to the stakeholders using appropriate methodologies and procedures. there is no direct relationship between funding and public higher education quality assessment outcomes. however, it is inevitable that the results of the quality assessment which are published and reported to the public bring about a direct impact on the internal income and privilege of the heis. the cyclical quality review in ethiopian heis is not done as planned because of resource constraints. the major setback for quality assessment in ethiopian heis is the found to be a shortage of both human and financial resource. the weak ties and networks the agency has with the international quality assurance agencies also can bring about a lag behind in keeping the international standards because the system may be in devoid of funding, a share of knowledge and practice from international agents. in addition, the analysis of the models of higher education quality assessment reveals that the more heis are free from influences and responsible for their quality (more mature, in other words), the more effective and less costy the quality assessment system. compared with the international quality agencies and networks’ standards of quality assurance set by herqa, although https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 29 the documents have full contents of the standards, the standards are not clearly and explicitly stated in such a way that they can be used to assess the internal and external higher education quality clearly and objectively. studies confirm that even though making the internal quality assurance agents more responsible and accountable for higher education quality assurance is preferred internationally, the governments are losing trust in internal quality assurance personnel. for this reason, the attention is highly given to other stakeholders (students, employers) on one hand and the outcomes of the higher education on the other hand. 5.3. recommendation depending on the analysis of the data and review of the literature and the findings of the study, the following recommendations were forwarded: 1) this study suggests the quality assessment agent (herqa) independent from government and other third-party influences on the work of quality assessment and decisions; 2) it is recommended that peer review should be effectively used for quality assessment, for example, external experts (both national and international) can be invited to make a review; 3) this study also recommends that larger number of experts should be trained in quality assessment to supply the adequate number of human resource. these experts should also be a mix of national and international experts in order to share the experiences in the area. 4) the diversified methodologies are recommended to be used in quality assessment in heis (experience from the usa). rigorous specific programme reviews and interdisciplinary reviews that are done by internal and external experts in the field is found to be more effective and better be used; 5) the quality assurance assessment should be highly depended on satisfying the needs of stakeholders; 6) the culture of cultivating the ownership in the university community and other stakeholders for the quality issues is the effective method; 7)the process of quality assurance should base itself on outcomes of higher education and the emphasis should be given to changes o students’ learning and https://doi.org/10.4995/muse.2020.10512 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 30 changes in economy; and 8)this study also recommends that further studies using different methodologies should be conducted in the ethiopian higher education quality assessment especially in the areas of specific models of higher education quality assessment as related to outcomes of heis. 6. references aelterman, g. (2006). sets of standards for external quality assurance agencies: a comparison. quality in higher education, 12(3): 227-233. https://doi.org/10.1080/13538320601050996. african standards and guidelines for quality assurance in higher education (asg-qa), draft version for consultation (2017): african union. retrieved from https://haqaa.aau.org/wpcontent/.../06/haqaa_asgqa_draft_310517_en-1.pdf asnake, m. t. (2013). assessment of the implementation of higher education quality assurance guidelines in public universities in ethiopia (unpublished doctoral dissertation). the catholic university of eastern africa, nairobi, kenya. billing, d. (2004) international comparisons and trends in external quality assurance of higher education: commonality or diversity? higher education, 47(1): 113-137. https://doi.org/10.1023/b:high.0000009804.31230.5e. craft, a. (2005) quality assurance in higher education: proceedings of an international conference hong kong, 1991. el-khawas, and elaine, h. (1998). quality assurance in higher education: recent progress; challenges ahead. washington, dc: world bank. human development network/education. european association for quality assurance in higher education (2015). standards and guidelines for quality assurance in the european higher education area. brussels higher education relevance and quality agency herqa institutional quality audit: herqa qa05/06/v1 (2006). retrieved 27 january 2017, from http://www.higher.edu.et jeliazkova, m. and westerheijden, d. f. (2002). systemic adaptation to a changing environment: towards a next generation of quality assurance models. higher education, 44 (3/4): 433-448, http://www.jstor.org/stable/3447497 https://doi.org/10.1023/a:1019834105675 kahsay, m. n. (2012). quality and quality assurance in ethiopian higher education: critical issues and practical implications (doctoral dissertation, addis ababa university, addis ababa, ethiopia). https://doi.org/10.4995/muse.2020.10512 https://doi.org/10.1080/13538320601050996 https://doi.org/10.1023/b:high.0000009804.31230.5e http://www.higher.edu.et/ http://www.jstor.org/stable/3447497 https://doi.org/10.1023/a:1019834105675 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.10512 social and technological sciences issn: 2341-2593 wariyo (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 1-31 | 31 lagrosen, s. seyyed‐hashemi, r, & leitner, m. (2004). examination of the dimensions of quality in higher education. quality assurance in education, 12(2): 61-69. https://doi.org/10.1108/09684880410536431. materu, p. (2007). higher education quality assurance in sub-saharan africa: status, challenges, opportunities, and promising practices. washington, dc: world bank. https://doi.org/10.1596/978-0-8213-7272-2 noaman, a. y., ragab. a. h., madbouly, a. i., khedra, a. m. & fayoumi, a. g. (2015). higher education quality assessment model: towards achieving educational quality standard. studies in higher education, 42(1): 23-46. https://doi.org/10.1080/03075079.2015.1034262. ryan, t. (2015). quality assurance in higher education: a review of literature. higher learning research communications, 5(4). https://doi.org/10.18870/hlrc.v5i4.257. saketa, k. n. (2014). quality assurance practices in ethiopian public and private higher education institutions (unpublished doctoral dissertation). university of south africa, south africa. sarrico, c. s. rosa, m. j. teixeira, p. n. & cardoso, m. f. (2010) assessing quality and evaluating performance in higher education: worlds apart or complementary views? minerva, 48(1): 35-54. https://doi.org/10.1007/s11024-010-9142-2. van vught, f. a. & westerheijden, d. f. (1994). towards a general model of quality assessment in higher education. higher education, 28(3): 355-371. https://doi.org/10.1007/bf01383722 weber, l. mahfooz, s. b. hovde, k. (2010). quality assurance in higher education : a comparison of eight systems. europe and central asia knowledge brief; volume no. 35.world bank, washington, dc. © world bank. https://openknowledge.worldbank.org/handle/10986/10148 license: cc by 3.0 unported. weert, e. (1990). a macro-analysis of quality assessment in higher education. higher education, 19(1): 57-72. https://doi.org/10.1007/bf00142023. westerheijden, d. f. (2005). walking towards a moving target: quality assurance in european higher education. quality of higher education, 2: 52-69. retrieved from https://eric.ed.gov/?id=ej874242 https://doi.org/10.4995/muse.2020.10512 https://doi.org/10.1108/09684880410536431 https://doi.org/10.1596/978-0-8213-7272-2 https://doi.org/10.1080/03075079.2015.1034262 https://doi.org/10.18870/hlrc.v5i4.257 https://doi.org/10.1007/s11024-010-9142-2 https://doi.org/10.1007/bf01383722 https://doi.org/10.1007/bf00142023 https://eric.ed.gov/?id=ej874242 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 23 working on critical thinking skills using the computer lab works of an engineering subject juan giner-navarro1 , águeda sonseca1 , josé martínez-casas1 , javier carballeira1 1department of mechanical engineering and materials science, universitat politècnica de valència, 5e building, camí de vera, s/n, 46022, valencia, spain. emails: juanginer@upv.es; agsonol@upvnet.upv.es; jomarc12@mcm.upv.es; jacarmo@mcm.upv.es (corresponding author) received: 13 july 2022; accepted: 22 august 2022; published: october 2022 abstract a practical intervention was devised in order to promote and to assess critical thinking in undergraduate students of engineering. first, in collaboration with specialists in education, the concept of critical thinking, and its skills and dispositions on which to focus on, was determined. second, a teaching strategy was designed in order to be as effective as possible, regarding the acquisition of the competence by students, considering the limitations of the intervention. this strategy took advantage of the computer lab sessions where discussion and questioning encouraged the development of critical thinking. then, the instruments to assess the acquired skills and attitudes of the students were developed. finally, a quantitative analysis of the results was conducted in order to evaluate the validity and reliability of the strategy. this paper presents a full description of the intervention carried on for two years. besides the desired effects on the students’ performance, some conclusions regarding the development of appropriate instruments to deal with a large group of students are drawn. this intervention has proven to be effective in order to help the students to develop their critical thinking skills, and it is particularly suitable for large groups. keywords: critical thinking; learning assessment; teaching strategies; engineering education; engineering students. to cite this article: giner-navarro, j., sonseca, a., martínez-casas, j., carballeira, j. (2022). working on critical thinking skills using the computer lab works of an engineering subject. multidisciplinary journal for education, social and technological sciences, 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://orcid.org/0000-0002-0513-3625 https://orcid.org/0000-0002-2776-4399 https://orcid.org/0000-0001-5706-4951 https://orcid.org/0000-0002-4179-8499 https://doi.org/10.4995/muse.2022. multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 24 1. introduction engineering problems in real-life are usually ill-structured and complex because they possess "conflicting goals, multiple solution methods, non-engineering success standards, non-engineering constraints, unanticipated problems, distributed knowledge, collaborative activity systems, the importance of experience, and collaborative activity that rely on multiple forms of problem representation" (jonassen et al. 2006). it has also been defined as "a purposeful, reflective judgment, which manifests itself in reasoned consideration of evidence, context, methods, standards, and conceptualization in deciding what to believe or what to do" (facione 1990). critical thinking (ct) skills such as analysis, evaluation, interpretation and self-regulation are essential to develop the engineering judgement through solid decision-making and problem-solving abilities in order to address these problems (pierce et al. 2013; baytiyeh and naja 2017). therefore, ct has always been considered as a key competence for engineers and, in general, a crucial learning outcome for higher education (stassen et al. 2011; association of american colleges and universities (aacu) 2009; hart research associates 2013). this competence is hence highly valued by employers to optimize recruitment (andrews and higson 2008). furthermore, these skills are recognised to be critical in preparing the students for increasing social responsibility and their own individual career development (kelly 2001; rieckmann 2012). sustainable development problems such as climate change, poverty and resource scarcity share those characteristics with engineering problems, which is what makes them really challenging for engineering students (lönngren and svanström 2015; llopis et al. 2022). so, having well-developed ct skills is not only important for students to get a good job, but also to become better professionals and citizens (paul 1995). developing ct skills and dispositions can improve the students’ learning, moreover when there is such an amount of information available through the internet (naimpally 2012; douglas et al. 2018). the relation between ct and problem-solving abilities is especially significant in engineering (itabashi-campbell 2011). therefore, most of the frameworks in engineering education include these skills, together with some other generic competencies (young and chapman 2010; sursock and smidt 2010). https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 25 nevertheless, several studies emphasize that students need to be able to evaluate and solve problems rather than memorize and deploy algorithms to meet workplace demands (marshall and horton 2011; dunne 2015), and there is evidence that this need is not presently being met. prior research endorsed that ct skills appear to be lacking among fresh engineering graduates (jackson 2010; schulz 2008). some interviews made by dunne (2015) reported that, in general, employers contend that university graduates ct skills required in the workplace. in this line, the recent crithinkedu european project on the ct skills and dispositions needed in different professional fields, states that there is a gap between the technical skills and the ct skills of the graduates (dominguez et al. 2018). it was emphasised by the employers that, while students have strong technical skills, their ct skills have not been developed sufficiently. this gap was also recognised some years ago in the governing bodies of the authors’ university – technical university of valencia, spain (upv). so, an institutional project to foster innovative learning and teaching activities related to generic competencies, including ct, was implemented (upv 2020). these findings demonstrate that ct is still a pivoting and demanded skill for students to get a good job, leading to the need to incorporate ct into traditional teaching methodologies in order to enhance their abilities to identify and solve problems within the professional context. one main issue to deal with generic competencies is their assessment (zlatkintroitschanskaia et al. 2015). engineering academics feel confident about the methods to teach and to evaluate technical skills, but they have more trouble to do the same with ct, in part because there is a wide range of understanding of the ct skills. engineering academics have more difficulties to define what ct is than academics in non-technical disciplines (ahern 2012). this leads to the use of a variety of pedagogical strategies for its promotion and assessment (michaluk 2016; guerra and holgaard 2016; hagerty and rockaway 2012), and this variety makes it difficult to measure or monitor their effectiveness (ahern 2019). furthermore, we decided carefully which skills and dispositions of ct to focus on, as there were some limitations to our intervention. fortunately, our university took this first step for us. within the institutional project mentioned above, and in collaboration with specialists in education, a set of definitions and learning outcomes associated with the generic competencies was given. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 26 this definition of ct includes the ability to apply logical and rational processes to analyze the components of an issue, and to think creatively to generate innovative solutions. it also states that developing a disposition toward ct goes along with being interested in the foundations on which the ideas, actions and judgments, both their own and those of others, are based. ct goes beyond the skills of logical analysis, since it involves questioning the underlying assumptions in our habitual ways of thinking and acting. it refers to being prepared to think and do differently, based on that critical reasoning. consequently, we will say that the students have developed it to the extent that they question themselves about things and are interested in the foundations of the ideas, actions, assessments, and judgments. according to this definition, the main ct skills we focused on are interpretation, analysis and evaluation, and the main disposition to promote is willingness to questioning. from the authors’ point of view, developing these skills can be very helpful for the students to deal with technical subjects in which they have to put their cognitive capacities into play. so, a positive correlation between ct level and students’ performance should be expected. therefore, the main goals of our intervention were: to promote and assess ct; to develop appropriate evaluation instruments to deal with a large group of students; and to check if there was a positive correlation between ct and students’ performance. to reach these goals, we had to decide how explicit we would be about teaching ct. we could have used one of the three following options: a general approach and teach ct separately from the subject matter; an infusion approach and teach ct embedded in the subject matter, or an immersion approach and teach ct "involving students in a dialogue where they are prompted to consider, analyze and evaluate different points of view" (angeli and valanides 2009). then, in order to promote the ct skills and dispositions, the learning activities that better fit our approach were planned: problem-based learning, case-based studies, lecture discussion, open-ended questions, roleplay, essays, interviews, observations, and so on. a careful assessment of the acquired skills and attitudes must be designed to measure the effectiveness of the teaching strategy (liu et al. 2014). conventional tests within the subjects not only provide ease and timeliness in quantifying students’ knowledge, but they also come at the expense of helping them become better critical thinkers (shuman et al. 2004). nevertheless, not to https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 27 have separate instruments for assessing the students' ct prevent educators from gaining insight to students’ learning processes (birenbaum 1996). these instruments will be more effective when designed for learning (angelo and cross 1993), trying to challenge the students to learn by constructing new meanings, developing strategies, thinking critically about solution implications, and evaluating their own thinking during the task (mccormick et al. 2014). the idea is to use the assessment to interpret students’ reasoning and ways of thinking critically about the problem. depending on the approach and the activities, the instruments for assessing the students may be different. the use of more than one measure is strongly recommended to give consistency to the further data analysis (ahern 2019). our approach for the assessment of ct is based on the knowledge that students must really understand the concepts to pass the exams in courses for such challenging degrees as engineering ones. from this approach, we would state that the instrument is practical for our purposes when there is a good agreement between the ct grading and the students’ performance in the subject, i.e. students with better grades in the subject should also have a higher score in ct. nevertheless, we will assume that most of our students will earn grade high on ct, as they are already students who have undergone a very selective process to access to this degree. however, ct are not the only skills and dispositions necessary to be successful. therefore, a direct relationship between ct grading and the students’ performance in the subject should not necessarily be expected. 2. method the intervention was implemented in the second year of the bachelor’s degree in aerospace engineering. it was carried on during 2018 and 2019 in mechanics, a subject related to kinematics and dynamics of rigid bodies and mechanical systems. the intervention involved 116 students in 2018 and 103 in 2019, divided into six subgroups for the lab sessions. the global demographics: 21% were female, 74% were freshman, 22% were sophomores and 4% were juniors, 45% were in high academic performance groups. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 28 2.1. teaching strategy mechanics is traditionally considered one of the toughest subjects of the degree by most of the students (evaluations compared with other subjects of a technical nature certify this) since it requires developing knowledge related to complex mathematical and physical concepts. the objective of the teaching strategy proposed is to try to find a way in which it wouldn't be even more difficult while we work ct. both the technical nature of the subject compels to a rethinking of the way of teaching ct (ahern 2012), as does the large number of 100 students. once we clarified what skills and dispositions need to be taught and considered the practical limitations described, we decided to use the computer lab works to develop ct using an immersion approach. this approach allows teaching ct within the normal progression of the subject matter, without causing readjustment to the contents of the teaching guide (which is already aligned with the number of hours available to the subject). in our course, the theory sessions are carried out dividing the total group into two: the high academic performance (hap) group, with about 45% of the students, and the remaining 55% (no high academic performance, nhap), so that both have more than 50 students each. the design of any intervention associated with the development of ct for such large groups was not feasible if we pretend to complete the teaching guide. in addition, each of these theory groups are separated into three subgroups for the laboratory component of the course because the number of computers per laboratory is limited to 12. the students work in pairs along seven sessions (2 hours each) with the objective to solve different applied cases from which they had to extract kinematic and dynamic results. they make use of the commercial simulation software adams/view© to do some exercises and practical demonstrations that help them to visualize and understand some of the main concepts in the subject, i.e. the software is used as a virtual lab. these computer lab works are designed to be suitable to work on ct by putting the ct skills and dispositions into play. using software with several alternatives to solve the exercises forces the students to analyze, understand and evaluate the information they are given, and to think critically about the choices they make to accomplish their goals. moreover, students can check most of their results using analytical methods they have learnt in the theory sessions with real data provided. so, a comparison between what they should expect and what they get from the software is available, which allows them to be critical users of numerical tools in contrast with real experiments. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 29 2.2. intervention design the fundamental advantage of having limited capacity to a maximum of 24 students per lab session is that the practical application of the intervention becomes feasible. each pair is given a different exercise with geometry and inertial data differing from the reference problem developed and solved in the available tutorial. the intervention is organised and planned in three phases that last in total the 2 hours of each session. the distribution is as follows: phase 1: introduction to the lab work. during the first 20 minutes, the instructor introduced the main goals of the session, detailed the problem that the students faced and how to approach it through the simulation software, in line with the physical conceptual framework previously worked on during the theory sessions. the students had the support of a tutorial to help them in the use of the software. key indications for the most general difficulties in the software implementation (based on the experience of previous years) were also given. phase 2: segmented debates. the students had 80 min to implement the dynamic model, and they could ask for the instructor's support twice as maximum per work pair, forcing them to be selfsufficient, to try to overcome the problems they encountered with the available means, and to critically select the questions they would later ask the lecturer. the lab works are designed so that the students face one or two critical moments that require decisions on their criteria sustained on their physical background. the instructor takes advantage of these critical-based steps to work the competence within reduced groups of two or three work pairs separately in order to have a more direct treatment with the students. the lecturer tries to compel each student to participate giving their critical opinion focusing the critical step to make progress in the dynamic modelling (rodriguezlargacha 2014). in this way, a proper interpretation of results, that may raise to substantial doubts, is ensured. some of the reasoning that emerges might be really interesting thus, the instructor takes note of each student interaction through observation. in the following subsection, we describe the assessment instruments adopted and justify why this observation tool was only used in the first year for the assessment of ct. phase 3: group intervention. the last 20 min of the session were reserved for a group analysis of the modelling process, the key decisions that each work pair made based on critical criteria, the results obtained, their comparison and the critical discernment of their plausibility based on the https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 30 problems addressed in the theory sessions and reality experience. again, the lecturer tried to get all the students, or at least all the pairs, to intervene in a debate. if some of the decisions made by each pair led to erroneous results, their reasons were analysed as was how to re-approach the dynamic model. that is why, in order not to condition lab scores with the development of ct, the students had a week to finish the lab work with the appropriate corrections, spurring the critical questioning of decisions previously made even after the lab session. the lab work grade is the average of the seven sessions, evaluating the technical skills on the computer lab works. since the requested results are numerical, an automated correction is available (assuming a 5% of tolerance margin) in the intranet of the university website, through which the students complete each lab work. 2.3. assessment instruments to measure the effects of our intervention and the influence of ct on the students’ performance, two assessment methods were devised in the first year of the intervention: a set of open-ended questions to be answered by the students in the last lab session, and the observation during the lab sessions. these methods would allow us to check if there was an improvement in the ct skills and dispositions of the students, and if there was a correlation between these skills and the performance in such a challenging subject as mechanics. there were three instructors involved every year, so all these decisions were coordinated and discussed together in order to minimize the instructor bias. within the institutional project (upv 2020) a rubric was developed to define the learning outcomes associated with this competence in the first years of bachelor’s degree (see table 1). each of these learning outcomes are divided into four levels of achievement, with the corresponding numerical values interval: not reached (d: 0–3.9 points), in development (c: 4–5.9), good/adequate (b: 6–7.4, excellent/exemplary (a: 7.5–10). these intervals have worked well in other subjects when discriminating the degree of acquisition of other evaluated competencies, so it has been decided to keep them in this case. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 32 adapting this rubric to our subject has required a critical rethinking of the assessment instruments. two of these developed instruments were implemented in the first year experienced: 4item open questionnaire and observation. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 31 table 1 rubric to assess the learning outcomes of critical thinking for the two first years of the bachelor´s degrees (upv 2020) learning outcome d. not reached c. developing b. good / adequate a. excellent / exemplary evidence the student shows a critical attitude towards reality: the student wonders why things are happening the student does not manifest any kind of critical spirit: he never questions the situation or the reality in which he lives. it assumes as true any information it receives the student questions certain situations of the reality in which he lives. but it is incapable of issuing judgments and valuations of its own. need the help of others to get answers the student asks the question of things and investigates to get answers autonomously. but it is influenced by issuing its own judgments and assessments the student reflects and investigates the why of things, and is able to find answers and argue them objectively in exercises or problems with real proposals, include questions that invite the student to question: is the result obtained in the previous section coherent (units, physical sense)? could it have been solved in another way? detects inconsistencies or contradictions in other people's speech or in a text the student is not able to detect inconsistencies or contradictions in a speech or text the student is able to detect some inconsistencies but does not know how to explain why the student detects inconsistencies and contradictions, and provides arguments to demonstrate the same the student detects inconsistencies and contradictions, provides arguments, and coherently reformulates contradictory statements to evaluate solving problem process; to suggest problems with multiple solutions; to justify the methodology and data used differences made from opinions, interpretations or valuations the student shows a reflective attitude towards another people's discourse. does not distinguish facts from opinions the student normally distinguishes facts from opinions, but can accept judgments or decisions based on opinions the student differences facts from opinions, interpretations or assessments in the arguments of others the student questions judgments or decisions based on opinions, evaluations, etc. and detects fallacies and ambiguities from press news or texts related to the subject, ask students to differentiate between objective facts, and interpretations of the author delves into a topic with logic and impartiality, contrasting information in reliable sources the student is not able to delve into a subject. it uses a single source and does not contrast the information the student resorts to diverse sources, but it does not verify the reliability of the same ones the student consults different sources and contrasts the information of them to verify their reliability the student consults reliable sources, contrasts the information and provides his personal assessment ask the students to do brief documentation work on a topic related to the subject, and to contribute the bibliography https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 32 2.3. open questionnaire first, a competence whose nature demands a solid reasoning from the students when addressing an open problem requires an open-format assessment tool (liu et al. 2014). it should allow the student to develop and synthesize a chain of ideas with a plot line that goes outside the framework of the acquired theoretical knowledge. however, we were dealing with large groups of students, so we needed an instrument that could be quickly completed by students and quickly assessed. therefore, we opted for a short questionnaire with open questions as the appropriate trade-off between an open-format and time (hong and choi 2015). the rubric shown in table 1 is a general-purpose instrument (upv 2020), and so, the generic descriptors provided could not be directly applied in mechanics. thus, these descriptors had to be adapted and converted into open questions. these questions forced the students to analyze, understand and evaluate as a result of handling the software used in the lab sessions. this assessment instrument intends to assess the originality of their answers beyond the technical knowledge acquired, also proposing new cases as well as the advantages and disadvantages to implement them in the software. the students had 20 minutes to respond to the four open questions. the questionnaire was delivered in the last session, following the format presented in table 2. a set of guidelines was discussed to score the answers to each question in order to obtain a quantitative measurement of the students' ct level. the grade in the competence questionnaire was not included in the averaging of the course. table 2. assessment instrument: open 4-item questionnaire 1. “as the world’s most famous and widely used multibody dynamics (mbd) software, adams improves engineering efficiency and reduces product development costs by enabling early system-level design validation. engineers can evaluate and manage the complex interactions between disciplines including motion, structures, actuation, and controls to better optimize product designs for performance, safety, and comfort. along with extensive analysis capabilities, adams is optimised for large-scale problems, taking advantage of high-performance computing environments.” with which affirmations of the text belonging to the adams’ brochure do you agree more and with which disagree? argue your reasons. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 33 2. explain which, in your opinion, are the advantages and disadvantages of using adams/view© to solve a parabolic shooting problem with air friction. 3. what are, from your point of view, the factors that can cause the numerical results to move away from the real measures? 4. which parameters do you consider most important to carry out a correct dynamic analysis? why? item 1. it requires the critical evaluation of a promotional text of the software brochure, indicating with which statements students agree/disagree based on their experience. this item involves questioning the underlying assumptions and asks students to differentiate between objective facts and interpretations.  rubric 1. it is expected that the students question some statements from the brochure, such as the ease of use and flexibility of the software in some specific circumstances and especially “adams is optimised for large-scale problems”, since they have not seen such problems in any of the cases studied in the sessions. item 2. it raises the problem of parabolic shooting considering air friction, a question solved analytically in the theory sessions. this allows students to evaluate the problem-solving process for a situation with multiple solutions, requiring the justification of the methodology used.  rubric 2. we want to make students think whether, for a preliminary study, the time invested in a model in the software is profitable, being able to solve the problem analytically with a shorter procedure; the considerations about the complexity of modeling friction with air and other physical phenomena are also valued. this question opened a debate about the most optimal way to address a problem in which the aerodynamic forces completely condition the trajectory of the body under study. item 3. it focuses on the causes of divergences between the numerical results and the actual measurements, encouraging the students to take a critical perspective in the assumptions considered during the modelling process. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 34  rubric 3. it sought to expand on expectations versus reality regarding the energy dissipation mechanisms and predictable and unpredictable factors that deviate measurements with respect to the software results. numerical precision of the software in the time integration can also be mentioned. item 4. the students are asked to choose the key parameters for solving dynamic cases, especially the ones that they consider crucial for the analysis.  rubric 4. the idea is that the students take control of the parameters that influence the dynamic study. the rigor and completeness of the answers have been well scored. 2.4. observation as the size of the groups was reduced substantially to a maximum of 24 students, an observation instrument was conducted only in 2018 as a direct measurement of the interactions and critical participation of the students in the lab sessions. through debates applied to subgroups of two or three work pairs, the lecturer tried to encourage the students to question critically about whether, the tutorial approach was appropriate for their specific problem or they would have adopted other methodologies. some students proposed alternatives that involved saving user time and/or allowing better visualization of the results and, hence, contrasting them in a more direct way. 3. results data analyses are presented in this section to check the validity and the reliability of this implementation, to assess the influence of the ct intervention on the students’ performance. table 3 shows the average score of all the students (total), for the hap and nhap groups for each performance indicator, i.e. lab work, theory and global grades, together with the rates of the assessment based on observation (obs.) only applied in 2018, and the open questionnaire instrument (quest.) for 2018 and 2019. average observation for ct showed that there were not statistically significant differences between the hap and nhap students (7.54 and 7.53, respectively). regarding the open questionnaire, the ct rates are apportioned into the four items, among which the averag e https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 35 of the item 4 in 2018 was significantly higher than the three previous questions and led us to check the origin of this deviation. it was decided to delete it after concluding that this item generated memory-based responses instead of reasoning-based ones as will be discussed later. in 2018, there was a noticeable difference between hap and nhap students in the performance indicators around 1.3 points over 10, which was reflected with a deviation of 0.84 points for the ct assessment favorable to hap students. there was detected a considerable increase of the average of the performance indicators in 2019, especially in the theory and global marks with more than 1 point. this increment led to a reduction in the difference between the hap and nhap average (0.62 points in the global marks), also shown in the ct rate with a shortened but still remarkable difference of 0.6 points favorable to hap students. hereby, the table demonstrates a preliminary trend in line with our expectations: students with better ct skills earn higher scores in the subject. table 3 average of the questionnaire (indexed per items), lab work, theory and global marks for 2018 and 2019 item 1 item 2 item 3 item 4 questionnaire observation lab theory global 1st year total 6.53 6.20 6.55 7.30 6.64 7.54 7.39 4.56 5.32 hap 7.25 6.67 6.94 7.56 7.11 7.54 8.15 5.28 6.04 nhap 5.94 5.81 6.23 7.09 6.27 7.53 6.77 3.97 4.73 2nd year total 6.77 6.44 6.57 6.32 7.73 5.77 6.35 hap 7.03 6.57 6.79 6.79 8.11 6.13 6.67 nhap 6.58 6.33 5.75 6.19 7.35 5.44 6.05 to take a further step on the information that the results of the instruments used can provide, the correlation between performance and ct measured through both tested instruments can be visualised in the figures above. first, figure 1 allows to check that observation within the immersion approach experienced in the subject does not give any noticeable correlation or trend. the experience corresponding to 2018 made us quickly realize that the notes taken by the instructors and, finally, the score based on the observation unconsciously ended up depending more on how extrovert or not the students were than on their real abilities and dispositions regarding the competence under study. subjectivity played a role that could hardly be avoided despite any established guideline or rubric, https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 36 which increased with the number of instructors in lab works. furthermore, under what criteria did we evaluate students who barely participated? could we have created a situation that would have allowed us to have information to evaluate them? we had no time enough for individual interviews without interrupting the progress of the work in pairs during the lab session. so, the lack of correlation obtained could be expected and led us not to repeat this instrument in the second year. figure 2 compares the open questionnaire results implemented in 2018 (3 items, figure 2(a)) and 2019 (4 items, figure 2(b)) separately. the results for the 4-item questionnaire illustrate that, in general terms, scoring high in ct is indicative of a good performance in the subject; the trend line, which crossed at the origin, makes this more obvious. similar correlations for the lab sessions and the theory exam were also obtained (not shown in this paper for conciseness). the suppression of the item 4 in the questionnaire allowed obtaining better results in the second year. comparing figures 2(a) and (b), it is observed a significant lower number of anomalous points out of the 2points range lines traced parallel to the trend one: 31% against 22% for 2018 and 2019, respectively. in fact, only three results can be classified as very anomalous in 2019 (highlighted in black), showing a clear discordance between ct and global score. for lab work and theory exam (not shown), the percentage are similar, with about 10% less out-of-range point for the second year. figure 1 observation instrument rates vs. global marks of the subject for the first year (2018) 0 2 4 6 8 10 0 2 4 6 8 10 g lo b a l m a rk observation https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 37 (a) (b) figure 2 questionnaire vs. global scores for the 4-item in 2018 (a) and 3-item (b) instruments conducted in order to delve into this trend, we divided the performance scores into failure (d': 0–4.9 points), sufficient (c': 5–6.9), good (b': 7–8.9) and excellent (a': 9–10). the numerical values for the ct were obtained by scoring the 4-item questionnaire at 80% and observation at 20% for the first year; the 3-item questionnaire involves 100% of the ct grade. these values are classified within the a, b, c and d levels of achievement defined in the previous section. from these intervals, we considered an ideal correlation to get the pair a-a' for outstanding students, b-b' for the remarkable ones, c-c' for who barely passed the subject and d-d' for the failed ones. distributing the students' scores within these intervals, we obtain figure 3 for both years experienced. in the first year, it was surprising to find in table 4(a) the only 2 students with a' qualification in the subject with c grade in ct, or 12 d' with a. a 76% of the students were located in the upper left corner that, although partially explained due to the lower mean of the global score average of more than 1 point, permits to visualize a poor correlation between both subject and ct scores. in contrast, we observe a significant shift in the distribution of the second year, more diagonalbased, with no-over 10 students in any box out of the main diagonal or sub-diagonals of table 4(b). only 4 students with a' in the subject got a in ct, and there were 14 students in the b-b' box and 20 in the c-c' one. the jarring note was getting 3 students with a who did not pass the subject. it is also interesting to highlight that only a 9% got lower grades in ct than in the subject, suggesting that the restrictive selection criteria of students in engineering degrees promote the access of students with good critical abilities. 0 2 4 6 8 10 0 2 4 6 8 10 g lo b a l sc o re 4-item questionnaire (2018) 0 2 4 6 8 10 0 2 4 6 8 10 g lo b a l sc o re 3-item questionnaire (2019) https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 38 table 4 ct vs. global performance distributed in the intervals defined previously for the: (a) first year (2018), (b) second year (2019) experienced a 12 18 12 0 a 3 9 11 4 b 16 18 8 0 b 1 21 14 0 c 12 8 3 2 c 8 20 9 0 d 4 3 0 0 d 0 2 1 0 ct d' c' b' a' ct d' c' b' a' subject subject (a) (b) for the analysis of the instructor bias in the ct correction, it is proposed a correlation factor defined through the weight matrix w presented in table 5. it weights with 1 the number of students located in the diagonal boxes (a-a', b-b', c-c', d-d'), and the farther the rest of the boxes from the diagonal is, the lesser the weight is (0.5, 0.25 and 0). by multiplying box-to-box the 4x4-matrix w with weights of each box (table 5), and the 4x4-matrix p with the number of each box (table 4), we get the factor: 𝜌 = ∑ ∑ 𝑊𝑖𝑗𝑝𝑖𝑗 4 j=1 4 i=1 ∑ ∑ 𝑝𝑖𝑗 4 j=1 4 i=1 (1) which gives 1 if all the ct-subject points are contained in the diagonal boxes (perfect correlation using this estimative criterion) and 0 if they are in the a-d' and/or d-a' positions (the worst possible correlation). table 5 weight matrix w to compute the ct vs. scores correlation from eq. (1) a 0 0.25 0.5 1 b 0.25 0.5 1 0.5 c 0.5 1 0.5 0.25 d 1 0.5 0.25 0 ct d' c' b' a' subject https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 39 table 6 gathers this correlation factor for the three performance indicators considered, in average and separated per instructor. first, it is confirmed that the correlations for the second year are significantly more consistent in line with figures 1 and 2 (excluding the specific case of instructor 2 for the theory exam). the standard deviation is only 0.02 (2018) and 0.01 (2019) for the lab sessions, while this value is slightly more significant for the theory and global scores. internally, these results indicate us that successive meetings and feedback between lecturers minimize the instructor bias, especially in the second year. table 6 correlation  −eq. (1)− between ct vs. lab work, theory and global scores for the two years (2018 and 2019) experienced, with the average and the standard deviation  from the three instructors theory instructor 1 instructor 2 instructor 3 avg.  1st year 0.41 0.47 0.33 0.39 0.07 2nd year 0.54 0.46 0.49 0.51 0.04 lab work 1st year 0.62 0.62 0.65 0.63 0.02 2nd year 0.68 0.67 0.69 0.68 0.01 global 1st year 0.52 0.49 0.40 0.46 0.06 2nd year 0.66 0.61 0.71 0.65 0.05 4. discussion 4.1. open questionnaire the poor correlations given by figure 2(a) for the open questionnaire in the first year indicated us that the design had to be reviewed. we concluded that the item 4 was not well outlined, since the students made a list of the parameters that influence the dynamic study without discerning the key elements or the plot line needed to address general dynamic cases. this generality of the question led to memory-based answers without reasoning implications. after deleting this question, the second https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 40 year brought about better correlations (see figure 2(b)), revealing that ct plays a significant role in the skills that the student needs to pass the subject, as we expected. the influence of the instructors was also critical to explain these better results. there were three instructors in the lab sessions, who were responsible for promoting and assessing ct in their respective groups. the results section analyze the bias per instructor and year (see table 6), giving low values for both years, but better correlation for the second one. the coordination achieved in successive meetings to prepare the subject allowed the understanding required to minimize the bias. by analyzing the results in detail, more important deviations are observed (poor correlations) in students with lower performance in the subject (see table 4). these outcomes could suggest that some students with good ct have not prepared the subject properly to address the exams. perhaps, they consider that the skills related to ct that they are aware of possessing will facilitate them to pass the exams without the need of a great effort in preparing the subject. on the contrary, other students, who are aware of their difficulties in reasoning skills, need to spend more time on the subject even though their learning is based more on memorization than on understanding the underlying physical concepts. these are hypotheses that should be contrasted with individual interviews to a representative sample of the student body. 4.2. observation as part of the intervention activities in the immersion approach followed, debates were promoted within two or three pairs in order to look for strategies to overcome the problems arisen during the lab session. but these were revealed insufficient to observe the individual way of thinking of each student. although we tried to promote the participation of the most reluctant students to speak, many of them were masked in the arguments of the most participative ones. furthermore, working in pairs often results in the reasoning of one of the components being opaque. and we must include the difficulty of scoring objectively with a score from 0 to 10 the competence, based on the quality of their arguments and not on how often they intervene in the debate. sometimes, to separate these concepts is a challenging task, especially in the global debates at the end of the lab session. those factors explain the really indiscernible correlations of the observation between the subject https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 41 indicators in the first year (see figure 1), which led us not to repeat this assessment instrument in the second year. 5. conclusions a practical intervention has been carried out for promoting and assessing ct in a technical subject for two years. an immersion approach has been used to work on the skills and to encourage the dispositions of ct. two assessment instruments have been developed to deal with the specific characteristics of the subject, with more than one hundred students every year, and to obtain some quantitative feedback in order to measure the validity and reliability of the intervention. the correlation between ct development and students’ performance has been analyzed from a qualitative point of view. the main conclusions that can be drawn from this work are:  the need for an appropriate design of the assessment instruments for the specific evaluation of ct in technical subjects with a large group of students. assessing a complex competence such as ct in a challenging subject such as mechanics requires a careful trade-off. the instruments should be able to measure the cognitive skills and dispositions of the students related to ct, but they should be embedded in the contents of the subject to avoid using extra-time for this evaluation.  a reliable way to confront a cognitive competence as ct in large groups in engineering courses is to implement an immersion approach in lab sessions with a limited number of students (up to 25). in this experience, computer lab works are designed to offer plenty of opportunities to put the ct skills and dispositions into play.  the open questionnaire has revealed a promising instrument to assess ct, adapting the associated learning outcomes to the technical nature of the engineering courses. the observation tool needs a more individualised intervention that is not feasible for the size of the lab groups which we dealt with. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 42  developing good ct skills provides a better opportunity to pass the subject successfully, but they are not the only skills required. from a qualitative point of view, a good correlation has been observed between the ct grade and the students’ performance for those with the best grades in the subject. however, a rather poor correlation has been obtained for those students with an average performance in the subject. anyway, it is expected that the prolongation of the experience in successive courses will confirm that the evaluation tool detects improvements in the development of the competence and that the direct relationship with the performance in the subject could allow in future courses the evaluation of this competence directly from a correlation with the mark of the subject. further research is needed to check the relevance of promoting ct skills in the students’ achievements.  an effective coordination of the instructors is essential to minimize the bias in the teaching and in the assessment of ct. an important effort should be done to agree a set of learning outcomes and how to grade them. because of the importance of ct for engineering students, further research on its promotion and assessment will continue in the next years for developing better strategies and instruments. a suitable design of any intervention should be done to ensure that it is aligned with the learning outcomes desired both for the technical and the ct skills. reducing the influence of the instructors should be a priority when dealing with large groups of students. acknowledgements: the authors acknowledge the financial contribution by the universitat politècnica de valència through the project pime/2018/dpto. imm. author contributions: conceptualization, j. g and j. c; methodology, j. g, a. s, j. m-c., j. c.; validation, j. g, a. s, j. m-c., j. c. ;formal analysis, j. g, a. s, j. m-c., j. c.; investigation, j. g, a. s, j. m-c., j. c.; resources, j. c. ; data curation, j. g.; writing original draft preparation, j. g., a. s, j. m-c., j. c.; writing review and editing, j. g., a. s, j. m-c., j. c.; visualization, j. g., a. s, j. m-c., j. c. funding: this research was funded by the universitat politècnica de valència through the project pime/2018/dpto. imm. conflicts of interest: “the authors declare no conflict of interest.” “the funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results”. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 43 references ahern, a., dominguez, c., mcnally, c., o’sullivan, j. j. and pedrosa, d. (2019). a literature review of critical thinking in engineering education. studies in higher education, 44, 816–828. https://doi.org/10.1080/03075079.2019.1586325 ahern, a., o’connor, t., mac ruairc, g., mcnamara, m. and o’donnell, d. (2012). critical thinking in the university curriculum: the impact on engineering education. european journal of engineering education, 37 (2), 125–132. https://doi.org/10.1080/03043797.2012.666516 andrews, j., and higson, h. (2008). graduate employability, ‘soft skills’ versus ‘hard’ business knowledge: a european study. higher education in europe, 33, 411–422. https://doi.org/10.1080/03797720802522627 angeli, c., and valanides, n. (2009). instructional effects on critical thinking: performance on ill-defined issues. learning and instruction, 19 (4), 322–334. https://doi.org/10.1016/j.learninstruc.2008.06.010 baytiyeh, h. and naja, m. k. (2017). students’ perceptions of the flipped classroom model in an engineering course: a case study. european journal of engineering education, 42 (6), 1048–1061. https://doi.org/10.1080/03043797.2016.1252905 birenbaum, m. (1996). assessment 2000: towards a pluralistic approach to assessment. alternatives in assessment of achievements, learning processes prior knowledge. m. birenbaum and f. j. r. c. dochy, eds., kluwer, boston, 3–30. https://doi.org/10.1007/978-94-011-0657-3_1 dominguez, c. coord. (2018). a european collection of the critical thinking skills and dispositions needed in different professional fields for the 21st century. itad, vila real. douglas, k. a., fernández, t., purzer, s., fosmire, m., and van epps, a. (2018). the critical-thinking engineering information literacy test (celt): a validation study for fair use among diverse students. international journal of engineering education, 34 (4), 1347–1362. dunne, g. (2015). beyond critical thinking to critical being: criticality in higher education and life. international journal of education research, 86–99. https://doi.org/10.1016/j.ijer.2015.03.003 facione, p. a. (1990). critical thinking: a statement of expert consensus for purposes of educational assessment and instruction, executive summary, the delphi report. the complete american philosophical association delphi research report, california academic, millbrae, ca. guerra, a., and holgaard, j. e. (2016). enhancing critical thinking in a pbl environment. international journal of engineering education, 32 (1(b), 424–437. hagerty, d. j., and rockaway, t. d. (2012). adapting entry-level engineering courses to emphasize critical thinking. international journal of stem education, 13 (2), 25–34. hart research associates. (2013). it takes more than a major: employer priorities for college learning and student success. accessed february 10, 2021. http://www.aacu.org/liberaleducation/lesp13/hartresearchassociates.cfm. hong, y. c. and choi, i. (2015). assessing reflective thinking in solving design problems: the development of a questionnaire. british journal education technology 46 (4): 848–863. https://doi.org/10.1111/bjet.12181 itabashi-campbell, r., perelli, s., and gluesing, j. (2011). engineering problem solving and knowledge creation: an epistemological perspective. technology management conference (itmc), ieee international, san jose, california, 777–789. https://doi.org/10.1109/itmc.2011.5996057 https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1080/03043797.2012.666516 https://doi.org/10.1080/03797720802522627 https://doi.org/10.1016/j.learninstruc.2008.06.010 https://doi.org/10.1080/03043797.2016.1252905 https://doi.org/10.1016/j.ijer.2015.03.003 https://doi.org/10.1111/bjet.12181 https://doi.org/10.1109/itmc.2011.5996057 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 44 jackson, d. (2010). an international profile of industry-relevant competencies and skill gaps in modern graduates.” international journal of management education 8 (3): 29–58. https://doi.org/10.3794/ijme.83.288. jonassen, d., strobel, j. and beng lee c. (2006). everyday problem solving in engineering: lessons for engineering educators. journal of engineering education 92 (2): 139–151. https://doi.org/10.1002/j.2168-9830.2006.tb00885.x kelly, a. (2001). the evolution of key skills: towards a tawney paradigm. journal of vocational education & training 53 (1): 21–36. https://doi.org/10.1080/13636820100200149 liu, o. l., frankel, l. and roohr, k. c. (2014). assessing critical thinking in higher education: current state and directions for next-generation assessment. ets research report series 1: 1–23. lönngren, j., and svanström, m. (2015). systems thinking for dealing with wicked sustainability problems: beyond functionalist approaches. in proc., 7th international conference on engineering education for sustainable development (eesd’15) 1–7. university of british columbia. https://doi.org/10.1007/9783-319-32933-8_14 llopis-albert, c., rubio, f., zeng, shouzhen, z., grima-olmedo, j., grima-olmedo, c. (2022) the sustainable development goals (sdgs) applied to mechanical engineering. multidisciplinary journal for education social and technological sciences 9(1), 59-70. https://doi.org/10.4995/muse.2022.17269 mccormick, m., lawyer, k., wiggins, j., swan, c. w., paterson, k. g. and bielefeldt, a. r. (2015). sustainable engineering assessment using rubric-based analysis of challenge question responses. journal of professional issues in education and practice 141 (2). michaluk, l. m., martens, j., damron, r. l. and high, k. a. (2016). developing a methodology for teaching and evaluating critical thinking skills in first-year engineering students. international journal of engineering education 32 (1(a)), 84–99. naimpally, a., ramachandran, h. and smith, c. (2012). lifelong learning for engineers and scientists in the information age. elsevier, amsterdam. https://doi.org/10.1016/c2010-0-67009-x paul, r. (1995). critical thinking: how to prepare students for a rapidly changing world, foundation for critical thinking, santa rosa, ca: ritthenhouse book distribu. pierce, c. e., gassman, s. l., and huffman, j. t. (2013). environments for fostering effective critical thinking in geotechnical engineering education (geo-effects). european journal of engineering education 38 (3), 281–299. https://doi.org/10.1080/03043797.2013.800021 rieckmann, m. (2012). future-oriented higher education: which key competencies should be fostered through university teaching and learning? futures 44: 127–135. https://doi.org/10.1016/j.futures.2011.09.005 rodríguez-largacha, m. j., garcía-flores, f. m., fernández-sánchez, g., fernández-heredia, a., millán, m. a., martínez, j. m., vilardaga, i. and bernaldo, m. o. (2015). improving student participation and motivation in the learning process. journal of professional issues in education and practice 141 (1). https://doi.org/10.1061/(asce)ei.1943-5541.0000209 schulz, b. (2008). the importance of soft skills: education beyond academic knowledge. windhoek, namibia: dept. of communication, polytechnic of namibia. https://doi.org/10.1016/0006-3207(93)90452-7 sirotiak, t., and sharma, a. (2019). problem-based learning for adaptability and management skills. journal of professional issues in education and practice 145 (4). https://doi.org/10.1061/(asce)ei.19435541.0000420 stassen, m. l. a., herrington, a., and henderson, l. (2011). defining critical thinking in higher education. chapter 10, to improve the academy, j. miller, ed., vol. 30, jossey-bass, san francisco, ca, 126–141. https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ https://doi.org/10.1002/j.2168-9830.2006.tb00885.x https://doi.org/10.1080/13636820100200149 http://dx.doi.org/10.1007/978-3-319-32933-8_14 http://dx.doi.org/10.1007/978-3-319-32933-8_14 http://dx.doi.org/10.1016/c2010-0-67009-x https://doi.org/10.1080/03043797.2013.800021 https://doi.org/10.1016/j.futures.2011.09.005 http://dx.doi.org/10.1061/(asce)ei.1943-5541.0000209 http://dx.doi.org/10.1016/0006-3207(93)90452-7 http://dx.doi.org/10.1061/(asce)ei.1943-5541.0000420 http://dx.doi.org/10.1061/(asce)ei.1943-5541.0000420 multidisciplinary journal for education https://doi.org/10.4995/muse.2022.17908 social and technological sciences e-issn: 2341-2593 giner-navarro et al. (2022) mult. j. edu. soc & tec. sci. (2022), 9(2), 23-45. https://doi.org/10.4995/muse.2022.17908 45 sursock, a., and smidt, h. (2010). trends 2010: a decade of change in european higher education. european university association, brussels. upv (2020). transversal competencies. accessed june 01, 2020. http://www.upv.es/contenidos/comptran/index-en.html young, k., and chapman, e. (2010). generic competency frameworks: a brief historical overview. education research and perspectives 37 (1): 1–24. zlatkin-troitschanskaia, o., shavelson, r. j. and kuhn, c. (2015). the international state of research on measurement of competency in higher education. studies of higher education 40 (3): 393–411. https://doi.org/10.1080/03075079.2015.1004241 https://doi.org/10.4995/muse.2022.17908 https://doi.org/10.4995/muse.2022.17908 http://polipapers.upv.es/index.php/muse/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://www.crossref.org/ http://dx.doi.org/10.1080/03075079.2015.1004241 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 the effect of the reutilization of an open educational experience on the improvement of the reflexive process of education learning in university students. ramirez, c.*, lorenzo, j., jiménez.a.c., rivilla-garcía, j. polytechnic university of madrid * corresponding author: email: cristina.ramirez@upm.es; phone: + 34 680128027 abstract the aim of this experience has been to analyze the effect on the improvement of the process of education learning, the interest and the perception of the student body that generates the reutilization of an open didactic resource. for that 115 students of the degree in sciences for physical activity and sport, realized a recording of an educational lesson. after the recording, the students in his teachers’ role, did a reflexive and personal analysis on there verbal communication, the answer of the students on the educational speech, body language and didactic aspects. afterwards a group reflection was made about on the tape recording of the rest. for the evaluation of the results, students completed an evaluation questionnaire on a scale 1 to 5, on ten educational and methodological aspects, about the experience of learning from the reutilization of open resources. results shows a score over 3,5 in the item referred to the pedagogic interest of the experience (4,12 ± 1,09); in the item referred to the utility of the resources used (3,92 ± 0,81), in the item of the perception that the experience is positive to the learning process (4,02 ± 0,93), as well the help that these resources suppose to reach the learnings (3,88 ± 0,76). these variables obtained a positive valuation from 3 in a very high percentage: pedagogic interest (95,65 %), perception of positive experience (94,78 %) and utility of resources used (93,91 %). in equal importance is that the majority of the students thought that the contents were identical to the aims of the subjects (3,8 ± 0,72). all this confirms the importance of the use of new open didactic resources is a motivating element in the university teaching, in the context of the sports education. keywords open resources, educational experience, educational innovation ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 141 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 1. introduction open access to information allows its universal diffusion in a free and open way. the idea of open educational resources (oer) has numerous working definitions. the term "open educational resources" was first adopted at unesco's 2002 forum on the impact of open courseware for higher education and they were defined as “materials offered freely and openly for educators, students and self-learners to use and re-use for teaching, learning and research”. (unesco, 2002). according to santos-hermosa et al. (2012), after elaborating a repository of the open educational resources, claims that their essential characteristics are: • the accessibility in terms of resource’s availability to be located and use in any place or time. • the reusability, the possibility to be used and modified in diverse learning contexts. • the interoperability or the facility to be adapted or interconnected between different hardware, devices or tools. • the sustainability, a proper working despite the change in software, versions, etc. • the metadata, or descriptions which allow its indexation, storage, search and recovery. the reusability paradox in open resources claims that if the content is “reusable”, it has to be specific in its context, but if it is too much specific, it do not tend to be “reusable” (wiley, 2006). the european higher education area (ehea), with its new pedagogical model, promotes the use and reuse of the open educational resources for teachers and students. the figure of the teacher in educational resources is essential for starting with the reforms of education systems and for achieving a quality education. “as an education figure, the ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 142 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 teacher comes in society as the result of his interaction with the context in which he works” (alfaro, 2013). the ehea emphasizes new teaching methodologies in detriment of the traditional master classes. within the new methodologies we have the “continuous evaluation”, in which the teacher has to do a daily continuation to the student, and the ehea proposes two methods for that: the use of the ict and internet , tutorials and a practicing teaching, which tries to find a more active intervention of the student, with group works, exercises, professional practices, etc. new technologies are a very powerful teaching resource. according to de la torre et al. (2008) the mains purposes that teachers make lie in to plan courses, designing teaching and learning strategies, find and prepare teaching materials and resources, motivate students, provide information, advice on the use of resources, guide the completion of activities, student mentoring, evaluating, management jobs, contact with the environment, and promote a positive attitude towards ict from a critical perspective of the students and staff continuous training to improve teaching skills. as teachers, we should reflect on our teaching practice, before, during and after class sessions. in this reflection we should ask ourselves how do to perform them. among the authors who have delved into this issue we can include esteve (2004), who defends the crucial role of reflection and its positive influence on teacher action through the use of observation and portfolio. one of the common used self-observation instrument in teaching are video recordings. but in the field of teaching future teachers, it also helps the teacher to issue feedback to the most effective student, of being able to analyze, sequence the document, etc. the latest investigations about the perceptive and conduct changes, shows that selfreflection spaces are created when the recorded images are analyzed. these spaces are based on the idea of their time contemplation, which create intrinsic processes which are enrolled in three levels: 1. receptive level. it corresponds with the awareness (realism). 2 ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 143 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 dynamic level. it is related with the motivation, interest and satisfaction. 3 learning level. it is related with the achievement of the skills. the objective of this experience has been to analyze the effect of the process of teaching and learning, the interest and the perception of university students who generate the reuse of an open educational resource. 2. method for the development of this experience an open resource has been used "the recording video of the teaching practice, a strategy for the development of the reflexive teaching" (calzada, 2010). this resource is an experience placed in the national school for teachers of kindergarten in an institution of higher education that has the objective of the training of teachers for preschooler. the current curriculum studies of this school, as the author says, emphasizes the importance of that "the new teacher should have a reflexive attitude that would allow him to recognize the complex of the teaching practice and to visualize that in the presence of an uncertain reality he must keep an attitude of research of alternatives in the work with preschoolers children. the election of this resource to be reuse in our faculty is justified under the research of meaningful reflexive learnings, and above all because with the new curriculum studies we are encouraging the experience in the teaching practice since the first years in the degree. in this experience there were 115 students from the subject "swimming and aquatics skills" from the first year of the degree in sport science and physical activity (polytechnic university of madrid) who took part. the experience was developed in the academic year 2010-2011, in which there were 246 students enrolled in the subject ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 144 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 distributed in 8 groups. the experience was made with 4 groups out of 8. there were 128 students who took part in the recording videos, 13 of them did not finish the experience so did not participate in the analysis and reflection of the teaching practice. the 128 students were distributed in four groups of 27, 33, 34 and 34 students per group. the open resource has been modified according to the abilities and characteristics of the students in the first year of the degree in sport science and physical activity as well as the objectives of the subject "swimming and aquatics skills". the possibility of filming the teaching practice allows the students to visualize repeatedly of some aspects of the video that, due to their importance, need to be analyze in detail: a dialogue, an action, a reaction of a group of students to the educational objectives or a didactic sequence and their results. for the reutilization of the resource we have follow the methodology of the author, modifying, to our context in the classroom. filming of the students was realized giving class of aquatic activity in a acclimatized swimming pool. the video were recorded by the teacher responsible for the subject. the teacher raised different situations of class and the students developed the educational practice while they were filmed. the presence of the chamber in the meetings can generate tension and edginess, when feels the student who is going to be evaluated across the recording, which provokes a change of behavior with regard to the rest of the meetings. once recorded the video, and under the authorization of the student body, lodged at the channel youtube of the upm (polytechnic university of madrid), and the link was published in the platform of tele education moodle of the upm, to which they had access all the students. ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 145 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 later the students realized an individual reflection on how they were considering his forms of expression, the use of the body, the ways of interacting, as well as on the response of the companions to his indications. inside this first reflection it was asked them to analyze: • verbal communication: clear, opened, expressive, enthusiast, shy, serious, aggressive, cutting, tax, etc. • tone of voice: effects of the tone of voice, volume, speed, intonation of agreement to the circumstances, etc. • capacity of leadership. • classmates’ answers to the speech of teacher: they answer, take part, are indifferent, pay attention, are opposed, interpret adequately the slogan. etc. • body expression/not verbal communication: natural, agile, enthusiast, projects diverse states of mind, calls with his forms of expression and with his attitudes for the participation of all, rejects, is closed, demonstrates insecurity, coyness, distress, etc. – global didactic aspects. • sensations, feelings and ideas that it generates turn in to an action opposite to the group. • analysis of the swimming skills. after the individual analysis one proceeded to the accomplishment of a reflection in group, in which all the pupils were listening initially to the analysis of the companions who took part as teachers in the practice, and later they were informing all. the aims chased with this experience (to generate a major reflection on the capacities of the educational futures, to analyze his verbal language and gestural, and to think about the safety aspects that a teacher must fulfill in aquatic facilities) are directly related to some general and specific jurisdictions of the subject swimming and aquatic skills, as well as to the results of learning of the same one, since we can see in the table 1. ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 146 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 table 1. competitions and results of learning of the subject. general competence especific competence learning result cg 5 to manage with efficiency and efficiency the information proceeding from different sources integrating his relevant aspects for the fulfillment of the proposed aims ce 3 to identify and to anticipate the risks that stem for the health of the students in the practice of physical activities inadequate or wrong realized to develop a correct application of the education of the aquatic activities in the different areas of application, regarding the characteristics both of the group and of the individual interests of the components of the above mentioned group. 3. results students filled in a questionnaire in which they were expected to value in a scale from 1 to 5 (being 1 totally disagree, 2 mostly disagree, 3 in agreement, 4 agree mostly, 5 fully agree) 10 educational and methodological aspects of the open resources reutilization (oer) learning experience. table 2 shows the different items in the questionnaire. ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 147 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 table 2. items in the evaluation questionnaire for the oer experience. # question 1 the organization of the experience (individual work, group work…) is adequate 2 the contents and the format (text, pdf, power point presentation, video, etc.) of the experience match the objectives posed previously to the students. 3 the time spent in carrying out the experience is suitable. 4 the resources used in the experience have pedagogical interest. 5 the resources used are of a suitable technical quality. 6 in comparison with the equipment normally used by professors in class, the open resources used are useful. 7 the experience has positively improved the level of knowledge on the topic of study. 8 the evaluation method used has been adequate. 9 the use of pedagogical resources of different nature (texts, software, video, etc.) helped to reach the learning goals. 10 the use of open materials in a language other than spanish complicates the understanding of the taught concepts. first of all we must mention that this study involved editing, selecting, creating and reusing multimedia teaching resources with high interest for the development of the subject, and that may be used as open teaching resources in following years. results for variables regarding interest, usefulness and appraisal of the experience were very high. more specifically, results were above 3,8 out of 5 for pedagogical interest (3,97±0,92), usefulness of the resources (3,83±0,96) and positive effect of the experience on learning (4,02± 0,93), as well as for the aid these resources provide towards the learning goals (3,88± 0,99). for these items, students giving a positive opinion (above 3) ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 148 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 was very high: pedagogical interest 95,65%, perception of the experience as positive 94,78% and usefulness of the resources 93,91%. equally noticeable was the fact that most students found the contents followed the subject’s goals (3,81± 0,91). these results seem to support the importance of new open materials as a motivational element in college teaching, in the context of the sports education. the students’ view was moderately positive as regards the organization of the experience (3,41 ± 0,92), the time spent on it (3,44 ± 1,09), the suitability of the evaluation methods (3,56± 0,98) and the technical quality of the resources (3,79 ± 0,95). however, it was made clear that a difficulty rose when the materials were in a language other than spanish (3,21± 1,33). this is probably due to the lack of communication capabilities in other languages. 4. conclusions the filming of didactic sequences for analysis is a very powerful tool to achieve the best possible results in the teaching/learning process of water activity subjects. the experience conducted has improved the teamwork amongst students as well as the professor-student relationship, increasing in-class participation and individual tutoring for the subject. the main gain has been the higher level of learning students have achieved. this experience has forced them to acknowledge the meaning and function of the open resources, as well as the importance of being able to access this information and reusing them as a learning method. this involves deliberation on the possibility of creating resources for this course and being able to make them public so that other teachers can conduct the same experience. ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 149 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 this experience should have been conducted for a small number of students. however, conducting it during the class period means it must be adapted to large groups, which complicates the control of the activity. 5. references alfaro, a.p. (2013). estudio del uso de las tecnologías de la información y la comunicación en la formación permanente del profesorado para la mejora de la práctica docente. tesis doctoral no publicada. departamento de------universidad autónoma de madrid, madrid. . calzada, b.e.i. (2010). la video grabación de la práctica docente, una estrategia para el desarrollo de la docencia reflexiva. a 'cidd: ii congrés internacional de didàctiques 2010'. girona: universitat. recuperado de: http://hdl.handle.net/10256/2883. camerino, o. (2011). evaluación formativa y tecnologías de la información y la comunicación (tic). en v. m. lópez (ed.) evaluación formativa y compartida en educación superior. propuestas, técnicas instrumentos y experiencias. (pp. 127-141). madrid: narcea. de la torre, s., oliver, c., & sevillano, m. l. (coords.) (2008). estrategias didácticas en el aula. buscando la calidad y la innovación. madrid: uned. esteve, o. (2004): la observación en el aula como base para la mejora de la práctica docente”, en d. lagasabaster, y j. m. sierra (eds.) loera, a. (2006). la práctica pedagógica video grabada. méxico: upn. lópez, f. (2006). metodología participativa en la enseñanza universitaria. madrid: narcea. lópez, v. m. (coord.) (2011). evaluación formativa y compartida en educación superior. propuestas, técnicas instrumentos y experiencias. madrid: narcea. rivilla, j., lorenzo, j., rodríguez-romo, g., jiménez, a.c., ramírez, c. y ferro, a. (2011). use of ict in the development of perception and analysis capacity in sports by undergraduates.congreso internacional inted 2011 (international technology, education and development conference). valencia. ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 150 http://hdl.handle.net/10256/2883 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2253 social and technological sciences eissn: 2341-2593 rué, j. (2007). enseñar en la universidad. el eees como reto para la educación superior. madrid: narcea. santos-hermosa, g., ferran-ferrer, n., abadal, e. (2012). recursos educativos abiertos: repositorios y uso. el profesional de la información, 21(2), 136-145. recuperado de http://dx.doi.org/10.3145/epi.2012.mar.03 wiley, d. (2006). on the sustainability of open educational resource initiatives in higher education. ecd. recuperado de http://www.oecd.org/dataoecd/33/9/38645447.pdf ramírez et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 141-151 | 151 http://dx.doi.org/10.3145/epi.2012.mar.03 http://www.oecd.org/dataoecd/33/9/38645447.pdf abstract the aim of this experience has been to analyze the effect on the improvement of the process of education learning, the interest and the perception of the student body that generates the reutilization of an open didactic resource. for that 115 studen... open access to information allows its universal diffusion in a free and open way. the idea of open educational resources (oer) has numerous working definitions. the term "open educational resources" was first adopted at unesco's 2002 forum on the impa... according to santos-hermosa et al. (2012), after elaborating a repository of the open educational resources, claims that their essential characteristics are:  the accessibility in terms of resource’s availability to be located and use in any place or time.  the reusability, the possibility to be used and modified in diverse learning contexts.  the interoperability or the facility to be adapted or interconnected between different hardware, devices or tools.  the sustainability, a proper working despite the change in software, versions, etc.  the metadata, or descriptions which allow its indexation, storage, search and recovery. 2. method  verbal communication: clear, opened, expressive, enthusiast, shy, serious, aggressive, cutting, tax, etc.  tone of voice: effects of the tone of voice, volume, speed, intonation of agreement to the circumstances, etc.  capacity of leadership.  classmates’ answers to the speech of teacher: they answer, take part, are indifferent, pay attention, are opposed, interpret adequately the slogan. etc.  body expression/not verbal communication: natural, agile, enthusiast, projects diverse states of mind, calls with his forms of expression and with his attitudes for the participation of all, rejects, is closed, demonstrates insecurity, coyness, dist...  sensations, feelings and ideas that it generates turn in to an action opposite to the group.  analysis of the swimming skills. table 1. competitions and results of learning of the subject. 3. results table 2. items in the evaluation questionnaire for the oer experience. 4. conclusions multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 92 so similar and yet so different: reasons for online purchasing, a comparative research in colombia and mexico using nonparametric association tests juan carlos rentería-garcía economics, finance, and business administration school, universidad ean, bogotá, colombia mauricio sabogal-salamanca economics, finance, and business administration school, universidad ean, bogotá, colombia yesica mayett-moreno agribusiness and management degree programs, universidad popular autónoma del estado de puebla, puebla, mexico * corresponding author: jcrenteria@universidadean.edu.co; phone: +571 593 6464 ext. 2247 received: 11 november 2018; accepted: 27 september 2019 abstract the reasons for buying online have occupied the literature of consumer behavior on the internet for quite some time. different theories seek to understand the relationships between different variables of purchasing behavior using this channel. however, the literature has not been sufficiently focused on making comparative analyses between countries, especially between emerging and latin american countries. for this reason, it is relevant to carry out a mailto:jcrenteria@universidadean.edu.co https://orcid.org/0000-0003-1061-6039 https://orcid.org/0000-0002-2633-731x https://orcid.org/0000-0002-7585-6060 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 93 comparative study that seeks to understand the differences among the reasons to purchase online. in 2017, a survey with 552 students, professors, and administrative staff in two universities was conducted: one in bogotá, colombia, and the other one in puebla, méxico, using a convenience sampling. we designed a semi-structured questionnaire with seven items, mostly categorical. using nonparametric association tests, the results indicate that in five of the six questions that are asked, there are significant statistical differences in the two countries such as why customers buy on internet, the payment methods, shipment preferences, and the reasons why they would not buy on the internet. the implications for marketing for those companies that are interested in penetrating or improving their participation in these markets, is that they should be aware of their differences and use, depending on the country, adapted strategies. keywords: online shopping behavior, e-commerce, comparative study, digital marketing, nonparametric tests 1. introduction emerging countries in latin america have shown enormous dynamism. according to international data corporation (idc, 2019), during the last six years the e-commerce market has tripled in the region, going from usd $ 30,000 million in 2011 to usd $ 106,000 million in 2017. within this panorama, it is striking that the two countries with the highest growth in electronic commerce were colombia (growing 27.1% in 2017) and mexico (growing 27.3% during the same year). both countries share similar positions in the competitiveness index in the world economic forum and in the doing business report multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 94 of the world bank (schwab, 2017), as well as similarities in terms of their culture according to the dimensions of hofstede (hofstede, n.d.). there is a long tradition in the analysis of the online purchase process and the different variables that motivate it. theories such as the planned behavior of purchase and the theory of adoption of technology are part of an extensive theoretical framework that combines various variables in the analysis. however, there are not so many studies that have analyzed differences in online purchases, especially in countries with similar characteristics, in emerging economies. at the same time, it has been found that there is even less literature for countries in latin america. the marketing areas of companies operating in this region could use comparative studies that seek to analyze these variables in order to design tailored marketing strategies by country. this study develops part of the quantitative analysis of online shopping habits between colombia and mexico that was performed jointly by researchers from universidad popular autónoma del estado de méxico in puebla, mexico, and universidad ean in bogotá, colombia. the fieldwork is part of a survey that was applied in both countries in 2017, where almost all the variables were categorical (nominal), reason why the chi-square and intensity of association tests were used (using the ϕ2 statistic proposed by agresti & liu [1999]), in order to be able to answer the question of whether there are differences by country in six categorical variables chosen from the survey. as will be shown, in most of the questions that were analyzed, there are significant differences between both countries. this document is structured as follows: after a first introductory part, a summary of the different conceptual approaches is developed in order to explain the purchasing behavior on the internet; therefore, it is a description of the methodology, including the types of questions, the sample, and the statistical procedures that are proposed for the analysis. in a later part, the results are presented and discussed, with some conclusions and recommendations from the marketing perspective. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 95 2. literature review this study focuses on understanding the relationships between two countries in terms of online shopping behavior. for this reason, the main contributions of the marketing theory in terms of purchasing behavior are presented below, previously making some basic definitions. 2.1 e commerce according to laudon (2017), electronic commerce (e-commerce) can be defined as the use of the internet, the web, and software applications to do business. it includes digital business transactions that occur between organizations (b2b), individuals (c2c), as well as between organizations and individuals (b2c). it also includes commerce using social networks (s-commerce) and mobile devices (m-commerce). for strauss and frost (2014) there is a difference between e-commerce and e-business, the latter being the use of information technologies within the company’s business model, not necessarily and specifically for online transactions. 2.2 online consumer behavior according to solomon, bamossy, & hogg (2009), consumer behavior encompasses the study of the processes in which individuals or groups select, buy, use, or discard products, services, ideas, and experiences to meet their needs and desires. this behavior is usually studied online and offline. purchasing behavior, one of the branches of study of consumer behavior, influences multiple aspects, and the amount of information online is relevant to consumer decision-making in its steps of identifying and evaluating alternatives. likewise, these authors mention the success of internet sites based on their own design (navigability, information provision, order processing, appropriate customization, and selection of appropriate product), reliability (appropriate description of the product, delivery of the appropriate product in the promised time), security and privacy (security for card payments multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 96 and handling of consumer information), and customer service (response times to customer requests). on the other hand, there is information that users produce, such as product ratings and reviews that influence buying decisions and build trust. previously, when consumers needed information, they turned to marketer-generated sources, looked at third-party certifications, or sought advice from friends and/or relatives in conversations “over the backyard fence” (king, racherla, & bush, 2014). 2.3 planned internet behavior, attitudes, and trust ajzen (1991) proposed the theory of planned behavior. according to this theory, intentions to perform different types of behaviors can be predicted with great precision from the attitudes regarding the behaviors, some subjective norms and also the perceived control of the behavior. under this approach, the consumer becomes a rational actor, and intentions capture the motivational factors that influence behavior, so the stronger the motivation to perform a certain behavior, the more likely it is that such behavior manifests itself. more recently, pavlou & fygenson (2006) proposed that this approach, where intentions predict behavior, could be used accurately for online purchase. he tests the initial model proposed by ajzen for two buying behaviors, getting information and buying from online retailers. the results show the importance of trust and the adoption variables of the technology (perceived utility and perceived ease of use) as the most important beliefs to predict the adoption of e-commerce. likewise, additional features of the website related to delays in downloading, site navigability, and protection of user information, consumer skills, time and money disposition, as well as the value of the product, also resulted predictive for adopting e-commerce in the study that was conducted in the united states. regarding trust and online shopping, the study by kim and peterson (2017), makes a detailed review of 150 empirical studies that analyze this variable in the e-commerce in multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 97 b2c clients ("business to consumer", business that serve the final consumer directly), finding that trust in online transactions has an important relationship with the background of the website (perceived privacy, perceived quality of service) and its consequences (loyalty, repeated purchase intention). in the same direction, the study by oliveira et al. (2017), found a strong relationship between high confidence and high intention to buy online, modeling trust in its three dimensions: competence (ability of the company to fulfill promises they made to customers), integrity (which suggests that the company acts in a consistent, reliable, and honest way), and benevolence (which is the ability of a company to have the client's interest above its own interest). the risk in the reasons to buy online can be mediated by culture. hofstede's model on the six different dimensions of culture, which may be different between countries, highlights one of its dimensions: risk aversion, which is defined as the degree to which members of a culture feel threatened by the uncertainty (hofstede, hofstede, & minkov, 1991). based on the above, the study by yu, hudders, and cauberghe (2018) reveals a negative relationship between the level of risk aversion in a country and the level of advantage in making transactions online. 2.4 unified theory of acceptance and use of technology a more recent and general approach regarding the use of internet for purchases is derived from the planned behavior model made with the unified theory of acceptance and use of technology (venkatesh et al, 2003), which analyzes the theory of rational action, the theory of technology acceptance, the motivational model, and the theory of planned behavior. after testing the four models and their variables, he proposes a model that improves the statistical effectiveness of the previous ones, and that is based on the expectations of the technology's performance (degree in which individuals believe that using technology or the system will improve the performance of their work), the expectations of effort of using that technology (perceived degree of effort in the use of the system), the social influence (the multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 98 degree to which the individual believes that other persons, who are important to him, think that he should use the technology), and the facilitating conditions for the use of that technology (beliefs of the person about the existence of a support infrastructure). although these four are determinants of the technology use behavior, they are mediated by variables such as gender, age, experience, and the voluntary nature of the use of technology (the person's belief on whether the use of technology will be voluntary). in line with this study, singh and matsui (2018) have used this model to see the effect of long tail and the confidence in buying online, validating the usage of the model given the fact that purchasing online is a type of activity of technology adoption. the study states, for the purchase of books and airline categories in urban areas in japan, that trust, again, is an important factor in the intention to purchase on internet, and long tail is important for the purchase of physical products, but not for virtual products. 2.5 beliefs, attitudes, and intentions the study by izogo & jayawardhena (2018), establishes a model of determinants of the online shopping process, which is composed of three parts: beliefs, attitudes, and intentions. in the case of beliefs, there are some determinants related to the product or service, such as the seller's prices on the internet, the beliefs about the handling of complaints, and the quality of the product. there are other determinants related to the website, which are related with convenience, functionality, relational experience, and the enjoyment of the shopping experience on the company's website. these determinants form the dimension of buying attitudes and shape the online shopping experience. finally, the dimension of intention, which is related to the beliefs that generate attitudes towards the website, is determined by the variables trust, e-word of mouth, or ewom1 (including advice from the client and/or the seller, inquiry, recommendation, and dissuasion), the 1 the authors refer to word of mouth when they talk of voice-to-voice power, or any comment made by a client about a brand, that is available through the internet for other consumers or companies. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 99 regrets or withdrawals on the products, the internal response to failures in the service, all in turn determining the external response of the client to failures in the service. 2.6 other motivators that lead to choose e-commerce salazar, mondaca, and cea (2018) set out a model in which the sociodemographic factors (gender, age, ethnicity, educational level, marital status), economic factors (level of income, access to a credit card), familiarity factors (frequency of internet use and internet use skills), and trust and familiarity with electronic commerce (frequency of purchase, experience with use and personal data, and experience of payment by credit card), are the factors that influence the purchase online. in the case of chile, this study suggests that the variables that determine the purchase through internet are gender, age, educational level, access to credit card, frequency of use, ability of using data and not having a previous negative experience delivering personal data over the internet. although section five of the survey goes in the same direction of this model, it does not concentrate on all demographic variables; it only gives importance to the country, and in turn includes some other variables that are not contemplated by salazar et al. (2018), such as advantages, delivery conditions and problems, and bad experiences in previous purchases. the study by fernández-poyatos and papí-gálvez (2017), presents the behavior of buying tourism products online, where the level of income and educational level explain or predict the online purchase behavior for europe, while other variables such as gender and age are not significant. even in the case of purchase between consumers or c2c, one of the few studies on this type of transaction is the study by mariani et al. (2017), which shows that social influence and trust are key predictors of the intention to purchase online in this modality. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 100 sobia tufail et al. (2018), come up with a strong and positive relationship between the lifestyle and personality of consumers and the intention to buy online, mediated by the attitudes they have towards brands. 2.7 other valued aspects and preferred forms of payment on the internet dixit (2018) points out that price and quality are the most important factors that influence the purchase behavior of buyers, whether they buy online or not. proper advertising, variety, effective communication about discounts, and proper post-sale service were also important for customers to decide to buy online. devi (2017) shows that for consumers in india, the preferred means of payment are mainly credit card (36.67%) and payment on delivery (27.3%). this same author finds that the attractive prices (34.0%) and the consistency of the channel (23.0%) were the most important reasons to buy in such population. the literature that was reviewed has few options for comparison with other studies on aspects that are related to online shopping, since most of them refer only to a single market, with few exceptions (fernández-poyatos & papí-gálvez, 2017; kim & peterson, 2017). likewise, although there are studies on emerging markets (devi, 2017; dixit, 2018; izogo & jayawardhena, 2018) and there are some from latin america (salazar et al., 2018), there is a necessity for more comparative studies from emerging countries that examine variables related to online shopping reasons. the study below seeks to fill part of that gap. rather than defining what kind of theoretical orientation this article should follow, the state of the art leads us to the need to understand the different variables that are commonly used to study online shopping behavior. some of these variables were condensed in the instruments that were elaborated (such as those related to trust, perceived risk, prices, custom or previous purchases, customer service, and means of payment), as well as others that are not so frequently reviewed in the literature but were included to have some other multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 101 sources of information (delivery or return conditions, online shopping problems, online shopping convenience). finally, some other state of the art variables related to some attitudes and beliefs about online purchases were not included in this research, because they were not considered as relevant for the present analysis (effort, social influence, and ewom, for example). 2.8 similarities between mexico and colombia 2.8.1 macroeconomic and competitiveness data according to schwab (2017), mexico and colombia have similar positions in the ranking of medium competitiveness in internet users and broadband subscriptions. the mexican economy showed little more macroeconomic performance than the colombian one, but both belong to the group of middle-income countries, located as countries that are guided by efficiency according to the pillars of the index of competitiveness (table 1). table 1 – main macroeconomic and competitiveness internet related data variable colombia méxico population (millions) 48,7 122,3 gross domestic product – gdp (millions of us$) 282,4 1.046 gdp per capita (us$) 5.792 8.554 position in the competitiveness ranking 66 51 internet users as a proportion of the population (position in the ranking) 72 68 bandwidth measured as kbps per user (position in the ranking) 32 82 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 102 subscriptions of mobile broadband width (position in the ranking) 92 70 source: schwab (2017). 2.8.2 internet sales the growth of sales on internet doubled in 2017 and it is expected to remain so in 2018 in both countries. according to international data corporation (idc, 2019), sales on internet in colombia grew by 27.1% in 2017 and it is expected to grow 24.5% in 2018. in mexico, sales on internet grew by 27.3% in 2017 and it is expected to grow by 28.4% in 2018. 2.8.3 hofstede culture dimensions in terms of the dimensions of culture measured by the six-dimensional model of geert hofstede (first presented in hofstede, 1984), the two countries present some similarities that are worth highlighting (table 2). table 2 – culture dimensions in the hofstede model – comparison between colombia and mexico variable colombia méxico distance of power (degree to which the less powerful members of a society accept that power is unequally distributed) 67 81 individualism (preference for close ties, in order to take care of oneself and one’s family) 13 30 masculinity (preference for achievement, heroism, assertiveness, and material achievement) 64 69 risk aversion (degree of discomfort in situations of risk) 80 82 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 103 long term orientation (low scores mean more traditional societies, higher scores more pragmatic societies that believe in education as a way to prepare for the future) 13 24 indulgence (degree of importance to variables of life enjoyment and fun) 83 97 source: hofstede, (n.d.). both countries have quite similar variables, despite the fact that mexico scores higher than colombia in all of them. they are traditionalist countries, oriented to achievement, with high aversion to risk, and with a high orientation to indulgence and fun. 3. methodology 3.1 design of the survey and components in order to collect primary data, we designed a structured survey, with open-ended closedended, likert scale, as well as multichotomous questions. the questionnaire included 31 questions divided into six sections. the following is a description of the type of questions that are included in each of the sections (annex 1): ● i online purchase process: section 1 contains questions of purchase habits in nominal scale to define frequency of online and offline purchase by category, preferred days, and preferred hours; there is also a question in an ordinal scale on purchase frequency by category, and one in the likert-type scale on purchase preference per hour. ● ii influence factors: includes a likert question on the grade of influence of family and friends in the purchase decision. ● iii use and abandonment of web pages: this section includes a question in the likert scale that seeks to measure agreement or disagreement on reasons to abandon multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 104 a purchase or not to buy online again; another question in the same scale that measures the importance of the characteristics of the purchase process, and two others in the same scale on the degree of ease of the purchase for the first time and on repeated purchases. ● iv online purchase and advertising: this section has two questions in a nominal scale on internet access habits and advertising recall, a likert-scale question on how to access a web page in terms of frequency, and another three in the same scale that measure the degree of perceived utility and preference over different types of advertising media. there is also a question in the same scale that collects some insights from the qualitative research in relation to advertising, in order to measure agreement or disagreement towards them. ● v reasons to buy online: this section covers two questions of nominal scale related to the advantages and disadvantages that are perceived when buying online, as well as valuations on the payment process, a question in ordinal scale to measure the type of preferred payment method, questions in an ordinal scale to assess the degree of importance of the conditions of delivery and/or return of the product, and three questions in a nominal scale to measure difficulties when buying online, problems experienced with past purchases, and reasons for not buying on internet. ● vi demographic data: it consists of information in nominal scales and proportion scales on marital status, age, gender, education, occupation, and income. given that this document will concentrate only on the last two sections, these two are the only ones included in the annex (annex one). for the general design of the survey, a preliminary review of the construction of each of the questions was made by the researchers involved in the entire project, as well as a pilot test among the researchers to review inconsistencies and confusions. the construction of the scales was based on hair, bush, and ortinau (2010), for a foundation from the quantitative research. however, aspects such as the purchase process steps, the characteristics that constitute a purchasing process, and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 105 the advertising methods were based on chaffey and ellis-chadwick (2016) and laudon, k. & traver, c. (2017). specifically, for section five, we used questions that are included in the report on online purchase in colombia, made for the colombian chamber of electronic commerce by the cocktail analysis (2013). 3.2 sampling plan the data that is analyzed in this document correspond to a fieldwork that was carried out in colombia, in the city of bogotá, during the months of june and july of 2017, and in september and october of 2017 in the city of puebla, mexico. a total of 552 surveys were collected, 257 in colombia and 295 in mexico. the surveys were administered face-toface, using a convenience sample to students and teachers of universidad popular autónoma de puebla (upaep) in puebla, mexico, and of the universidad ean in bogotá. 3.3 description of the variables and characterization of the data the collected data are part of a comparative study on consumer behavior and attitudes towards online shopping, among other aspects. related with online shopping, the survey included seven questions, and other nine related to demographic data. the 16 resulting variables respond to the following typology (table 3). table 3 – type of variables that were used group type type of answer / number of different choices to choose from consumer behavior regarding reasons to buy online (section v) advantages of buying online categorical nominal multiple / 8 options valued aspects at the time of payment categorical nominal multiple / 4 options multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 106 preferred forms of payment categorical nominal multiple / 7 options delivery conditions categorical nominal multiple / 6 options frequency in bad experiences categorical ordinal unique option / 4 options type of negative experience categorical nominal multiple / 8 options reasons for not buying online categorical nominal multiple / 11 options demographic data (section vi) gender categorical nominal unique option / 3 options age (years) quantitative does not apply marital status categorical nominal unique option / 5 options children categorical nominal unique option / 2 options (dichotomus) number of children quantitative does not apply education level categorical nominal unique option / 8 options multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 107 current occupation categorical nominal unique option / 4 options currently studying categorical nominal unique option / 2 options (dichotomus) amount of monthly internet purchases categorical ordinal unique option / 6 options (ranks) source: prepared by the authors based on the information in the survey, section vi. 3.4 statistical model applied a statistical test can be defined as a random variable that is calculated with sample data and used in hypothesis testing. these statistical tests are used to determine if a null hypothesis can be rejected (hair et al., 2010) and have, first of all, a theoretical distribution. in this order of ideas, the test must be compared with the theoretical distribution value, verifying that the test value does not exceed the theoretical value. in case that the value of the statistic exceeds the theoretical value, then it can be affirmed that there is statistical evidence that allows rejecting the null hypothesis or main hypothesis and accepts the alternative hypothesis (rodríguez, 2012). the above requires determining a level of significance that allows us to control an eventual "type one error", that is, rejecting the main hypothesis when it should have been accepted. in this case, the level of significance will be set at 5% (α = 0.05); therefore, the confidence level will be 95% (1-α). for this investigation, the hypotheses to be statistically proven were: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 108 ho the probabilities distribution between the samples from mexico and colombia is the same. h1 the probabilities distribution between the samples from mexico and colombia is different. where h0 is going to be referred to as the main hypothesis and h1 as an alternative hypothesis. in the case of the variables that are presented in this study, sections 5 and 6 have categorical variables, which do not allow the calculation of a statistical test using proportion data. agresti and liu (1999) propose a special test for a case like this, where data is obtained for a categorical variable with multiple selection possibilities and it is necessary to know if there is an association or not between the set of sub-variables that are derived from each response option and the groups or segments, in this case the countries colombia and mexico. from the marketing perspective, being able to test if this type of association is absolutely important, for example, to determine if a certain communication campaign can be applied without great variations in one segment or another when high association is proven, or for understanding that it is necessary to use a completely different approach in terms of the marketing mix when the degree of association is minimal. the procedure described in rodríguez (2012) based on agresti and liu (1999) consists of: first, obtaining the observed and expected values for choosing and not choosing each of the response options in the different segments, in our case, the countries; and second, obtaining a statistical test according to the following formula: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 109 𝜑𝜑2 = � 𝑐𝑐 𝑗𝑗=1 � 𝑟𝑟 ℎ=1 � (𝑂𝑂ℎ𝑗𝑗1 − 𝐸𝐸ℎ𝑗𝑗1)2 𝐸𝐸ℎ𝑗𝑗1 + (𝑂𝑂ℎ𝑗𝑗0 − 𝐸𝐸ℎ𝑗𝑗0)2 𝐸𝐸ℎ𝑗𝑗0 � where 𝑂𝑂ℎ𝑗𝑗1 represents the observed value and 𝐸𝐸ℎ𝑗𝑗1 represents the expected value in group h who has chosen category j. similarly, 𝑂𝑂ℎ𝑗𝑗0 represents the observed value and 𝐸𝐸ℎ𝑗𝑗0 the expected value for group h, which did not choose category j. φ2 has a chi square theoretical distribution 𝑋𝑋𝑐𝑐(𝑟𝑟−1) 2 with c(r-1) degrees of freedom. in this equation, c represents the number of categories, while r represents the number of groups (rodríguez, 2012). the third step is to compare the value that was obtained for φ2 with the value of its theoretical chi-square distribution (x2) with c(r-1) degrees of freedom, where c is the number of response options and r the number of segments. in this way, if the statistical test φ2 is greater than x2, then h1 is rejected and h0 is accepted (alternate hypothesis), which means that preferences from one segment to another are significantly different (rodríguez, 2012). to test the previous procedure, the spss© version 25 was used to define multiple response sets for each of the survey questions with this characteristic. then, contingency tables that contained both the observed and the expected values were drawn up. these contingency tables were exported to excel. in the information analysis section, the detailed calculation of φ2 is shown for the categorical variable that describes the reasons of preference when buying on the internet. on the other hand, the critical value of x2 was obtained using the tables of critical values for the chi-square distribution. 4. information analysis 4.1 demographic of the respondents in terms of gender, the percentages of respondents are similar, with more men in colombia and more women in mexico (table 4). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 110 table 4 – distribution of respondents by gender gender colombia mexico total frequenc y percentag e frequenc y percentag e frequenc y percentag e male 135 45,76% 136 52,9 % 271 49,09% female 154 52,20% 117 45,53% 271 49,09% not respondin g 6 2,03% 4 1,56% 10 1,81% total 295 100% 257 100% 552 100,00% source: prepared by the authors based on the information from the survey, section vi. in terms of marital status, 88% in both cases is single, while between 5% and 7% are married, the divorced respondents represent less than 1%, and those in consensual union represent between 2% and about 3% (table 5). table 5 distribution of respondents by marital status marital status colombia mexico total frequenc y percentag e frequenc y percentag e frequenc y percentag e single 260 88,14% 227 88,33% 487 88,22% married 16 5,42% 18 7,00% 34 6,16% divorced 2 0,68% 1 0,39% 3 0,54% multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 111 consensual 10 3,39% 6 2,33% 16 2,90% not responding 7 2,37% 5 1,95% 12 2,17% total 295 100% 257 100% 552 100% source: prepared by the authors based on the information from the survey, section vi. when asked if they have children, 89% of the respondents in the two countries said they did not have, while 5% on average responded having them (table 6). table 6 distribution of respondents by having or not children option colombia mexico total frequenc y percentag e frequenc y frequenc y percentag e percentag e yes 17 5,76% 14 5,45% 31 5,62% no 265 89,83% 229 89,11% 494 89,49% not responding 13 4,41% 14 5,45% 27 4,89% total 295 100,00% 257 100,00% 552 100,00% source: prepared by the authors based on the information from the survey, section vi. in terms of educational level, less than 2% in the two countries said they had an elementary or basic secondary education, the majority said they had a full high school education (50 to 57%), university education (between 29 and 39%), while graduate education (diploma and master degree) only approximately 7% in both cases (table 7). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 112 table 7 distribution of respondents by education level (frequencies / percentages) education level colombia mexico total frequenc y percentag e frequenc y frequenc y percentag e frequenc y elementary school 2 0,68% 1 0,39% 3 0,55% junior high 5 1,69% 1 0,39% 6 1,09% high school 170 57,63% 128 50,20% 298 54,18% university 88 29,83% 100 39,22% 188 34,18% diploma 22 7,46% 9 3,53% 31 5,64% master degree 2 0,68% 8 3,14% 10 1,82% not responding 6 2,03% 8 3,14% 14 2,55% total 295 100% 255 100% 550 100% source: prepared by the authors based on the information from the survey, section vi. as it can be seen, the two samples in the demographic variables of the two countries have similar structures in terms of the participation of each of the variables, which helps support the use of the φ2 statistic, presented by agresti and liu (1999). 4.2 responses related with payment entering into the different parts of section v of the survey, it can be observed that the option of receiving a credit card is highly valued (close to or greater than 80%), while security is multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 113 in the second place. in the case of accepting the payment against delivery, the positive valuation is higher in the case of the colombian clients compared to the mexican clients (table 8). table 8 regarding payment, what do you value when buying a product or service online? select the ones that apply relative frequencies (percentages) what i value what i do not value colombi a mexic o colombia mexico to receive a credit card 80% 79% 20% 21% it is a secure website 68% 72% 32% 28% to accept payment against delivery 62% 41% 38% 59% other 2% 4% 98% 96% source: prepared by the authors based on the information from the survey, section v. when asked about what means of payment they prefer, colombian consumers replied that their first option was payment against delivery, followed by credit card and debit card, while mexican consumers prefer credit card in the first place, secondly the debit card, and in the third place payment in the physical store, even above payment against delivery, which is in the fourth place. payment with bank deposit2 is also different in the two countries; in the case of colombia, it is 29% in the first option while for mexico it is 13% (table 9). 2 in latin american countries, the payment option through bank deposit is very common. in this means of payment, the costumer receives a bill that must be printed, and then he/she needs to go to a bank, do the payment and send a scanned copy of the payment receipt with the seal of the bank to the retailer's e-mail. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 114 table 9 what means of payment do you prefer when paying for your purchases online? indicate which of the following are your first and second options relative frequencies in percentage first option second option don’t know / not responding col mex col mex col mex credit card 44% 53% 38% 27% 18% 20% payment against delivery 55% 26% 26% 33% 19% 41% debit card 43% 45% 35% 31% 22% 24% payment in the physical store of the brand 29% 28% 36% 32% 34% 39% payment in third-party intermediaries 28% 18% 40% 34% 32% 48% bank deposit 29% 13% 43% 35% 28% 52% other 3% 4% 1% 4% 96% 92% source: prepared by the authors based on the information from the survey, section v. 4.3 responses related to the conditions of delivery and return, disadvantages and reasons for not buying regarding the conditions of delivery and/or return, the most important aspects for the respondents were to choose the day and time of delivery (39% in first and second place for colombia, 38% for mexico) and a promise of clear delivery (38% and 29% respectively), followed by free shipping (27% and 29% respectively), as seen in table 10. there is a higher proportion of colombian customers who value less the pick up of the product in the multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 115 store (45%) than surveyed mexican clients. as it can be seen, there are differences between the numbers among the countries, although it is not yet evident that these are statistically significant. table 10 regarding the conditions of delivery and/or return, what is the most important when buying a product online? order the following options, number one (1) being the most important relative frequencies in percentage option 1 (most important) 2 3 4 5 6 (less important) col mex co l me x co l me x co l me x co l me x col mex easy store pick up 5% 11% 6% 4% 2% 10% 9% 7% 17% 12% 45% 35% choose day and time of delivery 25% 19% 14% 18% 16% 11% 12% 12% 15% 15% 3% 5% know the status of the order 10% 11% 21% 19% 21% 15% 16% 14% 12% 16% 6% 5% clear delivery promise 19% 13% 19% 16% 18% 16% 15% 18% 11% 9% 4% 7% ease to make changes or refunds 19% 7% 13% 11% 14% 18% 18% 17% 15% 17% 5% 7% free shipping 18% 20% 9% 9% 14% 10% 12% 11% 13% 10% 18% 18% source: prepared by the authors based on the information from the survey, section vi. when asked if they have experienced any problems with online purchases, the highest positive responses are delays in delivery (44% for colombia and 39% for mexico, where multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 116 the percentages differ the most) and that they received a different product (33 % and 25% respectively), with similar yes and no proportions in almost all the answers (table 11). table 11 have you ever experienced any problem or difficulty with purchases online? check only one option. yes no colombia méxico colombia méxico i did not receive the product and the price was charged 14% 22% 86% 78% the store charged a price that is higher than the one that is advertised 14% 16% 86% 84% i received a defective product 29% 25% 71% 75% problems with the online service 21% 20% 79% 80% complications for returning the product 18% 16% 82% 84% i received a different product 33% 25% 67% 75% delay in delivery 44% 39% 56% 61% product did not meet expectations 25% 25% 75% 75% source: prepared by the authors based on the information from the survey, section vi. most respondents answered that they do not buy online because they prefer to see the physical product (21% in colombia and 25% in mexico), and for lack of habit (25% and 23% respectively) as seen in table 12. the percentages in which they differ the most are in previous bad experience (13% in colombia, 18% in mexico), i want the product immediately (6% in colombia, 15% in mexico), i do not trust the means of payment (13% in colombia and 22% in mexico), and i am concerned about sharing personal information (13% in colombia and 25% in mexico). table 12 why don’t you buy online? you can choose several options. yes no colombia méxico colombia méxico previous bad experience 13% 18% 87% 82% i want the product immediately 6% 15% 94% 85% i prefer the advice of stores 9% 15% 91% 85% lack of habit 25% 23% 75% 77% multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 117 i haven’t had the necessity to buy online 13% 17% 87% 83% i do not trust the means of payment 13% 22% 87% 78% i prefer to see the physical product 21% 25% 79% 75% i am concerned about sharing personal info 13% 25% 87% 75% i do not see advantages when buying online 2% 8% 98% 92% i do not know how to buy on the internet 1% 5% 99% 95% others 2% 4% 98% 96% source: prepared by the authors based on the information from the survey, section vi. 4.4 advantages of online purchasing for respondents in both countries, the main advantage of buying through the internet is the "convenience of buying from their home or office", and they also agree on the second and third main advantages when doing an analysis ordering the percentages from highest to lowest. on the other hand, they disagree on the advantages from the fourth position to the seventh, being the option "possibility of reviewing comments from third parties" the one with the most disagreement, which was considered an advantage for 58% of the respondents in colombia and 44% of respondents in mexico (table 13). table 13 what are the main advantages for you of buying online? select the ones that apply. relative frequencies in percentage by country its and advantage is not an advantage colombi a méxic o colombia méxic o convenience of buying from home or office 91% 86% 9% 14% convenience of buying at any time 82% 71% 18% 29% possibility of finding better prices 55% 53% 45% 47% possibility of accessing brands that are not available locally 71% 70% 29% 30% possibility of comparing 55% 44% 45% 56% multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 118 possibility of reviewing comments from third parties 58% 44% 42% 56% possibility of buying before the product is available in stores 26% 33% 74% 67% other 3% 4% 97% 96% source: prepared by the authors based on the information from the survey, section vi. the frequency tables in sections (b) to (d) allow us to visualize, from the marketing point of view, preferences in the means of payment, conditions of delivery, reasons for buying or not buying through the internet, as well as the motivators that lead a person to choose the online shopping channel, his order of preference, and if there are important differences in that order between the respondents in both countries. however, the frequency tables do not help to determine if the country has any influence on the motivations for choosing the online sales channel, or the motivators to choose this channel and its differences by country. if the objective of the research is that one, then, statistical tests of association for categorical variables are required, which is exposed in the next section. 4.5 statistical significance for the advantages of internet purchasing discriminated by country the test statistic φ2 was calculated for each of the questions that were analyzed in parts b to d of this section, taking into account that the criterion of differentiation of the two samples was the countries. for each question, the observed value and the expected value by answer option were calculated, and then, the results of each response option by country were added (annex 2). as it is derived from the previous procedure, the statistical test φ2 is greater than the critical value of x2 with 8 degrees of freedom, which rejects the main hypothesis in favor of the alternative hypothesis, that is, the distribution of probabilities between mexico and colombia is different, or that the different variables related to reasons to buy or not to buy, conditions to pay, preferred means of payment, and terms of delivery and/or refund, are multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 119 significantly different between colombia and mexico. the only variable that did not obtain significant differences by country is the variable problems with purchases over the internet, where the value of the test was lower than that of x2. what can be seen in the results of table 14 is the demonstration of statistically significant differences, but which had already been presented in the previous sections of results (table 14). table 14 result of the statistical tests for all the categorical variables with multiple responses set of variables φ2 χ2 d. of f. conclusion reasons for buying on internet 33,6248 15,507 3 8 h1 is accepted; there are significant differences between the groups that were analyzed. conditions for payment 25,3660 9,4880 4 h1 is accepted; there are significant differences between the groups that were analyzed. preferred means of payment for purchases on internet 123,678 5 14,067 0 7 h1 is accepted; there are significant differences between the groups that were analyzed. terms for delivery and/or return 85,9924 12,592 0 6 h1 is accepted; there are significant differences between the groups that were analyzed. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 120 problems with internet purchases 14,1608 15,507 3 8 ho is accepted; there are no significant differences between the groups that were analyzed. reasons for not buying on internet 60,4101 19,675 0 11 h1 is accepted; there are significant differences between the groups that were analyzed. source: prepared by the authors based on the information from the survey, section vi. 5. conclusions and implications for marketing in terms of purchasing behavior on the internet, for the majority of the categorical variables that were analyzed (5 out of 6), the differences between the two countries are significant, with only one variable: "problems with internet purchases". in this case, the answers in proportions show very similar percentages, being the delay in delivery, followed by "received a different product" the main problem in both countries. regarding the conditions for payment, the main difference is the greater relative valuation of the colombian consumers that were surveyed to the payment against delivery. this is reinforced by the payment preference, in which the main means of payment preferred by colombians who were surveyed is payment against delivery, while in the case of mexicans it is a credit card. in terms of delivery and/or return conditions, the most important for colombian customers is to choose the day and time, while for mexicans is free shipping. the least important for colombians is easy to pick up at the store, as well as for mexicans, but with a difference in the proportion of responses. about the reasons to buy or not to buy, which also had significant differences, the main ones are in the response to the phrase “i am concerned about sharing personal information”, multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 121 being the most important for mexico, and "i do not trust the forms of payment", very important for mexico as well. in terms of the advantages and disadvantages of buying online, the most important variables examined for both countries are convenience buying from home or work, convenience buying at any time, and accessing to brands that are not found locally. however, the proportion of colombian consumers who consider that these are advantages in relation to mexican consumers is significantly higher. the previous differences are very important from the marketing perspective since, although colombia and mexico are relatively similar in terms of their culture (hofstede, n.d.), the behavior of the consumer when buying on the internet differs between both, at least in the tests made with the φ2 statistic for the samples that were examined. therefore, those companies that are interested in penetrating or improving their participation in these markets and that have operations in both countries must be aware of their differences and adapt their strategies accordingly. it is also important to say that e-commerce eliminates trade barriers between markets; however, its implementation at multinational scales cannot be taken lightly because, although two countries share similar cultural and economic characteristics, consumer expectations and behaviors may not be so similar. further studies could examine the differences in online purchasing behavior of countries with similar cultures, specifically in latin america, with the usefulness of being able to serve multinational companies that serve a latin american market (so called “multilatinas”), or online platforms with regional reach that can define and adjust the marketing strategy, depending on what each consumer in each country considers most relevant. on the other hand, the limitation of this study is that it was done with a non multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 122 probabilistic sample for convenience, in only two cities and urban areas, which prevents the generalization of the results to the population under study. 6. references agresti, a. & liu, i. m. (1999). modeling a categorical variable allowing arbitrarily many category choices. biometrics, 55(3), 936-943. retrieved from https://doi.org/10.1111/j.0006-341x.1999.00936.x ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision process, 50(2), 179–211. retrieved from https://doi.org/10.1016/07495978(91)90020-t chaffey, d. & ellis-chadwick, f. (2016). digital marketing (6th ed.). mexico: pearson. devi, m. n. (2017). consumer buying behavior towards online shopping in thoothukudi district. outreach journal, 10, 23–33. retrieved from http://www.voccoutreach.in/vol10/4.pdf dixit, g. s. (2018). a study of the factors affecting online shopping of products in indore city. shabd braham international research journal of indian languages. 6(4), 88-94. retrieved from http://shabd-braham.com/shabdb/archive/v6i4/sbd-v6-i4-sn18.pdf fernández-poyatos, m. d. & papí-gálvez, n. (2017). eturismo: estudio de criterios de segmentación clásicos del usuario online que compra por internet. ícono, 15 (2), 168189. retrieved from https://doi.org/10.7195/ri14.v15i2.1066 hair, j. f., bush, r. p., & ortinau, d. j. (2010). investigación de mercados en un ambiente de información digital. mexico: mcgraw hill. https://doi.org/10.1111/j.0006-341x.1999.00936.x https://doi.org/10.1016/0749-5978(91)90020-t https://doi.org/10.1016/0749-5978(91)90020-t http://www.voccoutreach.in/vol10/4.pdf http://shabd-braham.com/shabdb/archive/v6i4/sbd-v6-i4-sn18.pdf https://doi.org/10.7195/ri14.v15i2.1066 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 123 hofstede (n.d.). compare countries, hofstede insights, helsinki, finlandia: hofstede insights. retrieved from https://www.hofstede-insights.com/product/comparecountries/ hofstede, g. (1984). cultural dimensions in management and planning. asia pacific journal of management, 1(2), 81-99. retrieved from https://doi.org/10.1007/bf01733682 hofstede, g., hofstede, g. j., & minkov, m. (1991). culture and organizations: intercultural cooperation and its importance for survival. new york: mcgrawhill. izogo, e. e. & jayawardhena, c. (2018). online shopping experience in an emerging eretailing market. journal of research in interactive marketing, 12(2), 193-214. retrieved from https://doi.org/10.1108/jrim-02-2017-0015 international data corporation. (2019). retrieved from http://mx.idclatin.com/ kim, y. & peterson, r. a. (2017). a meta-analysis of online trust relationships in ecommerce. journal of interactive marketing, 38, 44-54. retrieved from https://doi.org/10.1016/j.intmar.2017.01.001 king, r. a., racherla, p., & bush, v. d. (2014). what we know and don’t know about online word-of-mouth: a review and synthesis of the literature. journal of interactive marketing, 28(3), 167-183. retrieved from https://doi.org/10.1016/j.intmar.2014.02.001 laudon, k. & traver, c. (2017). e-commerce 2017 business, technology, society (13th ed.). usa: pearson. mariani, m., muhamad, a., & lamarauna, i. (2018). the impact of social influence and trust on customer-to-customer online shoppers’ purchase intention: an empirical study https://www.hofstede-insights.com/product/compare-countries/ https://www.hofstede-insights.com/product/compare-countries/ https://doi.org/10.1007/bf01733682 https://doi.org/10.1108/jrim-02-2017-0015 http://mx.idclatin.com/ https://doi.org/10.1016/j.intmar.2017.01.001 https://doi.org/10.1016/j.intmar.2014.02.001 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 124 in indonesia. gstf journal on computing, 5(3), 1-6. doi: 10.5176/22513043_5.3.372 oliveira, t., alhinho, m., rita, p., & dhillon, g. (2017). modelling and testing consumer trust dimensions in e-commerce. computers in human behavior, 71, 153-164. https://doi.org/10.1016/j.chb.2017.01.050 pavlou, p. & fygenson, m. (2006). understanding and predicting electronic commerce adoption: an extension of the theory of planned behavior. mis quarterly, 30(1), 115-143. https://doi.org/10.2307/25148720 rodríguez, w. h. (2012). respuesta múltiple en datos categóricos. una prueba de hipótesis. pre-impresos estudiantes, (7), 14. retrieved from https://doi.org/10.2307/25148720 salazar, c., mondaca, c., & cea, j. (2018). comercio electrónico en chile: ¿qué factores inciden en la decisión de compra? revista academia y negocios, 4 (1), 1-14. retrieved from https://dialnet.unirioja.es/descarga/articulo/6599114.pdf schwab, k. (2017). the global competitiveness report 2017 – 2018. world economic forum. retrieved from http://www3.weforum.org/docs/gcr20172018/05fullreport/theglobalcompetitivenessreport2017%e2%80%932018.pdf singh, m., & matsui, y. (2018). how long tail and trust affect online shopping behavior: an extension to utaut2 framework. pacific asia journal of the association for information systems, 9(4). https://doi.org/10.17705/1pais.09401 sobia tufail, h., afzal humayon, a., shahid, j., murtza, g., luqman, r., & riaz, h. (2018). impact of life style and personality on online purchase intentions of internal auditors through attitude towards brands. european online journal of natural and https://doi.org/10.1016/j.chb.2017.01.050 https://doi.org/10.2307/25148720 https://doi.org/10.2307/25148720 https://dialnet.unirioja.es/descarga/articulo/6599114.pdf http://www3.weforum.org/docs/gcr2017-2018/05fullreport/theglobalcompetitivenessreport2017%e2%80%932018.pdf http://www3.weforum.org/docs/gcr2017-2018/05fullreport/theglobalcompetitivenessreport2017%e2%80%932018.pdf https://doi.org/10.17705/1pais.09401 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 125 social sciences: proceedings, 7(3), 72-83. retrieved from http://europeanscience.com/eojnss_proc/article/viewfile/5479/2580 solomon, m., bamossy, g., askegaard, s., & hogg, m. (2009). consumer behavior: a european perspective (4th ed.). usa: pearson. strauss, j. & frost, r. (2014). e-marketing (7th ed.). usa: pearson. the cocktail analysis. (2013). la compra online en colombia. diciembre de 2013. retrieved from https://www.ccce.org.co/sites/default/files/biblioteca/the%20cocktail%20analysis. %20compra%20online%20en%20colombia.pdf venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). "user acceptance of information technology: towards a unified view". mis quarterly, 27(3), 425-478. retrieved from https://doi.org/10.2307/30036540 yu, s., hudders, l., & cauberghe, v. (2018). selling luxury products online: the effect of a quality label on risk perception, purchase intention and attitude toward the brand. journal of electronic commerce research, 19(1), 16-35. retrieved from http://www.jecr.org/sites/default/files/19_1paper2.pdf http://european-science.com/eojnss_proc/article/viewfile/5479/2580 http://european-science.com/eojnss_proc/article/viewfile/5479/2580 https://www.ccce.org.co/sites/default/files/biblioteca/the%20cocktail%20analysis.%20compra%20online%20en%20colombia.pdf https://www.ccce.org.co/sites/default/files/biblioteca/the%20cocktail%20analysis.%20compra%20online%20en%20colombia.pdf https://doi.org/10.2307/30036540 http://www.jecr.org/sites/default/files/19_1paper2.pdf multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 126 annex 1 – survey on habits / on-line purchase process, and effects of advertising on said medium sections five and six section vreasons for online purchase: in this section you will find questions related to the reasons for preferring or discarding the internet as a means to purchase products and services. 3. in your opinion, what are the main advantages of buying online? select the ones that apply option a) convenience of purchasing from my home or place of work. b) convenience of purchasing at any time. c) possibility of accessing better prices than in physical stores. d) access to brands, stores, products, or services that are not available near my place of residence or work. e) being able to compare different suppliers, web stores, prices, products ... f) being able to review the comments of others who have already purchased products or services that are similar to the one i am looking for. g) being able to buy something before it goes out to the market in physical stores h) other. which one?____________________ 4. regarding payment, what do you value the most when buying a product or service online? select the ones that apply. option a) that they receive credit or debit card, or the payment means of my preference. b) that the website seems safe (padlock icon, address with https: //, etc.) c) that the provider accepts payment against delivery. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 127 d) other. which one?_________________ 5. which means of payment do you prefer when paying for your purchases online? indicate which of the following are your first and your second option. first option second option a) credit card b) payment against delivery c) debit card or direct transfer to the provider's account d) payment at the physical store of the brand or seller e) online purchase with payment in physical points (drugstores, stores) f) bank deposit g) other. which one? ____________________ 6. regarding the conditions of delivery and/or return, what is the most important when buying a product online? organize the following options with one (1) being the most important. order a) that i can collect the product in a physical store b) that i can choose the day and time of delivery c) that i can track down my order at any time in order to know what state it is in. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 128 d) that the website is clear on its promise of delivery e) that the website offers facilities to request a change or a refund f) that the shipment is free 7. have you ever experienced any problems or difficulties with purchases made online? check only one option: option a) yes, many times b) yes, more than once c) only once d) i have never had problems 8. if you had a problem in the past, what was the problem? check the corresponding option (s). option a) i never received the product, and they still charged me for it. b) they charged me more than the price that was specified on the web page. c) the product was defective when it arrived. d) problems with the service online. e) the return process was very complicated. f) the product was not exactly what i bought: different characteristics, sizes, colors ... g) delays in the delivery of the order. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 129 h) the product was not what i expected. 9. what has been the main reason for you not to buy on the internet? choose only one option. option a) i had a bad experience. b) when i want something, i want it at that moment. c) i prefer the advice of salespeople at the store. d) lack of habit. e) i haven’t had the necessity to buy online. f) i’m not convinced or i don’t trust the payment methods. g) i prefer to see/touch the products before buying them. h) i’m concerned about sharing personal information. i) i don’t find any advantage on buying online. j) i don’t know how to buy online. k) other. which one? _____________________ 10. any other reason or reasons? choose more than one option without repeating the main reason chosen in the previous question. option a) i had a bad experience. b) when i want something, i want it at that moment. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 130 c) i prefer the advice of salespeople at the store. d) lack of habit. e) i haven’t had the necessity to buy online. f) i’m not convinced or i don’t trust the payment methods. g) i prefer to see/touch the products before buying them. h) i’m concerned about sharing personal information. i) i don’t find any advantage on buying online. j) i don’t know how to buy online. k) other. which one? _____________________ section vi demographic data: finally, please answer some information related to demographic data. 11. gender male______________ female________________ other ___________________ 12. age (in years): _____________________ 13. marital status: option a) single b) married c) cohabiting d) divorced e) widowed 14. do you have children? yes ________ no ________ how many? ___________ 15. what is the last level of education you completed? option a) preschool b) primary c) junior high d) high school multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 131 e) university f) postgraduate certificate program (not master) g) master’s degree h) doctorate 16. tell us about your occupation: option a) employee b) self-employed / business owner c) unemployed d) work from home 17. do you study at the university? yes ____ no _____ 18. from the following range, please indicate the average amount spent on monthly online purchases: option less than usd $ 30 usd $ 30 to usd $ 90 usd $ 90 to usd $ 160 usd $ 160 to usd $ 330 usd $ 330 to usd $ 660 more than us$ 660 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 132 annex 2 – table 13 – estimation test value (φ2) for answers 1 to 8 and final sum multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.11048 social and technological sciences issn: 2341-2593 rentería-garcía et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 92-133 | 133 source: prepared by the authors. 3. methodology multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 19 may schools develop their students’ intuition? jordi villoro i armengol1, santiago estaún i ferrer2 1professor at the esic business&marketing school. barcelona university campus. passeig santa eulàlia, 2. 08017-barcelona. teléfono: 93 414 44 44 2emeritus professor at the autonomous university of barcelona. department of basic, evolutionary and educational psychology. uab campus. calle de ca n’altayó, s/n. 08193-bellaterra-cerdanyola del vallès (barcelona). teléfono: 93 581 13 77. corresponding authors: jordi.villoro@esic.edu; santiago.estaun@uab.cat received: 2018-04-15; accepted: 2018-08-28 abstract the main purpose of the research has been to dictate the role intuition plays in the decision-making of undergraduate marketing students. the study and process of decision-making has always been focused on statistics and mathematics by the business world and universities, while most psychological aspects have been avoided. in recent years emotions, cognitive processes, intuition, etc., have increased their importance in the decision-making process; but they still have a long way to go. the research, which this article is based on, has been performed under a sample of 404 individuals aged 20 to 25 years; half of which are marketing students in esic business & marketing school. their decisions are analysed and compared with other collectives to observe if the information is an accurate significant variable for their decision-making. a questionnaire was administered to the participants in which they were asked the possible success or failure of specific products if they were to be launched on the market. these products were real and chosen by marketing experts. the participation of the individual in physical, cultural, etc. activities was also taken into account as well as their personal profile and psychographic data. the results obtained vary significantly from a statistical point of view and prove that training, background, participations on social activities and the perception of one's intuitive capacity are keys to success when it comes to decision-making. keywords intuition, marketing, decisions, university education, social activities. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 20 1. introduction decision making is a process of thought and action that concludes in a behavioural choice. a decision involves a person having to make a pick from a set of mutually exclusive alternatives. maturana (1979) maintains that it is a choice among many possible acts. five basic elements must be involved in the decision making process: information (relevant data), knowledge (comprehension, understanding), experience (mental record of past experiences), analysis (how the elements are linked) and the criteria (parameter used to evaluate these). in decision making, one may affirm that more value is given in academic and entrepreneurial settings to knowledge based on logic and analysis of all the variables than knowledge based on intuition, a term that shall be defined later on. the reason for this is that if an executive makes a decision based on reports and studies and the result is not that expected, the source of the information may be held liable. on the other hand, if he decides to make a decision contrary to the studies based on his own intuition and the outcome is negative, he cannot avoid his responsibility. the aim here is not to determine whether the corporate setting is the appropriate one for intuition based decision making, or whether one must always resort to an exhaustive analysis of the information, to a validated process with scientific academic rigor. there are numerous examples of major failures by powerful companies based on copious, costly information. the aim is to corroborate some factors that may boost intuitive decision making success in order that, considering the results obtained, one may encourage debate regarding how these are evaluated and developed in the field of education. as an anecdote regarding how intuition is represented in the field of psychology, robin m. hogarth (2001) analysed the submissions to psychinfo over ten years. he compared multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 21 the word “intuition”, with 2,941 instances, against others such as “attitude” (116,108 instances), “instinct” (22,023) or “insight” (9,087). the dual-process model (chaiken & trope, 1999) is fully accepted by psychology and states that we have one brain, but two minds, one analytical and the other intuitive and holistic, that work in parallel (gilovich and griffin, 2002). this theory is based on the case that there are two ways of processing information, called system 1 and system 2 (evans, 2008; stanovich and west, 2000). system 1 is basically based on unconsciously applied heuristics. these reduce and simplify general processing of the information (gigerenzer, 2000). the processes are swift, automatic, unconscious and scarcely demanding in cognitive terms. they are able to generalise and relate, and based on experience, to make implicit suppositions (lakoff & johnson, 1999). system 2 is formed by logical inferences and algorithmic reasoning. the processes are analytical, conscious, explicit and rule based. they require cognitive effort to process and deliberate all the information (epstein, 2008). this implies that personal differences are visible in cognitive capacities (evans, 2008; stanovich, 2000). which of the two systems prevails in processing information and the decision making is conditioned by the nature of the situation, the degree of emotional involvement and the personal characteristics with regard to styles of thought (epstein, 2008). system 1 system 2 implicit reasoning process intuitive: feelings not analysed swift decision making process acquired by associative learning may cause illusions or deceit explicit reasoning process rational: analysis of logical implications slow decision making process acquired by formal learning allows illusion or deceit to be overcome characteristics of the two reasoning systems (tubau, 2005) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 22 intuition is a difficult process to describe because it does not correspond with the rational information process, although it provides knowledge and guides decision making. introspective information use arises with intuition that allows one to reach correct answers that appear to be based on insufficient data. the first difficulty lies in being to describe exactly what intuition is and what it is not; what intuition is and what distinguishes it from randomness, for example. the major challenge lies in preparing a measuring instrument that allows each person’s level of intuition to be quantified. intuition is connected to the memory acquired over the years and to experience. the brain stores data in the unconscious and that information suddenly appears, giving rise to an insight. intuition improves with experience and may be more reliable in uncertain environments. jagdish parikh (1993) considers intuition provides “access to internal reserve of skill and experience accumulated over the years, and obtaining a response, or an impulse to do something, or an alternative among several, without being aware of how it was obtained”. timothy wilson (2002) describes the adaptive unconscious as the part of the brain that extracts swift conclusions, and he contrasts it with rational thought based on conscious perceptions. weston agor (1989) defines intuition as the capacity to integrate and use information stored in either side of the brain. antonio damasio (1994) considers intuition is a means of reasoning that does not follow the usual phases of the conscious process, that does not go through the logical phases of thought production. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 23 according to hammond (1996), intuition is “a cognitive process that somehow produces an answer, solution, or idea without the use of a conscious, logically defensible, step-bystep process”. gigerenzer (2007) affirms that people know much more than they are able to express. intuition is a judgement characterised by: • appearing rapidly in the awareness. • one not being sure of the reasons that generate it. • being consistent enough to lead to action. kirsten volz (2007) maintains that the brain processes without the person being aware of doing so. she noted that when people had to answer certain questions, a specific region of the brain was triggered, but also the areas that process abstract information, which means that when the brain faces difficult situations, it is not only governed by learned rules of conduct, but also by those it has created over time for swift decision making in critical situations. intuition is related to experience-based memory, which is acquired over time and through experience. the brain stores data and information in the unconscious, which it later recovers and also processes unconsciously. that information is the basic element of intuition. however, considering the enormous amount of information within easy reach, the doubt lies in knowing which information must be taken into account (in quality and quantity) to consider that a rational, objective decision has been made. the traditional educational model instructs children to accumulate data and think a lot before acting. it concentrates on the conscious part and values logic and analysis. it rewards a precise answer obtained by exercising their memory, and does not invite them to reflect and experiment. malcom gladwell (2005) maintains that correct decisions may be made based on sensation or feelings. he also affirms that it is possible to identify and understand the reason for erroneous decisions based on intuition and thus it is possible to train one’s intuition. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 24 factors such as personality and culture (beliefs, customs, values ...) act as perceptive variables (prejudices) that distort reality. the interpretation of a fact is subjectivised by cognitive biases (kahneman & tversky, 1972). when the situation is too complex for logical analysis, intuitive mental processes are observed (neisser, 1963). in that sense, complexity arises from the degree of uncertainty of the problem and the risk associated with the decision. complex decisions intervene in knowledge, experience and emotions. dane & pratt (2007) and khatri & alvin (2000) describe the basic properties of intuition: • intuition involves holistic associations in which the environment stimulates profound unconscious sensations. patterns are generated by linking stimuli instead of by logical considerations. • cognitive biases hinder analysis of problems. however, cognitive biases are also present in a rational process, so intuition may be more reliable under uncertain conditions, or with expert decision-makers. the complexity of situations is due to the large amount of information and difficulty and the difficulty in taking it all into account. • intuition is swift, generates automatic responses. it mixes all the experience and learning by the person and instantly provides a response that is perceived as “the” best option. • it is an unconscious process that makes use of knowledge and experiences stored in the memory, but which are not accessible during a rational remembering process. one may sum up by stating that intuition:  allows access to a major reserve of knowledge one is not aware of.  is the way of thinking that works before applying rational thought.  is an attribute of the human being that includes both the cognitive as well as emotional level. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 25  is motivating and responsible for success in creativity and innovation.  may be developed and accessed, not at random, but as a result of a process. intuition must not be imposed as a preferential resource in decision making, but rather must be taken advantage of by reason. reason interprets, analyses and takes advantage of the unconscious knowledge. it is due to this that teaching must include (or continue to include) methods based not only on study and learning knowledge, but also those that work on skills that are harder to measure, but no less important. 2. literature review a basic aspect of decision making involves agreeing the criteria that determines what is considered a good decision. vlek (1984) considers that in situations of uncertainty, a good decision is that which complies with the statistical model (according to the probability theory) although such decisions do not guarantee a good result. maier (1980) considers it necessary to consider two dimensions to evaluate the potential correctness of a decision. on one hand, there is the quality of the decision that will depend on the available information, and on the other, acceptance of the people who will have to implement it. majone (2010) considers that the results of a decision are uncertain, so it is difficult to determine the correct decision. when faced with the same problem, diverse valid options and a variety of opinions may appear that hinder evaluation of the result. thus, majone emphasises the importance of studying the decision process and not the result alone. three important concepts appear in these contributions that influence the efficiency of a decision. that is why it is necessary to emphasise the notion of uncertainty, regarding information and the problems. according to the model provided by hellriegel, jackson & slocum (2002) decisions, according to the characteristics of the problem to be solved, may be placed in three categories: certainty, uncertainty and risk. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 26 • certainty conditions are decisions without risk in which the options are clearly defined and identified. all the information is available, the valid alternatives are known and one may identify the impact of the decision with nearly total precision. the aim is to choose the solution that provides the best result. • risk conditions arise when there is scarce information and it is ambiguous. a certain amount of information is available that allows the problem to be defined, to identify possibilities and evaluate probabilities of results. the decisions are based on objective and subjective probabilities. • uncertain conditions are when decisions are made in ignorance (yates, 1990). there is little or no information available that may have an effect on the difficulty of clearly defining the problem. alternatives may be identified, but one cannot precisely measure the impact of each alternative. one may only make reasonable assumptions. mintzberg & westley (2001) or bransford & stein (1987) among other authors, maintain that the first step in the decision making process is defining the problem. a problem is the deviation between an established norm and the reality observed, the cause of which is not known and the importance of which makes one want to know it. a classic approach to problems sorts them in two categories: structured and unstructured problems. • structured problems are repetitive, common, with scarce risk, having a well defined procedure. • unstructured problems are new, there are few references available, there is uncertainty and risk, and thus they are subject to a point of view. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 27 routine decisions are valid for structured problems, those that are known and that any person may decide and act on. they are simple, quick decisions. structured problems unstructured problems information sufficient insufficient and dispersed experience there is experience in the solution little or no experience in the solution algorithm there is a proven algorithm for the solution there is no proven algorithm for the solution problem features innovative decisions are required for unstructured decisions. there is no established procedure and thus each situation requires specific processing. one requires capacity to propose solutions from different points of view. framing1 is when the same information is formulated in different ways. the same information may be interpreted differently according to the structure of the message. plous (1993) maintains that the choice between two options shall depend on whether the message focuses on profit or loss. there are numerous studies that support that affirmation. edwards, elwyn, covey, mathews & pill (2001) conducted research in the field of health and noted that patient response was different according to how the doctor presented the alternatives. although it is the same to say an operation has a 90% success rate as there is a 10% chance of dying, as a general rule, patients preferred the positive approach. elizabeth loftus & john palmer (1974) observed different responses regarding vehicle speed after displaying some images and making a minor change (one word) in the way the question was asked: “about how fast were the cars going when they contacted each other?”, “about how fast were the cars going when they hit each other?”, “about how fast were the cars going when they smashed into each other?”. the differences fell within a range of twenty km/h. 1 maintains that comprehension of a message is the result of how it has been explained; a theory, initially studied and developed in the media and political discourse. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 28 other research carried out by robert cialdini (cialdini, vincent, leeis, catalán, wheeler & darby, 1975) approached the same situation in two different ways. they first asked subjects to perform as chaperones for a group of juvenile delinquents on a trip to the zoo and obtained a 17% affirmative response. they then asked another group whether they would be willing to spend two hours a week as volunteer counsellors at a juvenile detention centre for the following two years, obtaining an almost unanimous rejection response. they then asked that same group the same question as they had asked the first group (to chaperone a group of juvenile delinquents from a detention centre on a trip to the zoo) and obtained 50% compliant responses. the framing questions the use of reasoning in a choice, as it invalidates the variance and independence of irrelevant alternatives. people are sensitive to the way in which a problem is presented. thus, the decisions are the result of the conclusions reached after processing the information. based on the information available, the subject obtains conclusions that allow him to make decisions. when the information is relevant and exact, valid conclusions may be reached. when one may not be sure of the information, either due to its veracity or insufficiency, and one cannot anticipate the consequences of the alternatives, holyoak & nisbett (1988) suggest that processes of inference must be applied. kahneman & tversky (1972) describe the existence of shortcuts that are used when it is necessary to evaluate complex situations. they proved the use of patterns of conduct that are far from judgement and a rational choice process under certain conditions. they attributed such deviations to heuristics and biases that lead to systemic errors. gerd gigerenzer (2008) maintains that heuristics is an adaptation mechanism that does not follow the rules of logic, of probability or rational thought. these are shortcuts to predict and make decisions in uncertain situations. it is a procedure that provides the solution to a problem, but not in a justified manner. heuristics works with scarce relevant information. it is based on experience and on the available information. empirical knowledge is stored in the deep limbic system and is more available than rational thought. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 29 tversky & kahneman2 affirm that people do not perform complex analyses, nor exhaustive probabilistic reasoning in all the decisions they make. on many occasions, they follow a path that allows them to simplify the process. miles & sadler-smith (2014) worked with personnel recruitment managers and observed that in situations with lack of specific information or contradiction in the tests, they considered it positive, fair and valid to evaluate the “overall impression” of the candidate. beilock, bertenthak, mccoy & carr (2004) conducted an investigation of golf players to question or validate the appropriateness of their reflection process in decision making. they distinguished two groups: expert players and beginners. they were asked to strike the ball under two scenarios. first, they were asked to be aware of each and every one of their movements; in the other, they were asked to hit it without thinking. the results were that the beginners hit better in the first case; in the second case, the experts drove best. unconscious skills may be harmed by excessive thought. nalini ambady (1993) analysed how long a student needs to decide whether a professor is good or not. to do so, he showed a group of students a ten second mute video of a professor and asked them to evaluate him. he then asked another group who had had the same professor for a full year to evaluate him. the valuations were quite similar. to explain the use of heuristics, gigerenzer (2007) asked a group of students in the united states which city had the larger population, detroit or milwaukee and 60% of the students gave the correct answer that it was detroit. the fact is that at that time, the difference between both cities was not very significant, which was what led some students to be mistaken. however, the relevant part of the research is that he posed the same question to a group of german students, with a significantly higher correct answer rate. the germans, who had never heard of milwaukee, or who had few references, did not have the same doubts the american students did. the explanation gigerenzer provided is that the germans with less knowledge of american geography acted by the rule that if something is well known it is more important. if the name of one city is known and the other is not, one must infer that the well-known city has the larger population. 2 daniel kahneman, a professional psychologist, was awarded the nobel prize in economic sciences in 2002 for linking psychological research to the science of economics, especially in everything related to decisions in uncertain environments. he developed prospect theory jointly with amos tversky. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 30 all this made kahneman affirm that a certain degree of ignorance may be positive. trusting in that which is known may be highly useful on numerous occasions. research into heuristics does not yet explain how that skill is acquired, nor under what conditions the mechanism must be used to be effective. hunt and agnoli (1991) suggest that heuristics is used by children under 8 or 9 old, and jacobs and potenza (1991) describe heuristic conduct in children of six years. rationality and logic take second place and people use intuition because one prioritises the most accessible feature or characteristic of the matter (first impression effect); because emotion appears and attention is diverted toward the most remarkable (positive or negative) feature; because the choice has already been made (even without having performed any kind of reflection or analysis. gigerenzer (2008) affirms that having little information is better as long as there is a positive degree of ignorance; cognitive limitations are observed; there is excessive information; a major number of alternatives are being handled. gigerenzer considers that intuitions based on a single argument are usually right when one has to predict a future that is difficult to foresee, when little time is available, or when one has limited information available. on the contrary, logical analysis is required to speak of the past, to discuss a highly foreseeable future, when there is a large amount of relevant information. the researcher proposes that the mind acts according to some general rules and a set of personal skills, and it takes advantage of the unconscious to simplify. fact and theory do not always coincide, so what is important is not to determine whether or not one may trust intuition, but rather to ascertain when one may do so and when it is not convenient. the intelligence of the unconscious involves ascertaining what may work better at each specific moment. organisations in present day society (such as schools) encourage accumulation of knowledge and people may be overloaded by so much information. short term memory is limited. more information and more cognition is not always the best and, on occasions (on many occasions) less may be more. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 31 3. methodology our objective was to determine elements or factors that have an effect on the intuitive process that is specified in the following hypotheses: the main one is whether a specific training provides the skills to make the best decisions. the secondary one is whether sex, one’s own perception of personal intuition and the fact of participating in social activities are factors that influence intuitive skill. the participants were two age groups between 20 and 25 years, one formed by marketing students at the esic, and the other formed by two subgroups: students of other degree subjects and professionals. the first is comprised exclusively of students from official and private degree courses in marketing, advertising and public relations on the esic campus in barcelona, madrid and valencia. three classes were selected from each campus, following the criteria of accessibility and the participants were chosen by attendance. the control group was formed by youths resident in barcelona, with two differentiated profiles. on one hand, there were students (degree or diploma cycles) and, on the other hand, workers (employed or unemployed). the students were grouped by specialism. four groups of students were defined: corporate graduates, scientific or technical graduates, humanistic or literary graduates and training cycles. the workers were asked for their main occupation to be their employment (or job seeking) although they might be studying or have completed a degree. the following table summarises the participants: group amount esic students 202 others students 125 workers 77 (321+452) total sample 404 1 with marketing studies 2 without marketing studies multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 32 by sex, there were 205 women and 199 men. the average of the participants was 22.25 years. by groups, the average ages were: group age esic barcelona (students) 22,60 esic madrid (students) 21,43 esic valencia (students) 21,48 business (students) 22,37 technical (students) 21,96 arts (students) 22,04 professional training (students) 21,56 sales (workers) 23,87 administration (workers) 23,47 productions (workers) 23,95 average 22,25 average ages by groups the sample was of non probabilistic and intentional (lohr, 2000). participation was voluntary and with informed consent. the research design considered that a questionnaire was the most appropriate instrument for qualitative assessment of a sample of 404 people. double option (yes/no) closed questions and valuation questions (from 1 to 5) were used to increase the effectiveness of the questionnaire. the content and drafting of the questionnaire was validated according to the triangulation procedure and 90% matching the text drafted by experts in the field of research, by professors with experience in marketing, by psychologists and professionals from the entrepreneurial field. in order to decide which products would be included in the questionnaire, what was considered a successful product was defined in a first phase to then proceed to prepare the product list. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 33 we based the product definition on the radar de la innovación3 that defines all products that provide a new feature to the category as an innovation and considers the innovation to be successful when it exceeds the penetration and average repetition index in the category. in a second phase, an exhaustive list of products launched on the market in recent years was drawn up, based on criteria from the radar de la innovación. the list was then submitted to focus group discussion and brainstorming by a group of experts, who selected a sample of products that were classified as a success or failure. lastly, a trial was carried out on a pilot group of 15 people representing the group to be studied (ten students from the esic and five unrelated youths, all chosen at random) in order to determine whether they had prior knowledge of the products and whether the results provided significant data. three non existent products taken from esic master student graduation projects were included as a control element. the result of the trial was positive and validated the idealness of the products chosen. a questionnaire was prepared with four kinds of questions: a first block, formed by questions to identify the demographic profile of the participant: age, sex, place of birth ... a second section intended to identify the leisure and cultural activities the respondent practices or has practiced: association, musical group, day-tripper... a third group of questions matches the main objective of the research and aims to determine the number of correct answers the participants have obtained with regard to ascertaining whether the products mentioned have been a success or failure. an appraisal was requested for a total of nine products. 3 first continuous tracking of innovation in launching major consumer goods. it analyses more than 1,000 skus launched on the market each year and identifies the successful innovations. a sku (stock-keeping unit) is an identifier to recognise the products or services offered on the market. the sku is the reference number linked to a product, brand, service ... during 2014, 108 major consumer goods were launched on the market and, according to the data from radar de la innovación, less than 20% were a success. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 34 happyjama agua onura yogur mcspaghetti kitchen entrées bragas bic fresa plátano donuts fresquito maxiroll the last block included questions with the most significant variables that affect intuition in order to establish the intuitive profile prospectively. these evaluate creativity, attitude to risk, the methodology, reasoning, resistance to change, improvisation, ... a scale has been established with five options from “definitely not true of myself” (1 point) to “definitely true of myself” (5 points), according to a scale established by pacini & epstein (1998) that contained options such as “i believe in trusting my hunches” or “when it comes to trusting people, i can usually rely on my gut feelings”. three types of different profiles have been established with cut-off points calculated by the k-means4 algorithm: • high (from 32 to 45 points): people who fully trust their intuition in decision making. • medium (from 25 to 31 points): people who, even when trusting their intuition, require other additional factors for decision making. 4 classification algorithm that maximises the differences between cases of the three clusters created. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 35 • low (from 9 to 24 points): thoughful people who find it difficult to make decisions unless they have all kinds of information as the basis for such. the fieldwork was carried out during september and october 2015. the questionnaire was mainly administered in business schools, universities or on company training courses. the questionnaire was administered in a group to all the students at the esic and students at other schools, and meetings were called for the other groups. the relevant information was provided on the object of the research and they were told they could decide not to answer without the need to justify their decision. they were informed that it was anonymous, exclusively for research use, and their consent was requested. the research consisted of asking the participants about the probability of success or failure of some products launched on the national market. they were informed that the products might be real or fictitious, to minimise the conditioning, although finally all the products chosen were real. the following were used to analyse the results: a) the mann-whitney u test for comparisons between ordinal or quantitative variables between groups by two level factors; b) the kruskal-wallis test for comparisons between groups by factors with more than two levels. these two nonparametric tests have been opted for because, in general, some of the variables do not allow normal distribution (e.g.: the number of correct answers is a discrete number between 4 and 9; the normal model takes continuous values without specific lower or upper thresholds). one may check that the kolmogorov-smirnov and shapiro-wilk tests reject the null hypothesis that the normal model is acceptable (p < 0.001 for correct answers, p < 0.007 for time and p < 0.001 for the intuitive profile in the two tests): multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 36 test of normality kolmogorov-smirnova shapiro-wilk statistic df sig. statistic df sig. correct answers ,180 404 ,000 ,925 404 ,000 time ,070 404 ,000 ,990 404 ,007 profile ,075 404 ,000 ,985 404 ,000 lilliefors significance correction a linear multiple regression model has been adjusted for the response time. it has been validated that the residuals in the model admit normal distribution (kolmogorov-smirnov with p-value = 0.186, shapiro-wilk with p-value = 0.849), in this case there is no need to avoid parametric procedures. bivariate correlations between quantitative variables have also been used. the pearson correlation is reported, as it is the must usual, due to the fact of not detecting linearity assumptions violated, and because it has been checked that the conclusions do not vary if nonparametric correlations are used (kendall’s tau-b, or spearman’s correlation coefficient). the sample size (n = 404) and the fact that the quantitative variables are sufficiently symmetric, assure (due to the well-known central limit theorem) that the conclusions would be similar, even if one had opted for parametric variables in all the cases. software: the statistical analysis of the data was performed using the spss v. 22 suite. demographic studied: youths from 20 to 25 years. sub-set of esic students: 540 students. demographic size of other groups in the same age range in barcelona: 95,000 youths. global sample: 404 youths. segmented sample: 202 esic students + 202 youths among students from other degree courses or training cycles and workers (employed or unemployed) from barcelona. margin of error: for a confidence level of 95.5%, the margin of error for the global sample is ±4.87%. in the different segments, the margin of error is ±5.46% for the sub-set of esic students and ±6.89% for the sub-set of other groups in the same age range. neutrality of the sampling conditions: p=q=0.5. geographic area: the cities of barcelona, madrid and valencia. technical file multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 37 4. analysis and results table 1 records the average number of correct answers in the sample, distinguishing the three groups analysed: esic students, students in other degree courses or cycles and workers. correct average standard deviation minimum percentile 25 median percentile 75 maximum number of cases esic 6.86 1.10 4 6 7 8 9 202 students 6.21 1.19 4 5 6 7 8 125 workers 6.42 1.25 4 6 6 7 9 77 table 1 a first analysis of these results leads one to think that indeed specific training in marketing provides cognitive resources that provide a person the skills to obtain better results. this thus confirms the hypothesis that marketing students may have more correct intuitions in related to their scope of study. the distribution of the number of correct answers between the esic group and the non-esic group is different, as is shown in graph 1 and the relevant statistical analysis (p < 0.001. mann-whitney test). graph 1 a detailed analysis of the three groups shows that the differences are statistically significant (p<0.001, kruskal-wallis test). the differences lie between esic and students (adjusted p-value < 0.001) and between esic and workers (p = 0.022). no differences are observed between students (not from the esic) and workers; p = 0.743 (graph 2). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 38 graph 2 the diversity of the academic and labour situations of the members of these groups requires a more detailed analysis. graph 3 shows that students of economics and humanistics degree courses and workers in the sales field have a greater tendency to obtain correct results than those from more technical studies and those from the administrative and operations fields. graph 3 the groups that have a level of correct answers above the average are from esic, with an index of 6.9 (the average is 6.57), workers in the sales field and students of literary subjects. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 39 with regard to esic students, it is confirmed that specific training enables them to make more correct intuitive decisions (related to the scope of the studies). with regard to workers from the sales field, there are two reasons that may explain the high level of correct answers. on one hand, the average age is higher (23.9 years against the global average which is 22.2 years) and on the other 65% of the members of the group have marketing-related studies. the explanation regarding students of humanistic or literary subjects who have obtained a high level of correct answers may lie in the capacity to interpret conduct and analyse nonstandard situations. on the contrary, groups that have obtained a lower level of correct answers are the students from technical or scientific courses (this may be explained because they are used to applying mechanical methodologies) and the students in vocational training cycles (5.9 right) even though this case one must specify that if only company related specialities are taken into account (marketing and international commerce), the average number of correct answers is higher (7.1). it is significant to discriminate the results by the workers in the sales department according to whether or not they have studies directly related to marketing. graph 4 shows these results and one notes that the sales force who have such studies are those with a greater number of correct answers (7.2 on average) more than the target public of the research (6.57 esic students). graph 4 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 40 this datum is significantly sufficiently significant to reinforce the hypothesis of the study, and it also coincides with the information gathered from review of the literature. experience is a determining factor in decision making in uncertain situations. the fact of having specific training is complemented by having professional experience in the sales and/or marketing environment. the professional experience provided by the years employed in performing professional tasks or activity, even though one may acquire experience in transversal personal skills for any task. in that sense, participating in certain social activities undoubtedly provides baggage and resources to face situations of conflict in which decisions must be made. it is due to this that we wished to observe the relation between the level of correct answers (and thus intuition) and participation in social activities. table 2 and graph 5 show that relation. correct average standard deviation minimum percentile 25 median percentile 75 maximum number of cases no activities 5,68 0,88 4 5 6 6 8 176 activities 7,26 0,91 4 7 7 8 9 228 table 2 graph 5 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 41 the activities considered and frequency of replies is shown in table 3. activity frequency percentage team sports 323 79.95% music group 120 29.70% voluntary work 100 24.75% excursion group 92 22.77% ngo 36 8.91% association 35 8.66% political party 8 1.98% none 42 10.40% table 3 the fact of participating in cultural, sports or leisure activities is positively related and one may even say that it has an explicit influence on non-academic intelligence and thus develop personal and social skills. comparing the quantity of activities performed with the average of right answers, one observes a trend in the sense that the more activities are performed, the greater the percentage of right answers. in that sense, as may be seen in graph 6, people who have affirmed that they perform (or in recent years have performed) five of the activities considered, obtained the highest number of write answers (8.1), exceeding the global average. thus, this group, comprised of members of other groups, is that which achieves the best result. graph 6 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 42 at the other extreme, there are those who say they have not performed, or do not perform any activity. that result (5.5) is the lowest of all, and it is under the group who obtained the worst results (5.8) that was that of students from technical or scientific degree courses. nearly perfect attendance has been shown in the other groups, so one affirms that the more social activities, the greater number of correct answers are obtained in decision making, at least with regard to the subject of this research. the people who achieved nine correct answers (or 7 or 8) all performed or had performed five of the seven activities considered. graph 7 shows the activities performed by all the persons who obtained correct answers equal to or exceeding seven. in this graph, one may see there are some activities that prevail over others. in that sense, it is necessary to state that “team sport” has been excluded from activity, as it had a majority response (nearly 90%) and that provides it scarce interest and significance. the reason for that majority response is that they performed some sport at school and in that case the activity, not being a personal option, loses significance in the results. among the rest of the activities, there are three than exceed 40%, such as “musical group” (choir, group, band …), “volunteering” and “day-trip group”. there is a response of about 15% by those who say they have belonged to or collaborated with an “ngo” or an “association” (neighbours, to defend animals, against evictions ...); and only 3% say they have belonged to a “political party”. one may consider that the fact of participating in social activities, of whatever type they may be, determines the type of social and promise skills. in any case, one may affirm that the people in that sample who have performed the most activities have obtained more multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 43 right answers in their response to the questionnaire. statistically significant differences in number of right answers have been detected between performing activities or not (p < 0.001) in favour of those who perform activities. highly related to these personal and social activities performed by the person, there is their own perception of themselves and their level of secureness and confidence in their own capacity. in that sense the level of confidence and self-esteem was linked to the number of right answers, as it was considered that the way to face a situation and thus the decision making process may vary according to the person’s psychological characteristics. in graph 8, the participant’s intuitive profile is linked to the average number of correct answers. the correlation between both terms is 0.121. the higher the intuitive profile, the greater number of correct answers. it is a statistically significant correlation (different from 0 and positive; p=0.015). graph 8 the trend is clear in that sense: the more intuitive the person is described as, the greater the number of correct answers. this coincides with the information noted in the bibliographic review, that the fact of trusting in one’s own skills and managing emotions has a highly positive contribution to gaining in predictive efficiency. the skills considered in the questionnaire are closely related to cognitive and behavioural skills present in decision making. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 44 5. discussion and conclusions the results obtained allow us to affirm that the working hypothesis is valid. effectively, it is noted that marketing degree students may make better decisions related to their speciality than other youths. notwithstanding this, the best results were provided by the group of workers from the sales force with studies in the company speciality. that fact does not invalidate the results, but rather quite the contrary. as reflected in the bibliographic review, experience is absolutely necessary for intuition to act efficiently. if that specific experience has specific training added, the best combination is generated. it has been noted that the fact of performing (or having performed) some kind of social activity, contributes to improving correct decision making. this is not due so much to the unique nature of the situation the decision concerns, but rather the actual decision making process. it is not a matter of whether or not one has knowledge or experience of the matter concerned, but rather the acquired skills that favour correct decision making. and these skills influence the cognitive processes that are activated by intuition. the fact of belonging to a cultural, sports or leisure group ... develops a kind of skills that are closely linked to intuition. it has also been noted that having a favourable perception of oneself regarding the intuitive skill improves the results. there are numerous studies in the world of medicine, sports and the corporate world that back the importance of focussing one’s thought adequately to achieve the objective. self-confidence, security as to the records or accepting the actual result (whether positive or negative) are factors that boost success. in this investigation, it is noted that persons who are described with inherent intuition characteristic s obtained a greater number of correct answers. moreover, they also answered quicker and spent less time on reflecting regarding the answer. the validation of these hypotheses leads one to think that if the training enables one to decide correctly in an intuitive manner, the aim is to motivate students to use and let themselves be guided by their intuitions. the work lies in helping them to distinguish the situations when they may trust their intuition, and those when a logical, rational, conscious process must be followed. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 45 schools must continue to include methodologies and objectives to develop overall intelligence and not concentrate exclusively on academic intelligence (logicalmathematical skill and verbal skill) that are the ones that have been taken as a reference to measure personal intelligence. it is fully accepted that intelligence is plural and that it is comprised of different elements, so one must speak of multiple intelligences. intelligence is a skill that may be developed independent of genetic or cultural factors. knowing how to recognise and manage ones’ own emotions is a necessary skill to avoid biases in decision making. thus, the problem does not lie in deciding whether one must develop students’ intuition or not. the answer is undoubtedly affirmative. one must. the difficulty lies in how to do so and in how to include this in the training curriculum. in that sense, one must complement traditional teaching based on study, memorisation and repetition with develop of other complementary ones that stimulate both parts of the brain, both reasoning systems. one must explore the issue and whether it is considered appropriate to include new trends in training methodologies: gamification, power pupils, happy & healthy, lean entrepreneurship, techno-craft, crowd power… social activities are predicating factors of more correct decisions, so they must be considered and included in education. the aim is not to debate who is responsible (parents or educational centres) or whether or not these must be evaluated and form part of the curriculum. these must be encouraged, and the students should be motivated to participate and become involved in any kind of activity that obliges them to engage with each other and to develop the greatest number of personal skills. and due to one’s own perception of oneself influencing the results, it is necessary to generate favourable conditions that contribute to increase safety and confidence in oneself. concepts that are closely linked to emotional intelligence affect intuition. one must effectively extend its application throughout students’ training, providing them resources to manage their emotions more positively (frustration resistance, for example). due to the fact that the validity and idealness of use of intuition under certain circumstances, has been proven, lines of research are proposed based on certain challenges for which a satisfactory response has not yet been provided. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 46 • unifying and validating the concept of intuition based on academic rigor, clearly establishing the difference between that concept and others such as fate, premonitions, ... • preparing an instrument to measure intuition that may be validated and used in a scientific and academic environment. • provide a theory (methodology, pedagogical resources, teaching guide ...) that really develops intuition in practical terms. references agor, w. (1989). intuition in organizations: leading and managing productively. evanston: sage publications. ambady, n. and rosenthal, r. (1993). half a minute: predicting teacher evaluations from thin slices of nonverbal and physical attractiveness. journal of personality and social psychology, 64, nº 3. 431-441. bechara, a., damasio, h., tranel, d., and damasio, a. r. (2005). the iowa gambling task and the somatic marker hypothesis: some questions and answers. trends in cognitive sciences, 9 (4), (pp.159-162). bransford, j. d. and stein, b. s. (1988). solución ideal de problemas. barcelona: labor. beilock, s.l., bertenthal, b.i., mccoy, a.m. and carr, t.h. (2004). haste does not always make waste: expertise, direction of attention, and speed versus accuracy in performing sensorimotor skills. psychon bull rev. 2004 apr: 11 (2): 373-9. burke, l. a., and sadler-smith, e. (2006). instructor intuition in the educational context. academy of management learning and education, 5, (pp.169−181). chaiken, s., and trope, y. (1999). dual-process theories in social psychology. new york: guilford press. cialdini, r.b., vincent, j.e., lewis, s.k., catalán, j., wheeler, d. and derby, b.l. (1975). reciprocal concessions procedure for inducing compliance: the door-in-the-face technique. journal of personality and social psychology. vol. 31. n. 2 (pp. 206-215). damasio, a.r. (1994). descartes error: emotion, reason and the human brain. nueva york: harper collins multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 47 damasio, a. r. (1996). the somatic marker hypothesis and the possible functions of the prefrontal cortex. philosophical transactions of the royal society b: biological sciences, 351(1346), 1413-1420. dane, e. and pratt, m. g. (2007). exploring intuition and its role in managerial decision making. academy of management review, 32, 33−54. edwards, a., elwyn, g., covey, j., matthews, e., and pill, r. (2001). presenting risk information – a review of the effects of “framing” and other manipulations on patient outcomes. journal of health communication. 6 (1): 61–82. epstein, s., lipson, a., holstein, c. and huh, e. (2008). irrational reactions to negative outcomes: evidence for two conceptual systems. journal of personality and social psychology, nº 62, 328-339 evans, j. (2008). dual-processing accounts of reasoning, judgment, and social cognition. annual review of psychology, 59, 1−24 gardner, h. (1993). multiple intelligences: the theory in practice. new york: basic books. gigerenzer, g. (2007).the intelligence of the unconscious. nova york: viking press. gilovich, t., griffin, d. and kahneman, d. (2002). heuristics and biases. the psychology of intuitive judgment. cambridge: cambridge university press. gladwell, m. (2006). inteligencia intuitiva. ¿por qué sabemos la verdad en dos segundos? madrid: punto de lectura guilera, ll. (2002).vías de acceso conceptual a la resolución de problemas: importancia de los estímulos sensoriales. tesis doctoral. uab. hammond, k.r. (1996). human judgment and social policy: irreductible uncertainty, inevitable error, unavoidable injustice. new york: oxford university press hellriegel, d., jackson, s.e. and slocum, j.w. (2002). administración, un enfoque basado en competencias. mèxic: paraninfo. helmholtz, h. (1860/1962). handbuch der physiologischen optik. southall, j.p.c. vol. 3. new york: dover. hogarth, r. (2001). educar la intuición. barcelona: editorial paidós. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 48 holyoak, k. j and r. e. nisbett (1988). induction. en sternberg, r.j y e.e smith: the psychology of human thought. editorial cambridge, univ. press. n.y hunt, e. and agnoli, f. (1991). the worfian hypothesis: a cognitive psychology perspective. psychological review, 98 (3). pp 377-389. jacobs, j.e. and potenza, m. (1991). the use of judgement heuristics to make social and object decisions: a developmental perspective. child development. 62, february. (pp. 166–178). kahneman, d., slovic p. and tversky, a. (1982). judgments under uncertainty. heuristics and biases. new york: cambridge university press. lakoff, g. and johnson, m. (1999). philosophy in the flesh. the embodied mind and its challenge to western thought. basic books, new york. khatri, n. and alvin, h. (2000). the role of intuition in strategic decision making. human relations, 53 (1), 57-86 maturana, h (1979). autopoiesis and cognition: the realization of the living. boston studies in the philosophy of science. paperback. loftus, e. f., and palmer, j. c. (1974). reconstruction of automobile destruction: an example of the interaction between language and memory. journal of verbal learning and verbal behavior, 13(5), 585–589. maier, r. (1980). toma de decisiones en grupo. méxico: editorial trillas majone, g. (2010). foundations of risk regulation: science, decision-making, policy learning and institutional reform. eur. j. risk regul. 1 (2010), pp. 5–19 mayer, r.e. (1992). thinking, problem solving, cognition. new york: freeman. metcalfe, j and wiebe, d. (1987). intuition in insight and non insight problem solving. memoryand cognition, 15 (3). 238-246. mintzberg, h and westley, f. (2001). decision making: it's not what you think mit sloan management review; 42. pg. 89 miles, a. and sadler-smith, e. (2014). with recruitment i always feel i need to listen to my gut: the role of intuition in employee selection, personnel review, vol. 43 issue: 4, pp.606-627 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9958 social and technological sciences eissn: 2341-2593 villoro and estaún (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 19-49 | 49 neisser, u. (1963). the multiplicity of thought. british journal of psychology. 54. (pp. 1-14) parikh, j., lank, a. and neubauer, f. (1993). intuition: the new frontier of management. wiley-blackwell plous, s. (1993). the psychology of judgment and decision making. mcgraw-hill. sadler-smith, e., allinson, c. w. and hayes, j. (2000). cognitive style and learning preferences: some implications for cpd. management learning, 31, 239−256. stanovich, k. e., and west, r. f. (2000). individual differences in reasoning: implications for the rationality debate? behavioral and brain sciences, 23, 645−726. tversky, a. and kahneman, d. (1983). extensional versus intuitive reasoning: the conjunction fallacy in probability judgment. psychological review, 90, 293-315. vlek, ch. (1984). social decision methodology for technological projects. dordrecht (netherland) kluwer academic publishers. volz, k. g. (2007). finding culturally (in) dependent levels of self-representation. workshop. max-planck-institute for human cognitive and brain sciences wilson, t. (2002). strangers to ourselves: discovering the adaptive unconscious. cambridge: harvard university press. yates, j.f. (1990). judgment and decision making. englewood cliffs, new jersey, prentice-hall inc. << /ascii85encodepages false /allowtransparency false /autopositionepsfiles true /autorotatepages /none /binding /left /calgrayprofile (dot gain 20%) /calrgbprofile (srgb iec61966-2.1) /calcmykprofile (u.s. web coated \050swop\051 v2) /srgbprofile (srgb iec61966-2.1) /cannotembedfontpolicy /error /compatibilitylevel 1.4 /compressobjects /tags /compresspages true /convertimagestoindexed true /passthroughjpegimages true /createjobticket false /defaultrenderingintent /default /detectblends true /detectcurves 0.0000 /colorconversionstrategy /cmyk /dothumbnails false /embedallfonts true /embedopentype false /parseiccprofilesincomments true /embedjoboptions true /dscreportinglevel 0 /emitdscwarnings false /endpage -1 /imagememory 1048576 /lockdistillerparams false /maxsubsetpct 100 /optimize true /opm 1 /parsedsccomments true /parsedsccommentsfordocinfo true /preservecopypage true /preservedicmykvalues true /preserveepsinfo true /preserveflatness true /preservehalftoneinfo false /preserveopicomments true /preserveoverprintsettings true /startpage 1 /subsetfonts true /transferfunctioninfo /apply /ucrandbginfo /preserve /useprologue false /colorsettingsfile () /alwaysembed [ true ] /neverembed [ true ] /antialiascolorimages false /cropcolorimages true /colorimageminresolution 300 /colorimageminresolutionpolicy /ok /downsamplecolorimages true /colorimagedownsampletype /bicubic /colorimageresolution 300 /colorimagedepth -1 /colorimagemindownsampledepth 1 /colorimagedownsamplethreshold 1.50000 /encodecolorimages true /colorimagefilter /dctencode /autofiltercolorimages true /colorimageautofilterstrategy /jpeg /coloracsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /colorimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000coloracsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000colorimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasgrayimages false /cropgrayimages true /grayimageminresolution 300 /grayimageminresolutionpolicy /ok /downsamplegrayimages true /grayimagedownsampletype /bicubic /grayimageresolution 300 /grayimagedepth -1 /grayimagemindownsampledepth 2 /grayimagedownsamplethreshold 1.50000 /encodegrayimages true /grayimagefilter /dctencode /autofiltergrayimages true /grayimageautofilterstrategy /jpeg /grayacsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /grayimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000grayacsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000grayimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasmonoimages false /cropmonoimages true /monoimageminresolution 1200 /monoimageminresolutionpolicy /ok /downsamplemonoimages true /monoimagedownsampletype /bicubic /monoimageresolution 1200 /monoimagedepth -1 /monoimagedownsamplethreshold 1.50000 /encodemonoimages true /monoimagefilter /ccittfaxencode /monoimagedict << /k -1 >> /allowpsxobjects false /checkcompliance [ /none ] /pdfx1acheck false /pdfx3check false /pdfxcompliantpdfonly false /pdfxnotrimboxerror true /pdfxtrimboxtomediaboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxsetbleedboxtomediabox true /pdfxbleedboxtotrimboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxoutputintentprofile () /pdfxoutputconditionidentifier () /pdfxoutputcondition () /pdfxregistryname () /pdfxtrapped /false /createjdffile false /description << /ara <feff06270633062a062e062f0645002006470630064700200627064406250639062f0627062f0627062a002006440625064606340627062100200648062b062706260642002000410064006f00620065002000500044004600200645062a064806270641064206290020064406440637062806270639062900200641064a00200627064406450637062706280639002006300627062a0020062f0631062c0627062a002006270644062c0648062f0629002006270644063906270644064a0629061b0020064a06450643064600200641062a062d00200648062b0627062606420020005000440046002006270644064506460634062306290020062806270633062a062e062f062706450020004100630072006f0062006100740020064800410064006f006200650020005200650061006400650072002006250635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e0635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e> /bgr <feff04180437043f043e043b043704320430043904420435002004420435043704380020043d0430044104420440043e0439043a0438002c00200437043000200434043000200441044a0437043404300432043004420435002000410064006f00620065002000500044004600200434043e043a0443043c0435043d04420438002c0020043c0430043a04410438043c0430043b043d043e0020043f044004380433043e04340435043d04380020043704300020043204380441043e043a043e043a0430044704350441044204320435043d0020043f04350447043004420020043704300020043f044004350434043f0435044704300442043d04300020043f043e04340433043e0442043e0432043a0430002e002000200421044a04370434043004340435043d043804420435002000500044004600200434043e043a0443043c0435043d044204380020043c043e0433043004420020043404300020044104350020043e0442043204300440044f0442002004410020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200441043b0435043404320430044904380020043204350440044104380438002e> /chs <feff4f7f75288fd94e9b8bbe5b9a521b5efa7684002000410064006f006200650020005000440046002065876863900275284e8e9ad88d2891cf76845370524d53705237300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /cht <feff4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef69069752865bc9ad854c18cea76845370524d5370523786557406300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /cze <feff005400610074006f0020006e006100730074006100760065006e00ed00200070006f0075017e0069006a007400650020006b0020007600790074007600e101590065006e00ed00200064006f006b0075006d0065006e0074016f002000410064006f006200650020005000440046002c0020006b00740065007200e90020007300650020006e0065006a006c00e90070006500200068006f006400ed002000700072006f0020006b00760061006c00690074006e00ed0020007400690073006b00200061002000700072006500700072006500730073002e002000200056007900740076006f01590065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f007400650076015900ed007400200076002000700072006f006700720061006d0065006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076011b006a016100ed00630068002e> /dan <feff004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000620065006400730074002000650067006e006500720020007300690067002000740069006c002000700072006500700072006500730073002d007500640073006b007200690076006e0069006e00670020006100660020006800f8006a0020006b00760061006c0069007400650074002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /deu <feff00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f006300680077006500720074006900670065002000500072006500700072006500730073002d0044007200750063006b0065002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /esp <feff005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f00730020005000440046002000640065002000410064006f0062006500200061006400650063007500610064006f00730020007000610072006100200069006d0070007200650073006900f3006e0020007000720065002d0065006400690074006f007200690061006c00200064006500200061006c00740061002000630061006c0069006400610064002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /eti <feff004b00610073007500740061006700650020006e0065006900640020007300e4007400740065006900640020006b00760061006c006900740065006500740073006500200074007200fc006b006900650065006c007300650020007000720069006e00740069006d0069007300650020006a0061006f006b007300200073006f00620069006c0069006b0065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740069006400650020006c006f006f006d006900730065006b0073002e00200020004c006f006f0064007500640020005000440046002d0064006f006b0075006d0065006e00740065002000730061006100740065002000610076006100640061002000700072006f006700720061006d006d006900640065006700610020004100630072006f0062006100740020006e0069006e0067002000410064006f00620065002000520065006100640065007200200035002e00300020006a00610020007500750065006d006100740065002000760065007200730069006f006f006e00690064006500670061002e000d000a> /fra <feff005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f0075007200200075006e00650020007100750061006c0069007400e90020006400270069006d007000720065007300730069006f006e00200070007200e9007000720065007300730065002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /gre <feff03a703c103b703c303b903bc03bf03c003bf03b903ae03c303c403b5002003b103c503c403ad03c2002003c403b903c2002003c103c503b803bc03af03c303b503b903c2002003b303b903b1002003bd03b1002003b403b703bc03b903bf03c503c103b303ae03c303b503c403b5002003ad03b303b303c103b103c603b1002000410064006f006200650020005000440046002003c003bf03c5002003b503af03bd03b103b9002003ba03b103c42019002003b503be03bf03c703ae03bd002003ba03b103c403ac03bb03bb03b703bb03b1002003b303b903b1002003c003c103bf002d03b503ba03c403c503c003c903c403b903ba03ad03c2002003b503c103b303b103c303af03b503c2002003c503c803b703bb03ae03c2002003c003bf03b903cc03c403b703c403b103c2002e0020002003a403b10020005000440046002003ad03b303b303c103b103c603b1002003c003bf03c5002003ad03c703b503c403b5002003b403b703bc03b903bf03c503c103b303ae03c303b503b9002003bc03c003bf03c103bf03cd03bd002003bd03b1002003b103bd03bf03b903c703c403bf03cd03bd002003bc03b5002003c403bf0020004100630072006f006200610074002c002003c403bf002000410064006f00620065002000520065006100640065007200200035002e0030002003ba03b103b9002003bc03b503c403b103b303b503bd03ad03c303c403b503c103b503c2002003b503ba03b403cc03c303b503b903c2002e> /heb <feff05d405e905ea05de05e905d5002005d105d405d205d305e805d505ea002005d005dc05d4002005db05d305d9002005dc05d905e605d505e8002005de05e105de05db05d9002000410064006f006200650020005000440046002005d405de05d505ea05d005de05d905dd002005dc05d405d305e405e105ea002005e705d305dd002d05d305e405d505e1002005d005d905db05d505ea05d905ea002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e05d005de05d905dd002005dc002d005000440046002f0058002d0033002c002005e205d905d905e005d5002005d105de05d305e805d905da002005dc05de05e905ea05de05e9002005e905dc0020004100630072006f006200610074002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e> /hrv (za stvaranje adobe pdf dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. stvoreni pdf dokumenti mogu se otvoriti acrobat i adobe reader 5.0 i kasnijim verzijama.) /hun <feff004b0069007600e1006c00f30020006d0069006e0151007300e9006701710020006e0079006f006d00640061006900200065006c0151006b00e90073007a00ed007401510020006e0079006f006d00740061007400e100730068006f007a0020006c006500670069006e006b00e1006200620020006d0065006700660065006c0065006c0151002000410064006f00620065002000500044004600200064006f006b0075006d0065006e00740075006d006f006b0061007400200065007a0065006b006b0065006c0020006100200062006500e1006c006c00ed007400e10073006f006b006b0061006c0020006b00e90073007a00ed0074006800650074002e0020002000410020006c00e90074007200650068006f007a006f00740074002000500044004600200064006f006b0075006d0065006e00740075006d006f006b00200061007a0020004100630072006f006200610074002000e9007300200061007a002000410064006f00620065002000520065006100640065007200200035002e0030002c0020007600610067007900200061007a002000610074007400f3006c0020006b00e9007301510062006200690020007600650072007a006900f3006b006b0061006c0020006e00790069007400680061007400f3006b0020006d00650067002e> /ita <feff005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f00620065002000500044004600200070006900f900200061006400610074007400690020006100200075006e00610020007000720065007300740061006d0070006100200064006900200061006c007400610020007100750061006c0069007400e0002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /jpn <feff9ad854c18cea306a30d730ea30d730ec30b951fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e305930023053306e8a2d5b9a306b306f30d530a930f330c8306e57cb30818fbc307f304c5fc59808306730593002> /kor <feffc7740020c124c815c7440020c0acc6a9d558c5ec0020ace0d488c9c80020c2dcd5d80020c778c1c4c5d00020ac00c7a50020c801d569d55c002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /lth <feff004e006100750064006f006b0069007400650020016100690075006f007300200070006100720061006d006500740072007500730020006e006f0072011700640061006d00690020006b0075007200740069002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b00750072006900650020006c0061006200690061007500730069006100690020007000720069007400610069006b007900740069002000610075006b01610074006f00730020006b006f006b007900620117007300200070006100720065006e006700740069006e00690061006d00200073007000610075007300640069006e0069006d00750069002e0020002000530075006b0075007200740069002000500044004600200064006f006b0075006d0065006e007400610069002000670061006c006900200062016b007400690020006100740069006400610072006f006d00690020004100630072006f006200610074002000690072002000410064006f00620065002000520065006100640065007200200035002e0030002000610072002000760117006c00650073006e0117006d00690073002000760065007200730069006a006f006d00690073002e> /lvi <feff0049007a006d0061006e0074006f006a00690065007400200161006f00730020006900650073007400610074012b006a0075006d00750073002c0020006c0061006900200076006500690064006f00740075002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006100730020006900720020012b00700061016100690020007000690065006d01130072006f00740069002000610075006700730074006100730020006b00760061006c0069007401010074006500730020007000690072006d007300690065007300700069006501610061006e006100730020006400720075006b00610069002e00200049007a0076006500690064006f006a006900650074002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006f002000760061007200200061007400760113007200740020006100720020004100630072006f00620061007400200075006e002000410064006f00620065002000520065006100640065007200200035002e0030002c0020006b0101002000610072012b00200074006f0020006a00610075006e0101006b0101006d002000760065007200730069006a0101006d002e> /nld (gebruik deze instellingen om adobe pdf-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. de gemaakte pdf-documenten kunnen worden geopend met acrobat en adobe reader 5.0 en hoger.) /nor <feff004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d00200065007200200062006500730074002000650067006e0065007400200066006f00720020006600f80072007400720079006b006b0073007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /pol <feff0055007300740061007700690065006e0069006100200064006f002000740077006f0072007a0065006e0069006100200064006f006b0075006d0065006e007400f300770020005000440046002000700072007a0065007a006e00610063007a006f006e00790063006800200064006f002000770079006400720075006b00f30077002000770020007700790073006f006b00690065006a0020006a0061006b006f015b00630069002e002000200044006f006b0075006d0065006e0074007900200050004400460020006d006f017c006e00610020006f007400770069006500720061010700200077002000700072006f006700720061006d006900650020004100630072006f00620061007400200069002000410064006f00620065002000520065006100640065007200200035002e0030002000690020006e006f00770073007a0079006d002e> /ptb <feff005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020006d00610069007300200061006400650071007500610064006f00730020007000610072006100200070007200e9002d0069006d0070007200650073007300f50065007300200064006500200061006c007400610020007100750061006c00690064006100640065002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /rum <feff005500740069006c0069007a00610163006900200061006300650073007400650020007300650074010300720069002000700065006e007400720075002000610020006300720065006100200064006f00630075006d0065006e00740065002000410064006f006200650020005000440046002000610064006500630076006100740065002000700065006e0074007200750020007400690070010300720069007200650061002000700072006500700072006500730073002000640065002000630061006c006900740061007400650020007300750070006500720069006f006100720103002e002000200044006f00630075006d0065006e00740065006c00650020005000440046002000630072006500610074006500200070006f00740020006600690020006400650073006300680069007300650020006300750020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e00300020015f00690020007600650072007300690075006e0069006c006500200075006c0074006500720069006f006100720065002e> /rus <feff04180441043f043e043b044c04370443043904420435002004340430043d043d044b04350020043d0430044104420440043e0439043a043800200434043b044f00200441043e043704340430043d0438044f00200434043e043a0443043c0435043d0442043e0432002000410064006f006200650020005000440046002c0020043c0430043a04410438043c0430043b044c043d043e0020043f043e04340445043e0434044f04490438044500200434043b044f00200432044b0441043e043a043e043a0430044704350441044204320435043d043d043e0433043e00200434043e043f0435044704300442043d043e0433043e00200432044b0432043e04340430002e002000200421043e043704340430043d043d044b04350020005000440046002d0434043e043a0443043c0435043d0442044b0020043c043e0436043d043e0020043e0442043a0440044b043204300442044c002004410020043f043e043c043e0449044c044e0020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200431043e043b043504350020043f043e04370434043d043804450020043204350440044104380439002e> /sky <feff0054006900650074006f0020006e006100730074006100760065006e0069006100200070006f0075017e0069007400650020006e00610020007600790074007600e100720061006e0069006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b0074006f007200e90020007300610020006e0061006a006c0065007001610069006500200068006f0064006900610020006e00610020006b00760061006c00690074006e00fa00200074006c0061010d00200061002000700072006500700072006500730073002e00200056007900740076006f00720065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f00740076006f00720069016500200076002000700072006f006700720061006d006f006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076016100ed00630068002e> /slv <feff005400650020006e006100730074006100760069007400760065002000750070006f0072006100620069007400650020007a00610020007500730074007600610072006a0061006e006a006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b006900200073006f0020006e0061006a007000720069006d00650072006e0065006a016100690020007a00610020006b0061006b006f0076006f00730074006e006f0020007400690073006b0061006e006a00650020007300200070007200690070007200610076006f0020006e00610020007400690073006b002e00200020005500730074007600610072006a0065006e006500200064006f006b0075006d0065006e0074006500200050004400460020006a00650020006d006f0067006f010d00650020006f0064007000720065007400690020007a0020004100630072006f00620061007400200069006e002000410064006f00620065002000520065006100640065007200200035002e003000200069006e0020006e006f00760065006a01610069006d002e> /suo <feff004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f00740020006c00e400680069006e006e00e4002000760061006100740069007600610061006e0020007000610069006e006100740075006b00730065006e002000760061006c006d0069007300740065006c00750074007900f6006800f6006e00200073006f00700069007600690061002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a0061002e0020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /sve <feff0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400200073006f006d002000e400720020006c00e4006d0070006c0069006700610020006600f60072002000700072006500700072006500730073002d007500740073006b00720069006600740020006d006500640020006800f600670020006b00760061006c0069007400650074002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /tur <feff005900fc006b00730065006b0020006b0061006c006900740065006c0069002000f6006e002000790061007a006401310072006d00610020006200610073006b013100730131006e006100200065006e0020006900790069002000750079006100620069006c006500630065006b002000410064006f006200650020005000440046002000620065006c00670065006c0065007200690020006f006c0075015f007400750072006d0061006b0020006900e70069006e00200062007500200061007900610072006c0061007201310020006b0075006c006c0061006e0131006e002e00200020004f006c0075015f0074007500720075006c0061006e0020005000440046002000620065006c00670065006c0065007200690020004100630072006f006200610074002000760065002000410064006f00620065002000520065006100640065007200200035002e003000200076006500200073006f006e0072006100730131006e00640061006b00690020007300fc007200fc006d006c00650072006c00650020006100e70131006c006100620069006c00690072002e> /ukr <feff04120438043a043e0440043804410442043e043204430439044204350020044604560020043f043004400430043c043504420440043800200434043b044f0020044104420432043e04400435043d043d044f00200434043e043a0443043c0435043d044204560432002000410064006f006200650020005000440046002c0020044f043a04560020043d04300439043a04400430044904350020043f045604340445043e0434044f0442044c00200434043b044f0020043204380441043e043a043e044f043a04560441043d043e0433043e0020043f0435044004350434043404400443043a043e0432043e0433043e0020043404400443043a0443002e00200020042104420432043e04400435043d045600200434043e043a0443043c0435043d0442043800200050004400460020043c043e0436043d04300020043204560434043a0440043804420438002004430020004100630072006f006200610074002004420430002000410064006f00620065002000520065006100640065007200200035002e0030002004300431043e0020043f04560437043d04560448043e04570020043204350440044104560457002e> /enu (use these settings to create adobe pdf documents best suited for high-quality prepress printing. created pdf documents can be opened with acrobat and adobe reader 5.0 and later.) >> /namespace [ (adobe) (common) (1.0) ] /othernamespaces [ << /asreaderspreads false /cropimagestoframes true /errorcontrol /warnandcontinue /flattenerignorespreadoverrides false /includeguidesgrids false /includenonprinting false /includeslug false /namespace [ (adobe) (indesign) (4.0) ] /omitplacedbitmaps false /omitplacedeps false /omitplacedpdf false /simulateoverprint /legacy >> << /addbleedmarks false /addcolorbars false /addcropmarks false /addpageinfo false /addregmarks false /convertcolors /converttocmyk /destinationprofilename () /destinationprofileselector /documentcmyk /downsample16bitimages true /flattenerpreset << /presetselector /mediumresolution >> /formelements false /generatestructure false /includebookmarks false /includehyperlinks false /includeinteractive false /includelayers false /includeprofiles false /multimediahandling /useobjectsettings /namespace [ (adobe) (creativesuite) (2.0) ] /pdfxoutputintentprofileselector /documentcmyk /preserveediting true /untaggedcmykhandling /leaveuntagged /untaggedrgbhandling /usedocumentprofile /usedocumentbleed false >> ] >> setdistillerparams << /hwresolution [2400 2400] /pagesize [612.000 792.000] >> setpagedevice multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 89 the effects of active recreational maths games on maths anxiety and performance in primary school children: an experimental study homoud mohammed n alanazi college of education and arts, northern border university (nbu). po box 1321, arar, northern border province 91431 kingdom of saudi arabia email: halanazi@nbu.edu.sa received: 02 november 2019; accepted: 08 march 2020; published: april 2020 abstract this study investigates the effect of active recreational maths games (armg) on 1st grade male students’ maths anxiety and performance. the sample was divided into two groups: an experimental group (n=28), which was taught maths with armg in addition to traditional teaching methods, and a control group (n=30), which was taught with traditional teaching methods only. the armg consisted of 24 sessions taking place over a period of two months. students participated in three sessions per week, each of which lasted 45 minutes. the findings revealed a significant negative correlation (r=-0.482; p<0.05) between maths anxiety and students’ performance. a significant difference (α= 0.05) in maths anxiety and performance emerged between the experimental and control groups. the experimental group obtained lower maths anxiety scores and higher performance scores than the control group. considering these results, the study proposes several recommendations and suggestions to develop maths teaching with armg. keywords: recreation games, maths anxiety; maths performance; primary school; teaching strategy. https://doi.org/10.4995/muse.2020.12622 mailto:halanazi@nbu.edu.sa https://orcid.org/0000-0002-1597-5092 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 90 1. introduction the twenty-first century teacher must impart knowledge to students, but also develop successful teaching strategies to facilitate the learning process (johnson, 2013). in the past decades, educators have been increasingly aware of the necessity of examining which teaching strategies are the most effective (eady & lockyer, 2013; johnson, 2013). educators identify different learning methods to assist them in creating and developing the right strategies to deal with their target groups and realizing their educational objectives by imparting relevant knowledge and skills, and the necessary positive attitudes towards learning (arthur et al., 2017; kelly & pohl, 2018). according to harris (2000), a “[t]eaching strategy is a generalized plan for a lesson or lessons which includes structure, desired learner behavior in terms of goals of instructions and an outline of planned tactics necessary to implement the strategy” (p. 12). teaching strategies refer to methods, techniques, procedures and processes used by a teacher to help students learn the desired course content and develop achievable goals in the future (tiwari, rathar, & singh, 2007). primary education is the first phase of compulsory education in many countries and the starting point of the education system (alharbi & madhesh, 2018). educators have indicated that one of the most essential academic periods is the primary stage, for much of the cognitive, social, emotional, cultural and physical skills, alongside the personality and general development of a child, takes place during this period (rosalind, 2011). beliefs, values, and feelings, either positive or negative, towards school and different subjects are often formed in primary school (hacieminoglu, 2016; anggoro, 2017). for this reason, most advanced countries have directed special efforts toward the development of primary education (rosalind, 2011). maths is an essential subject of the school curriculum and is important in daily living (yükselşahin, 2008). moreover, it is an instrument for the development of all other sciences (divjak & tomic, 2011). however, there is a common belief that the majority of students dislike maths due to its very nature (luttenberger, wimmer, & paechter, 2018). more specifically, geist (2010) has reviewed a number of psychological studies and concluded that maths anxiety was the main factor for the aversion of maths. psychological explanations suggest that maths anxiety stems from https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 91 weaknesses in the curriculum, negative experiences with maths, pressure and family expectation, peer pressure, teaching styles, and learning environments (geist, 2010; luttenberger et al., 2018). this study investigates the effectiveness and positive outcomes of active recreation on maths anxiety and students’ performance. to achieve this, the study seeks to fulfill two objectives: • to find out whether maths anxiety influences students’ performance. • to examine the difference between the maths anxiety and performance scores of the control group who used traditional teaching instruction (tti) and the experimental group who used traditional teaching instruction supported by active recreational maths game (armg). results of the study will help students, teachers, counselors, and parents gain greater insight into how to avoid the negative effects of anxiety in maths instruction. in addition, the study assists instructional and recreational game designers by creating information about the required game components to reduce maths anxiety, optimize academic performance and avoid negative effect on future career choice. moreover, it offers a novel comparison between results obtained in saudi arabia, a developing country, and results from foreign studies. 2. literature review maths anxiety is commonly defined as “a feeling of tension, apprehension, or fear that interferes with maths performance” (ashcraft, 2002, p.181). more recently, lorenzen (2017) described maths anxiety as a state of discomfort that one experiences when involved in situations requiring the use of mathematics and that can affect people of all ages. therefore, it can be regarded as negative psychological reactions involving fear, apprehension, low confidence, and tension towards any activities related to maths skills or numerical manipulations (luttenberger et al., 2018). maths anxiety has been a concern for researchers because of its proven negative effect on students’ performance (wu, barth, amin, malcarne, & menon, 2012; siebers, 2015; puteh & khalin, 2016). geist (2010) and luttenberger et al. (2018) have found evidence that maths anxiety may cause low performance because children with high maths anxiety tend to avoid maths-related tasks. https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 92 understanding maths anxiety during childhood is most important. studies have found that maths anxiety increased with age among 1st to 6th grade school children with ages ranging from 7 to 12 years old (wood et al., 2012). furthermore, preventing a growing negative attitude towards maths is especially crucial considering that it has been shown to affect the child in the long term and into adulthood (kendall, safford, flannery-schroeder, & webb, 2004; scarpello, 2007; dunn & goodyer, 2006; siebald, khandaker, zammit, lewis, & jones, 2016). maths anxiety in childhood also affect career choices. scarpello (2007) has found that seventy-five percent of americans stop studying maths and avoid careers that related to mathematics due to childhood maths anxiety. espino, pereda, recon, perculeza, and umali (2017) reached the same conclusion in relation to students in the philippines. they found that maths anxiety has a significant effect on further studies and career choice of grade 11 students. because of maths anxiety, most of the respondents tended to pursue humanities and social sciences in their education. hence, furner and gonzalez-dehass (2011, p. 226) stressed that “maths anxiety is a real issue that can impact a child goals, many career-related decisions they may make in life and their overall future”. therefore, researchers have focused on developing effective teaching tools to improve students’ academic performance in maths to offer alternatives to traditional teaching strategies. there is solid evidence that enjoyment is one of the most important components of effective teaching (zosh et al., 2017; liu et al., 2017). games, for instance, will facilitate the learning process and satisfy most educational needs in childhood (zosh et al, 2017). therefore, studies have tested strategies to make the learning process more enjoyable, such as computer and video games, to improve students’ maths performance (sayan, 2015; hieftje, tyra, kyriakides, gilliam, & fiellin, 2017; chizary & farhangi, 2017). with such games, children can learn graphs, charts, shapes, colors, as well as gain positive values and experiences (liu, 2017). according to hurd and anderson (2011), recreation is defined as an activity that individuals engage in to satisfy their needs for excitement, fun, amusement, and enjoyment. recreational activities can be broadly categorized as passive or active. passive recreation refers to nonmotorized activities for which a person does not exert any significant physical energy (e.g. going to the cinema, watching television, playing computer and video games). active recreation refers to activities that require physical exertion (e.g. sports, walking, dancing, playing on a playground or any enjoyable physical activity (hurd & anderson, 2011; salmon et al., 2019). https://doi.org/10.4995/muse.2020.12622 https://en.wikipedia.org/wiki/amusement multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 93 the general consensus is that passive recreation (e.g. television and video games) is negatively correlated with well-being, while physically active recreation activities positively contribute to the psychological well-being of children (holder, coleman, & sehn, 2009; eime, young, harvey, charity, & payne, 2013; çetinkaya, sahin, & yariz, 2017; laidley & conley, 2018; salmon et al., 2019). they can enhance self-efficacy and self-esteem (nieman, 2002), self-concept and self-confidence (scully, kremer, meade, graham, & dudgeon,1998), provide opportunities for more positive social interactions (eime et al., 2013), and reduce depression, anxiety, and symptoms related to attention deficit disorders (biddle & asare, 2011; goldstein, 2012; paggi, jopp, & hertzog, 2016). generally, the desire to progress in the game, improve the way of playing and win are constructive and educational aspects of games (cagiltay, ozcelik, ozcelik, 2015). they motivate passive students to think (burguillo, 2010; cagiltay et al., 2015). findings of empirical studies revealed that a combination of game playing and friendly competitions resulted in strong motivation and increased learning effectiveness (burguillo, 2010; divjak & tomic, 2011; cagiltay et al., 2015). researchers suggest that a successful computer game must meet pedagogical criteria and be exciting in order to have a positive effect on students (kebritchi, hirumi, & bai, 2010; divjak & tomic, 2011). literature on recreational maths games, more specifically, has focused on the effects of passive recreation games such as computer games (kebritchi et al., 2010; al-mashaqbeh & al dweri, 2014; sayan, 2015; ezeugwu, onuorah, asogwa, & ukoha, 2016; chizary & farhangi, 2017), and video games (abdullah, abu-bakar, ali, faye, & hasan, 2012; ibrahim & abu-hmaid, 2017; hieftje et.al, 2017) on students’ mathsematical performances. in addition, literature on maths anxiety has concentrated on the reasons for maths anxiety and its relationship with students’ performance (siebers, 2015; puteh & khalin, 2016; reali et al, 2016; luttenberger et al., 2018; kundu & kar, 2018). other researches have examined the effects of maths anxiety on future career choice (scarpello, 2007; espino et al., 2017), while others investigated gender differences in maths anxiety (yüksel-şahin, 2008; keshavarzi & ahmadi, 2013; yeo, tan, & lew, 2015; hill et al., 2016). https://doi.org/10.4995/muse.2020.12622 https://www.researchgate.net/profile/abdollah_keshavarzi multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 94 in summation, the use recreational games to improve maths performance has been limited to passive games such as computer and video games. studies have not addressed the potential of active recreational maths games to treat maths anxiety and improve maths performance. however, children are more likely to prefer sport and active recreational activities than passive recreation activities (stodolska, shinew, & li, 2010; ullenhag, krumlinde-sundholm, granlund, & almqvist, 2014; smith, hannon, brusseau, fu, & burns, 2016). in addition, as we have seen, active recreational activities contribute to reduce anxiety (biddle & asare, 2011; goldstein, 2012; birturk & karagun, 2015; paggi et al., 2016). participation in active recreational maths games may change negative attitudes toward maths and improve maths abilities, since negative experiences with maths contribute to maths anxiety and poor performances (jansen et al., 2013; luttenberger et al., 2018). given this, the theoretical framework of this study is represented in the following figure (1). • maths anxiety • maths performance control group: treatment with traditional teaching instruction teaching method experimental group: treatment with armg figure 1. theoretical framework to examine the effectiveness of active recreation games on maths anxiety and students’ performance to conclude, this study contributes to the existing literature by exploring the effects of active recreational maths games on the alleviation of maths anxiety and the improvement of students’ maths performance. 3. materials and methods this study is experimental in nature. its design includes two groups – the experimental and the control group – a pre-test and a post-test. the population sample consisted of students studying for one semester. sixty students were recruited from the first year of a primary school in saudi independent variable dependent variables https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 95 arabia. a permission was taken from the ministry of education as well as the participants' parents to carry out the study. a parental consent form was also sent to their parents in order to get their approval. the consent form included the main objective and significance of the study. the parents were informed that their sons' participation will be treated confidentially and all information will be kept anonymous, meaning that no one will be able to obtain the answers provided by their sons. out of 66 parental consent forms sent, 60 ones were received. the participants were randomly divided into two groups (32 in the control group, 28 in the experimental group) by flipping a coin for each student. as stated by price, jhangiani, chiang, leighton, & cuttler (2017), random assignment is a key element of all experimental research that allows researchers to assign participants to the different conditions groups. random assignment should meet two criteria. each student must have a 50 percent chance of being assigned to each condition and must be assigned to a condition independently of other students (price et al., 2017). the sample were chosen from one government primary school and was taught by the same teacher to ensure uniformity and equivalence in the traditional way of teaching maths, so as to ensure comparability between the groups. maths anxiety and performance were pre-tested in both groups to further ensure uniformity and equivalence between the two groups. the values of mean and standard deviations in table 1 show that the scores of both groups on the maths anxiety scale and the maths performance test were normally distributed and almost equal. the calculated t-value (less than table value) demonstrates that the difference between the two groups was not significant. table 1. difference between the mean of the pre-test scores of the control group and the experimental group. * significant at p≤ 0.05, table value at 0.05 = 2.042. variable groups n mean sd df t-value p remark maths anxiety experimental 28 2.42 0.271 58 0.498 0.147 not significant control 32 2.44 0.268 maths performance test experimental 28 12.58 2.413 58 0.598 0.496 not significant control 32 13.04 2.862 https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 96 the armg sessions were organized for the experimental group over a period of two months, while the control group was taught with traditional maths teaching methods only. the sessions lasted 45 minutes each and took place 3 times a week, for a total of 24 sessions. each session included a warm-up, a main activity and a cool down. the two groups took a post-test at the end of the experiment, close to the end of the semester, to compare students’ performance and anxiety level in both groups. the 60 questionnaires were analyzed using the statistical package for social sciences (spss) version 20.0, and the following statistical methods of data analysis were used: • descriptive statistics were gathered: means and standard deviations (std). • pearson correlation coefficient was used to find out the correlation between maths anxiety and students’ performance in mathematics. • independent sample t-test was used to determine the differences between the maths anxiety scores of the control group who received tti and the experimental group who received traditional teaching instruction supported with armg. 3.1 active recreational maths game design several design elements have been explored to improve the benefits of games such as competition, challenge, feedback, uncertainty, goals, fantasy, learner control, cooperation, interactivity, flexibility, and fairness (burguillo, 2010; zosh et al., 2017; chizary & farhangi, 2017). cagiltay et al. (2015) recommend six necessary features of games: (i) rules, (ii) variable and quantifiable outcomes, (iii) value assigned to outcomes, (iv) player effort to influence outcomes, (v) participant attached to outcomes, and (vi) negotiable consequences of activities. to guide the participants towards the objectives of the game, the goals and rules to be followed by the participant need to be provided explicitly (burguillo, 2010; cagiltay et al., 2015). in the present study, the design of the armg takes several of the abovementioned components into consideration and focused on 1st grade maths objectives and curriculum in saudi primary school students, including counting, subtraction, additions, shapes, numbers order etc. (see saudi arabian 1st grade maths (year 1) curriculum in appendix). one of the activities exercised in this study to make learning addition and subtraction more enjoyable was by dividing the students into two teams (14 each). a movement story was read containing addition – subtraction questions. https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 97 for example, there are (9) baby birds in a nest, four of them flew away, how many are stayed in the nest? a circle was drawn representing the nest for each trailer. each of them performs the movement story, (9) birds of each trailer get into the nest while (4) players fly away imitating birds sounds while leaving. the trailer that gets the results of the story first is declared winner. stories of abstraction and addition are repeated using other animals like cats and dogs, imitating their sounds, in order to get the result of such math game. to ensure the adequacy of the recreational maths games in relation to the physical skills and maths curriculum of 1st graders, the sessions were reviewed by six experts in leisure and recreation field and six experts in maths education. all experts held doctorates and their academic rank ranged between associate professors and professors. sessions were revised and modified based on the experts’ feedback and recommendations. 3.2 the maths performance test the researcher used a maths performance test to measure the 1st grade students’ maths performance. the maths performance test was developed by the researcher. questions were developed based on the objectives of the maths textbook units. the test consisted of 18 questions covering the maths curriculum. the total mark for the test was 55. to ensure the validity and reliability of the performance test, a panel of six experts in maths education (trustees validity) reviewed the test and offered some suggestions to improve it. to assess the reliability of the test, a pilot study was implemented. pilot subjects were excluded from the main study. three weeks later, the same subjects took the test again and the results were correlated using pearson’s formula. the score obtained, r-value = 0.86, indicated a good reliability. 3.3 maths anxiety scale the original mathematics anxiety scale for children was developed by chiu and henry (1990) in order to assess the anxiety level of children during certain situations involving maths. the scale has been standardized for primary school populations and has adequate test/retest reliability, internal consistency, and concurrent validity (wu et al., 2012; jansen et al., 2013; keshavarzi & ahmadi, 2013; hoorfar & taleb, 2015). https://doi.org/10.4995/muse.2020.12622 https://www.researchgate.net/profile/abdollah_keshavarzi multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 98 the chiu and henry’s scale has been modified and translated into arabic language by adnan and ibrahim (1994) to assess the mathematics anxiety of arab children in primary schools. the modified arabic version of the maths anxiety scale includes 23 items, has been used by many researchers (sawalha & asfa, 2008; ladjal, 2016; ali, 2017), and was found to be reliable and adequate to conduct surveys with primary school students in arab countries. cronbach’s alpha in the present study was measured as 0.88. children rated their level of anxiety in the situation presented in each statement on a 3-point likert-type scale (1 = not nervous, 2 = a little bit nervous, 3 = very nervous). a high score on the maths anxiety scale was indicative of high maths anxiety. when requested, participants received clarification on the meaning of any item. 4. results 4.1 the effect of maths anxiety on students’ performance pearson correlation coefficient was used to assess the relationship between maths anxiety and students’ maths performance. table 2. correlation level between maths anxiety and students’ maths performance in the sample as a whole * correlation is significant at the 0.05 level, p ≤ 0.05 (2-tailed). the findings in table 2 revealed that sig (significance level) is significant at 0.05 error level. with 95 percent certainty, the result showed negative correlation between maths anxiety and students’ performance (r = -.482, p < 0.05). students who had high maths anxiety performed poorly in maths. 4.2 the difference in maths anxiety levels between the experimental and control group tables 3 and 4 offer descriptive data of the pre-test and post-test scores. the means and standard deviations of the scores obtained on the maths anxiety scale were calculated. variable maths performance n r remark maths anxiety 60 -.482* significant https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 99 table (3) pre-test scores of control group and experimental group on maths anxiety scale. no item groups weighted mean sd interpretation rank 1 getting a new maths book experimental 2.06 0.269 moderate 21 control 1.96 0.269 moderate 23 2 reading and interpreting graphs and charts experimental 2.15 0.306 moderate 20 control 2.42 0.278 high 13 3 listening to another student explain a maths problem experimental 2.17 0.280 moderate 19 control 2.19 0.273 moderate 22 4 watching a teacher work on maths problems on the chalk board experimental 2.77 0.259 high 1 control 2.65 0.269 high 3 5 solving a fraction problem experimental 2.44 0.291 high 12 control 2.49 0.278 high 9 6 looking through the pages of a maths book experimental 2.74 0.255 high 3 control 2.76 0.266 high 1 7 starting a new chapter in a maths book experimental 2.30 0.270 moderate 16 control 2.43 0.288 high 12 8 thinking about maths outside of class experimental 1.94 0.305 moderate 23 control 2.24 0.273 moderate 21 9 picking up a maths book to begin working on a homework assignment experimental 2.56 0.266 high 8 control 2.58 0.270 high 6 10 working on a problem, such as "if i spend $3.87 at the store, how much change would i get from a $5?" experimental 2.37 0.295 high 15 control 2.47 0.251 high 10 11 solving an addition problem experimental 2.51 0.310 high 9 control 2.35 0.281 high 18 12 listening to the teacher in maths class experimental 2.45 0.244 high 11 control 2.39 0.269 high 15 13 being involved in a competition including numerical manipulations experimental 2.62 0.291 high 5 control 2.59 0.253 high 5 14 being told how to interpret mathsematical statements experimental 2.60 0.236 high 6 control 2.64 0.278 high 4 15 being given many difficult maths problems as a homework assignment due the next day experimental 2.73 0.283 high 4 control 2.55 0.282 high 7 16 thinking about a maths test one day before the test experimental 2.60 0.301 high 6 control 2.51 0.279 high 8 17 solving a subtraction problem experimental 2.22 0.245 moderate 18 control 2.40 0.267 high 14 18 taking a maths quiz experimental 2.28 0.277 moderate 17 control 2.30 0.287 moderate 19 19 getting ready to study for a maths test experimental 2.50 0.268 high 10 control 2.46 0.272 high 11 20 being given a maths quiz that you were not told about experimental 2.44 0.318 high 12 control 2.36 0.264 high 17 21 waiting to get a maths test returned when you expect to have done well experimental 2.02 0.302 moderate 22 control 2.27 0.277 moderate 20 22 taking an important test in maths class experimental 2.38 0.286 high 14 control 2.39 0.257 high 15 23 being asked to answer a maths problem on the chalk board in front of your colleagues experimental 2.76 0.279 high 2 control 2.74 0.289 high 2 overall weighted mean experimental 2.42 0.271 high control 2.44 0.268 high https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 100 no item groups weighted mean sd interpretation rank overall weighted mean for all groups 2.432 0.172 high legend: low = 1.00-1.66; moderate= 1.67-2.33; high = 2.34-3.00. the pre-test scores on the maths anxiety scale are provided in table 3. the weighted means were used to describe the levels of maths anxiety. a mean between 1.00 and 1.66 was interpreted as a low level of maths anxiety, between 1.67 and 2.33 as a moderate level, and between 2.34 and 3.00 as a high level. the overall weighted mean for the experimental group was 2.42 with a standard deviation of 0.271, and was interpreted as a high level of maths anxiety. for the control group, the overall weighted mean was 2.44 with a standard deviation of 0.268, and was interpreted as a high level of maths anxiety. the overall weighted mean for all groups was 2.432 with a standard deviation of 0.172, and was interpreted as a high level of maths anxiety. these results demonstrate that 1st grade students suffer from high-level maths anxiety. table (4) post-test scores of the control group and the experimental group on the maths anxiety scale. no item groups weighted mean sd interpretation rank 1 getting a new maths book experimental 2.16 0.298 moderate 6 control 2.61 0.197 high 22 2 reading and interpreting graphs and charts experimental 1.98 0.306 moderate 13 control 2.78 0.205 high 13 3 listening to another student explain a maths problem experimental 1.99 0.297 moderate 12 control 2.78 0.214 high 13 4 watching a teacher work on a maths problem on the chalk board experimental 2.27 0.304 moderate 1 control 2.87 0.231 high 3 5 solving a fraction problem experimental 2.09 0.311 moderate 8 control 2.82 0.223 high 8 6 looking through the pages of a maths book experimental 2.01 0.299 moderate 11 control 2.77 0.205 high 16 7 starting a new chapter in a maths book experimental 2.24 0.313 moderate 2 control 2.78 0.213 high 13 8 thinking about maths outside of class experimental 1.90 0.309 moderate 18 control 2.77 0.227 high 16 9 picking up a maths book to begin working on a homework assignment experimental 2.14 0.323 moderate 7 control 2.80 0.218 high 10 10 working on a problem, such as "if i spend $3.87 at the store, how much change would i get from a $5?" experimental 1.93 0.316 moderate 16 control 2.76 0.199 high 18 11 solving an addition problem experimental 1.88 0.295 moderate 19 https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 101 no item groups weighted mean sd interpretation rank control 2.71 0.210 21 12 listening to the teacher in maths class experimental 2.22 0.307 moderate 3 control 2.81 0.219 high 9 13 being involved in a competition including numerical manipulations experimental 1.81 0.301 moderate 21 control 2.79 0.213 high 11 14 being told how to interpret mathsematical statements experimental 1.96 0.298 moderate 14 control 2.79 0.221 high 11 15 being given many difficult maths problems as a homework assignment due the next day experimental 2.03 0.312 moderate 10 control 2.85 0.219 high 6 16 thinking about a maths test one day before the test experimental 1.74 0.322 moderate 23 control 2.74 0.211 high 20 17 solving a subtraction problem experimental 1.96 0.307 moderate 14 control 2.75 0.216 high 19 18 taking a quiz in maths experimental 1.75 0.335 moderate 22 control 2.83 o.217 high 7 19 getting ready to study for a maths test experimental 1.86 0.324 moderate 20 control 2.88 0.224 high 2 20 being given a maths quiz that you were not told about experimental 2.18 0.301 moderate 5 control 2.86 0.209 high 4 21 waiting to get a maths test returned when you expect to have done well experimental 1.92 0.317 moderate 17 control 2.53 0.214 high 23 22 taking an important test in maths class experimental 2.09 0.311 moderate 8 control 2.89 0.198 high 1 23 being asked to answer a maths problem on the chalk board in front of your colleagues experimental 2.19 0.321 moderate 4 control 2.86 0.217 high 4 overall weighted mean experimental 2.01 0.302 moderate control 2.78 0.212 high overall weighted mean for all groups 2.395 0.189 high legend: low = 1.00-1.66; moderate= 1.67-2.33; high = 2.34-3.00. as can be seen in table 4, the overall weighted mean for the experimental group was 2.42, with a standard deviation of 0.271, which was interpreted as a high level of maths anxiety. for the control group, the overall weighted mean was 2.44 with a standard deviation of 0.198, which was interpreted as a high level of maths anxiety. the overall weighted mean for all groups is 2.432 with a standard deviation of 0.172, which was interpreted as a high level of maths anxiety. it can be noted from the above tables 3 and 4 that the control group, on average, recorded a higher maths anxiety level on the post-test (m = 2.78; sd = 0.212) than they did on the pre-test (m = 2.44; sd = 0.268). in contrast, the experimental group, on average, recorded a lower level of maths anxiety on the post-test (m = 2.01; sd = 0.302) than they did on the pre-test (m = 2.42; sd = 0.271). the https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 102 difference between the post-test maths anxiety scores of the control and the experimental group was tested with the t-test and 0.05 was set as the significance threshold. table 5. significant difference between the mean of the post-test maths anxiety score in the experimental group and the control group. * significant at p≤ 0.05, table value at 0.05 = 2.042. table 5 shows that the computed t-value of 11.283 has a significance of 0.043 which is less than the 0.05 threshold. this means that there is a significant difference between the maths anxiety level of the experimental group and the control group according to the post-test. in other words, the experimental group scored significantly lower in maths anxiety than the control group as a result of the armg experiment. 4.3 the difference in maths performance between the experimental group and the control group the maths performance of the control group and the experimental group were compared to assess the effectiveness of armg on maths performance. the difference was tested with the t-test and the significance threshold was set as 0.05. table 6. significant difference between the means of the post-test performance scores in the experimental group and the control group * significant at p≤ 0.05, table value at 0.05 = 2.042. table 6 reveals that the computed t-value of 5.166 has a significance of 0.030 which is less than 0.05. therefore, there is a statistically significant difference between the performance of the variable groups n mean sd df t-value p remark maths anxiety experimental 28 2.01 0.302 58 11.283 0.043 significant control 32 2.78 0.212 variable groups n mean sd df t-value p remark performance test experimental 28 52.39 3.871 58 5.166 0.030 significant control 32 41.36 4.949 https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 103 experimental group and the control group on the post-test. this result implies that the amrg experiment significantly enhanced students’ performance. 5. discussion negative attitudes to maths are common to many countries and cultures. our study, which found that primary school students in saudi arabia report high levels of maths anxiety, is in line with previous studies on maths anxiety among primary education students in arab countries (ladjal, 2016; altakhayinah, 2018) and elsewhere (yüksel-şahin, 2008; olmez & ozel, 2012; hill et al., 2016; caviola, primi, chiesi, & mammarella, 2017). hill et al. (2016) state that maths anxiety begins at an early age. therefore, fostering positive experiences should be a priority in the early stages of maths learning to prevent children to develop negative feelings towards maths. furthermore, that control group, on average, scored higher on the maths anxiety scale on the post-test than on the pre-test. these measures corroborate two systematic reviews on maths anxiety which found that negative attitudes to mathematics tend to deteriorate linearly with age throughout childhood (dowker, sarkar, & looi, 2016; luttenberger et al., 2018). in addition, the results confirmed that maths anxiety has negative effects on maths performance (wu et al., 2012; siebers 2015; reali et al, 2016; luttenberger et al. 2018). more specifically, this study supports puteh and khalin’s (2016) argument that a high level of maths anxiety will lower students’ success in the subject because anxiety overloads and disrupts working memory during mathsematical tasks (dowker et al., 2016). on the contrary, low maths anxiety endows the tranquillity and comfort required for students to improve their maths performance. the mean of the post-test anxiety level scores in the experimental group is significantly lower than in the control group. it can be concluded that the armg experiment may reduce anxiety in the experimental group. this result is in agreement with biddle & asare (2011) and eime et al. (2013), who found that children who participate in active recreational activities report fewer symptoms of anxiety than inactive children. a number of recent systematic reviews suggest that active recreation is widely viewed as a key indicator of mental health and well-being (biddle & asare, 2011; eime et https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 104 al., 2013; rafferty et al., 2016; johnson, connolly, & tully, 2017). various studies have confirmed that recreation enhances self-concept, self-esteem, self-confidence and self-efficacy, provide social interactions opportunities, and reduces alienation, loneliness and isolation. all these elements can contribute to treat many forms of anxiety (goldstein, 2012; paggi et al., 2016; whitebread et al., 2017). the findings also show that the experimental group scored significantly higher than the control group on the maths performance post-test. this implies that recreational maths games may be effective to improve students’ maths performance. again, this result is consistent with the findings of cagiltay et al. (2015), who reported that designed games have the potential to create better learning environments to reach educational and training goals. in contrast, a study conducted by assad and wafi (2017) did not find a significant difference in post-test maths performance scores between the control group and the experimental group. the main reason for this difference could be the fact that they used computer games in their study, instead of active recreational games. this discrepancy could suggest that high enjoyment via recreational maths games and competitions contributes to children’s willingness to practice the subject, and to persist in the face of difficulties, thereby lowering maths anxiety and increasing maths performance. despite these new findings, this study presents some limitations that underline the need for further research. the study was limited to 1st grade students only. the results may not be generalized for students of other grades. future research might consider whether the same results would be achieved among students of higher or lower ability or interest in maths. in addition, the sample only consisted of male students. the results may not generalize to female classes. therefore, future research might include female students to determine whether there is a gender differences in maths anxiety and performance based on the experiment. the lack of longer-term analyses is an additional limitation which prevents from determining if the superior knowledge of the experimental group was retained over time. 6. conclusion the present study demonstrates that teachers need to be aware of the effects of maths anxiety on students’ performance and motivation. they should make an effort to create an environment in https://doi.org/10.4995/muse.2020.12622 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 105 which students feel at ease to reduce anxiety and allow them to learn in the best possible conditions. armg may be an important opportunity for children to experience positive attitudes towards maths. armg may positively predict positive outcomes and, by favoring enjoyment, prevent maths anxiety and consequently improve maths performance. challenges to the implementation of armg include the limitations of the maths curriculum, the limited recreational activities available to address the range of teaching goals and the breadth of knowledge that teachers need to learn to adapt such curricula in ways that address students’ recreation needs. limitations of the 1st grade maths curriculum for addressing learning aimed at transfer and higher order thinking and performance skills which caused maths anxiety and low maths performance. however, there is insufficient strategy supports for teachers to improve maths performance. creating effective recreational maths games is a challenging task, but one that is also likely to have great benefits for the development and well-being of children. references abdullah. m. r., abu bakar, z., ali, r. m., faye, i., & hasan, h. (2012). the impact of video games in children’s learning of mathematics. international scholarly and scientific research & innovation, 6(4), 607-613. https://search.mandumah.com/record/806335. adnan, a., & ibrahim, y. (1990). the translated arabic and modified version of maths anxiety scale: a valid and reliable instrument for arab children. dirasat: human and social sciences, 6(4), 145-161. https://search.mandumah.com/record/2796 alharbi, a., & madhesh, a. (2018). inclusive education and policy in saudi arabia. international journal of education research and reviews, 6(1): 946-956. http://www.internationalscholarsjournals.org/download.php?id=285094428401152048.pdf &type=application/pdf&op=1 ali, a. h. (2017). the effectiveness of a training program based on the strategy of creative problem solving in alleviating the maths anxiety among primary school students in saudi arabiataif city. faculty of education journal, assiut university, 33(6), 42-110. https://search.mandumah.com/record/86190 al-mashaqbeh, i., & al dweri, a. (2014). educational maths game software: a supporting tool for first grade students’ achievement. journal of education and practice, 5(5), 134-141. https://www.iiste.org/journals/index.php/jep/article/view/10963 altakhayinah, b. (2018). the impact of using the brain based learning strategy in the development of mathsematical thinking and reducing the anxiety of students in the basic stage in https://doi.org/10.4995/muse.2020.12622 https://search.mandumah.com/record/806335 https://search.mandumah.com/record/2796 http://www.internationalscholarsjournals.org/download.php?id=285094428401152048.pdf&type=application/pdf&op=1 http://www.internationalscholarsjournals.org/download.php?id=285094428401152048.pdf&type=application/pdf&op=1 http://www.aun.edu.eg/faculty_education/arabic/journal_versions.php https://search.mandumah.com/record/86190 https://www.iiste.org/journals/index.php/jep/article/view/10963 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 106 amman schools. journal of educational and psychology sciences, 26(1), 283-301. https://doi.org/10.12816/0048784 anggoro, s., sopandi. w., & sholehuddin, m. (2017). influence of joyful learning on elementary school students’ attitudes toward science. journal of physics: journal of physics: conference series, 812(1), 1-6. https://doi.org/10.1088/1742-6596/812/1/012001. arthur, c., badertscher, e., goldenberg, p., moeller, b., mcleod, m., nikula, j., & reed, k. (2017). strategies to improve all students’ mathematics learning and performance. waltham, ma: edc. ashcraft, m. h. (2002). maths anxiety: personal, educational, and cognitive consequences. current directions in psychological science, 11(5), 181-185. https://doi.org/10.1111/14678721.00196 assad, a. m., & wafi, o. r (2017). educational maths game software and students' achievement. journal of educational and psychological sciences, 4(3), 18-26. https://uqu.edu.sa/en/jep/75910 biddle, s. j. h., & asare, m. (2011). physical activity and mental health in children and adolescents: a review of reviews. british journal of sports medicine, 45, 886-895. http://dx.doi.org/10.1136/bjsports-2011-090185. birturk, a., & karagun, e. (2015). the effect of recreational activities on the elimination of statetrait anxiety of the students who will take the sbs placement test. educational research and reviews, 10(7), 894-900. https://doi.org/10.5897/err2015.2093. burguillo, j. c. (2010). using game theory and competition-based learning to stimulate student motivation and performance. computers & education, 55(2), 566-575. https://doi.org/10.1016/j.compedu.2010.02.018. cagiltay, n. e., ozcelik, e., & ozcelik, n. s. (2015). the effect of competition on learning in games. computers & education, 87, 35-41. https://doi.org/10.1016/j.compedu.2015.04.001. caviola, s., primi, c., chiesi, f., & mammarella, i. c. (2017). psychometric properties of the abbreviated maths anxiety scale (amas) in italian primary school children. learning and individual differences, 55, 174–182. https://doi.org/10.1016/j.lindif.2017.03.006. çetinkaya, g., şahin, k., yarız, k. (2017). leisure satisfaction level of active and passive participation in outdoor recreation activities and its relationship with public health. acta medica mediterranea, 2, 191-196. https://doi.org/10.19193/0393-6384_2017_2_027. chiu, l. h., & henry, l. l. (1990). development and validation of the mathematics anxiety scale for children. measurement and evaluation in counseling and development, 23(3), 121127. https://doi.org/10.4995/muse.2020.12622 https://doi.org/10.12816/0048784 https://iopscience.iop.org/journal/1742-6596 https://iopscience.iop.org/journal/1742-6596 https://iopscience.iop.org/journal/1742-6596 https://iopscience.iop.org/volume/1742-6596/812 https://doi.org/10.1088/1742-6596/812/1/012001 https://doi.org/10.1111/1467-8721.00196 https://doi.org/10.1111/1467-8721.00196 https://uqu.edu.sa/en/jep/75910 http://dx.doi.org/10.1136/bjsports-2011-090185 https://www.researchgate.net/journal/1990-3839_educational_research_and_reviews https://www.researchgate.net/journal/1990-3839_educational_research_and_reviews https://doi.org/10.5897/err2015.2093 https://doi.org/10.1016/j.compedu.2010.02.018 https://doi.org/10.1016/j.compedu.2015.04.001 https://doi.org/10.1016/j.lindif.2017.03.006 https://doi.org/10.19193/0393-6384_2017_2_027 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 107 chizary, f., & farhangi, a. (2017). efficiency of educational games on mathematics learning of students at second grade of primary school. journal of history culture and art research, 6(1), 232-240. http://dx.doi.org/10.7596/taksad.v6i1.738. reali, f., jiménez-leal, w., maldonado-carreño, c., devine, a., & szücs, d. (2016). examining the link between maths anxiety and maths performance in colombian students. revista colombiana de psicología, 25(2), 369-379. http://dx.doi.org/10.15446/rcp.v25n 2.54532. divjak, b., tomic, d. (2011). the impact of game-based learning on the performance of learning goals and motivation for learning mathematics-literature review. journal of information and organizational sciences, 35(1), 15-30. dowker, a., sarkar, a., & looi, c. y. (2016). mathematics anxiety: what have we learned in 60 years? frontiers in psychology, 7, 508. http://dx.doi.org/10.3389/fpsyg.2016.00508. dunn, v., & goodyer, i. m. (2006). longitudinal investigation into childhood and adolescenceonset depression: and adolescence-onset depression: psychiatric outcome in early adulthood psychiatric outcome in early adulthood. the british journal of psychiatry, 18, 216-222. https://doi.org/10.1192/bjp.188.3.216 eady, m. g., & lockyer, l. (2013). tools for learning: technology and teaching strategies. in m. g. eady & l. lockyer (eds.), learning to teach in the primary school (pp. 71-89). brisbane: queensland university of technology. eime, r. m., young, j. a., harvey, j. t., charity, m. j., & payne, w. r. (2013). a systematic review of the psychological and social benefits of participation in sport for children and adolescents: informing development of a conceptual model of health through sport. international journal of behavioral nutrition & physical activity, 10, 98-118. https://doi.org/10.1186/1479-5868-10-98. espino, m., pereda, j., recon, j., perculeza, e., & umali, c. (2017). mathematics anxiety and its impact on the course and career choice of grade 11 students. international journal of education, psychology and counselling, 2(5), 99-119. http://www.ijepc.com/pdf/ijepc2017-05-09-08.pdf ezeugwu, j. j. o., onuorah, j. c., asogwa, u. d., & ukoha, i. (2016). effect of mathematics game-based instructional techniques on students’ achievements and interest in algebra at basic education level. global journal of pure and applied mathematics, 12(4): 3727-3744. furner, j. m., & gonzalez-dehass, a. (2011). how do students’ mastery and performance goals relate to maths anxiety? eurasia journal of mathematics, science & technology education, 7(4), 227-242. https://doi.org/10.12973/ejmste/75209 geist, e. a. (2010). the anti-anxiety curriculum: combating maths anxiety in the classroom. journal of instructional psychology, 37(1), 24-31. https://doi.org/10.4995/muse.2020.12622 http://dx.doi.org/10.7596/taksad.v6i1.738 http://dx.doi.org/10.15446/rcp.v25n%202.54532 https://doi.org/10.3389/fpsyg.2016.00508 https://doi.org/10.1192/bjp.188.3.216 https://doi.org/10.1186/1479-5868-10-98 http://www.ijepc.com/pdf/ijepc-2017-05-09-08.pdf http://www.ijepc.com/pdf/ijepc-2017-05-09-08.pdf https://doi.org/10.12973/ejmste/75209 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 108 goldstein, j. (2012). play in children’s development, health and well-being. toy industries of europe. hacieminoglu, e. (2016). elementary school students’ attitude toward science and related variables. international journal of environmental & science education, 11(2), 35-52. https://doi.org/10.1177/1365480214521457. harris, b. (2000). a strategy for identifying the professional development needs of teachers: a report from new south wales. journal of in-service education, 26(1): 25-47. https://doi.org/10.1080/13674580000200109. hieftje, k., tyra pendergrass, t., kyriakides, t. c., gilliam, w., & fiellin, l. (2017). an evaluation of an educational video game on mathematics achievement in first grade students. technologies, 5 (2): 30-38. https://doi.org/10.3390/technologies5020030. hill, f., mammarella, i. c., devine, a., caviola, s., passolunghi, m. c., & szűcs, d. (2016). mathss anxiety in primary and secondary school students: gender differences, developmental changes and anxiety specificity. learning and individual differences, 48, 45–53. https://doi.org/10.1016/j.lindif.2016.02.006. holder, m. d., coleman, b., & sehn, z. l. (2009). the contribution of active and passive leisure to children’s well-being. journal of health psychology, 14(3), 378-386. https://doi.org/10.1177/1359105308101676. hoorfar, h., & taleb, z. (2015). correlation between mathematics anxiety with metacognitive knowledge. procedia social and behavioral sciences, 182, 737-741. https://doi.org/10.1016/j.sbspro.2015.04.822. hurd, a. r. & anderson, d. m. (2011). the park and recreation professional’s handbook. champaign, il: human kinetics, usa. ibrahim, b., & abu hmaid, y. (2017). the effect of teaching mathematics using interactive video games on the fifth grade students’ achievement. an najah univ. j. res. (humanities), 31(3): 471-492. https://repository.najah.edu/handle/20.500.11888/2954 jansen, b. r., louwerse, j., straatemeier, m., van der ven, s. h., klinkenberg, s., & van der maas, h. l. (2013). the influence of experiencing success in maths on maths anxiety, perceived maths competence, and maths performance. learning and individual differences, 24, 190– 197. doi: https://doi.org/10.1016/j.lindif.2012.12.014 johnson, a. (2013). effective methods for 21st century learning: a teacher action research project. meridian journal of k-16 school computer technologies, 16(2), 1-14. file:///c:/users/drabokhald/downloads/effectivemethodsfor21stcenturylearning.pdf johnson, a., connolly, p., & tully, m. a. (2017). school-based physical activity interventions and wellbeing in children: a systematic review and intervention complexity assessment. https://doi.org/10.4995/muse.2020.12622 https://doi.org/10.1177/1365480214521457 https://doi.org/10.1080/13674580000200109 https://doi.org/10.3390/technologies5020030 https://doi.org/10.1016/j.lindif.2016.02.006 https://www.researchgate.net/journal/1359-1053_journal_of_health_psychology https://doi.org/10.1177/1359105308101676 https://doi.org/10.1016/j.sbspro.2015.04.822 https://repository.najah.edu/handle/20.500.11888/2954 https://doi.org/10.1016/j.lindif.2012.12.014 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 109 retrieved from: https://campbellcollaboration.org/media/k2/attachments/ecg_johnson _title.pdf. kelly, j., & pohl, b. (2018) using structured positive and negative reinforcement to change student behavior in educational settings in order to achieve student academic success. multidisciplinary journal for education, social and technological sciences, 5(1), 17-29. doi: https://doi.org/10.4995/muse.2018.6370 kebritchi, m., hirumi, a., & bai, h. (2010). the effects of modern mathematics computer games on mathematics performance and class motivation. computers & education, 55, 427-443. doi: https://doi.org/10.1016/j.compedu.2010.02.007 kendall, p. c., safford, s. m., flannery-schroeder, e. c., & webb, a. (2004). child anxiety treatment: outcomes in adolescence and impact on substance use and depression at 7.4year follow-up. journal of consulting and clinical psychology, 72(2), 276–287. doi: https://doi.org/10.1037/0022-006x.72.2.276 keshavarzi, a., & ahmadi, s. (2013). a comparison of mathematics anxiety among students by gender. procedia social and behavioral sciences, 83, 542-546. doi: https://doi.org/10.1016/j.sbspro.2013.06.103 kundu, s. c., & kar, s. (2018). mathematics anxiety and its relationship with the performance of secondary school students. international journal of research and analytical reviews, 5(3): 451-455. http://ijrar.com/upload_issue/ijrar_issue_1674.pdf ladjal, s. (2016). comparative study of mathematics anxiety between normal pupils and those with mathematics learning disabilities in some primary schools. journal of psychological and educational sciences, 2(2), 43-64. https://arts.ksu.edu.sa/ar/journal-faculty-arts laidley, t., & conley, d. (2018). the effects of active and passive leisure on cognition in children: evidence from exogenous variation in weather. social forces, 97(1), 129-156. https://doi.org/10.1093/sf/soy020 liu, c., solis, s. l., jensen, h., hopkins, e. j., neale, d., zosh, j. m., hirsh-pasek, k., & whitebread, d. (2017). neuroscience and learning through play: a review of the evidence (research summary). the lego foundation, dk. lorenzen. j. k. (2017). the effect of instructional strategies on maths anxiety and performance: a mixed methods study of preservice elementary teachers. ph. d dissertation. the university of southern mississippi. luttenberger, s., wimmer, s., & paechter, m. (2018). spotlight on maths anxiety. psychology research and behavior management, 11, 311-322. https://doi.org/10.2147/prbm.s141421 nieman, p. (2002). psychosocial aspects of physical activity. paediatrics & child health, 7(5), 309-312. https://doi.org/10.4995/muse.2020.12622 https://campbellcollaboration.org/media/k2/attachments/ecg_johnson%20_title.pdf https://campbellcollaboration.org/media/k2/attachments/ecg_johnson%20_title.pdf https://doi.org/10.4995/muse.2018.6370 https://doi.org/10.1016/j.compedu.2010.02.007 https://psycnet.apa.org/doi/10.1037/0022-006x.72.2.276 https://www.researchgate.net/profile/abdollah_keshavarzi https://doi.org/10.1016/j.sbspro.2013.06.103 http://ijrar.com/upload_issue/ijrar_issue_1674.pdf https://arts.ksu.edu.sa/ar/journal-faculty-arts https://doi.org/10.1093/sf/soy020 https://doi.org/10.2147/prbm.s141421 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 110 olmez, i. b., & ozel, s. (2012). mathematics anxiety among sixth and seventh grade turkish elementary school students. procedia social and behavioral sciences, 46, 49334937. https://doi.org/10.1016/j.sbspro.2012.06.362 paggi, m., jopp, d., & hertzog, c. (2016). the importance of leisure activities in the relationship between physical health and well-being in a life span sample. gerontology, 62, 450-458. https:/doi.org/10.1159/000444415. price, p. c., jhangiani, r. s., chiang, i. a., leighton, d. c., & cuttler, c. (2017). research methods in psychology. 3rd american edition. pullman, wa: publisher. puteh, m., & khalin, s. z. (2016). mathematics anxiety and its relationship with the performance of secondary students in malaysia. international journal of social science and humanity, 6(2), 119-122.https:/doi.org/10.7763/ijssh.2016.v6.630 rafferty, r., breslin, g., brennan, d., & hassan, d. (2016). a systematic review of school-based physical activity interventions on children’s wellbeing. international review of sport and exercise psychology, 9(1),1-16. https://doi.org/10.1080/1750984x.2016.1164228 rosalind, c. (2011). understanding child development. (16th ed.) australia: wadsworth cengage learning. salmon, j., eime, r., brown, h., hodge, s., milton, k., & foreman, r. (2019). sport and active recreation: increase physical activity levels through sport and active recreation. blueprint for an active australia. (3rd ed.). melbourne, victoria: national heart foundation of australia. sawalha, m. a., & asfa, m. m. (2008). the effect of reinforcement processes for reducing maths anxiety among primary school students. umm al-qura university journal of social sciences, 20(2), 326-363. sayan, h. (2015). the effects of computer games on the performance of basic mathsematical skills. educational research and reviews, 10(22), 2846-2853. https://doi.org/10.5897/err2015.2172 scarpello, g. (2007). helping students get past maths anxiety. techniques: connecting education and careers, 82(6), 34-35. schwarzenegger, a., chrisman. m., & coleman, r. (2005). the health and social benefits of recreation: an element of the california outdoor recreation. scully, d., kremer, j., meade, m. m., graham, r., & dudgeon, k. (1998). physical exercise and psychological well-being: a critical review. british journal of sports medicine, 32(2), 111120. http://dx.doi.org/10.1136/bjsm.32.2.111 https://doi.org/10.4995/muse.2020.12622 https://doi.org/10.1016/j.sbspro.2012.06.362 https://doi.org/10.1159/000444415 https://doi.org/10.7763/ijssh.2016.v6.630 https://www.researchgate.net/journal/1750-984x_international_review_of_sport_and_exercise_psychology https://www.researchgate.net/journal/1750-984x_international_review_of_sport_and_exercise_psychology https://doi.org/10.1080/1750984x.2016.1164228 https://doi.org/10.5897/err2015.2172 http://dx.doi.org/10.1136/bjsm.32.2.111 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 111 siebald. c., khandaker, g. m., zammit. s, lewis, g., & jones, p. b. (2016). association between childhood psychiatric disorders and psychotic experiences in adolescence: a populationbased longitudinal study. comprehensive psychiatry, 69, 45-52. https://doi.org/10.1016/j.comppsych.2016.05.004 siebers, w. m. (2015). the relationship between maths anxiety and student performance of middle school students. ph. d dissertation. colorado state university. fort collins, colorado. smith, c., hannon, j. c., brusseau, t. a., fu, y., & burns, r. d. (2016). physical activity behavior patterns during school leisure time in children. international journal of kinesiology & sports science, 4 (1), 17-25. http://dx.doi.org/10.7575/aiac.ijkss.v.4n.1p.17 stodolska, m., shinew, k. j., & li, m. z. (2010). recreation participation patterns and physical activity among latino visitors to three urban outdoor recreation environments. journal of park and recreation administration, 28(2), 36-56. stones, e., and morris, s. (1972). teaching practice, problem and perspective. london, england: methuen and company ltd. tiwari, s. r., rathar, c. l. & singh, y. k. (2007). teaching methods of physical education. publishing corporation, 5 ansari road, darya ganj, new delhi. india: publishing corporation. ullenhag, a., krumlinde-sundholm, l., granlund, m. & almqvist, l. (2014). differences in patterns of participation in leisure activities in swedish children with and without disabilities. disability and rehabilitation, 36(6), 464-471. http://dx.doi.org/10.3109/09638288.2013.798360. whitebread, d., neale, d., jensen, h., liu, c., solis, s. l., hopkins, e., hirsh-pasek, k. zosh, j. m. (2017). the role of play in children’s development: a review of the evidence. the lego foundation, dk. wood, g., pinheiro-chagas, p., julio-costa, a., rettore-micheli, l., krinzinger, h., kaufmann, l., willmes, k., & haase, v. g. (2012). maths anxiety questionnaire: similar latent structure in brazilian and german school children. child development research, 1(1), 1–11. https://doi.org/10.1155/2012/610192 wu, s. s., barth, m., amin, h., malcarne, v., & menon, v. (2012). maths anxiety in second and third graders and its relation to mathematics performance. frontiers in psychology, 3, 162173. https://doi.org/10.3389/fpsyg.2012.00162 yeo, w. l., tan, c. k., & lew, s. l. (2015). mathematics anxiety among male and female students. international journal of psychological and behavioral sciences, 9(8), 2830-2835. yüksel-şahin, f. (2008). mathematics anxiety among 4th and 5th grade turkish elementary school students. international electronic journal of mathematics education, 3(3), 179-189. https://doi.org/10.4995/muse.2020.12622 https://doi.org/10.1016/j.comppsych.2016.05.004 http://dx.doi.org/10.7575/aiac.ijkss.v.4n.1p.17 http://dx.doi.org/10.3109/09638288.2013.798360 https://doi.org/10.1155/2012/610192 https://doi.org/10.3389/fpsyg.2012.00162 multidisciplinary journal for education, https://doi.org/10.4995/muse.2020.12622 social and technological sciences issn: 2341-2593 mohammed (2020) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 7 nº 1 (2020): 89-112 | 112 zosh, j. m., hopkins, e. j., jensen, h., liu, c., neale, d., hirsh-pasek, k., solis, s. l., & whitebread, d. (2017). learning through play: a review of the evidence. the lego foundation, uk. https://doi.org/10.4995/muse.2020.12622 abstract 3. materials and methods multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 new trends outside the translation classroom s. martínez martínez*, c. álvarez de morales mercado*. department of german philology, university of seville, c/ san fernando, s/n 41004 sevilla, spain department of translation and interpreting, university of granada, c/ buensuceso, 11, 18002, granada, spain * email: smartinez4@us.es; phone: + 34 649915947 * email: cristinaalvarez@ugr.es; phone: + 34 677738839 received: 2013-12-31; accepted: 2014-08-11 abstract this paper is based on the study of different elements at the university of granada’s faculty of translation and interpreting and seeks to elaborate a prototype for a multilingual and accessible audio guide (audio description, sdhh and spanish sign language interpretation). we defend a new methodology, one that focuses on teaching the translation process from previous museum-based learning experiences in the translation classroom using qr codes. our main goal is to innovate translation-related teaching based on the new approaches acquired through learning workshop perspectives. in this sense, we will offer an ideal framework in developing the new concept of translation learning. this concept involves systemising a new means of learning and organising the realities of translation itself, encompassing objectives, competences, methodology and evaluation. keywords accessible audio guide, audio description; sdhh; museum-based learning; learning workshop martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 110 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 1. introduction this article is the result of a teaching innovation project called the development of content for a universal, low-cost cross-platform system on the description, location and guide to university of granada (ugr) buildings. desam (for its spanish-language acronym), code 12-53, developed over the past two years by the tracce (the spanishlanguage acronym for accessible translation) research group within the university of granada’s faculty of translation and interpreting. first of all, the desam project, based at the ugr, launched both a line of teaching innovation and a line of research that develop different kinds of universally accessible cross-platform systems and devices in different languages, transmitting the description, accessibility and guide to ugr buildings. this university’s department of computer languages and systems has been responsible for the elaboration of the cross-platform system. further, this project continued with a line of universally accessible contentdevelopment (subtitling and sign language interpreting for deaf persons and audio description for visually impaired persons) in different languages (english, french, german and arabic), which had been initiated in previous projects, like the r&d project tacto. tacto (its spanish-language acronym), translation and accessibility: accessible dissemination of science/science for everyone, with code 09-105, was launched by the tracce research team. the goal of this project has not been merely to improve a situation, but also to create a product that offers an absolutely innovative service – a service that does not yet exist and is unparalleled in any other institutional building on the national or international level. basically, it involves incorporating a series of cross-platform devices into buildings, devices that are capable of providing a location and a description of these buildings and include a simple guide through their interiors, as well as the functions (instructional, service-based, or artistic elements) of each space. the primary objective of this teaching innovation project has been to apply different types of accessible translation (soler 2010) to the ugr’s architectural areas as instructive martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 111 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 tools in training translation and interpreting students. likewise, the goal of rendering buildings accessible – and describing them in different languages for different purposes and objectives – presented an ideal context for the students to move beyond the classroom and experience the epistemology of the translation process first-hand. the students developed a global translation assignment that consisted of producing content toward creating a product similar to an accessible guide, which would serve as a final result; this guide was intended for different types of users, with particular attention to those suffering from a range of sensorial disabilities. in this way, desam adopts an interdisciplinary approach in order to construct a theoretical framework that supports its research; key resources include studies on museum accessibility and the socio-constructivist theory. according to museum accessibility we must say that from their earlier role as exclusive and elitist institutions, museums are now at the service of the society and community. the museum of today is a space for communication (hernández hernández 1998) where the exhibitions constitute the museum message or discourse (serrat antolí) and the exhibited artifact "...becomes the bearer document of conceptual information" (garcía blanco 1999: 20). in recent years and in line with the new museology the museums also start to become objects of negotiation, participation and social integration (flórez crespo 2006: 232). the communicative function of museums consists of translating the scientific or artistic discourse encoded in the language of the exhibited artifacts (garcía blanco 1999) to a discourse that makes the expert knowledge of the former accessible to different kinds of receivers. these museum and exhibitory discourses are multimodal, as they use different semiotic codes to construct the complete meaning, and it constitutes the source text of translation and interpretation processes that operate in museums to make them accessible to different audiences. from this semiotic perspective, the accessible translation and interpretation (t&i) of the museographic multimodal event brings together different modalities that can be martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 112 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 classified into two broad categories: monomodal or monosemiotic t&i and multimodal or multisemiotic t&i. multimodal t&i refers to translation processes that involve more than one semiotic mode, either in the source text or in its translation. in translation courses, which are the classes most frequently assigned to us, one of the most important objectives pursued is for the student to acquire not only conceptual knowledge, but also procedural and attitudinal notions in order to independently perform certain tasks. thus, we offer an alternative teaching model to the traditional one (fedor de diego 2003): its primary methodological goal is to innovate in the instruction of translation through creating an ideal context for autonomously learning the epistemology of translation, as well as for developing the student’s decision-making and teamwork capacities. the ugr’s different buildings have played a fundamental role in this structure by serving as the facilitators of the architectural space (the situational macro context) that contains the various interior spaces (the micro-texts) and their functions (the macro-text). these buildings and their interiors comprise the principal textual units from which the workflow will emerge. in the vocabulary of translation, they constitute the source text – and, in this case, a source text that is multimodal and innovative in nature. the inclusion of a guide through the buildings, following their location and description, enables users to navigate different environments, including multimodal ones. in addition to this learning methodology, we must also highlight the effort undertaken by students in different areas: students who have endeavoured to work productively with their fellow students in other areas of the university of granada, with the professionals and experts involved, and with users – all towards securing the success of the products in terms of their accessibility. martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 113 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 2. objectives the project’s general objectives are the following: a) learning about and spreading accessibility in public spaces. b) competence training through transversality and interdisciplinary approaches. c) creation and launching of new professional profiles. based on these general objectives, the following specific objectives also emerge: a) learning how to use and manipulate new technologies in concrete mobile devices and qr codes. given that the adapted information will be multimedia in nature, the translation for and adaptation to different users will pose a spectacular opportunity for carrying out an absolutely innovative translation process. in this way, accessibility will improve in the ugr infrastructure, making use of technology and accessible multimedia content, crystallising into directed operation at two different centres: the etsiit (university technical school of computing and telecommunication engineering) and the fti (department of translation and interpreting). b) create a type of self-learning in translation and interpreting that emulates the job markets, simulates companies’ translation strategies, and strengthens virtual education, through skilland competence-based learning. c) create samples of multimodal teaching materials on translation for universal sensorial accessibility. d) move beyond the classroom as an instructional space in such a way that the student becomes aware of the fact that her learning may be conceptualised in connection with the most immediate environment, her local environment (in this case, ugr buildings), and that only in this way can an expansion into the global environment become a reality. martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 114 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 e) train professors in order to connect their research context with teaching tasks and their applicability to the information society. of the specific objectives delineated here, this article will focus on three – b), c) and d) – which encompass the other two and constitute the essence of the desam r&d project. these objectives were carried out in the second phase of the project, which began in the second quarter (february to june) of the 2012 academic year. objective a) is the one developed wholly by the school of computing team that has collaborated on the r&d project, which is why we will not elaborate further on this subject. objective e) is present in the others, so we will discuss it as part of the analysis we undertake on each one. in order to create a form of self-learning in translation and interpretation that emulates the reality of the job market, students in the t7 courses (specialised translation in spanishenglish, german-spanish and italian-spanish) received a semi-professional translation assignment from their professors and carried out a series of individual tasks in a translation company focused on strengthening visual learning. in this sense, the students worked as a team, assuming the roles of documentalist, translator, editor and copy editor in each group; thus, the translation process resembled the actual process in all ways. through the trial and error method (risku 1998), the student acquired universal translation strategies that could be extrapolated to other translation assignments. the relevance of the translation task is, therefore, a highly important factor in the development of the translation process. we believe that if the student implements a particular practice in a real context, she will more successfully confront her mission, as she will feel motivated and can apply the translation strategies she has learned. multimodal museum texts have been innovatively used so that students will understand the analysis and interpretation process intrinsic to the mechanisms of meaningconstruction in the source text – since these texts must integrate and interpret the different semiotic resources present in the exhibition in order to access the discourse it seeks to transmit. further, in being used as documental resources, the multimodal texts and events (rivière 1993) enable access to knowledge and strengthen learning processes by virtue of martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 115 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 their similarity to processes of human perception. all of these multimodal texts have accessible multilingual content (spanish, english, french, german and arabic) that refer to spaces, functions, location, distribution, history and other academic information from two ugr centres. 3. methodology the methodology used throughout the project has adhered to the following principles: 1. motivation: is crucial for learning to be effective in the classroom and largely depends on professors’ abilities to spark and maintain the students’ interest. in the assignments that are part of this project, the professor involves the student in the translation process from the very beginning and motivates her through innovative experiences like working in a real context (different ugr buildings); working for recipients with specific needs (sensorial disabilities and tourists); and working with multimodal texts and documentation (descriptions of buildings; the art they contain, like photos, paintings and other elements), where they will learn through multi-sensoriality and experience (handling objects, using audio-visual tools, etc.). we must stress that the faculty of translation and interpreting of the university of granada is a neoclassical palace from the end of the 19th century, an in this sense it must be considered as a museum because inside it there are a lot of artistic elements from the history patrimonial. 2. professor as a facilitator of guided discovery: after fulfilling the prior objective, the professor, also referred to as a guide or adviser, introduces the student to and guides her through a learning-by-discovery process based on asking questions, debate and discussing ideas (leach and moon 2008). further, the professor will serve as the coordinator for the various work teams and their respective disciplines. martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 116 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 3. self-learning: in this phase, the professor will release the student to begin a process by which she will be able to develop skills in planning, action-organising, problem-solving, and the creation and review of learning and self-evaluation strategies. professors must maintain their role as adviser and guide during this phase and until the end of the learning process, which is the evaluation stage. within the framework of the faculty of translation and interpreting which it is also a neoclassical palace and therefore a sort of museum, the project has involved – and could not have done otherwise – the participation of students from the translation degree, which includes various of the courses entailed in its multilingual and two-way language combination. in order to fulfil the specific objective d) – that is, to move beyond the classroom as a learning space – we have insisted on how important it is for professors to receive training on the connection of their research context with teaching tasks and their applicability to the information society. we must also remember to raise the student’s awareness on how her own learning may be conceived in connection with the most immediate environment – the local one (in this case, ugr buildings) – and that only in this way can an expansion into the global environment become a reality. with respect to the previously mentioned courses, the following activities were carried out, both beyond the classroom and within it: i) beyond the classroom: the professors, along with their students, made multiple visits all over the faculty of translation and interpreting, pursuing two points of view: a) from the perspective of someone with a visual disability, the guided segments were repeated so as to verify that the source texts corresponded exactly to what was audio-described in them. various errors were detected and corrected along the way. b) from the perspective of tourists or visitors who wanted to know something else about an artistic element in front of them, and which they stopped to observe for the first martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 117 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 time. we must not forget that the faculty of translation and interpreting is a nineteenthcentury palace and, as such, contains many artistic heritage items. these visits were videotaped (and then uploaded onto the sawd1 platform, which is a resource for university of granada professors) in such a way that the thematic field was covered from the very beginning. the video helped all students who missed class, but it also served as a documentation resource for the texts to be translated later. in this way, the instrumental and technical fields were largely resolved. moreover, the interest shown by the students during the visit to the faculty, their continual questions on the subject at hand, determined the psycho-physiological field. ii) in the classroom: in all selected classes, an identical protocol was followed during the translation process. first, students were given the source texts in spanish in order to analyse them exhaustively. next, the students searched for the parallel texts in the corresponding languages so as to elaborate a glossary according to the previously established norms, using the textual corpus specified by their professor. they were advised to expand this corpus by consulting other spanish-language texts on subjects like heraldry and artistic heritage, as well as other parallel texts in english/italian, and by using online dictionaries. finally, a translation of technical texts was undertaken, based on the professor’s assignment of an accurate translation. the assignment involved the publication of the professor-facilitated texts in english for translation in science and technology a-b (english)’s class, in italian for scientific translation 5 a-c (italian)’s class, and in german for translation in science and technology a-b (german)’s class in their spanish-language online versions. 1 the swad (web teacher support system) technological collaboration platform at the university of granada, which is organised as a course-management system that helps educators to create online learning communities. it promotes a social constructivist pedagogy (cooperation, activities, critical reflection, etc.). martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 118 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 and finally, to complete the educational unit, a student evaluation phase was implemented in the following ways: through direct observation of students in the classroom, making note of anything considered important (general or specific aspects of each student, the pertinence of her questions, problem-solving skills, etc.). through analysing the translations produced by each work team. through observing the level of objective-fulfilment, likewise considering other important aspects like the level of interest shown in the activity undertaken. 4. results the project desam can be defined as a methodological experiment that has gathered students from different areas of knowledge and levels of access to that knowledge. this phenomenon in itself already implies, among other factors, a substantial need for adaptation to the educational setting. further, the project involved working with mixed groups from different disciplines, incorporating working methods as diverse as the applied sciences, social sciences, and humanities. each area, according to socioconstructivism has contributed its own conception of the tasks’ content and distribution. in the same way, and with a view to the professionalization of the students’ self-learning, both students and professors will make contact with the direct users of their products, remember that in the socio-constructivist theories there were three elements which act with a close relationship, the teacher, the student and the topic to be learned. as we explained in the theoretical framework the socio constructivist theories try to make the process of learning in translation making the students participate in the experience has partly involved taking on all phases of the translation process in a real labour environment – from the assignment of the translation to the submission of each complete text – simulating, at all levels, the real relationship between translator and client. martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 119 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 in this way, they have entered a semi-professional realm, and they have had to adapt to the concrete needs of a user for whom the translation task will be carried out – which is why the job’s quality and success will be continuously evaluated. all of this is enough of a guarantee to affirm that the project involves highly rigorous measurement tools to prove the students’ educational evolution. in fact, from the methodological perspective, the project is based on the collaboration between the university of granada, companies focused on access to the media (aristia producciones and espectáculos), and two associations of people with sensorial disabilities: the regional administration of once (the spanish-language acronym for spain’s national organisation of the blind) in granada and the faas (andalusian federation of associations of the deaf). synergy has increased among research groups, increasing impetus for applied research. although the initial objective is that the proposed works should be simple ones, the challenge of applying knowledge acquired and adding innovation implies an incentive for research on access-related subjects for professors and students. moreover, an evaluation of the project’s output was undertaken during its execution and on its conclusion. most of the information necessary for conducting this evaluation and reframing was obtained through a) team meetings, b) standardised records, and c) multilateral contacts. further, in order to analyse the quality and results of the knowledge transmitted and acquired by the students, periodic evaluations were conducted on basic information related to the subjects in question, as well as on the techniques being used. external evaluation at the end of the project, and based on the documents produced by each group, another document will be created to narrate the experience. it will describe the results to groups at martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 120 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 other centres so they may appreciate its exportability and usefulness in their own work environments. however, the key measurements of the external evaluation will focus on two aspects: (a) professional evaluations, and (b) user evaluations. with respect to the professionals’ evaluations, various companies in the sector have been contacted on multiple occasions; the idea is that they offer their evaluation in exchange for travel costs, if they are based outside of granada, and for sharing the results. the second form of evaluation has been undertaken by following the study methodologies of museum visitors. the goal of this evaluation is to collect the experiences, opinions and needs of people with functional, sensorial, visual, and auditory diversity regarding the accessibility of the buildings in general and of the ones we present in particular. the subject sample for the study will, in principle, contain a single prerequisite for participating in this evaluation: the participants must either have a functional sensorial disability, to varying degrees, or must come from a different culture and/or speak a different language from those offered. thus, the subjects have been divided into three groups: (a) individuals with a visual disability, (b) individuals with an auditory disability, and (c) individuals from a different culture or language. as the final step of the desam r&d project, which will be carried out in the coming months, surveys will be conducted and, in order to facilitate contact and encourage participation in the study among individuals with this profile, we will make contact with organisations and associations of people with sensorial disabilities: the once (regional administration for the once in granada) and the faas (andalusian federation of associations of the deaf) in granada; and the company signovisión, the directors of martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 121 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 which were presented with the project and asked to support the implementation of the studies. further, pilot tests were carried out with small groups of subjects, the goal being to verify the comprehensibility, relevance, and appropriateness of the content, formulation and duration both of each question and of the survey itself. 5. references brinckmann, w. e., and wildgen, j. s. (2003). desafíos para los estudiosos del turismo: la construcción de la “sociedad inclusiva” y del “turismo accesible”. cuadernos de turismo, (11), 41-58. darcy, s., cameron, b., and pegg, s. (2010). accessible tourism and sustainability: a discussion and case study. journal of sustainabletourism, 18(4), 515 – 537. doi: 10.1080/09669581003690668 fedor de diego, a. (2003). sobre las propiedades metacognitivas y constructivistas de la terminología y sus consecuencias didácticas. r. muñoz martín (ed.). aieti. actas del i congreso internacional de la asociación ibérica de estudios de traducción e interpretación, granada: aieti, 19-31. garcía blanco, ángela (1999). la exposición, un medio de comunicación. madrid: akal hernández hernández, francisca (1998). el museo como espacio de comunicación. gijon: trea. jiménez hurtado et al. (2010). un corpus de cine. teoría y práctica de la audiodescripción. granada: tragacanto. leach, j. and moon, b. (2008). the power of pedagogy. london: sage publications. cambridge: cambridge university press. risku, h. (1998). translatorische kompetenz. kognitive grundlagen des übersetzens als expertentätigkeit. tubingia: stauffenburg. martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 122 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2229 social and technological sciences eissn: 2341-2593 rivière, g. (1993). la museología: cursos de museología, textos y testimonios, madrid: akal. santacana mestre; n., serrat antolí, j. (2005) (coordinators). museografía didáctica, barcelona: ariel. soler, s., et al. (2010). la traducción accesible en el espacio multimodal museográfico y su aplicación a la formación de traductores. in: la traducción en contextos especializados. ii seminario permanente de formación de formadores en traducción e interpretación, 317327. soler, s. (2012). traducción y accesibilidad en el museo del siglo xxi. traducción accesible. granada: tragacanto. varine-bohan, h. (1974). los museos del mundo. barcelona: salvat. martínez and álvarez (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 110-123 | 123 http://polipapers.upv.es/index.php/muse/ teaching methodologies for combustion science within the european higher education area multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 the impact of continuous assessment on a temporal perspective: the results of a pioneering experiment at the university of barcelona (spain) daniel montolio* dpt. d’economia pública. universitat de barcelona, av diagonal 690, 09034 barcelona, spain. * corresponding author: email: montolio@ub.edu; phone: + 34 934021812 received: 2013-12-22; accepted: 2014-08-14 abstract this study completes the works by gallardo, montolio and camós (2010) and gallardo and montolio (2011) and brings new evidence on the impact of continuous assessment on students’ results. we use a complete dataset with information regarding both the subjects taught and the results obtained by students at the public administration and management diploma course of the university of barcelona (spain) that was a pioneering experiment at this university in implementing the guidelines of the european higher education area (ehea) back in 2004. more precisely, we have information for seven academic years (2001/2002 – 2007/2008) on i) the lecturer who taught each subject; ii) the definition of the continuous assessment contained in the teaching plans when they were introduced; and iii) the students’ marks in each subject. with this information we compare the results obtained by students before and after the implementation, following the bologna process, of the continuous assessment controlling for who was responsible of the subject. the results present new evidence on the impact of continuous assessment taking into account a temporal perspective. it is generally accepted that the implementation and development of continuous assessment has been one of the most difficult changes in adapting to the ehea guidelines. moreover, it is at the same time a key and a controversial aspect of the adaptation process itself. indeed, there is no agreement on how continuous assessment in higher education is defined and how lecturers should implement the new assessment procedure. the present study aims to provide further information in this complex process of changing the evaluation process in our universities. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 128 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 keywords european higher education area, continuous assessment, student’s results, teaching plans, public administration and management diploma course 1. introduction and motivation one important and difficult task when trying to evaluate the implementation of the european higher education area (ehea, hereafter) is to assess if these guidelines have achieved the desired impact on students performance. the ehea implies a new way of looking at the teaching-learning process. this, in turn, involves a number of changes to the student assessment system (aqu, 2003; delgado et al., 2005; delgado and oliver, 2006; cadenato and martínez, 2008). as the students themselves have become the centre of the learning process, the assessment system should be designed as a tool to assist them in this process; e.g. providing them the opportunity to know throughout the term what they have (or have not) learned, to what extent they have achieved objectives of the subject or activity, in what areas they need to improve, and so on. this new learning process, therefore, demands that student assessment be continuous. in general, one would expect that those students who actively participate in (a well defined) continuous assessment process should have a priori better results in comparison with previous assessment procedures implemented before the ehea era (basically assessment through a unique final examination). nevertheless the task of understanding if this goal has been achieved is far form easy. many factors can influence the results obtained by the students and, moreover, to observe the counterfactual state, that is what would have happened to a student with different systems of evaluation, it is simply impossible. even being aware of these limitations, in this paper we complete the works by gallardo, montolio and camós (2010) and gallardo and montolio (2011) that have assessed both the process of the implementation of the continuous assessment and its impact on students’ results. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 129 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 on one side gallardo, montolio and camós (2010) analyse how the continuous assessment system was defined in the diploma in management and public administration at the university of barcelona (one of the first degrees at the university of barcelona in adapting ehea criteria). the authors gathered data directly from the teaching plans written by the lecturers of each subject for the 2007-2008 and 2008-2009 academic years. they observe that, progressively, the continuous assessment system was accepted to such an extend that nowadays is the assessment formula used by practically the whole of the lecturers. nevertheless, the authors find a high dispersion in how continuous assessment was defined in those academic years. it seems that there was (and maybe there still is) not a unified view of how to perform continuous assessment. this dispersion, although diminished over time, suggested some interesting questions about the need to collectively agree on the criteria that defines the continuous assessment, taking always in consideration the potential of the continuous assessment as a pedagogical tool. on the other side gallardo and montolio (2011) examine whether a better definition of the continuous assessment reverts positively in the results obtained by students. first, they also use the teaching plans of all the subjects taught in same diploma for the academic courses 2007-2008 and 2008-2009, to analyse the definition of continuous assessment that different lecturers used in their subjects. next, they asked a panel of experts (academics with extensive careers in teaching innovation) their assessment of the definition of continuous assessment that the lecturers of this degree defined in their teaching plans. this yields a ranking of subjects depending on the definition of the continuous assessment. to relate this valuation of the continuous assessment with the performance of students they used information on grades obtained per student and per subject in those two academic years. they obtain that during these two academic years the definition of continuous assessment improved. moreover, this improvement seems to be correlated with a better student performance. these previous works also highlighted the difficulty to analyse the impact of continuous assessment on student’s performance. first, the evaluation process itself was (and maybe d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 130 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 it still is) not well established by many factors such as the lack of clear guidelines as to what is meant by continuous assessment or how to apply it following “a common” criteria across lecturers; the existence of overcrowded classrooms implying that the number of students per group could be up to hundred students; lecturers not properly trained for the change in the assessment procedure; huge time requirements of dedication from lecturers, not only to plan the new way of assessing students but also to design and implement it (completion and correction activities, continuous feedback, etc.); the low value of teaching (and teaching innovation) in the evaluation criteria to access contractual figures provided by the spanish university system; and reluctance of students to change the evaluation system given that the continuous assessment implies active participation in a variety of activities throughout the semester (in front of a passive attitude possible in other types of assessment). second, it is very difficult to understand the ultimate determinants of a student performance. there are many factors influencing the final marks that the students obtain, being the assessment system just one of them. other factors such as personal circumstances, the difficulty of a given subject, or circumstances related with the lecturer teaching the subject could also influence the final performance of students. in this sense, it is impossible for us to account for the personal determinants of students’ grades, but we can try to control for factors related with the “supply of education” (that is, control for aspects form the lecturer side) to isolate the impact of the continuous assessment on students achievements.1 therefore, this paper aims to overcome some of these last issues by providing more evidence of the impact of continuous assessment on students’ performance but taking a long-run temporal perspective to isolate the impact that different lecturers could have in the obtained results of the students. 1 in fact, the idea for this paper comes from the comments received from various participants to the different conferences where the work by gallardo and montolio (2011) was presented. we thank all of them for positive and useful feedback. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 131 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 more precisely, we compare the results obtained by students before and after the implementation of the continuous assessment but controlling for who was the lecturer responsible of the subject. in this sense, we complement the results obtained in gallardo and montolio (2011) but being sure that the variability that the change in the responsible of a subject (as it happened many times) is neutralized and, hence, we reduce the variability of the factors that could affect the students’ results. although there are still many factors that could influence the observed results, we move forward in assessing the impact of the introduction of the continuous assessment in our universities. 2. objectives and methodology in order to achieve the above mentioned goal we use a complete dataset with information regarding both the subjects taught and the results obtained by students at the public administration and management diploma course of the university of barcelona that was a pioneering experiment at this university in implementing the guidelines of the european higher education area (ehea) back in 2004. in the first stage of implanting ehea guidelines, during the 2002-2003 academic year, the university of barcelona selected a small number of degree courses in which to implement a pilot scheme for adapting to the new regulations. these would serve as a guide for other courses, in what was labeled a convergence process. one of the programs selected was the public administration and management diploma course, a diploma course comprising 184 credits which has been offered by the university of barcelona’s faculty of law since the academic year 1991-1992 (see solé, 2009). in 2004, this course also received the recognition of the catalonian government’s department of universities as a pilot experiment for ehea adaptation. it was during the academic year 2004-2005 that all the subjects taught at the diploma, as a pioneering experience in the ub, applied (one way or another) the ehea guidelines, among which there was the need to move the assessment process towards a continuous assessment procedure (see ub, 2009). d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 132 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 the experiences gathered from these pilot experiences made the university of barcelona, in 2006, to defined new regulations concerning the assessment system in order to adjust it to the ehea guidelines for all the courses taught. since then, continuous assessment has been the accepted form of assessing students. as it is well know, this new definition of the assessment process induced an important change in the lecturers’ own methodology. they included in their courses a variety of activities to be assessed and thus constitute part of the student’s final mark. such a new environment called for a rethinking and redesigning of the entire teaching approach (sometimes this process was rather informal). in general, the students, used to a learning system which was totally passive, were required to do more autonomous work (completing various assignments, participating in class, etc.) in order to pass the subject while lecturers faced the challenge of designing an entire course to promote this type of continuous student work and the resulting learning process. the analysis of the teaching plans (for the academic year 2007-2008) revealed eight tasks that lecturers considered as being part of the continuous assessment. a global test is a final test (theoretical and/or practical) usually done at the end of the term. this task was common and can be understood as an inheritance from the former evaluation system, which was based completely on a global test. mid-term exams are theoretical or practical tests commonly done during the term. exercises and case studies consist of practical exercises or analytical case studies that students are assigned to do, normally in class. this type of task, if programmed weekly, is similar to daily work and enables a sustained assessment of students. essays are written assignments handed in by students and involving reading, comprehension and reasoning of a text. teamwork consists of a final essay usually done throughout the term by several students working together. oral presentations correspond to the performance of students when expounding an essay and/or exercise in class. “various activities” comprises those tasks defined by lecturers in a rather vague manner. the activities are not clearly defined and, a priori, it is not d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 133 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 possible to know the type of activity that students will be required to do. student progress relates to the student’s attendance and the interest shown in class. a part from understanding what continuous assessment really meant for lecturers when implementing it, we collect information for seven academic years (2001/2002 – 2007/2008) being the academic year 2004-2005 (the one of the application of the continuous assessment in the diploma) just in the middle of the time period. this way we have information on students’ results when the continuous assessment was not implemented at all (2001/2002), to be compared with those results obtained in an academic year with the continuous assessment fully implemented (2007/2008). in such a long time period although the structure of the diploma regarding the number of subjects and their contents have been fairly stable, this is not so with regard the lecturers teaching them. we have clearly identified 38 subjects (basically compulsory and optional courses) that were consistently taught from the academic year 2001/2002 to the academic year 2007/2008.2 however, analysing who taught those courses we find that during the period under analysis there was mobility of the lecturers. as shown in table 1, there is certain mobility of the lecturers responsible of each subject, this is particular true for the academic year 2005-2006, the one precisely after the implementation, as a pilot scheme, of the ehea guidelines, among them the requirement of writing up a teaching plan and the implementation of the continuous assessment to students. for the whole period under analysis, there is one subject that changed 4 times lecturer, 5 subjects that changed 3 times, 6 subjects that changed 2 times, 6 subjects that changed 1 time and 20 subjects experienced no changes in the lecturer teaching the contents of the subject. we precisely rely on those subjects that, in seven academic years, have not changed the lecturer and, hence, we can hypothesize that any change in the academic results obtained 2 the public administration and management diploma course has, on average, 52 subjects: 17 compulsory, 23 optional and 12 free-choice. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 134 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 by students between the academic year 2001-2002 (continuous assessment was not implemented at all) and the academic year 2007-2008 (continuous assessment was implemented)3 could be the consequence of a change in the assessment procedure (and by extension by changes in the teaching methodology). it is clear that the results we present in the next section are completely descriptive and there are still many unobserved factors (such as student’s characteristics) that we cannot control for and, hence, warns us to derive any conclusion regarding causality. nevertheless, the results here presented aim to be another brick in the assessment of the implementation of the ehea guidelines. table 1. lecturer changes. 2001 2002 2002 2003 2003 2004 2004 2005* 2005 2006 2006 2007 2007 2008 number of lecturer changes (with respect previous academic year) 0 7 4 3 10 7 7 % of changes over total subjects analysed (38) 0% 18.4% 10.5% 7.9% 26.3% 18.4% 18.4% note: * in the academic year 2004-2005 there ehea guidelines were implemented in the diploma in a university of barcelona’s pilot scheme. 3 as presented in gallardo, montolio and camós (2010) in the academic year 2007-2008 the vast majority of lecturers in the diploma course define the continuous assessment as a set of activities done throughout the semester, concluding that continuous assessment was extensively used. however, they observe a lack of homogeneity of criteria: different activities and different weights assigned to each task were used, depending on the subject. moreover, some activities were vaguely defined as “various activities” and were, therefore, uninformative, i.e. not properly described to the students at the beginning of the academic year. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 135 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 3. main results table 2 presents the students’ marks (by grade) obtained in each subject in the academic year 2001-2003; similarly, table 3 present the students’ results for the academic year 2007-2008. table 4 presents the differences between both years. the subjects presented in those tables are only those that did not have any change in the lecturer during the whole period. table 2. 2001-2002 students’ results. %e %vg %g %s %f %ent sr pr subject 1 5.1% 12.0% 36.8% 26.5% 7.7% 12.0% 91.3% 80.3% subject 2 0.5% 2.7% 15.8% 20.8% 27.9% 32.2% 58.9% 39.9% subject 3 0.5% 0.5% 10.5% 15.2% 25.1% 48.2% 51.5% 26.7% subject 4 0.0% 2.1% 6.3% 15.6% 28.1% 47.9% 46.0% 24.0% subject 5 0.0% 3.7% 11.1% 40.7% 7.4% 37.0% 88.2% 55.6% subject 6 0.9% 0.0% 17.2% 45.7% 16.4% 19.8% 79.6% 63.8% subject 7 0.0% 4.5% 15.0% 32.3% 30.8% 17.3% 62.7% 51.9% subject 8 1.6% 2.4% 17.7% 21.8% 15.3% 41.1% 74.0% 43.5% subject 9 0.0% 9.6% 14.9% 31.9% 20.2% 23.4% 73.6% 56.4% subject 10 0.0% 11.1% 33.3% 40.0% 0.0% 15.6% 100.0% 84.4% subject 11 3.7% 7.4% 13.0% 25.9% 14.8% 35.2% 77.1% 50.0% subject 12 2.3% 6.8% 22.7% 22.7% 4.5% 40.9% 92.3% 54.5% subject 13 0.0% 7.2% 9.9% 17.1% 33.3% 32.4% 50.7% 34.2% subject 14 0.0% 11.4% 22.9% 37.1% 5.7% 22.9% 92.6% 71.4% subject 15 0.0% 20.0% 20.0% 23.3% 10.0% 26.7% 86.4% 63.3% subject 16 6.7% 6.7% 6.7% 26.7% 13.3% 40.0% 77.8% 46.7% subject 17 0.0% 25.0% 50.0% 25.0% 0.0% 0.0% 100.0% 100.0% subject 18 0.0% 12.5% 12.5% 18.8% 25.0% 31.3% 63.6% 43.8% subject 19 6.3% 0.0% 6.3% 25.0% 12.5% 50.0% 75.0% 37.5% subject 20 0.0% 7.0% 14.0% 53.5% 14.0% 11.6% 84.2% 74.4% average 1.4% 7.6% 17.8% 28.3% 15.6% 29.3% 76.3% 55.1% note: e: "excellent" (matrícula de honor); vg: “very good” (excelente); g: “good” (notable); s: “sufficient” (aprobado); f: “fail” (suspenso); ent: “evaluation not taken” (no presentado). sr: “success rate” (defined as the total number of students that pass the subject (e+vg+g+s) divided the total number of registered students minus the number of students that have not taken the evaluation (ent)); pr: “performance rate” (defined as the total number of students that pass the subject (e+vg+g+s) divided the total number of registered students). source: own elaborations. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 136 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 from these tables we observe that there are no big changes on the results obtained in the subjects of the public administration and management diploma; more precisely, in those subjects that for seven academic years were taught by the same lecturer. to some extent this result is fairly normal given how many factors can affect the students results (that we cannot control for) and how difficult is to identify the impact of any educational policy on results. however, we must admit that two results need more attention. on one side it seems that the number of students evaluated, that is, taking the final exam in the academic year 2001-2002 or following the continuous assessment in the academic year 2007-2008, increased by 5.1%. this result is line with one of the aims of the continuous assessment that is to have students more involved during the course and to avoid students to get disconnected from the subject. table 3. 2007-2008 students’ results. %e %vg %g %s %f %ent sr pr subject 1 6.4% 20.5% 30.8% 25.6% 2.6% 14.1% 97.0% 83.3% subject 2 2.2% 2.2% 2.9% 20.4% 30.7% 41.6% 47.5% 27.7% subject 3 0.0% 1.6% 9.7% 24.2% 39.5% 25.0% 47.3% 35.5% subject 4 0.0% 3.2% 4.3% 13.8% 24.5% 54.3% 46.5% 21.3% subject 5 0.0% 9.3% 16.3% 30.2% 18.6% 25.6% 75.0% 55.8% subject 6 0.0% 2.2% 18.7% 44.0% 27.5% 7.7% 70.2% 64.8% subject 7 0.0% 3.7% 6.5% 28.7% 47.2% 13.9% 45.2% 38.9% subject 8 0.0% 2.7% 12.7% 30.0% 19.1% 35.5% 70.4% 45.5% subject 9 0.0% 11.5% 26.9% 26.9% 7.7% 26.9% 89.5% 65.4% subject 10 0.0% 19.2% 34.6% 15.4% 0.0% 30.8% 100.0% 69.2% subject 11 3.0% 15.2% 12.1% 15.2% 12.1% 42.4% 78.9% 45.5% subject 12 6.3% 10.4% 18.8% 31.3% 12.5% 20.8% 84.2% 66.7% subject 13 2.7% 4.0% 8.0% 54.7% 9.3% 21.3% 88.1% 69.3% subject 14 0.0% 16.0% 20.0% 40.0% 0.0% 24.0% 100.0% 76.0% subject 15 0.0% 27.3% 18.2% 18.2% 0.0% 36.4% 100.0% 63.6% subject 16 0.0% 2.9% 17.1% 37.1% 25.7% 17.1% 69.0% 57.1% subject 17 3.8% 7.7% 53.8% 26.9% 0.0% 7.7% 100.0% 92.3% subject 18 0.0% 0.0% 11.1% 44.4% 33.3% 11.1% 62.5% 55.6% subject 19 5.0% 5.0% 15.0% 65.0% 0.0% 10.0% 100.0% 90.0% subject 20 0.0% 12.5% 30.0% 35.0% 5.0% 17.5% 93.9% 77.5% average 1.5% 8.9% 18.4% 31.4% 15.8% 24.2% 78.3% 60.1% note: see table 2. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 137 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 moreover, both the success rate (sr) and, especially, the performance rate (pr) increased between the two academic years. the performance rate, on average increased from 55.1% up to 60.1%. this was mainly due for a high percentage of students passing the subject while the percentage of the students failing the subject remaining nearly unaltered. table 4. differences between academic year 2007-2008 and 2001-2002 in students’ results. %e %vg %g %s %f %ent sr pr average 2001-2002 1.4% 7.6% 17.8% 28.3% 15.6% 29.3% 76.3% 55.1% average 2007-2008 1.5% 8.9% 18.4% 31.4% 15.8% 24.2% 78.3% 60.1% differences 0.1% 1.2% 0.5% 3.1% 0.2% -5.1% 2.0% 4.9% note: see table 2. 4. conclusions this paper aims to bring more descriptive information to the academic area regarding the assessment of the ehea implementation and completes and complements the previous studies of gallardo, montolio and camós (2010) and gallardo and montolio (2011). we compare the results obtained by students before and after the implementation of the continuous assessment controlling for who was responsible of the subject taught. the results seem to indicate new evidence on the impact of continuous assessment taking into account a temporal perspective. more precisely it seems that the new assessment methodology reduced the number of students not being finally assessed and increased those students that pass the subject, that is, increase the success rate and specially the performance rate of students. it is generally accepted that the implementation and development of continuous assessment has been one of the most difficult changes brought about by adaptation to the ehea, while at the same time a key, though controversial, aspect of the adaptation d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 138 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 process itself. indeed, there is no agreement on how continuous assessment in higher education is defined and how lecturers should implement the new assessment procedure. the results of the present study would indicate that it is worth keep working on improve the definition and implementation of the this complex process of changing the evaluation process in our universities given that it seems that there is a positive impact on students results. 5. references aqu (2003). marc general per a l'avaluació dels aprenentatges dels estudiants. barcelona, agència per la qualitat del sistema universitari a catalunya. cadenato, a. and martinez, mª. (2008): “l'avaluació en el marc de l'espai europeu d'educació superior (eees)”, monogràfics ice de la universitat politècnica de catalunya. disponible en: http://www.upc.edu/ice/portal-derecursos/publicacions_ice/2_avaluacio.pdf delgado, a.mª and oliver, r. (2006): “la evaluación continua en un nuevo escenario docente”, revista de universidad y sociedad del conocimiento, 3(1), 1-13. delgado, a.mª. (coord.) (2005): competencias y diseño de la evaluación continua y final en el espacio superior. madrid: dirección general de universidades. disponible en: http://www.mec.es/univ/proyectos2005/ea2005-0054.pdf gallardo, e., montolio, d. and camós, m. (2010): “the european higher education area at work: lights and shadows defining continuous assessment”. revista d'innovació docent universitària, vol. 2, pp. 10-22. doi: 10.1344/105.000001524 gallardo, e. and montolio, d. (2011): “¿existe relación entre la evaluación continua y los resultados de los alumnos?”. e-pública, 8, pp. 63-79. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 139 http://www.upc.edu/ice/portal-de-recursos/publicacions_ice/2_avaluacio.pdf http://www.upc.edu/ice/portal-de-recursos/publicacions_ice/2_avaluacio.pdf http://www.mec.es/univ/proyectos2005/ea2005-0054.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3447 social and technological sciences eissn: 2341-2593 solé, m. (2009): “a pilot experiment for adapting university studies to the european higher education area: the public administration and management diploma course at the university of barcelona”. paper presented at inted 2009 conference held at valencia (spain). ub (2009): “memoria para la verificación del título de grado graduado o graduada en gestión y administración pública”. universidad de barcelona, p. 15. d. montolio (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 128-140 | 140 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 online review courses as preparation for first term remedial exams domínguez-reyes, ricardo*, meléndez, juan, hernández-pérez, aarón departamento de física, universidad carlos iii de madrid, 28911 leganés, (spain) * corresponding author: avd. de la universidad 30, 28911 leganés, madrid, spain, rdomingu@fis.uc3m.es, +34 91 624 6261 received: 2015-08; accepted: 2015-08-12 abstract at universidad carlos iii de madrid we have developed a review course (rcourse) with a spoc structure intended for students who failed the first term regular exams. the purpose of the course is to help students prepare for the remedial exams at the same time they study the second term courses. the r-course is implemented in an open-edx platform that holds digital documents, both theory and solved exercise videos, platform integrated exercises, and forums. the content of the course is divided in videos of less than 10 minutes in length, allowing students to include the r-course in any schedule and making it compatible with the second term courses. interactive platform integrated exercises have been devised under different types of methodologies to increase the motivation of the students, and the platform forums give students and tutors a space to discuss and interact for a well-rounded academic experience. keywords spoc, online course, remedial exams. domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 139 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 1. introduction within the last years moocs, massive open online courses,(cormier 2008, daniel 2012) have spread through the teaching system across the globe as a tool for educating a major amount of people on different topics and at different academic levels. the utility and efficiency of these courses is a matter of discussion since the very beginning of their existence(vardi 2012). opinions vary between considering moocs as a powerful tool to increase dramatically the reach of university teaching or considering them the beginning of the end of the academic system (breslow et al. 2013). as a natural evolution of moocs, or maybe as a possible origin of them, spocs (small private online courses) are also used as a teaching method but, in this case, since the target of these courses is naturally reduced (in opposition to moocs), one of the most common environments of their implementation is the universities (fox2013). whereas moocs are usually free, or at least they have a free modality choice, spocs are often pre-paid experiences as means of restricting the number of participants or funding the courses (pomerol et al. 2015). whatever the reason is, the reduced amount of participants is considered crucial for the improvement in the academic level for these courses, and a higher quality is a must if they belong to a university environment. there are similarities between moocs and spocs (reinhardt 2014) but one of the most important differences between them is that the first are usually developed focusing on the desired topic and releasing it to the public in order to attract students, whereas spocs are developed focusing on an already known target, and the content of the course is designed to cover a specific need that the target requires. the specificity of the content of the spocs makes them a potential solution for specific problems; despite this being a good issue, it also has a major drawback, the budget required to prepare this type of courses is significantly higher than in moocss and the resources, both human and material, required for the design, performance, and support of the courses are more numerous, along with the time needed to develop new material. these are the causes of the non-free character that spocs usually have, or maybe just the consequence of the specificity of the courses. these reasons might be the main disadvantage in the development of spocs as a standard academic tool. domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 140 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 2. difficulties and proposed methodology: review courses (r-courses) 2.1 difficulties since lisbon recognition convention in 1998, the starting point of unification of the higher education system in europe, and the so called bologna process in 1999 (bologna 1999, caddick 2008), numerous structural reforms have been carried out to achieve the european higher education area (ehea), a unified frame for the european university academic system (hinojo et al.2005). one of the less regarded consequences of these reforms is the modification of the academic calendar in many countries, especially in spain (mecd 2003, boe 2007,crue 2009). before the unification of the systems, the spanish university academic system was established so that the standard periods for the first and second terms in a degree were from september to february and from february to may respectively, and their corresponding remedial exams were held in september(see figure 1 a)). the current calendar has important differences compared to the previous one; for instance, the period in which remedial exams were usually held (september) has been moved to june(crue 2009).this change caused the extinction of the two month period (july and august)that students traditionally had to prepare for remedial exams, in exchange for a very short period of up to a few weeks between the second term regular exams (held in may) and the remedial exams (held in june) (see figure 1 b). the reduction of the study period for the preparation of the remedial exams is one of the worst disadvantages that students find in the current system, and could be one of the causes for failure in some courses, especially in the first course subjects. as a consequence of the reduction of this period, the students prepare their remedial exams during the second term while attending the lectures of that term and preparing for those courses’ exams. that situation presents a very obvious asymmetry: first term classes are over and usually no further support is given to prepare them, while second term courses have a complete calendar of lectures, and teachers can guide the study process. domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 141 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 figure 1. a) academic calendar previous to the unified frame for the european university academic system. b) academic calendar after to the unified frame for the european university academic system. 2.2 proposed methodology: review courses (r-courses) as a solution for this asymmetry between first and second term courses during the second term, and as a study tool for the students who failed the first term regular exams, at the universidad carlos iii de madrid we have developed online review courses called rcourses (where the “r” stands for review, and may be translated as “cursos r” in spanish) with the philosophy of spocs inside of a project that contains different moocs and spocs (delgado kloos et al. 2014). these courses target students who failed the first year’s first term regular exams in several engineering bachelor degrees: industrial electronics and automation, mechanical engineering, industrial technologies, energy engineering, and electrical power engineering (next editions of the course will cover all the engineering degrees from the university). first year students are the most suitable candidates for these courses since they are still getting used to the university system and the adaptation process from high school to university makes them vulnerable to certain mistakes, which entail poor performance in first term exams. the subsequent disappointment often makes them get domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 142 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 even worse results in the second term. thus, first year courses are those in which students perform worse, making them one of the main reasons for the students to drop out of university studies (cabrera et al.2006, alvarez et al. 2006). in this line, our r-courses cover two needs: they provide students with a tool set to improve their grades, and they give an instrument to the university to minimize the number of students who leave their studies. in the first year of implementation of these courses, the university decided to focus on the subjects that most first year students have trouble with, namely physics i (mechanics and thermodynamics) and algebra. these are two of the most failed subjects and two of the subjects that make the students leave their studies after failing the six chances they have to get a passing grade in any subject, so they are the perfect test for review courses. 3. structure, content and tutorship of the r-courses 3.1 structure since the target students found the conventional approach unsuccessful, in order to achieve the main purpose of the r-courses (help students pass the remedial exams), a different approach of the content must be used, including, at the same time, all the elements that the subject had during the first term, both theoretical and practical (exercises). the following structure corresponds to the physics i r-course developed by the authors of this work, which might be slightly different for each r-course. as the r-courses are oriented as a spoc and the content is hosted in an online platform (open-edx platform, (kolowich 2013)), it is obvious that the content will be focused on videos, documents and platform integrated activities. the characteristics of this content will influence directly the result of the course. since the main problem that students in the r-course face is that they are also attending second term courses, they must manage their time in doing many activities. therefore, the smaller the modules of the course, the better they will include them among other activities. consequently, the content has been divided into short topics in contrast to the structured content of a regular subject, where lectures are divided in periods of 50 minutes. following this principle, all the lessons are divided in less than 10 minutes length theory videos so a complete part of any topic can be reviewed any time the students have. in the same line, the theoretical notes provided to support the theory videos are short, simple and condensed, in order to allow students to go over the content of a topic quickly. domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 143 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 in order to develop a well-rounded course, numerous exercises have been also added to the r-course. these exercises of the course are not a compilation of the exercises of the first term subjects because that will be against the spirit of the r-course, which it is supposed to face the subject from a different viewpoint. new exercises have been developed for the r-course focusing not only on covering the subject, but also on correcting the main mistakes that students made during the first term. since solving exercises usually takes time, it cannot be expected that students dedicate only a few minutes to this activity. in order to speed up this activity, around half of the exercises are fully solved in concise videos of less than 10 minutes in length. thanks to these videos, with fully detailed and explained solutions, students become more skillful and are able to solve the rest of the exercises more comfortably, later they can also check their solutions thanks to the resolution of those problems that are available in the online platform that holds the video content. all the exercises are provided to the students at the beginning of the week with the title “docent material of the week”. using this content (theoretical notes and theory videos, and solved and proposed exercises) a full course of 14 weeks has been developed to guide students in their process of preparing their first term remedial exams. despite the guidance, students have to make a big effort to prepare the exams and follow the r-course, so some feedback has to be given to them in order to let them know that they are improving. because of this, during the 14-week course, 4 mock exams are held in the platform, including the content of the previous 3 to 4 weeks. these exams consist of a classical exam-type exercise list that is provided to the students, which are supposed to solve it without help. then, they can check their solutions with the online platform (as they were doing with normal exercises during the course) and have their feedback. these exams are not supervised and students are placed in a situation in which they could think about cheating, none the less since the result of the exam is a non-binding evaluation of their work, in opposition to the tense atmosphere of a classical exams, students learn to properly face exams or other trials with an easier attitude helping them to face a real exam. domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 144 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 figure 2.structure and content of a week of the r-course. as an example of the cited structure, figure 2 shows the structure of the fourth week of the physics i r-course. the topic of the week is “one particle dynamics” and is divided into different modules such as “newton’s laws of motion”, “types of forces”, and “applications”. the “docent material of the week” is composed of 7 pages of theoretical notes and an exercise list of 11 exercises. different amount of theory and exercise videos are present on each module. this week has a total of 13 videos (7 theory videos and 6 exercise videos) with a total length of 75 minutes. in the two last modules a total of 7 exercises are uploaded to the platform to be solved interactively by the students. the structure of the course has been proposed in a way that students may take advantage of its guided process and well scheduled distribution of the content. the course is supposed to be attended during the whole term, and the way of following it is explained at the beginning. these instructions are essential, as the philosophy of the r-course is domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 145 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 different than the regular classes. this is a summary of the instructions for a week of the course: • access and download the digital documents of the week’s material (“docent material of the week”). • after going over the theoretical notes, try to solve all the exercises on the exercise list. • watch the video content (theory videos and solved problems videos) comparing the obtained solution for the exercises with the one in the videos. • check the rest of the solutions in the online platform integrated exercises. • in case some theoretical parts were not understood or in case some exercises were wrong, go over the “docent material of the week” and the videos again until everything are understood and solved properly. • in case a mock exam is scheduled in that week, go over the digital documents again and solve the mock exam. check the solutions with the ones on the platform after solving it. contents are released every week, so that the student has an orientation as to the adequate pace of preparation, but after release they are open during the whole second term, until the week after the remedial exam. as can be seen from these instructions, there is a major difference between the regular course in the degree and the r-course. in the latter, students have already attended the regular classes, having a certain degree of knowledge about the subject. although that knowledge was not enough to pass the subject, some skills have been already acquired (along with some errors or mistakes). the main purpose for asking students to solve the exercises just after going over the “docent material of the week” and without having watched the video content is to make them realize what their level on that particular topic is and to let them know what skills they have and what they lack. this strategy promotes also an active approach to the subject, which is essential for a real learning. regardless of the result of their first attempt with the exercises, watching the video content improves their knowledge about the topic and allows them to focus on important points. the solved exercise videos allow students to check if their solution was the most domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 146 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 appropriate or, in case they were not able to solve a certain exercise properly, show how to solve it. with all the video content, students are provided with all the tools to acquire the necessary skills and knowledge needed to face all the exercises and mock exams, and in case that more in-depth study was needed to face them, they could go over the content as many times as needed. 3.2 content theoretical part as previously mentioned, the theoretical part of the course is composed of theory videos and theoretical notes. theoretical notes are based on notes for the ordinary course, but have been adapted to the structure and approach of the review course so that students can save time when going over them looking for some information. all the theory videos have been recorded specifically for the course, with a total running time of more than 11 hours, but duration is kept nearly always under 10 minutes (with an average length of 7 minutes). as already explained, the primary reason for this is to allow for a more flexible visualization schedule, but is also intended in order to focus on a single topic, in contrast to the long duration of regular classes (usually two sessions of 50 minutes in a row). this concentrates students on a single objective and improves their understanding, and it is also very convenient to save time when students want to solve specific doubts that could arise when studying certain topics. yet another reason to parcel the content in small units is the typical low attention span for videos. taking this into account, additional effort has been made to engage the viewer, using a variety of resources (animated handwriting, powerpoint-like parts, animations, cartoons that ask questions on behalf of the students, etc). the theory videos cover all the syllabus of the physics i course, but not only their length is more reduced compared to the first term classes but also their content is focused on concepts the students usually have trouble with and on the main points they usually do not understand. this issue is very important since students in the r-course have already failed one of the exams; this probably means they lack understanding of the basics of the subject. thus the emphasis is put on the conceptual difficulties. domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 147 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 practical part the practical part is composed of exercises, which are provided to the student in the form of an exercise list. as previously mentioned, all the exercises have been developed specially for the r-course. besides covering the whole subject, they have been designed and presented in the exercise list in a way that students find a constant increase in difficulty. this sequential increase of difficulty avoids students to face difficult exercises before having solved basic ones, preventing them to get frustrated and drop the course. in addition to this, the problems usually are module specific and focus on a single information item allowing students to acquire a specific piece of knowledge. in the case of problem-solving videos, this is also complemented with some appointments (that are placed in their own sections inside the videos) to clarify some points or to emphasize the importance of certain parts of the problem or the solution process. in order to motivate students to not simply watch the videos but also understand them, sometimes some questions are asked when a certain concept is being treated. in that moment, some optional answers are given (some of them containing the most common mistakes) and after a short time the correct answer is provided and explained. this allows students to check if they understand what they are watching. the exercises not solved in the videos, i.e. the proposed exercises, follow the same structure of the solved ones, and in most of the cases they are variations of the ones solved in the videos. this way the students can use what they have learned from the videos to solve the rest of the exercises in case they were not able to solve them at the beginning of the week when the exercises list was provided. in order to make the exercises less repetitive, and due to the different nature of the exercises, several types of exercises are available in the platform. this makes the rcourse a little bit more “friendly” to avoid lack of motivation. this variety of exercises goes from just filling fields with numerical solution or an expression, to “drag and drop” pictures or names over a figure, or even multiple-choice test-like exercises. mock exams while the exercises from the “docent material of the week” are single module focused, mock exams contain a mixture of exercises from the topics treated in the latest weeks of the course. moreover, as going further in the r-course (so more topics are explained) the exercises in mock exams combine more topics to force students relate the content from domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 148 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 different weeks (such as “single particle kinematics” and “single particle dynamics” or “single particle dynamics” and “rigid body dynamics” in the case of physics i). this combination of contents is an evolution of the content towards a more realistic exercise that could be found in a real exam such as the remedial exam that is the final target of the r-course. 3.3 tutorship even when the full course is well structured and the understanding of the content does not depend on external resources, there is a need for students to have access to tutoring in order to guide them through the spoc and to solve any question that could arise relative to the content or to the online platform. because of this, tutorship is available during the whole length of the course and is provided at several levels of interaction. the first grade of interaction is the availability of an online forum integrated in the online platform. the topics in the forum can be created from each module of the course using a “discussion tool”. by using this tool, the question or comment from the student can be seen and accessed by the rest of the students and tutors in the same page the related resource is, and also in the forum page itself. thanks to this, when students access a new content, they can check the comments that students who previously accessed that content wrote; and the tutors have access to all the comments in the forum page for global access to all of them. using these topics of the “discussion tool”, the students can answer their doubts among them under the supervision of tutors that can join in the discussion if needed to correct some incorrect answer to the comments. the second grade of interaction is the direct contact with the tutors via email. this closer approach allows students to ask questions to the tutor directly, generally to arrange a meeting in person to solve more in detail questions. this is important since conceptual questions, or questions that involve a fair amount of mathematics, may be hard to treat in a forum. personal contact with the tutor may also be motivating and reassuring for the student. the third grade of interaction is the periodic meetings in which all the students are summoned for a purpose. these sessions are scheduled near the mock exams dates and generally after them, to discuss the main problems that could have arisen during the exam and provide feedback. a specific topic in the forums are open a week before these domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 149 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 sessions in order to allow students to propose specific questions they would like to be treat in the sessions. the addition of these three grades of tutorship to the sequential opening of the material of the course provides the appropriate guidance through the whole process of preparation for the first term remedial exams that students need during the second term in order to make both terms subjects preparation compatible. 4. conclusions we have developed a review course intended for students who failed the first term regular exams. the 9 modules of the content of the course, 5 modules of mechanics and 4 of thermodynamics, are split though 14 weeks. these modules consists of two different kind of contents, video content (theory and exercises) and platform integrated content (theoretical notes and exercises list). during the 14 weeks duration of the review course, the content is made public to the students sequentially to guide their preparation that is also followed by the online tutors that are available during the full course. we propose this r-course as a solution for one of the main problems that the spanish university system currently have, that is the systematic failure of the remedial exams after the implementation of the united frame of the european academic system. this will not only benefit students but also universities, since failing first year courses is one of the main reasons for students to leave their studies prematurely. 5. acknowledgements the authors wish to acknowledge the financial and structural support from universidad carlos iii the madrid, in particular from the uteid (unidad de tecnología educativa e innovación docente) and the encouragement from professors carlos delgado-kloos and luis raúl sánchez fernández. 6. references álvarez pérez p. r.; cabrera pérez l.; gonzález alfonso m. c. and bethencourt benítez j. t. (2006). “causas del abandono y prolongación de los estudios universitarios”. paradìgma 2006, vol.27, n.1, 349-363 domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 150 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 bologna (1999). “the bologna declaration of 19 june 1999”. bologna, italy: european ministers of education boe (2007). “real decreto 1393/2007”, b.o.e. 30-10-07, http://www.boe.es/boe/dias/2007/10/30/pdfs/a44037-44048.pdf breslow, l. b., pritchard, d. e., deboer, j., stump, g. s., ho, a. d., and seaton, d. t. (2013). “studying learning in the worldwide classroom: research into edx's first mooc”. research&practice in assessment, 8, 13-25, 2013. cabrera pérez l., bethencourt benítez j. t., alvarez pérez p., gonzález afonso m. (2006). “el problema del abandono de los estudios universitarios”. relieve. revista electrónica de investigación y evaluación educativa 2006, 12 (2) http://www.uv.es/relieve/v12n2/relievev12n2_1.htm caddick s. (2008). “back to bologna.the long road to european higher education reform”.embo rep. 2008 jan; 9(1): 18–21, http://dx.doi.org/10.1038/sj.embor.7401149 cormier d. (2008). “the cck08 mooc–connectivism course, 1/4 way” . dave’seduc. blog, vol. 2, 2008. http://davecormier.com/edblog/2008/10/02/the-cck08-mooc-connectivism-course-14way/ crue (2009). “estudio sobre el calendario académico en el marco del eees”. febrero 2009, conferencia de rectores de las universidades españolas, http://www.um.es/convergencia/wp-content/uploads/2009/02/estudio-de-un-calendarioacademico-unico.pdf daniel j. (2012). “making sense of moocs: musings in a maze of myth, paradox and possibility”.j. interact. media educ., vol. 3, no. 0, dec. 2012. http://dx.doi.org/10.5334/2012-18 delgado kloos, c, munoz-merino, p.j., munoz-organero, m., alario-hoyos, c., perezsanagustin, m., parada g, h.a., ruiperez, j.a., and sanz, j.l. (2014). “experiences of running moocs and spocs at uc3m”. global engineering education conference (educon), 2014 ieee , 884 – 891, http://dx.doi.org/10.1109/educon.2014.6826201 fox a. (2013).“from moocs to spocs”.communications of the acm, vol. 56.no. 12, page 38-40, december 2013, http://dx.doi.org/10.1145/2535918 domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 151 http://polipapers.upv.es/index.php/muse http://www.boe.es/boe/dias/2007/10/30/pdfs/a44037-44048.pdf http://www.uv.es/relieve/v12n2/relievev12n2_1.htm http://dx.doi.org/10.1038/sj.embor.7401149 http://davecormier.com/edblog/2008/10/02/the-cck08-mooc-connectivism-course-14-way/ http://davecormier.com/edblog/2008/10/02/the-cck08-mooc-connectivism-course-14-way/ http://www.um.es/convergencia/wp-content/uploads/2009/02/estudio-de-un-calendario-academico-unico.pdf http://www.um.es/convergencia/wp-content/uploads/2009/02/estudio-de-un-calendario-academico-unico.pdf http://dx.doi.org/10.5334/2012-18 http://dx.doi.org/10.1109/educon.2014.6826201 http://dx.doi.org/10.1145/2535918 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3687 social and technological sciences eissn: 2341-2593 hinojo lucena f.j., alonso garcía s. (2205). “la adaptación al espacio europeo de educación superior y su incidencia en los recursos funcionales: el tiempo escolar”. english abstracts 22, vol. 8 (7) 2005, issn 1575-0965. kolowich, s. (2013).“how edx plans to earn, and share, revenue from its free online courses”.chronicle of higher education. february 21, 2013. http://chronicle.com/article/how-edx-plans-to-earn-and/137433/ mecd (2003). “la integración del sistema universitario español en el espacioo europeo de enseñanza superior”. febrero 2003, ministerio de educación, cultura y deporte, http://www.eees.es/pdf/documento-marco_10_febrero.pdf pomerolj.c.,epelboin, y., thouryc. (2015). “moocs design, use and business models”. 2015. 142 seiten, hardcover . isbn 978-1-84821-801-7 john wiley & sons reinhardt a. (2014). “torques: a turning point for teaching”.ethz. http://www.let.ethz.ch/projekte/concept_torque_ethz.pdf vardi moshe y. (2012). “will moocs destroy academia?” . communications of the acm, vol. 55 no. 11, page 5, september 2012, http://dx.doi.org/10.1145/2366316.2366317 domínguez-reyes et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 139-152 | 152 http://polipapers.upv.es/index.php/muse http://chronicle.com/article/how-edx-plans-to-earn-and/137433/ http://www.eees.es/pdf/documento-marco_10_febrero.pdf http://www.let.ethz.ch/projekte/concept_torque_ethz.pdf http://dx.doi.org/10.1145/2366316.2366317 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 the cooperative learning: understanding and increasing the knowledge of the facilities design without a professor extra effort c. ferrera1*, j. fernández2, a. c. marcos1 1dpto. de ingeniería mecánica, energética y de los materiales. escuela de ingenierías industriales. universidad de extremadura, avda. de elvas s/n, 06006 badajoz, spain. e-mail: cfll@unex.es*, tlf: +34924289300 (86172), fax: +34924289601 2dpto. de energía. escuela politécnica de mieres. universidad de oviedo c/ gonzalo gutiérrez quirós s/n, 33600 mieres received: 2013-12-15; accepted: 2014-02-28 abstract lecturing has been prevailing in higher education. this teaching and learning model hinders the understanding of fundamental concepts in practical courses. the cooperative learning allows an improvement in the student’s achievements, attitudes and persistence. the main goal of this work is to implement the cooperative learning in the teaching of the design of industrial facilities. this methodology aims to solve part of the problems of recently graduate students when they undertake engineering projects lacking knowledge. finally, the results of an end-of-course satisfaction survey, conducted to assess this experience, are also presented. keywords cooperative learning; distance learning; peer learning; project-based learning ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 1 mailto:cfll@unex.es* multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 1. introduction the main goal of a professor has always been to success in the fully knowledge transmission. this knowledge should be understood and it should remain in the students for a long time. the earliest ideas about how to transmit the information to be understood and kept in mind as long as possible were conceived as a cone (dale 1946). the base represented the most firmly understood knowledge. real experiences with a specific purpose are in the base and verbal communications are in the apex. so the model which warrantees a better comprehension is the one where the student receives the biggest audiovisual support1. this support has always depended on the era when it was being taught. however, it has always originated from a fundamental principle: our brain learns better if we practice cooperative learning. primitively, the education was based mainly in cooperative learning with the family and the senior citizens (luzuriaga 1977) but the increase of the knowledge caused the building of schools. one of the first methods was the dialogue-based learning (socrates). in this method the teacher guided the students to discover the knowledge by means of lecturing, reflecting and questioning. in the roman empire, it was discovered that a person learns better when it teaches “docendo discimus” (seneca 64). nowadays, the cooperative learning appears continuously. for example, in remote rural areas in india there are real evidences in which children discover the way to use a computer and to surf the internet (sugata et al. 2005). on the other hand, a teaching and learning model based in the oral communication has been prevailing in the university. this model, consciously or unconsciously, has been penalizing the communication from the student to the professor. this problem is due to the enormous difference between the professor and the student’s knowledge. some1 there are some studies where a memorizing percentage is added as a function of the support received (chi et al. 1989) but some papers do not agree with this addition (betrus and januszewski 2002). ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 2 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 times, the cause is the huge courses programs which force the professor to reduce this communication. therefore, it has been a transition from a learning model where the student was the main part of the process to another model where the student plays a secondary role. in addition, the courses programs have been enlarged with the help of audiovisual multimedia. so multimedia contents, instead of enforce the knowledge of the student, have been used wrongly to increase the contents of the subject. this practice can cause an unmotivated and bored set of students who listen to a professor teaching “a brilliant lecture”. in this situation, the best students can keep some concepts in their short-term memory (as it has its limits), then they memorize the rest and, finally, they can solve the problems introduced by the teacher. the teacher has not to intervene and they pass the subject with some effort. but the real question is: do they understand the underlying concepts? do they know how to resolve another type of problems? the answer is: they learn but they have conceptual failures which prevent them to solve problems whose formulation is different to the usual one (halloun and hestenes, 1985a). this fact, confirmed by other authors (halloun and hestenes, 1985b; mazur 1992), has brought us to a conclusion: “it is difficult to learn in the university because the students do not collaborate in their study” (smith 1998). therefore, it is very important that students practice cooperative learning, not only in their own study (chickering and gamson 1987) but also with their professors and classmates. one of the most significant examples among all the collaborative learning working groups existing in spain, are the interactive ones (aubert et al. 2000; aubert et al. 2008; castells et al. 1994; flecha et al. 1977; flecha 1997). these are groups of students, fathers and teachers who talk among peers improving the results, the persistence and the attitudes of the students (springer et al. 1999). these groups make them responsible of their own and their peers learning process (michaelsen et al. 2003; paris and turner, ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 3 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 1994; weimer, 2002). the first student, who understands a concept, improves his knowledge teaching his peers. the peers understand the concepts in a better way because they receive explanations from a person who has just created the strategies to comprehend them. so they prevent the invisible barrier of the vast amount of knowledge of the professors, they allow to reduce the student work, improve the comprehension and allow to work transversally another abilities (problem solving, working in groups, leadership ability, project management, capacity to analyze the peers work, capacity to resolve conflicts in a group and organization) (sheetz 1995; winchesterseeto 2002). on the other hand, the university teaching is very specific, with tools for the big groups (davis 1993; lewis 1994; mckeachie 2006). nevertheless, it is very difficult to work with reduced groups in massive attended lecture halls. this problem could be solved with the help of assistant professors, teaching monitors (nyquist et al. 1991), or dividing the groups (macgregor et al. 2000; michaelsen 1983; stanly et al. 2002). however this is not possible in the spanish universities, so we have to apply another techniques developed in other universities. in the university of harvard, the professor teaches asking and does not lecture. this technique is called peer instruction or peer learning (mazur 1997; crouch 1998; crouch and mazur 2001). the student, previous to the class, has to read a text and to answer a set of questions online. the professor reads the results and prepares himself to introduce the problematic concepts in the following session. then he reinforces those concepts and asks new questions during the lecture. if the percentage of successful answered questions is higher than 70% he continues introducing new concepts but if this percentage is between 30% and 70% he lets a few minutes for the students to talk each other about these answers. the collaborative learning appears in this moment because some students have just solved his difficulties assimilating a concept and can teach their peers how they did it. many students understand the con ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 4 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 cept, improving the percentage of correct answers (ogawa and wilkinson 1997; saye 1997; redish 2003) and this improves the professorship quality (jackson and bruegmann, 2009). the objective of this paper is to put in practice the best of the mentioned methodologies (lecturing, project-based learning and cooperative learning) in the teaching process of the design and measure of industrial facilities for a student body of more than one hundred students. 2. current situation and problem statement the subject object of our study is “industrial and commercial installations ii”. it is taught in the third course of the degrees of industrial engineering in the university of extremadura. in a semester the students have to learn the knowledge of a wide range of facilities starting from a generic base where they do not know the basic components of a facility. we have 140 students divided into three activity groups. these groups are further divided in small groups of 20-30 students for laboratory classes. the program of the subject is very long with directed laboratory practices where the students only have to follow a provided guide. they do not need to justify the selected option and they run commercial software, easy to use without the possibility for multiple changes. the practices are not specific and do not allow to check if a student has acquired the necessary skills. the consequence is that the students gain a quite general knowledge but do not have the ability to calculate and to accomplish a specific project. the main goal of the present work was to apply collaborative learning in this subject, to establish a bridge with a fourth course subject (entitled projects) and to increase the program of the subject to the specific projects. thus the students can understand the calculations required by an industrial installation and they gain new abilities (yedidia et ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 5 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 al. 2000). these abilities were roughly and transversely learned by the graduates in previous years. the activity was designed in a way both professors and students did not take on an enormous extra work. the students were compensated for the effort these kinds of experiences require. the activity was atemporal and was not limited to a specific lecture hall. the necessary material was available through the virtual campus of the subject. the only spatial and temporal limitations were the weekly supervision meetings of the different tasks imposed by the professors. 3. learning methodology the methodology is mainly based in project-based learning, combining lecturing and the peer instruction. the developed activity is to measure a specific hydraulic facility. it was introduced in the first class the professor attended the students and it was limited to the first thirty students enrolled. these students were divided into five groups and were free to attend the laboratory classes as a compensation for the extra work. however the professors recommend them to attend these classes to learn the knowledge taught there. each project corresponded to a different installation (hotel, housing block, residential area, sporting arena and swimming pool). the professors granted one point out of ten for the students involved if the project was calculated and presented successfully2. the students received a short and very specific task each week. the schedule for the different tasks was: 2 there are recent references where the students, who collaborate teaching other students, receive recommendation letters (downing and liu 2012). ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 6 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 1st week: the professor introduces the work to be done and give to each group a blueprint of one facility. task: design a cad pipe network with adequate pipe lengths for each section (figure 1). figure 1. cad design of a pipe network in a sporting arena 2nd week: the professor checks the different blueprints. task: to create a spreadsheet (figure 2) with the minimum flow rate for each pipe. the flow rate has to be calculated according with the current regulations for each device. 3rd week: the professor looks over the different spreadsheets. task: to calculate the simultaneous flow rates in groups of different devices. ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 7 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 4th week: to inspection previous work and to explain the concept of maximum water speed in pipes. task: calculate the inner and the nominal diameter of the pipes to accomplish the requirements of the current regulations. 5th week: the professor checks the diameters of the pipes. task: evaluate the head loss in pipes and the minor losses. 6th week: the professor teaches the need of a pressure group. task: the students rest this week. 7th week: to finish the projects. figure 2. example of the shared spreadsheet ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 8 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 the professor lectures the students the first three weeks in the computer lab. the attendance is mandatory. after the third week of instruction, the groups receive technical assistance during the office hours. the different spreadsheets are shared in google drive (figure 2). this is a useful tool for both the professor and the students. they can do different changes and the professor could check them and solve the doubts. the students’ activity developed into a project-base activity in the last weeks. they started to confront their ideas showing different alternatives to the calculations made, improvements, errors, etc. after a first phase based in lecturing and project-based learning, a second phase started. in this phase the main objective was to transmit all the knowledges to the rest of the class (110 additional students). our idea was to apply peer learning. the students involved in these projects taught the rest of the class with a very slight supervision of the professor. we freed them to explain the knowledge in their own way. so they not only publicly demonstrate them they have the knowledge to measure an industrial facility but also help the professors in the learning process of the class. this phase was performed in the context of a short general discussion session. the format of this meeting is lecturing and dialogue between students. a speaker was elected in each group. the selected speakers prepared an oral presentation. the presentation allowed them to teach the rest of the class the different steps associated with the development of any hydraulic system. the teacher reviewed the presentation content before the public exhibition. we would like to mention that the preparation to the content was fully agreed between the professor and the selected students. everybody had to reason the importance of introducing or deleting a concept. for this reason, some different concepts, the teacher had given for granted, were preserved at their request. the remaining members of the group made a poster (figure 3). its cost was covered by a programme to strength the ehea in the university of extremadura. the use of posters came to fill ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 9 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 gaps in the oral presentation, and allowed us to observe particular details untreated during exposure. the students helped their classmates with unresolved questions that had appeared during the oral presentation. the great majority of the questions were resolved by them without teacher involvement. this showed us the total assimilation of the concepts and the work done. besides, the students, who did not participate, asked all kinds of questions. they were interested in calculating the facility and rated positively the work of their peers. figure 3. image of one of the posters made ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 10 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 4. results the students have learned to analyze, synthesize and manage technical information provided by the professors. they have solved a complex problem in an autonomous way with a slight supervision of the teacher. working in groups has allowed them to develop interpersonal skills such as to discuss between equals, to tolerate mistakes and to criticize constructively their peers with the support of the analysis of the data, the results of their classmates and other sources. the rivalry between groups has encouraged them to give the best of themselves, which has resulted in an improvement in the quality of work and an increment in their knowledge. moreover, they have begun to speak in public. all the aforementioned has been done by using public domain or free software for students. they have been able to find, to select and to use appropriate calculation tools for projects. finally a satisfaction suvey (figure 4) was conducted between the participating students in order to get their about how this teaching methodology was performed. the survey consisted of eleven questions, which are shown in the final appendix of this article, with five possible answers (none, little, some, quite a lot) to each one of them. additionally a web form is added to save suggestions for improvement and to include possible errors. over 88% believe that the difficulty level and the methodology are quite or very adequate. the same percentage is satisfied with the additional point given to the task (questions 1, 5 and 7). all the students considered that weekly and group supervision is correct (questions 2 and 3). they also rate positively the extra formation received (question 6). a 60% report that the time spent on homework is excessive (question 4), so they propose that the initial blueprints are delivered in the format of the graphic design tool with which they work. they are demanding more information at the subject website to ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 11 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 reduce the number of visits to the teacher. this is quite contradictory, since 88% shows that it is useful to start from scratch in the project (question 9). in addition, the experience in the first two sessions, where freedom for calculations was given, showed us the huge majority of students preferred a personal assistance. they also propose other conflicting solutions, such as decreasing the number of components in the group, so they would have to do more work. regarding the explanation to other students, more than 75% considered the method effective. on the desirability of this methodology remains in the future, over 88% expressed should be done with all the subjects of the degree. figure 4. survey results 5. conclusions a mixed methodology between lecturing, project-based learning and cooperative learning on teaching projects of industrial facilities has been implemented for over a hundred students. it has been found that the practices may no longer be over-managed and can be closer to the reality of the engineers. the completion of this work reveals the creativity 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 11 % a ns w er s question number none little some quite a lot ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 12 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 of students. the use of free software, a virtual campus with accurate information and a weekly meeting with each group is enough to guide their work and not to overload the teacher. it is important to reward the extra temporary effort undertaken by the students when put up experiences like this. experience has shown the student satisfaction, who wants this activity to be extended to other subjects. acknowledgements the authors gratefully acknowledge the financial support provided by the servicio de orientación y formación docente (guidance and teacher education service) from university of extremadura under the context of actions for the consolidation of the european higher education at the university of extremadura. references aubert, a., flecha, a., garcía, c., flecha, r. and racionero, s. (2008). aprendizaje dialógico en la sociedad de la información. barcelona. ed. hipatia. aubert, a., medina, a. and sánchez, m., (2000). de las agrupaciones flexibles a los grupos interactivos. viii conferencia de sociología de la educación, madrid, españa. betrus, a. and januszewski, a. (2002). for the record: the misinterpretation of edgar dale's cone of experience. session given at the 2002 annual meeting of the association for educational communications and technology (aect). dallas, usa. castells m., flecha, r., freire, p., giroux, h., macedo, d. and willis, p. (1994). nuevas perspectivas críticas en educación. ed paredes. ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 13 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 chi, m. t. h., bassok, m., lewis, m. w., reimann, p. and glaser, r. (1989). selfexplanations: how students study and use examples in learning to solve problems. cognitive science, 13, 145-182. doi: 10.1207/s15516709cog1302_1. chickering, a. w. and gamson, z. f. (1987). seven principles for good practice in undergraduate education. wingspread journal. crouch, c. h. (1998). peer instruction: an interactive approach for large clases. opt. photonics news 9 (9), 37-41. doi: 10.1364/opn.9.9.000037 crouch, c. h. and mazur, e. (2001). peer instruction: ten years of experience and results. am. j. phys., 69, 970-977. doi: 10.1119/1.1374249. dale, e. (1946). audio-visual methods in teaching. new york: the dryden press. davis, b. g. (1993). preparing to teach the large lecture course, chapter 12 in tools for teaching. san francisco: jossey-bass. downing, c. e. and liu, c. (2012). getting students to teach each other: doing more with less in is education journal of information technology and application in education vol. 1 iss. 4, 195-206. flecha, r., lloret, c. and garcía, j. m. (1977). transformemos la escuela. ed. renacimiento. flecha, r. (1997). compartiendo palabras. el aprendizaje de las personas a través del diálogo. ed. paidós ibérica. halloun i. a. and hestenes, d. (1985a). the initial knowledge state of college physics students”. am. j. phys. 53, 1043-1056. doi: 10.1119/1.14030 halloun i. a. and hestenes, d. (1985b). common sense concepts about motion, am. j. phys. 53, 1056-1065. doi: 10.1119/1.14031 ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 14 http://www.doi.org/ http://www.doi.org/ http://dx.doi.org/10.1119/1.1374249 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 jackson, c. k. and bruegmann, e. (2009). teaching students and teaching each other: the importance of peer learning for teachers. american economic journal: applied economics 1 (4), 85-108. doi: 10.1257/app.1.4.85. lewis, k. g. (1994) teaching large classes (how to do it well and remain sane), chapter 25 in handbook of college teaching, edited by k.w. prichard and r.mcl. sawyer. westport, conn.: greenwood press. luzuriaga, l. (1977). historia de la educación y la pedagogía. ed. losada. macgregor, j., cooper, j. l., smith, k. a. and robinson, p. (2000). strategies for energizing large classes: from small groups to learning communities, new directions for learning and teaching, 81. mazur, e. (1992). qualitative vs. quantitative thinking: are we teaching the right thin?", optics and photonics news. mazur, e. (1997). peer instruction: a user’s manual. prentice hall series in educational innovation. mckeachie, w. j. (2006) teaching large classes (you can still get active learning!) chapter 19 in teaching tips, 12th edition. lexington, mass: heath, 2006. michaelsen, l. k., (1983). team learning in large classes. chapter 2 in learning in groups, new directions for teaching and learning series, 14. san francisco: josseybass. michaelsen, l. k., knight, a. b. and fink, l. d. (2003). preface. in l. k. michaelsen, a. b. knight, & l. d. fink (eds.), team-based learning: a transformative use of small groups, (pp. vii-xi). westport, ct: praeger. nyquist, j. d., abbott, r. d., wulff, d. h. and sprague, j. (1991). preparing the professorate of tomorrow to teach: selected readings in ta training. kendall/hunt. ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 15 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 ogawa, n. and wilkinson, d. e. (1997). let your students teach their class. the internet tesl journal, 3 (12). paris, s. g. and turner, j. c. (1994). situated motivation. in p. r. pintrich, d. r. brown, & c. e. weinstein (eds.), student motivation, cognition, and learning: essays in honor of wilbert j. mckeachie (pp. 213-238). hillsdale, nj: erlbaum. saye. d. (1997). an alternative technique for teaching mathematics: students teach. proceddings of the 23rd amatyc annual conference atlanta, usa. seneca, l. a. (64). epistulae morales ad lucilium 7, 9. sheetz, l. p. (1995). recruiting trends: 1995-1996. east lansing, mi: collegiate employment research institute, michigan state university. smith, f. (1998). the book of learning and forgetting. new york: teachers college press. springer, l., stanne, m. e. and donovan, s. s. (1999). effects of small-group learning on undergraduates in science, mathematics, engineering and technology: a metaanalysis. review of educational research, 69, 21-51. doi: 10.3102/00346543069001021. stanly, c. and porter, m. e. (2002). engaging large classes: strategies and techniques for college faculty . onker publishing. sugata, m., dangwal, r., chatterjee, s., jha, s., bisht r. s. and kapur, p. (2005). acquisition of computer literacy on shared public computers: children and the “hole in the wall”. australasian journal of educational technology, 21(3), 407-426. redish, e. f. (2003). teaching physics with the physics suite. john wiley & sons. ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 16 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 weimer, m. (2002). learner-centered teaching: five key changes to practice. san francisco: jossey-bass. winchester-seeto, t. (2002). assesment of collaborative work-collaboration versus assesment. invited paper. annual uniserve science symposium. university of sydney. australia. yedidia, m. j., gillespie, c. c., moore, g. t. (2000). specified clinic competencies for managing care. jama, 284 (9), 1093-1098. doi: 10.1001/jama.284.9.1093 ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 17 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3262 social and technological sciences eissn: 2341-2593 appendix satisfaction survey of this methodology 1. is correct the difficulty level? 2. do you agree with the weekly supervision in the lecture hall/professor’s office? 3. is correct the supervision of the different groups separately? 4. do you consider excessive the time spent in this work? 5. are you satisfied with the extra point? 6. do you find useful the extra formation received? 7. is correct the methodology applied? 8. are you interested into extend this kind of practices to the rest of this subject? 9. is it useful to start from scratch the project design? 10. do you believe that to create presentation/posters is an effective way to transmit all the knowledge learned to the rest of the students? 11. are you interested into extend this kind of activities to different subjects in the degree? could you write down any suggestions to improve or to correct errors? ferrera et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 1-18 | 18 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 uncovering patterns of interest in useful plants frequency analysis of individual students’ interest types as a tool for planning botany teaching units peter pany1, 2*, christine heidinger1 1austrian educational competence centre for biology (aeccbio), university of vienna. porzellangasse 4, 1090 vienna, austria. 2botanical garden of the university of vienna. rennweg 14, 1030 vienna, austria. * corresponding author: e-mail: peter.pany@univie.ac.at; phone: +43(0)6802125676 abstract the paper presented examines how useful plants can help counteracting “plant blindness” – a phenomenon leading people to overlook plants in everyday-life. recent research indicates that people are most likely interested in useful plants, hence this group of plants could be used to trigger interest in botanical content in general. this study has investigated the structure of interest in five subgroups of useful plants (medicinal plants, stimulant herbal drugs, spice plants, edible plants, and ornamental plants). for this purpose, the fein-questionnaire (fragebogen zur erhebung des interesses an nutzpflanzen = questionnaire acquiring interest in useful plants) was filled in by n = 1,299 pupils from grade 5 to 12. data analysis shows (for all age groups and both genders) that medicinal plants and stimulant herbal drugs trigger high interest while spice plants, edible plants and ornamental plants raise only lower interest. however, mean values do not allow conclusions on an individual level (e.g. in a school class). in order to gain information about the interest structure in a specific target group teachers deal with in practice, we have analysed the interests on an individual level using frequency analysis of different interest types. results show that stimulant herbal drugs seem to strongly polarize students, whereas medicinal plants are interesting for almost the whole sample. eventually, medicinal plants turned out to be well suited to introduce botanical content by means of plants catching the interest of as many students as possible. therefore, medicinal plants should be established as flagships counteracting plant blindness. keywords interest types; medicinal plants; plant blindness; questionnaire; students’ interest; useful plants pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 15 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 1. introduction teaching botanical content is one of the hardest tasks in biology lessons (greenfield, 1955). a prominent reason for this fact is a phenomenon called “plant-blindness”, described about twenty years ago (bozniak 1994; wandersee and schussler 2001). plant blindness leads people to overlook plants in everyday life and therefore they do not gain knowledge about them (wandersee and schussler 1999; schussler, link-pérez, weber and dollo 2010). furthermore, studies on students’ interest in biological topics show that botanical issues are the most boring for students (elster 2007; lindemann-matthies, 2005). during adolescence their interests shift from animal biology rather to human biology and the interest in plants decreases even more (baram-tsabari and yarden 2007; baram-tsabari, sethi, bry and yarden 2010; osborne and collins 2001; tamir and gardner 1989). moreover, plants are often seen as inferior creatures compared to animals (flannery 2002) and are perceived only as a kind of scenery for animal life (schussler and olzak 2008; wandersee and schussler 1999). though, according to educational psychology research pre-existing interests are an important key for connecting new information to existing knowledge (hidi and baird 1986; hidi 1990; krapp 1999) and interest is an important basis for the development of intrinsic motivation to deal with a subject and thereby gain deeper knowledge (deci 1992; deci and ryan 1993). therefore, students’ lack of interest in plants is a big challenge for biology teachers, especially when the high educational value of botanical knowledge is taken in to account. knowledge about plants is an important prerequisite for the understanding of central biological concepts like evolution (wandersee and schussler 1999), lifecycles (schussler and winslow 2007) or the role of plants in ecological cycles like the carbon cycle (wandersee and schussler 1999). without profound botanical proficiency students develop a restricted view on nature which may also affect their attitudes towards their environment or environmental problems (dillon et al. 2006). pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 16 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 hence, plants have to be placed at the centre of humans’ perception of nature. but how can biology education endeavour to accomplish this purpose? the chosen approach in the present study is to start from students’ interests. as educational science research (e.g. deci and ryan 1993; hidi and baird 1988; krapp 1989) has extensively pointed out, it makes sense to distinguish between two different forms of interest. whereas individual interest in a subject develops gradually, is composed of subject knowledge and values and is regarded as a long lasting preference for a certain topic, on the other side, situational interest is a specific state which has its origin in a certain stimulus. it occurs spontaneously in different situations and is of only short duration. therefore, in our studies we focused on the more stable individual interests. in our literature review we pursued the question whether there are any groups of plants which are interesting for students. these groups could then be used by teachers as gateways to botanical content. unfortunately most studies on students’ interest in biological topics do not investigate systematically the interest in different groups of plants (e.g. wandersee 1986). quite on the contrary, plants are rather treated as a homogenous group in these studies. for example in the international investigation of students’ interest in science topics called “relevance of science education” [rose] students were asked very general questions with regard to their interest in botanical topics, e.g. „how plants grow and reproduce“, „plants in my locality“ (schreiner and sjøberg 2004). however, first hints that the group of useful plants could be worth examination came from (mayer and horn 1993) who showed that students prefer living organisms which are of value for human use. in addition krüger and burmester (2005) found that beside the “look of plants”, the “usefulness of plants” is the most prominent category students use to put plants into order. hammann (2011) partially supports the hypothesis that the group of useful plants is interesting for students by showing that students are highly interested in medical plants. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 17 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 following this trace, we systematically explored the interest of students in useful plants. as recent research has shown that questionnaires are appropriate tools for examining students’ interests (urhahne, jeschke, krombaß and harms 2004) we developed the fein-questionnaire (fragebogen zur erhebung des interesses an nutzpflanzen = questionnaire acquiring interest in useful plants) in order to explore the interest in different subgroups of useful plants in different age groups and genders. for this purpose, the fein-questionnaire was filled in by n = 1,299 austrian students (age ranging from 10–18 years). data analysis showed that the structure of interest in useful plants resulting from a pca, followed the botanical differentiation into the five subgroups medicinal plants, stimulant herbal drugs, spice plants, edible plants and ornamental plants which all raise different degrees of interest (sales-reichartzeder, pany and kiehn 2011; pany 2014). the means of interest of the whole sample show that medicinal plants were the most interesting group, followed by stimulant herbal drugs and spice plants. all three plant groups attracted above average interest of students of all age groups and both genders. edible plants and ornamental plants attracted less interest (see table 1). furthermore, there were significant differences with regard to what degree different age groups were interested in the five plant subgroups. the interest in medicinal plants was high in younger students (10–12 years) and older students (17–19 years), but lower in the age groups between 12–16 years, whereas the interest in stimulant herbal drugs showed no significant differences between the four age groups. the interest in edible plants, ornamental plants and spice plants was significantly higher in younger students (10–12 years) than in the other age groups. furthermore, ornamental plants showed strong gender differences in all ages, they are significantly more interesting for girls than for boys. table 1. means (m) and standard-deviation (sd) of interest in different plant groups measured with the fein-questionnaire; means above 2.5 indicating above average interest. from: pany (2014) plant group m sd medicinal plants 3.09 0.75 stimulant herbal 2.90 0.88 pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 18 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 drugs spice plants 2.56 0.78 edible plants 2.43 0.78 ornamental plants 2.32 0.89 as present research (baram-tsabari et al. 2010; strgar 2007) has pointed out, it is important to gain as much information as possible about the interest profiles prevailing in the target group in order to connect the science curriculum and its content to students’ interests. baram-tsabari and yarden (2009) used for example cluster-analysis to identify groups of students with similar interests in a large scale study and proposed using such data as support for the choice of content in science classes. they actually called it a “shadow-curriculum” supposed to assist teachers in complying with their respective national science curricula. comparing the data presented above, it may be concluded that medicinal plants as well as stimulant herbal drugs are suited as gateways to botanical content meeting students’ preexisting interests, and hence could be recommended as exemplary content of school lessons. both plant groups attract above average interest of students of all age groups and both genders. nevertheless, this inference has to be treated with caution. as valsiner (1986) has clearly pointed out, population data – as for example means or correlational data – do not allow conclusions on an individual level. however, most data on students’ interest reported in science education literature, including our own investigations up to now, are calculations on population level (e.g. elster 2007; pany 2014; sjøberg and schreiner 2010). hence, they allow drawing conclusions and making predictions only on this level and not on an individual level whereas direct information on an individual level is necessary for planning a botany lesson that is interesting for as many students as possible. while working in the classroom a teacher actually does not deal with a group of “mean students” but with a group of students each having diverse individual interests. inspired by such reasoning, the present study analyses students’ interest in useful plants pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 19 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 on an individual level in order to get a suitable answer which subgroup of useful plants is the most promising key to counteract plant blindness in the classroom. 2. material and methods 2.1 questionnaire the fein-questionnaire tests five scales which measure the interest in edible plants, spice plants, stimulant herbal drugs, medicinal plants and ornamental plants. each of these plant groups is represented by three items; the whole questionnaire contains 15 items. the design of the items follows the rose-questionnaire (=relevance of science education), an instrument used in one of the largest international comparative studies investigating students’ views on science and science education in 41 countries (schreiner and sjøberg 2004). the items are formulated as headlines describing the object of interest, e.g. “plants to improve my room” or “plants curing a sore throat”. similar to rose the fein-questionnaire uses a four-stage likert-scale (1-not interested, 2-rather not interested, 3-rather interested, and 4-very interested). additionally, the following demographic data were collected in the questionnaire: sex, age, grade, school. without any time pressure, filling in the questionnaires took approximately 10 to 15 minutes. 2.2 survey participants from march to may 2010, fifteen secondary schools voluntarily participated in the present study, each of these located in a different viennese district and two outside vienna, altogether providing a representative cross section of secondary schools in and around vienna. the questionnaires were filled in voluntarily during the students’ biology lessons. a total of 1,417 students answered the questionnaire; 118 of them were excluded due to missing, double, or obvious hoax answers (e.g. zigzag patterns), which resulted in pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 20 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 a final number of 1,299 participating students. these 1,299 questionnaires were filled in by 51% male and 49% female secondary school students. the sample was then divided into four subgroups: students between 10 and 12 years (age group 1), students between 13 and 14 years (age group 2), students between 15 and 16 years (age group 3) and students between 17 and 19 years (age group 4). exact numbers are given in table 2. table 2. descriptive data of the investigated sample (n = 1,299) age group group 1: 10–12 y group 2: 13–14 y group 3: 15–16 y group 4: 17–19 y total male students 245 197 159 62 663 female students 236 193 137 70 636 total 481 390 296 132 1299 percent of the sample 37 30 23 10 mean of age (y) 11.2 13.51 15.53 17.55 14.40 2.3 data treatment and statistics in order to draw conclusions on an individual level, it was necessary to reduce the complexity of the data per participant, which consisted of five means of interest (one for each plant subgroup, each ranging from 1 to 4 in ten possible steps, resulting from the likert-scale of the questionnaire).the method of complexity reduction was developed stepwise in order to reach a reduction level allowing meaningful conclusions from the data and therefore suitable as a basis for planning botany lessons. the challenge was to develop a procedure which takes into account the variation of the individual interest structure of each student but at the same time clusters the students to larger units, showing overlapping patterns of interest. so the aim of this procedure was to group the individuals in homogenous clusters of interest types – based on their individual interest structure. the process of complexity reduction and the development of this procedure are described in the following section. rank order pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 21 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 the first method of complexity reduction we applied was the calculation of the rank order of interest in the five groups of useful plants per participant. for this purpose, an interest rank order of the five subgroups (in order medicinal plants, stimulant herbal drugs, spice plants, edible plants and ornamental plants) for each participant was calculated, which resulted in a rank-order-code (e.g. 15342 means medicinal plants: first rank, stimulant herbal drugs: fifth rank, spice plants: third rank, edible plants: fourth rank and ornamental plants: second rank). in the end, the frequency of each rank-order-code in the sample was counted. this procedure still dealt with a number of possible combinations (55 = 3125) too large to give results which could be interpreted by identifying interest types (groups of students with similar interest structure) within the sample, because 394 of these possible combinations were actually realised in the population. besides, these rank orders showed that only 1 % (13 of 1299 individuals) of the whole sample had an interest rank order identical with the one calculated from the means of the population (12345, see table 1). ninety-nine percent of the target group showed a deviating interest ranking (393 different rank-order-codes) of the five subgroups of useful plants. remarkably, most of the rank-order-codes (211) were represented only by 2–5 individuals, 124 of the realised rank orders were represented by only one individual, which indicates a very high diversity of the population. therefore, it was inevitable to further reduce the complexity of the data. in a next step, categories were generated from the ten possible interest values for each subgroup resulting in three interest levels per useful plant subgroup: “high interest (values ranging from 3 to 4) – level 3”, “medium interest (values between 2 and 3) – level 2” and “low interest (values ranging from 1 to 2) – level 1” (exact values are given in table 3). transforming the data in this way, the number of possible combinations of the resulting rank-order-code now was 35 = 243, which led to 208 realised combinations (none of them representing more than 4.3 percent of the sample), which was still too much to allow conclusions which might be helpful in planning botany lessons. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 22 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 because of these reasons we decided to take into account only those subgroups of useful plants which best allow to differentiate between the interest structures of different individuals. chi-square-tests on the distributions of interests in the five subgroups were calculated in order to select only the subgroups of useful plants which clearly deviate from an equal distribution (see fig. 1 a-e and table 3 and 4) and therefore show a distinct pattern of interest. following this procedure, only interests in the subgroups of medicinal plants, stimulant herbal drugs and ornamental plants were used to characterize interest types in the sample. figure 1. frequencies (percent) of highly/medium/lowly interested students for all subscales of useful plants pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 23 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 table 3. numbers of individuals that have high/medium/low interest in the five subgroups of useful plants medicinal plants stimulant herbal drugs spice plants edible plants ornamental plants frequency percent frequency percent frequency percent frequency percent frequency percent high interest (3-4) 852 65.6 743 57.2 485 37.3 432 33.3 391 30.1 medium interest (2<interest<3) 284 21.9 253 19.5 408 31.4 365 28.1 305 23.5 low interest (1-2) 163 12.5 303 23.3 406 31.3 502 38.6 603 46.4 table 4. chi-square-values for the distribution of high, medium and low interest for each subgroup of useful plants medicinal plants stimulant herbal drugs spice plants edible plants ornamental plants chi-square 625.085 335.797 9.372 21.677 108.656 df 2 2 2 2 2 significance <0.001 <0.001 <0.01 <0.001 <0.001 now another rank-order-code was created that was based on the three students’ interest levels in medicinal plants, stimulant herbal drugs and ornamental plants (see table 5). this new code represents the characteristic interest type of each individual. for example, a code of “331” means this individual has high interest in medicinal plants and stimulant herbal drugs, but only low interest in ornamental plants. finally, using these reductions of complexity, the possible combinations of the rank-order-code were reduced to 33 = 27, which promised to be a number large enough to represent the variation within a target group, but small enough to allow meaningful conclusions for planning lessons. subsequently, we calculated the frequencies of all these rank-order-codes in the sample to identify frequent interest types. all statistical analyses were performed using spss™ for windows, version 16.0. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 24 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 3. results when analysing the data, the ten most frequent interest types (representing 74.7 percent of the sample) were chosen to give an impression of the sample (see table 5). as can easily be seen in table 5, there are approximately 12 percent of the sample (151 individuals) who have high interest in all subgroups. but on the other hand, most of the students (64 %) have low interest in at least one of the subgroups of useful plants (“331”, “313”, “131”, etc.). moreover, five of the interest types are not evenly distributed among the four age groups within the sample (see table 5 and 6), indicating different implications for teaching botany. all these results suggested going into more detailed data analysis. table 5. characteristics of the ten most frequent interest types in the whole sample. marked interest types (*) are not evenly distributed among the age groups within the sample interest in… medicinal plants stimulant herbal drugs ornamental plants frequency percent 331 high high low 18.9 * 333 high high high 11.6 * 332 high high medium 8.9 313 high low high 6.7 * 231 medium high low 6.2 323 high medium high 5.5 * 311 high low low 4.8 131 low high low 4.3 * 321 high medium low 4.2 232 medium high medium 3.8 table 6. chi-square-values for the distribution of the ten most frequent interest types among the four age groups within the whole sample 331 333 332 313 231 323 311 131 321 232 pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 25 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 chi-square 12.008 15.385 4.443 22.003 7.465 8.156 3.169 10.198 4.813 2.215 df 3 3 3 3 3 3 3 3 3 3 significance 0.007* 0.002* 0.217 <0.001* 0.058 0.043* 0.366 0.017* 0.186 0.529 note: marked interest types (*) are not evenly distributed in the sample in order to make the four age groups comparable in a clearly arranged way, we did not go on working with all of the ten interest types calculated from the whole sample but chose only those interest types which represent more than five percent of an age group (illustrated in fig. 2). comparing the frequency data of the rank-order-codes, one feature immediately catches attention: the most frequent type in all age groups is code “331” (students who have high interest in medicinal plants and stimulant herbal drugs, low interest in ornamental plants), covering between 15.3 % in age group 2 and 25.3 % of the sample in age group 3 (see fig. 2). another quite similar type, “332”, which represents students with high interest in medicinal plants and stimulant herbal drugs and medium interest in ornamental plants, appears in only two age groups within the first five ranks (age group 1, rank four, 10 % and age group 3, rank two, 8.4 %). it can also easily be seen that type “333” (that means high interest in all three subgroups) represents the second largest group of students in the two lower age groups (16 % in age group 1 and 10.5 % in age group 2), losing importance in the higher age groups (6.4 % on rank 4 in age group 3 but again 10.6 % on rank three in age group 4). pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 26 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 figure 2. frequencies of all interest-rank-order-codes representing more than 5 % of an age group another interest type is important in the two lower age groups: “313”, indicating students with high interest in medicinal plants and ornamental plants, but low interest in stimulant herbal drugs. this type can be found on rank three (covering still 10.4 %) in age group 1 and on rank five (representing 6.9 %) in age group 2. in higher age groups this interest pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 27 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 type loses importance and can be found only on rank 13 in both age groups 3 and 4, representing only 2.4 % of each sample, which means that only few students can be found who show high interest in medicinal plants and ornamental plants while taking low interest in stimulant herbal drugs. the relatively important role of this interest type in age group 1 is supported by the appearance of a similar interest type (“323”, 7.7 %) which can be found within the first five ranks only in this age group, also indicating quite a large subgroup of students not interested in stimulant herbal drugs. an interest type that can be found only in age group 2 within the first five ranks is “131” (students with low interest in medicinal plants and ornamental plants, but high interest only in stimulant herbal drugs, covering 6.9 % of the age group). with regard to the other age groups, this type is located only between ranks 8 and 16, covering just 1.5 to 4.1 % of the subsamples. in age groups 2 and 3 (on rank three each) a similar interest type (“231”) can be found which also represents students with high interest in stimulant herbal drugs and low interest in ornamental plants but at least medium interest in medicinal plants, covering approximately 8 % each. furthermore, a characteristic interest type for both higher age groups seems to be “311”, which represents students with specifically high interest only in medicinal plants, but low interest in stimulant herbal drugs and ornamental plants. this interest type can be found on rank 5 in age group 3 (5.4 %) and one rank higher, on rank four in age group 4 (7.6 %). it is remarkable that the codes of the five highest ranks in age group 4 start with a “3”, indicating students having high interest in medicinal plants. what also seems to be important is the distribution of type “111”, representing students with low interest in all plant groups. this interest type can be found on rank 14 in age group 1 (1.7 %), on rank 11 in age group 2 (3.1 %) and on rank 7 in age group 3 (4.1 %) but does not occur in age group 4 at all. to sum up, it can be said that in all age groups medicinal plants are highly interesting for most of the students. students of age group 1 are generally highly interested, which can pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 28 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 be seen from the fact that all five first ranks contain the interest level “high – 3”at least twice. both lower and higher age groups are characterized by specific interest types that are very prominent (“313” for lower age groups and “311” for higher age groups). in age groups 2 and 3 an opposing type can be found, ranking stimulant herbal drugs high and other plant groups low or medium (“131” and “231”). moreover, there seem to be coincident appearances of high interest with regard to different plant groups, as the majority of students with high interest in ornamental plants (80 %) is also highly interested in medicinal plants, and 69 % of the students with high interest in stimulant herbal drugs also take high interest in medicinal plants. on the other hand, in many cases some interests seem to exclude each other, as only 36 % of the students who are highly interested in medicinal plants are also highly interested in ornamental plants, but 42 % of them show only low interest in ornamental plants and 51 % of the students who have high interest in stimulant herbal drugs show low interest in ornamental plants. 4. discussion in order to efficiently counteract plant blindness (hershey 2002), educators should introduce botanical content using exemplary plants considered interesting by students (hidi and baird 1986). such interesting teaching objects may be found in the group of useful plants (krüger and burmester 2005), which students differentiate in five subgroups. some groups of useful plants (medicinal plants, stimulant herbal drugs and spice plants) are on average significantly more interesting for students than others (edible plants and ornamental plants) (pany 2014). however, means of a population do not allow conclusions on single individuals (valsiner 1986), which would be important for the planning of teaching lessons in class. therefore interest-rank-order-codes were calculated to characterize each individual, containing the personal interest level of medicinal plants, stimulant herbal drugs and ornamental plants. the frequency analysis of these codes shows that interest type “331” (high interest in medicinal plants and stimulant herbal pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 29 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 drugs but low interest in ornamental plants) is the most frequent in all age groups, while other interest types appear only in specific age groups or at least on different ranks in different age groups. because of the outstanding importance plants have in nature the recommendation frequently made in literature (e.g. baram-tsabari and yarden 2009) to teach general biological phenomena above all in the context of animals for young students and human biology for older ones should not be unrestrictedly followed. exactly these teaching traditions have been identified as one of the main reasons for plant blindness (hershey 1996; link-pérez, dollo, weber and schussler 2010). quite on the contrary, it should be explored which botanical context students also might consider interesting in order to avoid the impression that plants are boring und not important. as our results show there are indeed plant groups that are considered interesting by many students. medicinal plants are clearly number one with regard to interest across all age groups and so they rather contradict findings on a more general level (“plants”) that imply botanical content is not interesting for students (elster 2007; sjøberg and schreiner 2010). moreover, students’ interest in medicinal plants does not follow the general trend of decreasing interest in biological topics with increasing age, as often described in the educational science literature of the last decades, (e.g. kattmann 2000; löwe 1987; osborne and collins 2001). hence, medicinal plants can definitely be recommended as an appropriate gateway into botany or even into general biological content. this is all the more applicable as they may also have a relation to the context of human biology which is interesting above all for older students (baram-tsabari et al. 2010; osborne and collins 2000) of course there are some restrictions to our results as it is for example impossible to distinguish between the two scenarios that a topic is marked as “interesting” because it is a passion of the test person or only from a current mood. moreover, students vary in their expression of enthusiasm, so “very interesting” for one person may mean the same as pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 30 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 “interesting” for another (thorndike and hagen 1969). it has also been taken into account that a certain number of persons may have reasons to fake their answers due to social desirability (bühner 2011). in addition to these general constraints of questionnaire-based data, we cannot presume that medicinal plants, although often marked as “very interesting” in the questionnaire, are indeed as interesting for students as animalor human-related contents because there were no such items in the questionnaire. within this framework, the method of frequency analysis of interest types nevertheless makes it possible to study the characteristics of a population in detail without calculating means or using cluster algorithms blurring the variation of the sample. the interest types worked out in these analyses can therefore efficiently support teaching botany, assisting the development of a “shadow-curriculum” (baram-tsabari and yarden 2009) based on detailed scale, individual level analysis. as the results show, there is a high diversity within the target group which is worth being taken into account when preparing botany lessons. starting from analysis with a “mean student”, which seemed to allow clear conclusions for school, it could be shown that there is not “the one and only way” of choosing exemplary plants. most of the frequent interest types in all age groups show high interest in medicinal plants. therefore choosing exemplary medicinal plants for imparting botanical content (even the general structure of plants or flowers) should be standard for biology lessons in and outside school. although ornamental plants are seen as highly interesting by a subgroup of the sample, they should preferably be used only as additional examples. the majority of students with high interest in ornamental plants (80 %) are also highly interested in medicinal plants (e.g. interest type “313”), which cannot be said vice versa (e.g. interest type “311” or “331”) as only 36 % of the students who are highly interested in medicinal plants are also highly interested in ornamental plants. furthermore, ornamental plants have additional disadvantages (e.g. significant gender differences), pointed out in former studies (pany 2014). pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 31 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 moreover, stimulant herbal drugs no longer seem to be on one level with medicinal plants. quite on the contrary, especially in lower age groups, there is a large subgroup in the population who clearly is not at all interested in stimulant herbal drugs, although they are highly interested in both other plant groups (interest type “313”). choosing exemplary plants from only stimulant herbal drugs therefore could lead to a reduction of their interest in botanical content. obviously, stimulant herbal drugs seem to polarize: on the one hand they are indeed very interesting for many students; on the other hand there is a subgroup in the sample – especially in lower age groups – which definitely shows low interest. however, it should be marked that for interest types with low or medium interest in medicinal plants and ornamental plants stimulant herbal drugs are often the most interesting plant group (e. g. “131”, “231”, “121”, etc.). these interest types appear especially in age groups 2 and 3, where students’ interests are generally on a lower level than in age groups 1 and 4. stimulant herbal drugs can therefore be used to trigger interest within parts of the target group who cannot be motivated otherwise. these results should be used as a basis for further research exploring the role of stimulant herbal drugs with regard to different age groups in order to clear up their special role in the subgroups of useful plants to reveal their potential usability as keys to botanical content. this study intensifies the knowledge of the structure of interest in useful plants. it could be shown that allegedly clear recommendations for teaching botany derived from a “mean student” should be treated with caution. although medicinal plants are still the leading group for many students, our analysis on an individual level showed that there are groups of students who have significantly high interest for other subgroups of useful plants and hence should be captured by presenting additional plants from other subgroups. the selection of examples should only be reduced to the group of medicinal plants when there are logistic constraints, if one has to choose only one or two exemplary species for reasons of availability or price. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 32 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 in order to sustainably implement botanical content in education, plants like sage (salvia officinalis), hawthorn (crataegus spp.) or marigold (calendula officinalis) can be preferentially recommended as impressive examples. offering additional plant species belonging to stimulant herbal drugs (such as tobacco nicotiana tabacum) and ornamental plants (such as primroses primula spp., especially for lower age groups) may help considering pre-existing interests of most students and build a gateway into botany. what still remains unexplored and a field open for prospective studies are, for example, experimental designs which explore the reactions of different interest types on various educational settings in order to test and secure the hypothesis that interesting study objects also raise higher interest in botanical content. in summary, dealing only with exemplary plants that correspond to students’ interests can be an opportunity to prevent them from perceiving plants only as a lifeless scenery for animals and to facilitate students to develop a more comprehensive view of nature, without disregarding a vast majority of the organisms forming the basis of life on earth. 5. references baram-tsabari, a. and yarden, a. (2007). interest in biology: a developmental shift characterized using self-generated questions. the american biology teacher, 69(9), 532–540. baram-tsabari, a. and yarden, a. (2009). identifying meta-clusters of students’ interest in science and their change with age. journal of research in science teaching, 46(9). 999–1022. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 33 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 baram-tsabari, a., sethi, r. j., bry, l. and yarden, a. (2010). identifying students’ interests in biology using a decade of self-generated questions. eurasia journal of mathematics, science and technology education, 6(1), 63–75. bozniak, e. c. (1994). challenges facing plant biology teaching programs. plant science bulletin, 40, 42–26. bühner, m. (2011). einführung in die test-und fragebogenkonstruktion. münchen: pearson studium. deci, e. l. (1992). the relation of interest to the motivation of behavior: a selfdetermination theory perspective. in: renninger, k. a., hidi, s. and krapp, a. (eds.). the role of interest in learning and development (pp. 43–70). new york: psychology press. deci, e. l. and ryan, r. m. (1993). die selbstbestimmungstheorie der motivation und ihre bedeutung für die pädagogik. zeitschrift für pädagogik, 39(2), 223–238. dillon, j., rickinson, m., teamey, k., morris, m., choi, m. y., sanders, d. and benefield, p. (2006). the value of outdoor learning: evidence from research in the uk and elsewhere. school science review, 87(320), 107. elster, d. (2007). student interests–the german and austrian rose survey. journal of biological education, 42(1), 5–10. flannery, m. c. (2002). do plants have to be intelligent? the american biology teacher, 64(8), 628–633. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 34 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 greenfield, s. s. (1955). the challenge to botanists. challenge, 1(1). resource document. https://secure.botany.org/plantsciencebulletin/psb-1955-01-1.php hammann, m. (2011). wie groß ist das interesse von schülern an heilpflanzen? zeitschrift für phytotherapie, 32(01), 15–19. hershey, d. r. (1992). making plant biology curricula relevant. bioscience, 42(3), 188– 191. hershey, d. r. (1996). a historical perspective on problems in botany teaching. the american biology teacher, 58(6), 340–347. hershey, d. r. (2002). plant blindness: “we have met the enemy and he is us.” plant science bulletin, 48(3), 78–84. hershey, d. r. (2005). plant content in the national science education standards. resource document. http://www.actionbioscience.org/education/hershey2.html hidi, s. and baird, w. (1986). interestingness-a neglected variable in discourse processing. cognitive science, 10(2), 179–194. hidi, s. and baird, w. (1988). strategies for increasing text-based interest and students’ recall of expository texts. reading research quarterly, 465–483. hidi, s. (1990). interest and its contribution as a mental resource for learning. review of educational research, 60(4), 549–571. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 35 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 kattmann, u. (2000). lernmotivation und interesse im biologieunterricht. in: bayrhuber, h. and unterbruner, u. (eds.). lehren und lernen im biologieunterricht (pp.13–31). innsbruck, wien, münchen: studien-verlag. krapp, a. (1989). interest, learning and academic achievement. in p. nenninger (chair), task motivation by interest. paper presented at third european conference of learning and instruction (earli), madrid, spain. krapp, a. (1999). interest, motivation and learning: an educational-psychological perspective. european journal of psychology of education, 14(1), 23–40. krüger, d. and burmester, a. (2005). wie schüler pflanzen ordnen. zeitschrift für didaktik der naturwissenschaften, jg. 11, 85–102. lindemann-matthies, p. (2005). “loveable” mammals and “lifeless” plants: how children’s interest in common local organisms can be enhanced through observation of nature. international journal of science education, 27(6), 655–677. lindemann-matthies, p. (2005). “loveable” mammals and “lifeless” plants: how children’s interest in common local organisms can be enhanced through observation of nature. international journal of science education, 27(6), 655–677. link‐pérez, m. a., dollo, v. h., weber, k. m. and schussler, e. e. (2010). what’s in a name: differential labelling of plant and animal photographs in two nationally syndicated elementary science textbook series. international journal of science education, 32(9), 1227–1242. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 36 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 löwe, b. (1987). interessenverfall im biologieunterricht. unterricht biologie, 124, 62– 65. mayer, j. and horn, f. (1993). formenkenntnis–wozu. unterricht biologie, 189(17), 4– 13. osborne, j. and collins, s. (2000). pupils’ and parents’ views of the school science curriculum. kings college london. resource document. http://www.kcl.ac.uk/sspp/departments/education/web-files2/ppt.pdf osborne, j. and collins, s. (2001). pupils’ views of the role and value of the science curriculum: a focus-group study. international journal of science education, 23(5), 441– 467. pany, p. (2014). students’ interest in useful plants: a potential key to counteract plant blindness. plant science bulletin, 60(1), 18–27. sales-reichartzeder, j., pany, p. and kiehn, m. (2011). opening the window on “plantblindness.” botanic gardens conservation international education review, 8(2), 23–26. schreiner, c. and sjøberg, s. (2004). sowing the seeds of rose. background, rationale, questionnaire development and data collection for rose (the relevance of science education)-a comparative study of students’ views of science and science education (acta didactica 4/2004). oslo: department of teacher education and school development, university of oslo. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 37 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 schussler, e. e. and olzak, l. a. (2008). it’s not easy being green: student recall of plant and animal images. journal of biological education, 42(3), 112–119. schussler, e. e. and winslow, j. (2007). drawing on students’ knowledge. science and children, 44(5), 40–44. schussler, e. e., link-pérez, m. a., weber, k. m. and dollo, v. h. (2010). exploring plant and animal content in elementary science textbooks. journal of biological education, 44(3), 123–128. sjøberg, s. and schreiner, c. (2010). the rose project: an overview and key findings. oslo: university of oslo. resource document. http://www.cemf.ca/%5c/pdfs/sjobergschreineroverview2010.pdf strgar, j. (2007). increasing the interest of students in plants. journal of biological education, 42(1), 19–23. tamir, p. and gardner, p. (1989). the structure of interest in high school biology. research in science & technological education, 7(2), 113–140. thorndike, r. l. and hagen, e. (1969). measurement and evaluation in psychology and education. new york. urhahne, d., jeschke, j., krombaß, a. and harms, u. (2004). die validierung von fragebogenerhebungen zum interesse an tieren und pflanzen durch computergestützte messdaten. zeitschrift für pädagogische psychologie, 18(3), 213–219. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 38 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2309 social and technological sciences eissn: 2341-2593 valsiner, j. (1986). between groups and individuals. in: valsiner, j. (ed.). the individual subject and scientific psychology (pp. 113–151). new york: plenum press. wandersee, j. h. and schussler, e. e. (2001). toward a theory of plant blindness. plant science bulletin, 47(1), 2–9. wandersee, james h. and schussler, e. e. (1999). preventing plant blindness. the american biology teacher, 61(2), 82–86. pany and heidinger (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 15-39 | 39 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 calculator: a hardware design, math and software programming project base learning f. criado, l. costero, p. cabeza, m. sánchez-élez* dpto. de arquitectura de computadores y automática. universidad complutense de madrid, avda. complutense s/n, 28040 madrid, spain. * corresponding author: email: marcos@ucm.es; phone: + 34 913947604 received: 2013-12-25; accepted: 2014-08-12 abstract this paper presents the implementation by the students of a complex calculator in hardware. this project meets hardware design goals, and also highly motivates them to use competences learned in others subjects. the learning process, associated to system design, is hard enough because the students have to deal with parallel execution, signal delay, synchronization … then, to strengthen the knowledge of hardware design a methodology as project based learning is proposed. moreover, it is also used to reinforce cross-subjects like math and software programming. this methodology creates a course dynamics that is closer to a professional environment where they will work with software and mathematics to resolve the hardware design problems. the students design from zero the functionality of the calculator. they are who make the decisions about the math operations that it is able to resolve it, and also the operands format or how to introduce a complex equation into the calculator. this will increase the student intrinsic motivation. in addition, since the choices may have consequences on the reliability of the calculator, students are encouraged to program in software the decisions about how implement the selected mathematical algorithm. although math and hardware design are two tough subjects for students, the perception that they get at the end of the course is quite positive. keywords active learning methods; project based learning; logic design; computer aided design criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 110 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 1. introduction students enrolled in any of the different computer degrees they will study computer mechanics (computation, communication, coordination, automation, and recollection), design principles (simplicity, performance, reliability, resolvability, and security) and practices (programming, engineering systems, modeling and validation, innovating, and applying) (denning 2003). these students will develop “computational thinking”, which shares elements with various other types of thinking such as algorithmic thinking, engineering thinking, and mathematical thinking (perkovic et al. 2010), in other words, they should develop the intellectual skills necessary to apply computational techniques or computer applications to problems and projects in any discipline. however, in our experience, to make it possible, in the subjects within the computer science degree, it is necessary to encourage interaction between computer science concepts and engineering and math concepts. we think this is favorable when the courses have cross-subject projects, within a project base learning (pbl) methodology, which is the main objective of this paper. we have chosen system design course as the main course to develop this cross-subject project. the students will design computing systems and computing components of products, develop and test their prototypes, and implement them to market. therefore, the goal of the methodology presented in this paper is achieve that the students correctly design a complex algorithm into hardware with the help of any of the existing computeraided design tools, without forgetting the interrelationship of this course with others subjects that are given in the degree. hardware design is very important for undergraduate curricula in computer science, (harrison et al., 1998) and (mcgettrick et al., 2003), and the students need to manage concepts from many different fields. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 111 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 moreover, eda industry has evolved towards the use of cad tools which allow a rapid prototyping and manufacturing of very complex designs. it requires the management of a wide knowledge in many new areas such as, hardware design languages (hdls), fpga-based cad tools, use of ip cores, soft cores, advanced synchronization techniques, design of virtual systems, … (sklyarov 2004). several authors have presented methodologies to deal with the challenges of teaching hardware design today. some of them still hold a traditional point of view of hardware design, applied to a new hardware target (fpga), such as (velez et al 2007). in addition, most digital design texts dealing with hdls (ashenden 1996), (roth 2007) present a classical view of digital design concepts merely translated into hdl. they commonly propose the structural design of a simple microcontroller as final project, which adds no significant understanding to the former schematic approach. although in system design courses instructors usually make use of practical examples in the form of guided lab exercises, the consequence, from the student point of view, is that the proposed guided designs and their results seem to be a black box with data to collect. this approach is far from real design problems. instead, a pbl methodology is more suitable for these courses as it creates a course dynamics that is closer to a professional environment (yadav et al. 2011). in the field of apply pbl on system design courses, the proposal of (araujo et al 2008) is interesting, although these authors make separate simulation and synthesis projects. these are complementary aspects of a whole design process and should be analyzed together from the beginning of the course, since a correct simulation does not guarantee a correct implementation on the target hardware, fpga. the work presented in (cristiangyozo haba et al 2011) uses pbl in a very attractive way, although they use vip technology, which requires sophisticated laboratory equipment. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 112 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 although the purpose of this paper is not directly teach math, we should not forget that in the different computer degrees subjects related with math are the toughest in the opinion of students, whose main complaint was the lack of any attempt to demonstrate the applicability of those areas of mathematics which they were being taught. this may be due to the methodology usually used; it involves mainly formal lectures, which alone, are not particularly effective in teaching mathematics. mathematical courses are built in such a way that if a student misses a key concept at the beginning of the lecture, the rest of the lecture can be lost for him or her (sazhin 1998). among the proposed solutions, authors in (beaudin 2002) demonstrate that teaching engineering mathematics with technology constitutes a good opportunity to teach classical subjects with a new taste. it allows teachers to adapt their teaching methods to the new technological reality. most important, technology helps the teacher to present live examples of what mathematics are, how beautiful they are. students will much more appreciate theorems and general results if they can visualize concrete examples. the difficulty to teach mathematics has carried surveys during the last few years in favor of less mathematically rigorous cs curriculum (sazhin 1998). they do not deny the importance of math, but emphasizes that if we were to include all the math that is useful for a cs major, it would result in a specialized math major, leaving us short on computer science itself. then math is an important subject that should be interrelated with programming and hardware design concepts. however, it is most usual for programming and not at all for hardware design. last but not least, designing and implementing their own calculator machine as the course project is a key aspect of the methodology, as the students are able to interrelate different pieces, which have been studied in the computer science or computer engineering degree, improving the perception that different courses would no longer be compartmentalized. moreover, the creation of their own calculator machine is a challenge criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 113 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 related to a reality that is well known for them compared to other possible hardware projects. most students find it a little hard in the beginning because they have to handle real problems present in hardware design. by the time they are designing their own project they become very involved in the course and work really hard (higley et al. 2001). the rest of this paper is organized as follows: section 2 describes the methodology implemented. section 3 presents details of the calculator case study, section 4 describes in detail the mathematical algorithms used in the system, and section 5 presents the conclusions of this work. 2. proposed methodology: integration of different teaching methods project based learning in engineering programs has been reported by several authors (glass 2000) and (gunstra 2001). pbl is designed to engage students in research and implementation of authentic problems and it is easily adapted to add cross-cutting concerns from other subjects to help solve the problem. project based learning emphasizes learning activities that are long-term, interdisciplinary and student-centered. unlike traditional, teacher-led classroom activities, students often must organize their own work and manage their own time in a project-based class. project-based instruction differs from traditional inquiry by its emphasis on students' collaborative artefact construction to represent what is being learned. projects can increase student interest because the involve students in solving authentic problems, in working with others, and in building real solutions, this makes that the pbl methodology is the perfect candidate to face problems (courses) that are hard for the students. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 114 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 the students that going to work following a pbl methodology have passed a basic course of digital design. therefore, they are already familiar with: laws of boolean algebra; binary number representation; basic combinational logic design, basic gates, multi-level digital design; combinational building blocks; basic sequential building block and finite state machine (fsm) design. a basic knowledge of vhdl is also desirable, so a 4 6 hours vhdl tutorial (sanchez-elez 2011) is given to the students, regardless of whether or not they have studied it in previous courses. the key focus of the methodology is on creating the expectation that at the end of the course, once they have mastered certain design challenges, the student are ready to implement a complex calculator machine. the other important aspect is to encourage students to use concepts learned in other subjects, like calculus, algebra or programming, to make design decisions that involve the realization of the most accurate calculations possible. the project is developed using vhdl as design language, spartan 3 (spartan 3) as prototyping platform and xilinx ise (ise) as cad tool. the cornerstones of this methodology are the following: • at the beginning of the course the instructors present an overall guideline of the theoretical concepts required for the implementation of different hardware modules, all of which may be combined to create any calculator machine. • the different hardware modules that the students develop have been specifically designed to present increasing challenges in order to capacitate them to finally implement their own calculator. these modules are: simple and complex asm; use of the fpga internal memories; and input interfaces (ps2 keyboard). • there is one week exclusively dedicated to explain different hw algorithms for multiplication and division (parhami 2000). criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 115 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 1 figure 1. calculator machine block diagram • midterm group discussion: students are requested to suggest calculator functionalities that may be implemented with the hardware modules developed along the course. this should be presented as a realistic project, that is, they have to evaluate the difficulty of the math functions implementation by identifying the necessary hardware elements, the amount of reusable code and deadlines. in some cases the teacher addresses the students to either simplify or increase the complexity of their project, in order to meet the educative goals and deadlines of the course. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 116 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 • along the course, think critically over the numerical approximations done. the students should present a revision of the accuracy of the math functions implemented. they may study the differences using matlab or programming their hardware approximation in other high level languages. they have to discuss the different notation algorithm and choose one. they could use recursive software algorithm for resolve math problems that have to translate to hardware. they should use backus normal form … 3. case study since the students face the project working in groups, it is useful done a division of different parts that form the calculator as well as the implementation of a communication protocol between the different modules. thus students within each group can distributed in a more simple way their workload, because implement a complete calculator of this kind by a single student could be a sufficiently complicated work that only would help to discourage them. the design of the calculator is formed by four modules as is shown in figure 1: • input module: data and operations are introduced into the calculator following postfix notation (reverse polish), it is a mathematical notation in which every operator follows all of its operands and it is usually parenthesis free. additionally, users of reverse polish calculators made fewer mistakes than for other types of calculator (agate et al. 1998). the input module is divided into three sub-modules (interface, bcd-bin and sya). the interface directly translates the key pressed into a number or an operator. the communication protocol between the modules is based on an intermediate buffer that stores the output of the previous module criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 117 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 and some control signals to implement a basic handshake. in order to process the input, three different errors are taking into account: overflow (the integer is very large), parenthesis bad balanced, and syntactically incorrect numbers (i.e. more than one decimal point). if an error occurs a signal is sent to the control unit. • output module: is very simple, the output has two modes, display input and display output. if a key is pressed it is passed to the first, if it is doing a calculation it is changed to the second. • control unit module: the control is fully dedicated to the evaluation of expressions in post-fix notation. communication with the input module is through the intermediate buffer, with signals of coordination between both (handshake algorithm). the communication with the output module is very simple: signals are sent with the result or the error to be displayed. the control unit sends the operators to the processing unit and waits for the result or the error signal. the control unit, manages input module errors, and also handles buffer overflow (math expression too long) or stack overflow (too many nested operators). moreover the processing unit send an error signal to the control unit when there is a failure in the execution of any math algorithm (i.e. try to calculate -1 square root). • processing unit module: is responsible for collecting the operands and the corresponding math function from the control unit. and lead the inputs to the module in charge of the operation. in order to make this task it uses a state machine. each module is responsible for managing their own mistakes and only processing unit redirects the error signal to the control unit. the processing unit is design allows to add new operations quickly and almost effortlessly, since each criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 118 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 math operation is implemented in a differentiated module inside the processing unit. 4. algorithm study the most complicated part of the calculator implementation is related with the number representation and the math algorithms. 4.1. number representation the first problem to be solved by students involves the representation of numbers, since the system is a calculator, the numbers are introduced in bcd code and have to be translated to two complementary code. however, students have to take into account that the numbers have an integer part and a decimal part, which are translated separately by different algorithms. the integer part is translated following the algorithm: 10n + x = (n<<1) + (n<<3) + x. this implies the use of two adders and two shifters. when the students attempt to convert the decimal part different algorithms arose, they may use the standard algorithm in which the decimal part is multiplied by two … however, the students who have participated in this project decided use the following algorithm: 1) the decimal part is converted to binary as if it were an integer. 2) the new integer is multiplied by two, if the result is lower than one then a zero is added to the representation and the algorithm repeats the step two. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 119 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 3) the results is larger than one then a one if added to the representation and a new value is used, the new value is the result of subtracting the result obtained in the step two minus one. 4) if the process has been repeated fewer times than the number of decimal digits the algorithm repeats the step two. the algorithm works properly when one is the binary representation of 10n, where n is the number of decimal places in bcd. moreover, a counter is required to implement step four. 4.2 input algorithm the user enters numbers and operators, to transform these inputs into a recognizable expression the system uses the dijkstra’s shunting yard algorithm (sya), it transforms infix notation into post-fix notation, which is the notation used by the system as explained before. the algorithm works as follows: • if the input is a number it goes to the following module • if the input is ( , it is pushed in the operators’ stack • if the input is ) , then the operators are popped until find a (. • if it is an operator, depending on the operator precedence it is pushed or it goes to the following module • it is necessary an end of expression symbol (eoe), in this case all the operators are popped. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 120 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 4.3 arithmetic algorithms for the most basic calculator operations, namely, addition, subtraction, multiplication, two's complement, we have implemented them by using the arithmetic operations of ieee libraries, properly encapsulating to manage the arithmetic errors. the problems arise when the students try to implement most complex arithmetic functions. in the case of the division there are several sequential algorithms as can be shown in (parhami 2000), the students can use any of them. but they need to take into account the number of decimal places of the representation to truncate the division algorithm when the decimal places are completed. moreover, overflow and division by zero have to be referred. it is very important, when implementing this module in the fpga, studying the delay and the hardware resource use. for the trigonometric operations the students can use the cordic algorithm (parhami 2000), but the instructor encourage them to test their students-made option. is this part of the development of the calculator one in which most clearly may need use concepts learnt in other courses. for example students could implement in software the trigonometric approximation and compare with the math results, and/or students could study the polynomial representation of the trigonometric functions and choose an approximation to implement in the fpga. the students that have participated in this project decided use a criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 121 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 polynomial approximation with sexagesimal degrees instead of radians. the applied formula is: 𝑠𝑠𝑠𝑠𝑠𝑠(𝑥𝑥) = �2 − � 𝑥𝑥 90 �� · � 𝑥𝑥 90 �. the error of this approximation is shown in figure 2.a, the maximum error value is 0.05. at this point the instructor can encourage the students to review their calculus concepts to reduce the error. the error is estimated as: ϵ=α•β(1-|β|) wherein β stands for the result of the previous approximation and α is a parameter. this parameter is with which students can play (program sin results for different α values). with α value of 0.25, which is easily implemented with a shifter, the error is shown in figure 2.b. finally, since the algorithm always divides by ninety, the division is transformed to a multiplication after calculating its binary value. (a) (b) figure 2. absolute error of the different approaches the students may develop other mathematical functions. in the book (parhami 2000), there are algorithms enough to implement math function in hardware. the student should check if their numeric representation suits with the algorithm described and check, as was criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 122 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 done for sin function, if a change in the original algorithm would be required. most of the students also implement natural logarithm, and square root. 5. analysis and conclusions the results presented in this section are related to relate to the students’ feelings about course development. the course is held at universidad complutense de madrid (spartan 3) with an average of 25 students per class. a greater number of students complicate the correct development of the course. this course in a university program meant to evaluate teaching performance. within this program, the results of teaching activities are translated into terms of progress in student learning and in the assessment expressed as perceptions or opinions by students, graduates, academic leaders and the academic staff. this evaluation is coordinated within a program of the spanish agency in quality assurance in higher education, aneca. this agency is a full member of european association for quality assurance in higher education (enqa) and a full member of the international network for quality assurance agencies in higher education (inqaahe). the results obtained for this course in this program are shown in table i (questions are made on a rating scale of 5). table 1. external evaluation of the teaching activities mean lessons are well organized and prepared 4.24 the concepts are explained clearly and understandably 4.00 arouses our interest in the course 4.35 used properly teaching resources 4.00 degree of fulfillment of the course syllabus 4.24 course workload 3.53 degree that have achieved the objectives of the course 4.13 criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 123 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 additionally, we have carried out a specific test about students’ perceptions and opinions. the results are shown in table ii. this test shows that students are satisfied with the course and feel confident on their skills for designing complex hardware, although for some of them the effort required was significant. it was quite surprising to us that although students consider mostly to study the subject following this methodology requires more effort, 100% of the students would return to attend it following this methodology. table 2. internal evaluation of students' perceptions this method implies that the course work is much higher than a traditional method 56% higher than a traditional method 38% similar to a traditional method 6% lower than a traditional method 0% once all the implications of the methodology are known, would you enroll in this course again? yes 100% no 0% which of these options reflects your perception of the course? i was surprised positively 25 % i liked it 56 % i found it tedious 19 % i feel disappointed 0 % my perception about the design skills acquired is that i feel capable of designing any other complex system 31 % i feel capable of designing systems of a similar nature 50 % i feel capable of designing systems of a lower complexity 19 % i still do not feel capable of facing the design of a complex system without assistance 0% i feel that choosing this course for my curriculum is not significant 0 % criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 124 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 was interesting but i do not think will help me find a job 38 % will provide more professional opportunities for me 38 % will help me find the type of job i am interested in 25 % 5. references agate s.j., drury c.g. (1980). electronic calculators: which notation is the better? applied ergonomics, volume 11, issue 1, march 1980, pages 2-6, issn 0003-6870 araujo a. j. and alves j. c. (2008), a project based methodology to teach a course on advanced digital systems design. wseas transactions on advances in engineering education, vol.5, no.6, pp. 437-446. ashenden p.j (1996). the designer’s guide to vhdl, morgan-kaufmann, 1996 beaudin, m. (2002). “teaching mathematics to engineering students with hand-held technology”. in 2nd international conference on the teaching of mathematics. cristian-gyozo haba, a., liviu breniuc, a. (2011). teaching fpga embedded system design by course correlation and with the help of vip systems. proceedings of the 8th fpgaworld conference 2011. denning, p. j. (2003). great principles of computing. communications of the acm, 46(11), 15-20. gajski d. (1996) principles of digital design, prentice-hall glass r. l. (2000) a new answer to ‘how important is mathematics to the software criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 125 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 practitioner’?. ieee software, november/december 2000, pp.135-136. gunstra n. (2001). universities aren’t serving the it workforce. potomac tech journal, july 9, 2001, available at http://www.potomactechjournal.com harrison c.g., jones p.l. (1998). experiences with fpga teaching. the teaching of digital systems (digest no. 1998/409), iee colloquium on , vol., no., pp.3/1-3/4, 18 may 1998, doi: 10.1049/ic:19980570 higley, k.a., marianno, c.m. (2001). making engineering education fun. journal of engineering education, vol 90, no. 1, pp105-107, january 2001, isbn-13: 9780135103821 ise design suite software manuals and help pdf collection www.xilinx.com/support/documentation/sw_manuals/xilinx12_1/manuals.pdf keleman c. and tucker a.b. () iticse audience survey, available at http://www.cs.geneseo.edu/~baldwin/maththinking/iticse-survey.html lethbridge t. (2007) software engineering education relevance survey, available at http://www.site.uottawa.ca/~tcl/edrel/ mcgettrick a., theys m.d, soldan d.l, srimani p.k. (2003) computer engineering curriculum in the new millennium, ieee transactions on education, vol. 46, no. 4, november 2003 parhami, b. (2000). computer arithmetic: algoritms and hardware design. oxford university press. 2000 perkovic l., settle a., hwang s. and jones (2010), j. a framework for computational criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 126 http://www.potomactechjournal.com/ http://www.cs.geneseo.edu/%7ebaldwin/maththinking/iticse-survey.html http://www.site.uottawa.ca/%7etcl/edrel/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2275 social and technological sciences eissn: 2341-2593 thinking across the curriculum, proceedings of the 2010 conference on innovation and technology in computer science education, 123-127. roth c.h.jr. (2007), digital systems design using vhdl, 2nd edition, university of texas, austin, isbn-10: 0534384625 isbn-13: 9780534384623 sanchez-elez m. (2011). introduccion a la programación en vhdl. available at http://eprints.ucm.es/26200/ sazhin, s. s. (1998). teaching mathematics to engineering students. international journal of engineering education, 14(2), 145-152. sklyarov v., skliarova i. (2004). teaching reconfigurable systems: methods, tools, tutorials, and projects. education, ieee transactions on, vol.48, no.2, pp. 290300, may 2005 doi: 10.1109/te.2004.842909 spartan-3 generation fpga user guide, (2011). available at www.xilinx.com/support/documentation/user_guides/ug331.pdf velez i., sevillano j.f. (2007). a course to train digital hardware designers for industry. education, ieee transactions on , vol.50, no.3, pp.236-243, aug. 2007, doi: 10.1109/te.2007.900027 yadav a., subedi d., lundeberg m. a., bunting c. f., (2011) problem based learning: influence on students' learning in an electrical engineering course, journal of engineering education, vol. 100,no.2, pp. 253-280. criado et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 110-127 | 127 http://eprints.ucm.es/26200/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 71 using active learning methodologies in physical chemistry in clil contexts recatalá, david department of physical and analytical chemistry, jaume i university avda. de vicent sos baynat, s/n. 12071 castelló (spain) e-mail: drecatal@uji.es; tel. (+34) 964 728094 received: 2015-04-03; accepted: 2015-8-8 abstract one of the main objectives of the european higher education area (ehea) is to promote a change toward a student-centred education model. this fact has led to the implementation of novel methodologies based on active learning, aimed at engaging students’ interest. this implementation has been usually accompanied by significant changes in both the teaching and learning processes in european universities. furthermore, teaching a subject through the medium of a foreign language has also been gaining attention over the past few years. more specifically, this approach commonly known as content and language integrated learning (clil) has been employed for the simultaneous learning of content and english in a number of european countries. in this contribution we report on the active learning methods implemented in a physical chemistry course, as well as the efforts devoted to content and english language integration in this subject. this research analyses a series of factors that can contribute to the global learning and teaching experience when both active learning and clil are implemented in the physical chemistry classroom. some examples of them include changes in attitudes towards the subject, engagement and motivation during the course, perception of english learning, and in general, students’ satisfaction with the learning process. keywords active learning methods; teaching methodologies; european higher education area; content and language integrated learning multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 72 1. introduction the european higher education area (ehea), also known as bologna process, emerged in 2010 as a new educational area, conceived to promote a shift from a teacher-centred to a student-focused educational model (ehea, 2010). the creation of the ehea has been accompanied by the introduction of the european credit transfer system (ects), which takes into consideration the total time that students employ in order to reach the learning objectives (european communities, 2009). in contrast, before the implementation of the ehea, only the time spent in lectures was taken into account to define the duration of a higher education course. the implementation of the bologna process has led to dramatic changes in teaching methodologies, with the aim of promoting students’ participation in the classroom, as well as engaging their interest towards learning. in this context, the term “active learning” has gained a growing interest over the past few years. this concept involves the combination of teacher lectures with other student-centred activities, which engage students in the learning process during the time they spend in the classroom (prince, 2004). collaborative learning is another form of active learning in which students work together in small groups to achieve a specific aim. this methodology lies emphasis on the interaction between students, in contrast to the traditional individualistic approach. in this context, google docs has been recently explored as a tool for promoting collaborative learning in laboratory courses (spaeth et al., 2012). cooperative learning is a variant of collaborative learning, in which each member of the group is assigned a particular task, although all of them pursue the same goal. this methodology was satisfactorily applied in an undergraduate laboratory course, in which each member of the group was asked to prepare a particular part of the experiment (smith et al., 1991). both collaborative and cooperative learning methods have been successfully employed to increase the overall marks of an organic chemistry course by over 20 – 30 %, as compared to the traditional course based exclusively on teacher lectures (paulson, 1999). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 73 problem-based learning (pbl) is another active learning methodology in which students are provided with interesting and challenging problems, and they are therefore actively involved in the search for the solution. the effectiveness of this approach in a physical chemistry laboratory course was investigated (gürses et al., 2007), and the results indicate that the pbl approach highly promotes students’ problem-solving skills. content and language integrated learning (clil) represents “a dual-focused educational approach in which an additional language is used for the learning and teaching of both content and language” (marsh et al., 2011). the implementation of clil throughout europe varies greatly from one country to another. in the particular case of spain, the interest in this approach has grown significantly over the past few years, and its implementation in bilingual communities, such as valencia, fosters multilingualism (frigols, 2007). different types of clil programmes can be found, ranging from full immersion to partial immersion (eurydice, 2006). research shows that clil presents a number of benefits for students, and that its success strongly depends on the context in which this approach is introduced (cambridge esol, 2010). teaching a subject through the medium of the english language has attracted a great deal of interest in spain due to its multiple benefits, since clil is supposed to improve students’ fluency in english, and can hence contribute to better prepare them for their future career. in this contribution, we report on the results obtained upon application of active learning methodologies in a physical chemistry course within the degree in chemistry at jaume i university (castelló, spain). the main topic of this course is spectroscopy, and the contents are those covered by a standard textbook, such as physical chemistry by r. a. alberty and r. j. silbey. the students had already some notions of physical chemistry, as they were in the third course of the degree in chemistry. in addition, we decided to integrate the english language into this subject by following a partial integration approach in a group of bilingual speakers (spanish / catalan). this choice was motivated by the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 74 multiple benefits of clil. the students’ satisfaction with the changes introduced in the subject were analysed by means of a likert-type questionnaire. 2. methodology active learning methodologies and content and language integrated learning have been implemented in 29 bilingual chemistry students (spanish / catalan), which were divided into two groups of 14 and 15 students, respectively. the methods employed in each case are detailed herein. 2.1. active learning methodologies a series of active learning methodologies have been implemented in a physical chemistry course over the past few years. as already stated in the introduction, this course, in which spectroscopy is the main topic, corresponds to the third year (second semester) of the degree in chemistry at jaume i university (castelló, spain), and corresponds to 6 ects credits. the students taking this course are supposed to have passed three previous physical chemistry subjects, which cover chemical thermodynamics (first year), kinetics (second year), and quantum chemistry (third year, first semester). nevertheless, having passed these subjects is not a prerequisite to enrol the spectroscopy course. in this spectroscopy course, the lectures sessions are combined with computer-based workshops in which the students have to solve problems. these sessions involve spreadsheets calculations, graphical analysis of data, as well as the use of web-based simulation packages to facilitate a better understanding of the theoretical concepts. the students attend a three-hour workshop approximately on alternate weeks. prior to the workshop, they are encouraged to do a series of activities at home, which serve as a reminder of the theoretical concepts that they are going to use. students have to hand in these activities to the teacher just before the beginning of the workshop in order to prove multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 75 that they have understood the concepts. in the workshop sessions, each student uses a computer provided with a spreadsheet program. each student is provided with a set of problems that has to solve individually by using a computer. at the beginning of the workshop, teachers give a brief explanation of what has to be done during the session. this explanation is followed by the individual work of students, who are encouraged to finish all the activities during the session (approximately 3 hours). having finished all the activities, the students have to answer a quiz through the moodle platform that we usually employ for uploading content related to the subject. the marks obtained by the students in these workshops represent the 20 % of the total marks of the subject. students are required to complete the quiz and to hand in a definitive version of the spreadsheet in one week. 2.2. content and language integrated learning (clil) with the aim of teaching physical chemistry through the medium of a foreign language, we decided to integrate the english language in a partial integration fashion. in this approach, the written materials were provided in english, whereas the oral language was catalan. this choice of foreign language was motivated by the multiple benefits of being fluent in english (frigols, 2007), and in general by the benefits of learning foreign languages (merritt, 2013; kruschewsky, 2014). 2.3. analysis of students’ satisfaction the factors than can contribute to a better global learning and teaching experience when both active learning and clil are implemented in the physical chemistry classroom were assessed by means of a likert-scale questionnaire, for which an integer value from 1 to 5 was assigned to each item (1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; 5 = strongly agree). this questionnaire was comprised of 20 items that can be classified into four different groups according to their nature, that is: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 76 1) quality and adequacy of the learning resources. 2) satisfaction with the methodology employed. 3) satisfaction with clil. 4) overall satisfaction with the course. the questionnaire is provided in table 1 for illustrative purposes. the main aim of the items included in this questionnaire was to evaluate changes in students’ attitudes towards the subject as a result of the implementation of active learning methods, their perception of english learning in the clil context, and in general, students’ satisfaction with the learning process. in addition, students were asked to make at least one positive and one negative comment about the methodology employed in the computer-based workshops. table 1. likert-scale questionnaire. note that the items can be rated from 1 to 5 (1 = strongly disagree; 5 = strongly agree) quality and adequacy of the learning resources 1. the workshops are synchronised with the theory. 2. the workload is adequate. 3. recalling the theoretical concepts before each session is useful for me. 4. solving problems promotes a better understanding of the theoretical concepts. 5. the time allotted for each session is adequate. satisfaction with the methodology employed 6. there is no difference between solving problems by using computers and solving them by traditional methods. 7. i find it attractive and motivating to solve problems by using computers. 8. the quizzes via the moodle platform are useful for the learning process. 9. the difficulty of the problems is appropriate. 10. in general, these computer-based workshops are interesting. 11. the proposed activities are clear and adequate. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 77 satisfaction with clil 12. by using english, i am gaining fluency in this language. 13. i would like that this subject be taught entirely in english. 14. the fact that the written materials are in english is an added difficulty to the subject. 15. my english level is adequate to follow this course. 16. my english level would be adequate to follow and participate in the classroom if this subject would be taught entirely in english. overall satisfaction with the course 17. my attitude toward physical chemistry has positively changed throughout this course. 18. in general, i am satisfied with the course and with the methodology employed. 19. i feel that i am learning quickly. 20. i find multimedia resources rather scarce. so far, we have presented the state-of-the-art methodologies that are being applied within the framework of the european higher education area, as well as our approach to teaching physical chemistry in this context. the results obtained in the questionnaire in table 1, and also the analysis of these data are presented in the following section. 3. results and discussion the questionnaire was anonymously answered by the 29 students (gender: 52 % males and 48 % females). relevant data are listed in table 2. the average values of the points given by the students to each group of items are mentioned herein: 1) quality and adequacy of the learning resources (items 1 – 5): 3.92  0.81 2) satisfaction with the methodology employed (items 6 – 11): 3.39  0.81 3) satisfaction with clil (items 12 – 16): 2.68  1.22 4) overall satisfaction with the course (items 17 – 20): 3.29  0.90 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 78 table 2. relevant data for the satisfaction survey. note that the items of the questionnaire are listed in table 1. sd stands for standard deviation. item average median mode sd for the average 1 4.45 4.00 5.00 0.57 2 3.72 4.00 4.00 0.84 3 3.69 4.00 4.00 1.14 4 4.17 4.00 4.00 0.76 5 3.55 4.00 4.00 0.74 6 2.21 2.00 3.00 0.98 7 3.48 4.00 4.00 1.09 8 3.14 3.00 4.00 0.88 9 3.97 4.00 4.00 0.57 10 3.79 4.00 4.00 0.73 11 3.72 4.00 4.00 0.65 12 2.93 3.00 4.00 1.31 13 1.90 1.00 1.00 1.11 14 3.45 4.00 4.00 1.21 15 3.00 4.00 4.00 1.31 16 2.10 2.00 1.00 1.18 17 3.45 4.00 4.00 1.02 18 3.86 4.00 4.00 0.83 19 3.21 3.00 4.00 1.08 20 2.66 3.00 3.00 0.67 in view of the latter results, the highest satisfaction level corresponds to “quality and adequacy of the learning resources”, whereas the lowest average satisfaction level corresponds to “satisfaction with clil”. it is also noteworthy that the highest rated item is item 1 (the workshops are synchronised with the theory), with 4.45  0.57 points, while the lowest rated item is item 13 (i would like that this subject be taught entirely in english) with 1.90  1.11 points. the latter result is in agreement with a negative attitude toward english language amongst higher education students (tsuda, 2003). interestingly, the items multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 79 with the highest standard deviation are items 12 and 15 (sd = 1.31), which means that there are striking differences in the english level of the students. the relative frequencies of the points obtained for each group of items are represented in figures 1 and 2. in both charts in figure 1, the maximum relative frequencies correspond to 4 points, which indicates that in general students are satisfied with the quality and adequacy of the learning resources and with the methodology employed. it is also noteworthy that in figure 1a, the sum of the relative frequencies corresponding to 4 and 5 points represent 73 % of the total. this percentage is significantly lower in figure 1b, for which this sum represents 56 %. (a) (b) figure 1. relative frequency in “quality and adequacy of the learning resources” (fig. 1a; items 1 – 5) and in “satisfaction with the methodology employed” (fig. 1b; items 6 – 11). note that “pts” stands for points. in figures 2a and 2b, the sum of the relative frequencies corresponding to 4 and 5 points represent in both cases less than 50 %, being this sum lower in fig. 2a (33 %) than in fig. 2b (43 %). the extremely low value obtained from the data in the group “satisfaction with clil” reveals the negative attitude of our group of students towards the english language. 1 pts 1% 2 pts 7% 3 pts 19% 4 pts 47% 5 pts 26% 1 pts 7% 2 pts 11% 3 pts 26% 4 pts 48% 5 pts 8% multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 80 (a) (b) figure 2. relative frequency in “satisfaction with clil” (fig. 2a; items 12 – 16) and in “overall satisfaction with the course” (fig. 2b; items 17 – 20). note that “pts” stands for points. as previously mentioned, the survey was undertaken by two groups of students (15 and 14, respectively). the standard deviation for the average values of the items in both groups of students differ by less than 0.50 points. the maximum difference in the standard deviation values is observed in item 13 (0.44 points), while the minimum corresponds to item 6 (0.01 points). in addition to the likert-type items, both groups of students (n = 29) were encouraged to make both a positive and a negative comment about the workshop sessions. approximately 62 % of the students contributed with valuable comments that may become extremely convenient to improve the course methodology in the near future. the average score for the questionnaire (3.32  0.93 points) indicates the students’ overall satisfaction with the course. 4. conclusions an active learning method based on workshops in which physical chemistry students have to solve problems by using computers has been reported. in addition, our efforts devoted to teaching physical chemistry through the medium of english language have been presented herein. students’ satisfaction with both the active learning methodology and 1 pts 29% 2 pts 16%3 pts 22% 4 pts 25% 5 pts 8% 1 pts 4% 2 pts 18% 3 pts 35% 4 pts 32% 5 pts 11% multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 81 clil has been assessed by means of a 20-item likert-scale questionnaire. the highest satisfaction level corresponds to “quality and adequacy of the learning resources”, with 3.92  0.81 points, whereas the lowest average satisfaction level corresponds to “satisfaction with clil” (2.68  1.22). the average score for the questionnaire has been 3.32  0.93. the low score obtained in “satisfaction with clil” suggests that a great deal of effort has to be devoted to the efficient integration of english with physical chemistry, and also to raise students’ awareness toward the english language. 5. acknowledgements all teachers who have participated in this project are greatly acknowledged. the financial support from the unitat de suport educatiu within jaume i university is also appreciated. finally, we are also thankful to the students who have participated in the survey. 6. references cambridge esol (2010). teaching knowledge test (tkt). content and language integrated learning (clil). handbook for teachers. retrieved march 9, 2015 from http://www.cambridgeenglish.org/images/tkt-clil-handbook.pdf european communities (2009). ects users’ guide. retrieved march 9, 2015 from http://ec.europa.eu/education/tools/docs/ects-guide_en.pdf european higher education area (2010). retrieved march 9, 2015 from http://www.ehea.info eurydice (2006). content and language integrated learning (clil) at school in europe. retrieved march 9, 2015 from http://www.indire.it/lucabas/lkmw_file/eurydice/clil_en.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 82 frigols, m. j. (2007). clil implementation in spain: an approach to different models. retrieved march 9, 2015 from http://arca.unive.it/bitstream/10278/1013/1/13frigols.pdf gürses, a., açıkyıldız, m., doğar, ç. and sözbilir, m. (2007). an investigation into the effectiveness of problem‐based learning in a physical chemistry laboratory course. research in science & technological education, 25(1), 99 – 113. doi:10.1080/02635140601053641 kruschewsky, g. (2014). 6 multilingual benefits that you only get if you speak another language. the huffington post. retrieved march 9, 2015 from http://www.huffingtonpost.com/2014/06/02/multilingual-benefits_n_5399980.html marsh, d., mehisto, p., wolff, d. and frigols, m. j. (2011). european framework for clil teacher education. retrieved march 9, 2015 from http://www.ecml.at/tabid/277/publicationid/62/default.aspx merritt, a. (2013). why learn a foreign language? benefits of bilingualism. the telegraph. retrieved march 9, 2015 from http://www.telegraph.co.uk/education/educationopinion/10126883/why-learn-a-foreignlanguage-benefits-of-bilingualism.html paulson d. r. (1999). active learning and cooperative learning in the organic chemistry lecture class. journal of chemical education, 76 (8), 1136 – 1140. doi: 10.1021/ed076p1136 prince m. (2004). does active learning work? a review of the research. journal of engineering education, 93(3), 223 – 231 smith m. e., hinckley, c. c. and volk, g. l. (1991). cooperative learning in the undergraduate laboratory. journal of chemical education, 68 (5), 413 – 415. doi: 10.1021/ed068p413 spaeth a. d. and black r. s. (2012), google docs as a form of collaborative learning. journal of chemical education, 2012, 89 (8), pp 1078–1079. doi: 10.1021/ed200708p multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3696 social and technological sciences eissn: 2341-2593 recatalá (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 71-83 | 83 tsuda s. (2003). attitudes toward english language learning in higher education in japan (2): raising awareness of the notion of global english. intercultural communication studies xii-3, 61 – 75. retrieved march 10, 2015 from http://www.uri.edu/iaics/content/2003v12n3/06%20sanae%20tsuda.pdf valuation of the training received in university regarding the utility for work by catalan graduates multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 valuation by catalan graduates of the training received at university in relation to its utility in the workplace s. fachelli* and d. montolio** grup d’innovació docent consolidat de gestió i administració pública (gid-gap) av. diagonal, 684, barcelona, spain. *departament de sociologia. universitat de barcelona. corresponding author: sandra.fachelli@ub.edu; phone: + 34 934021801 ** departament d’economia pública i institut d’economia de barcelona. universitat de barcelona. received: 2015-02-14; accepted: 2015-05-24 abstract this article examines the factors that influence graduate valuations of the education/training they received at university in terms of its utility or applicability in the workplace. drawing on the 2014 survey conducted by the agency for the quality of the catalan university system, among students that graduated in 2010, we test three hypotheses. the first states that graduate valuation of the training received at university in terms of its utility for the workplace is higher among those who are currently employed in high quality jobs; the second that this valuation is higher among graduates employed in higher occupational categories; and, the third, that higher valuations are given by individuals with better academic records. the methodology used to test the three hypotheses is based on both descriptive and econometric techniques that allow us to control for specific individual characteristics and specific characteristics of the degree subjects studied. preliminary empirical results allow us to verify two of the three hypotheses. the main contribution of this paper is to provide some initial insights into a relationship not frequently examined in the literature and to offer some empirical evidence that counters the typical “matching” standpoint taken on the relationship between education/training and level of employment. keywords higher education; training valuation; labour insertion; university graduates fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 14 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 1. introduction this article examines the factors that influence graduate valuations of the education or training they received at university in terms of the satisfaction they derive from their work. the question under examination is highly complex, especially given that we are dealing with a subjective, self-report measure (i.e., the satisfaction derived) of the training/education received at higher education institutions in relation to the jobs being undertaken by recent graduates. the question that concerns us is intimately connected with the broader analysis of “job satisfaction” and its determinants, an issue that has been addressed by many empirical studies conducted in countries around the world (see gamero, 2003, and references therein for a review). job or occupational satisfaction is not readily measured, nor are its principal determinants readily accounted for and explained. the complexity is in part attributable to the fact that we need to address the graduates’ particular working conditions, with special attention to the structural characteristics of the job (contract type and duration), the degree of match between the job and the graduate’s degree and salary, among others. in this paper, therefore, we focus specifically on how recent graduates from catalan universities value the training/education they received at university with respect to the work they are currently performing, a relationship that has not received direct attention in the broader literature examining job satisfaction. previous studies undertaken in catalonia, based on an indicator developed by mañé and miravent (2007), found that a high percentage of graduates from catalan universities, now in full-time employment, considered (interviews conducted in 2011) that they were appropriately educated for the job that they did (79.6%). at the same time, 13.6% believed they were over-educated, while the remaining 6.8% considered themselves to be mismatched. no significant differences were recorded in these responses by gender (see, planas and fachelli, 2010). fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 15 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 to gain insights into graduate valuations of the training received at university in relation with the satisfaction they derive from working, without overlooking the broader question of job satisfaction in general, it is useful to learn about the quality of the education received by graduates in higher education institutions and to determine whether the education provided enhances the welfare of individuals when working, which is, after all, very much part of the ultimate goal of the educational process. descriptive studies of the employability of graduates of catalan universities have been undertaken by the agency for the quality of the catalan university system (known as aqu) since 2001 via a survey conducted with more than 11,000 alumni. in this paper we draw on the results from aqu’s latest survey (2014) conducted with students graduating in 2010. we seek to test three hypotheses. the first concerns the current job being performed by the graduate and states that the graduates’ valuation of the training received at university in terms of its utility for the workplace is higher among those graduates who enjoy higher “occupational quality” (see definition below). the second hypothesis states that this valuation is higher among those graduates that have a better “occupational category” (see definition below). finally, the third hypothesis states that higher valuations of the training received and its utility for work are given by individuals with a higher “academic performance” (better academic records). a multivariate statistical analysis is conducted to test the three hypotheses. thus, in addition to contingency tables, we employ multiple linear regression models to compare the variables of interest (occupational quality, occupational position and academic performance) over a set of control variables deemed relevant for explaining the individual valuations: sex, age, educational level of parents and the time taken to complete the degree. the rest of the paper is organized as follows. section 2 briefly presents the theoretical framework underpinning the three empirical hypotheses tested. section 3 presents the empirical model, describing both the methodology and the data sources. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 16 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 section 4 reports the main empirical results found on performing the estimations. finally, section 5 discusses these results and concludes. 2. theoretical framework as discussed above, the determinants of graduate valuations of the training/education received while completing their degree courses share obvious ties with the more general subject of worker job satisfaction. this broader assessment of individual job satisfaction incorporates factors from psychology and sociology, including, personality traits, personal relationships, recognition and prestige in the workplace, social support, health and environmental conditions, social protection and exercise of fundamental labour rights, which hinder its analysis and make constant reassessments necessary. in this framework, the valuation of the training/education received in relation to the current job can be considered an important part of the assessment of job quality and as an aspect that needs to be evaluated from the perspective of employers and society in conjunction with the perception of the employee (international labour organization, 2004; seashore, 1974). there is a vast literature examining the determinants of job satisfaction but there is a general consensus that, among other factors, it is dependent on the qualifications required to be appointed to the post, i.e., that the functions that have to be fulfilled in the job are in accordance with the qualifications required (for instance, holding a university degree). in general, and in line with aqu (2014), it would seem that workers express a higher level of satisfaction with jobs that require specific higher education qualifications and a lower level of satisfaction with jobs that do not require a university degree. the reason for this may well be the valuation that workers give of the training received at an institution of higher education and which is required to access those jobs that require a specific degree. however, the literature examining the valuation of the training received in relation to the job held is far less abundant. in general, and apart from obvious individual fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 17 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 characteristics, the valuation of training/education with respect to occupation is considered to be determined via two main channels. first, and in line with richard freeman’s (1976) perspective that takes as its focus an individual’s “job position”, job satisfaction is intimately related with the ex ante achievement of the requirements needed to perform the tasks related to a job. thus, more specialized job positions are associated with the specific skills that a worker must have acquired before taking up the position. in such instances, job satisfaction is very closely related to the acquisition of these skills and, hence, with the workers’ valuation of the training/education received. gargallo (2008), in reviewing studies undertaken from this perspective, discussed how worker satisfaction has been analysed from within various disciplines, including that of psychology (argyle, 2001), sociology (hodson, 1985; kalleberg and loscocco, 1983) and economics (hamermesh, 1977, 2001). olarte (2011) similarly examines the different perspectives adopted in studying factors that determine job satisfaction and concludes, in line with ardouin et al. (2000), that the key determinants are having an intellectually stimulating job with fair rewards, favourable conditions of work and cooperative colleagues. robbins (2004), likewise, highlights the importance of the attractiveness of the tasks performed, the associated rewards, relationships with colleagues, and the working conditions to which employees are exposed. to these leal et al. (1999) stress the role played by wages, relationships with superiors and peers, and opportunities for promotion. finally, garcía et al. (2005) identify four basic factors determining job satisfaction: personal fulfilment in employment, working conditions, interpersonal relations and remunerations. second, and in line with pierre naville (1956), the valuation of some jobs cannot be explained in terms of the “job position” but rather with regard to more general job “competences”, i.e., workers’ general knowledge, abilities, attitudes, and values that are useful in performing the specific tasks associated with a given job. in this framework, the valuation of the training/education received is more indirect given that education systems (at least in spain) were not explicitly designed to promote these competences; hence, fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 18 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 while students may have acquired these skills, they are not always conscious of the corresponding learning processes, which makes it more difficult for workers to assess them in terms of their current job. in this instance, the main research approach involves examining the match between level of skills and job position (mcguiness, 2006; sattiner, 1993), what this strand of the literature refers to as the “matching perspective”; that is, for every level of education there is a corresponding employment category. in what is clearly a normative assumption, the match corresponds to the optimal exchange between employers and employees in the labour market. there has been much criticism levelled at this approach because of its attempts to establish a direct relation between workers’ education and training and the requirements of a job position, without taking into account the role played by agents and institutions in that relationship (raffe, 2001; planas, 2014). within this context, and as pointed out above, we seek to determine if the evaluation of such training is dependent on occupational quality, occupational position and the academic performance of graduates. in the case of this last variable, we would expect graduates with a better academic record to present a better match with their jobs (i.e., training vs. tasks performed), since they would enjoy a wider range of employment opportunities. it is also possible that these “more skilled workers” are capable of more sophisticated assessments of the utility of their prior training in relation to their current work. a similar reasoning could be applied to the case of employment quality and occupational position. in principle, we would expect to find a relationship between better job positions and the evaluation that workers have of the training received at higher education institutions, if that training was the means that allowed them to take up a specific job or position. however, the answers to these questions lie in the data and, so, our aim here is to determine whether these relations hold for the graduates of catalan universities. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 19 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 3. empirical model, methodology and data sources 3.1. empirical model and methodology the empirical counterpart of the aforementioned hypotheses can be expressed as follows: 𝑌𝑌𝑖𝑖 = 𝛼𝛼 + 𝛽𝛽1 ∑𝑋𝑋𝑖𝑖 + 𝛽𝛽2 ∑𝑍𝑍𝑖𝑖 + 𝜀𝜀𝑖𝑖 (1) where i refers to the individual graduates in our dataset and y is our dependent variable accounting for the individual valuation of the training received in relation to its utility for the current job. x is the vector containing the study’s three main variables, namely, variables accounting for the “occupational quality”, “occupational category” and “academic performance” of the graduates. z is a set of individual control variables that include sex, age, parents’ level of education, type of degree, degree duration and degree awarded/place of graduation. finally ε is an error term that ideally should be i.i.d. distributed. eq (1) is estimated using the traditional ols technique. the qualitative variables in our model are codified as dummy variables. the r2 informs us about the goodness-of-fit of each of the estimated models. 3.2. data sources and variables our data are taken from the 2014 survey conducted by aqu catalunya on the employability of graduates from catalan universities. this, the latest survey available, interviews individuals that graduated in 2010 (more than 17,000 interviews conducted). we restricted the overall sample of individuals to those that graduated from public universities (84.1%), and focus our study specifically on students graduating in the degree in public administration (henceforth gap) from the law faculty of the university of barcelona (ub) (identified in the empirical model by ‘degree awarded/place of graduation’, see table 1). our sample includes those graduates that were working (both full and part-time) at the time of the survey. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 20 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 our dependent variable is the average score given by graduates when they were asked to evaluate the quality of a specific competence acquired at university (from 1 very low to 7 very high) in relation to its applicability to his/her job at the moment of answering the survey. the 14 competences are: “theoretical learning”; “practical learning”; “oral expression”; “written expression”; “teamwork”; “leadership”; “problem solving”; “decision-making”; “creativity”; “critical thinking”; “management skills”; “computing skills”; “languages” and “documentation skills”. table 1: sample of catalan graduates by degree awarded/place of graduation and in employment sample total employees frequency percentage frequency percentage gap (ub law fac.) 37 0.3 32 0.3 law (ub law fac.) 114 0.9 98 0.9 other degrees ub law fac. 198 1.5 169 1.5 other degrees rest of ub 3,076 23.3 2,545 23.0 all degrees rest of public unis. in catalonia 9,755 74.0 8,239 74.3 total 13,180 100 11,083 100 source: based on aqu (2014) we work with three main variables. the first, the index of occupational quality (see corominas et al., 2007) summarizes the quality of the labour market outcomes of university graduates and combines the type and duration of the contract of employment, the salary, the match (suitability) between the graduate’s university studies and job, and job satisfaction in general. the index ranges between 0 and 100, with the lowest values reflecting a lower quality of labour market outcomes and the highest value representing a higher quality. the second variable, occupational category (see table 2), groups the nine categories making up the spanish national classification of occupations (nco) in three broad bands. these three occupational categories seek to group workers by overall skill level fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 21 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 (note that the nco itself classifies occupation categories from more skilled to less skilled). the third variable, academic performance, which originally comprised four categories (honours, excellent, good and pass) was regrouped into three by combining honours and excellent into a single group given the small number of students obtaining these two grades (see table 3). note, however, the variable academic performance is selfreported by the graduates, that is, it is not based on university records. as for the expected impact of this variable, planas and fachelli (2010) report that the entrance grade and the student’s academic record present a very weak relationship with the graduates’ mean assessment of their university education and the skills required in their job. table 2: occupational categories based on skill level occupational category national classification of occupation categories upper 1 directors and managers 2 technicians and scientific professionals and intellectuals medium 3 technicians; support professionals 4 accounting, administrative and other office employees lower 5 service and sales workers 6 skilled agricultural, forestry and fishery workers 7 craftsmen and skilled workers in manufacturing industries and construction 8 plant and machine operators, and assemblers 9 elementary occupations source: based on aqu and nco. in the case of the control variables, sex, age and socio-economic background (measured by parental education) are directly available from the dataset. the socioeconomic background has been analysed for the spanish case but there does not seem to be a statistically significant relationship between this variable and employability (see carabaña and de la fuente, 2015; torrents y fachelli, 2015; fachelli, torrents y navarrocendejas, 2015, navarro-cendejas, 2013; fachelli y planas, 2014). here, we analyse the impact of the graduates’ socio-economic background on their assessment of the training fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 22 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 received at university in relation with the satisfaction derived from their work. degree length was constructed by subtracting the initial year of matriculation from the year of graduation. the degree awarded/place of graduation variable differentiates, on the one hand, between the degree in public administration (gap) and the law degree (the graduates of which courses are the specific focus of this study) taught within the ub law faculty and, on the other, between all other degree courses taught at the ub law faculty, all other ub degrees and finally all other degrees taught within catalonia’s public universities. finally, and to test the robustness of our results we apply the same empirical model as that presented in eq. (1) but change the dependent variable; that is, we take as our dependent variable the results of a more general question regarding the graduates’ valuation of the education/training received but without linking this valuation to its utility in the actual job position occupied. table 3: explanatory variables independent variables control variables index of occupational quality (numerical) sex (male female) age (years) educational level of parents occupational category primary upper secondary medium higher lower time taken to graduate (in years) degree awarded/place of graduation academic performance gap degree ub law fac. pass law degree ub law fac. good other degrees ub law fac. excellent and honours other degrees rest of ub all degrees rest of public unis in catalonia fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 23 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 4. results 4.1. descriptive analysis first, we undertake a descriptive or exploratory analysis of the dependent variable: the assessment made by graduates of the training received during higher education with respect to its usefulness for their work. we also describe the main explanatory variables of this assessment by focusing on the degree awarded/place of graduation, which proves of particular interest. the average assessment of the usefulness of the set of tools received in higher education in relation to the tasks performed in the current job is 4.36 (on a scale with a maximum value of 7; that is 6.62 on a decimal scale). this can be interpreted as an acceptable outcome but one that falls well short of the standard of excellence expected from higher education institutions (see figure 1). on average, the assessment given by gap graduates coincides with the overall mean, while the ub’s law graduates are the ones that value most highly the skills taught at university in relation to the jobs they now perform. figure 1: assessment of the applicability of knowledge to work by degree awarded/place of graduation. source: based on aqu (2014). mean: 4.3502 3.8222 3.8407 4.2345 4.4274 fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 24 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 figure 2 presents the assessment of the applicability of knowledge received in university with respect to the job occupied by degree awarded/place of graduation. figure 2: assessment of the applicability of knowledge to work by occupational category, sex, academic performance and parental level of education. source: based on aqu (2014). when testing the alternative dependent variable (that is, the assessment of university training), on average, we obtain higher scores: 5.0 (or 7.14 on a decimal scale) (see occupational category upper: 5.2 occupational category media: 4.9 occupational category baja: 4.2 women: 5.1 men: 4.9 passed: 4.9 remarkable: 5.1 excellent: 5.3 honors: 5.3 up primary: 5.0 media sstudies: 5.0 higher education: 5.1 educational level of parents u ti lit y fo r w or k u ti lit y fo r w or k u ti lit y fo r w or k u ti lit y fo r w or k fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 25 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 figure a1 in the appendix). in this case, the gap and law graduates valuation of the knowledge received is on the average as well as the other degrees analysed. figure 3 directly compares the two dependent variables and plots the difference for each specific competence (that is, the difference between the graduates’ valuation of each component of training received at university and the appraisal of its utility for the current job). this comparison highlights certain deficits in university training if we consider that the use of each of the competences in the workplace should afford the graduates with a certain degree of satisfaction. however, this “mismatch indicator” may be called in to question on the grounds that the training provided at university may pursue multiple objectives, including the general education and personal development of individuals in all areas of life, and that labour skills constitute just one of the many areas that higher education should consider. however, the negative values reported in figure 3 can be seen as identifying deficits in university education/training if we restrict our interpretation solely to the demands of the labour market (see table a1 for the specific values reported in figure 3 for each competence and category and for degree awarded/place of graduation). fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 26 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 figure 3: difference between the graduates’ valuation of each component of training received at university and the appraisal of its utility for the current job. source: based on aqu (2014). interestingly the results presented by the gap graduates and the values recorded by those graduating at all other catalan public universities are similar; that is, deficits (negative values) in general with the exception (although not significant) of documentation skills, teamwork and theoretical learning. this means that graduates only consider their level of competence in these first two skills as being adequate for the work they currently undertake. most notably it is the ub’s law graduates that report the greatest deficits across the whole skill set with a marked “mismatch” in their valuations of computing skills, languages, decision-making and oral expression. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 27 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 4.2. regression analysis the regression analyses presented in this section allow us to examine our findings in greater depth. specifically they enable us identify more precisely the impact of our main variables of interest (“occupational quality”, “occupational category” and “academic performance”) while controlling for a set of other potential determinants of the valuation made by graduates of their training in relation to its applicability to their current jobs (see table 4).1 the final sample comprises the valuations of 8,813 graduates and the goodness of fit of the proposed model is around 20%. the results for the three main variables of interest are statistically significant. the index of occupational quality, as developed by corominas et al. (2007), has a positive impact, showing that the higher the occupational quality, the higher the graduates value the education/training received at university. the occupational category also presents a positive point estimate when the lower category is taken as our reference. this suggests that a higher occupational category is associated with a higher valuation of the adequacy of the training received for the tasks performed at work. finally, academic performance presents a positive impact: graduates with a better (self-reported) academic performance value more highly the education/training they received in relation to their current position. as for the other control variables, we find that the socio-economic background of recent graduates (measured by their parents’ level of education) does not seem to affect the valuations given; that is, it would seem that the valuation is independent of family background. meanwhile, the variables of age and degree length present a negative impact, indicating that younger graduates and graduates who made short degrees (or taking fewer years to complete their degree) value less the education/training received from the university in relation to the tasks they now perform in the workplace. 1 in the appendix we present (in table a2) and analyse the results when our dependent variable is the valuation of the training/education received at the university. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 28 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 table 4: multiple linear regression: testing the model of analysis. source: based on aqu (2014). in the case of the variable capturing degree awarded/place of graduation, we find that the valuations of the gap and law graduates from the ub law faculty do not differ from those of the reference category (i.e., all other graduates from catalonia’s public universities excluding the ub). however, in general, graduates in other degree courses offered at the ub law faculty and graduates in other degree courses offered at the ub show a lower (and statistically significant) appreciation of the utility of their training to their current jobs. overall, the index of occupational quality and occupational category explain most of the variance in our dataset with regards the valuation of education/training in relation to its utility for work. these two variables account for 18% of the overall variance (the coefficients typified b stand. error beta constant 3,606 ,076 47,199 0,000 occupational quality index (numerical) ,019 ,001 ,358 14,0 34,302 ,000 occupational category: low (*) upper ,504 ,039 ,217 4,9 13,019 ,000 medium ,394 ,041 ,153 -1,3 9,620 ,000 academic performance: approved (*) excellent and honors ,238 ,066 ,035 ,1 3,587 ,000 remark able ,146 ,022 ,068 ,6 6,672 ,000 sex: male (*) ,229 ,022 ,104 ,8 10,564 ,000 age (years) -,008 ,002 -,039 ,2 -3,731 ,000 educational level of parents: up primary (*) media studies ,031 ,026 ,013 ,0 1,196 ,232 higher education ,016 ,025 ,007 ,0 ,638 ,523 career length (in years) -,018 ,004 -,044 ,3 -4,167 ,000 place of graduation: public rest (*) gap degree -,025 ,177 -,001 ,0 -,142 ,887 law degree -,180 ,106 -,016 ,0 -1,696 ,090 rest degrees of law (ub) -,518 ,085 -,059 ,5 -6,071 ,000 other degrees of ub -,090 ,024 -,036 ,1 -3,668 ,000 cases number 8813 r2 20,2% r2 adjust 20,1% standard error 0,96607 control variables (*) reference category of independent variables dependent variable: assessing the training received in connection with the utility for working coefficients unstandardized beta * r pearson (in %) t sig. independent variables fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 29 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 index of occupational quality explaining 14% and the occupational category explaining 4%). 5. discussion and conclusions we have constructed a model that seeks to account for graduate valuations of the education/training they received at university in terms of its utility in the workplace. in so doing, we analysed the responses of 13,180 graduates from catalan universities (interviewed during the first quarter of 2014) and who were working at the time of the survey (84%). the average valuation provided by the interviewees for the training/education received with respect to the job they are currently performing is 4.36 on a scale with a maximum of 7. on average, the assessment given by gap graduates coincides with this overall mean, while the ub’s law graduates are the ones that value most lower the skills taught at university in relation to the jobs they now perform. the regression analysis confirms the descriptive findings and we observe that two variables explain most of the variance of our dependent variable (the graduate valuation of the training/education received at university in relation to their current job). these two variables are the “index of occupational quality” and the “occupational category” of graduates. thus, the results indicate, on the one hand, that the better the contract, salary and match between training and employment conditions, the higher the valuation graduates give of their training/education. on the other hand, graduates occupying the upper occupational categories in terms of job skills value their training/education more than those in lower categories. our results also show that women have, in general, a more positive valuation of the knowledge received at university in relation to their work, and that the highest valuations are given by those women with the best academic records. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 30 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 the graduates who made shorter degrees give lower valuations to the education/training received, which could be linked to the duration of the degree, a variable that shows a negative relationship with our dependent variable. in contrast, we find no differences in valuations when considering the educational level of parents. this result is interesting and shows further evidence that university graduates are becoming increasingly independent of the influence of their social origins as their education level increases (i.e., the higher their educational level, the weaker the influence of parental status). the degree awarded/place of graduation variable shows that graduates from the ub’s faculty of law, except for those graduating in gap and law itself, and all other ub graduates report a slightly lower valuation of their training than that reported by graduates from catalonia’s other public universities. here, it might be interesting to examine the influence of social class, an issue not explicitly addressed here, but one that should be addressed in future work. in short, given that the most important variables are the index of occupational quality and occupational category, we might summarise the findings of this study in the following sentence: “tell me what work you perform and i will tell you how you value the applied skills you acquired at university”. the theoretical contribution of this paper can be summarized as follows: on the one hand, we contribute detailed information and empirical estimates to a subject rarely discussed in the specialized literature, which tends to focus much more on questions of job satisfaction or the match between training and employment; on the other hand, by examining the match between graduate competences and employment, we are able to offer more than a purely technocratic vision of this issue and provide evidence that contradicts the purely “matching” approach. more specifically we, first, incorporate the institutional element into our empirical estimation by introducing the specific assessments that graduates offer of the institution at which they studied. second, we include an assessment of the “educational market” that comprises a variety of agents, and fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 31 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 which is typically overlooked from the matching perspective. third, we deal with two elements that are often confused and account for questions of both training supply (received in a specific institution) and labour supply characteristics (measured by the actual job positions held by the graduates), while we also include a link between the two elements by valuing the training received in relation to the specific applicability of these competences to the work done by the graduates at the time of answering the survey. in terms of education policy, these results are of obvious relevance because, generally speaking, graduates value positively their own past efforts, at the same time as they value positively the specific tools available to them in terms of the knowledge and skills that their university provided them with. moreover, this assessment has been carried out not within the field of higher education but within the labour market, and it clearly points to the positive impact of universities on the job satisfaction of graduates in terms of the valuation given to the training/education received in higher education institutions. references aqu (2014) universitat i treball a catalunya 2014. estudi de la inserció laboral de la població titulada de les universitats catalanes. barcelona agència per a la qualitat del sistema universitari de catalunya. ardouin, j., bustos, c., gayó, r., and jarpa, m. (2000) motivación y satisfacción laboral. documento en línea. disponible en www.apsique.cl argyle, m. (2001) the psychology of happiness. routledge, 2001. carabaña, j. and de la fuente, g. (in press 2015). facultad por facultad. origen familiar y empleo de los licenciados en ccss y humanidades de la ucm en el año 2003. revista complutense de educación. corominas rovira, e.; villar hoz, e; saurina canals, c.; fàbregas alcaire, m. (2007) el mercat laboral qualificat i la qualitat de l’ocupació. in educació superior i treball a catalunya. anàlisi dels factors d’inserció laboral. barcelona: agència per a la qualitat del sistema universitari de catalunya fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 32 http://polipapers.upv.es/index.php/muse http://www.apsique.cl/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 fachelli, s.; planas, i. (2014) inserción profesional y movilidad intergeneracional de los universitarios: de la expansión a la crisis. revista española de sociología (res). 21: 6998. http://ddd.uab.cat/pub/artpub/2014/125654/revespsoc_a2014n21p69ispa.pdf fachelli, s.; torrents, d.; navarro-cendejas, j. (2014) ¿la universidad española suaviza las diferencias de clase en la inserción laboral? revista de educación, 119–144. http://ddd.uab.cat/pub/artpub/2014/118532/revedu_a2014m4-6n364p119ispa.pdf freeman, r.b. (1976). the overeducated american, new york academic press. gamero, c. (2003) análisis económico de la satisfacción laboral. màlaga: universidad de málaga. [tesis doctoral] garcía, j., gargallo, a., marzo, m. amd torres, p. (2005) “job satisfaction: empirical evidence of gender differences”. women in management review, 20 (3-4), pp. 279-288 http://dx.doi.org/10.1108/09649420510599098 gargallo, a. (2008) la satisfacción laboral y sus determinantes en las cooperativas. en: de castro silva y díaz de castro (coords), universidad, sociedad y mercados globales. asociación europea de dirección y economía de empresa. international conference. salvador de bahía (pp. 563-575) hamermesh, d.s. (1977) economic aspects of job satisfaction. in orley ashenfelter &wallace oates, essays in labor market analysis. new york: halsted press. (pp. 53-72) hamermesh, d.s. (2001) the changing distribution of job satisfaction. journal of human resources, 36(1), pp. 1-30. http://dx.doi.org/10.2307/3069668 hodson, d. (1985) philosophy of science, science and science education. studies in science education, 12(1), pp. 25-57 http://dx.doi.org/10.1080/03057268508559922 international labour organization (2004) the future of work and quality in information society. geneva: international labour organization. kalleberg, a.l. and loscocco, d.a. (1983) aging, values and rewards: explaining age differences in job satisfaction. american sociological review, 48(1), pp. 78-90 http://dx.doi.org/10.2307/2095146 leal, a., alfaro de prado sagrera, a., rodríguez félix, l., and román onsalo, m. (1999) el factor humano en las relaciones laborales: manual de dirección y gestión. madrid: pirámide. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 33 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1108/09649420510599098 http://dx.doi.org/10.2307/3069668 http://dx.doi.org/10.1080/03057268508559922 http://dx.doi.org/10.2307/2095146 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 mañé, f. and miravent, d. (2007) l’adequació de la formació universitària en serra ramoneda (ed.). educació superior i treball a catalunya: anàlisi dels factors d’inserció laboral. cap. 4 (155: 186). barcelona: agència per a la qualitat del sistema universitari de catalunya (aqu). mcguiness, s. (2006) overeducation in the labour market. journal of economic surveys, 20(3), 387-418. http://dx.doi.org/10.1111/j.0950-0804.2006.00284.x navarro-cendejas, j. (2013) universidad y mercado de trabajo en cataluña: un análisis de la inserción laboral de los titulados universitarios. tesis doctoral. universidad uab. http://ddd.uab.cat/pub/tesis/2014/hdl_10803_134648/jnc1de1.pdf naville p. (1956) essai sur la qualification du travail, paris, editions marcel rivière. olarte, m. l. (2011) “los determinantes de la satisfacción laboral: una revisión teórica y empírica”. tesis. universidad jorge tadeo lozano. facultad de ciencias económicas. bogotá. planas, j. (2014). adecuar la oferta de educación a la demanda de trabajo. ¿es posible? una crítica de análisis adecuacionistas de relación entre formación y empleo. méxico: asociación nacional de universidades e instituciones de educación superior. planas, j. and fachelli, s. (2010) “catalan universities as a factor of equity and professional mobility”. barcelona: agència per a la qualitatat del sistema universitari de catalunya (aqu). http://ddd.uab.cat/pub/llibres/2010/114675/catunifacequ_a2010.pdf raffe, d. (2001) “the role of vocational training and education in the combat against youth unemployment”. in groth, c. and maennig, w. (eds) strategies against youth unemployment: an international comparison, pp. 91-104, berlin: peter lang. robbins, s.p. (2004) comportamiento organizacional. pearson educación. sattinger, m. (1993) assignment models of the distribution of earnings. journal of economic literature, vol. xxxi, june, pp. 831-880. seashore, s.e. (1974) “job satisfaction as an indicator of the quality of employment”, social indicators research, 1, p. 135-168. http://dx.doi.org/10.1007/bf00302885 torrents, d.; fachelli, s. (2015) el efecto del origen social con el paso del tiempo: la inserción laboral de los graduados universitarios españoles durante la democracia. revista complutense de educación, 26 2, 331-349. http://dx.doi.org/10.5209/rev_rced.2015.v26.n2.43070 fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 34 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1111/j.0950-0804.2006.00284.x http://ddd.uab.cat/pub/tesis/2014/hdl_10803_134648/jnc1de1.pdf http://ddd.uab.cat/pub/llibres/2010/114675/catunifacequ_a2010.pdf http://dx.doi.org/10.1007/bf00302885 http://dx.doi.org/10.5209/rev_rced.2015.v26.n2.43070 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 appendix figure a1: assessment of the knowledge received from higher education institutions by degree awarded/place of graduation. (all graduates) table a1: difference between the valuation of the education/training received (in general) and the education/training received with respect to its utility in the workplace. difference between valuations gap law other degrees in the law fac. rest of ub rest of total public unis. theoretical learning 0.8438 0.5306 1.4107 0.8905 0.8922 0.8963 practical learning -0.2813 -0.6804 0.0602 0.0012 0.2483 0.1791 written expression -0.4063 -0.7959 -0.4881 -0.4026 -0.5050 -0.4836 oral expression -0.9375 -1.6531 -0.9461 -1.0323 -0.9734 -0.9924 teamwork -0.0625 -0.7449 -0.4615 -0.6412 -0.4075 -0.4640 leadership -0.9063 -1.4694 -0.7679 -1.1115 -1.0611 -1.0714 problem solving -1.0938 -1.2551 -1.0000 -1.1727 -1.0057 -1.0464 decision-making -0.7188 -1.7653 -0.9394 -1.2080 -1.1840 -1.1896 critical thinking -0.2813 -0.2857 0.1024 -0.3496 -0.4158 -0.3912 creativity -0.9063 -1.2857 -0.4671 -0.7297 -0.6936 -0.7043 management skills -0.8438 -1.3265 -1.0419 -1.0146 -0.9795 -0.9911 documentation skills -0.1563 -0.9184 -0.2410 -0.2361 -0.2968 -0.2871 languages -1.5000 -2.0510 -1.5298 -1.6682 -1.6618 -1.6642 computing skills -1.1563 -2.3878 -1.6488 -1.3057 -0.9334 -1.0433 means: 4.9240 4.8921 4.3137 4.9449 5.0691 ass ess me nt of kn ow led ge rec eiv ed fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 35 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 table a2: multiple linear regression: testing training received at university. dependent variable: training received at university unstandardized coefficients coefficients typified t sig. b stand. error beta independent variables constant 4.364 .080 54.610 0.00 occupational quality index (numerical) .005 .001 .093 8.226 .000 occupational category: lower (*) upper -.244 .040 -.109 -6.029 .000 medium -.142 .043 -.057 -3.326 .001 academic performance: pass (*) excellent and honours .044 .069 .007 .636 .525 good .124 .023 .060 5.389 .000 control variables sex: male (*) .232 .023 .109 10.222 .000 age (years) .001 .002 .005 .479 .632 educational level of parents: primary (*) secondary .035 .027 .015 1.267 .205 higher -.089 .026 -.041 -3.418 .001 career lenght (in years) -.045 .005 -.115 -9.953 .000 place of graduation: rest public (*) gap degree -.104 .186 -.006 -.561 .575 law degree -.466 .111 -.044 -4.199 .000 other degrees – ub law fac. -.637 .089 -.075 -7.129 .000 other ub degrees -.176 .026 -.073 -6.913 .000 number of cases 8,817 r2 5.4% r2 adjust 5.3% standard error 1.01140 (*) reference category of independent variables regarding the results reported in table a2 the goodness of fit of the model is very low at around 5%. the results for the three main variables of interest are statistically significant except in the case of academic performance classed as excellent and honours. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 36 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3690 social and technological sciences issn: 2341-2593 the index of occupational quality has a positive impact, but the occupational category presents the expected opposite tendency. finally, academic performance also presents a positive estimate: graduates with a good (self-report) academic record value more highly the education/training they received at university. as for the other control variables, we find that socio-economic background of recent graduates (measured by their parents’ level of education) only affects the valuation when the parents have completed studies of higher education. age is not significant and degree length presents a negative impact: graduates who made shorter degrees value less the education/training received at university. in the case of the variable capturing degree awarded/place of graduation, we observe that gap graduate valuations do not differ from those of the reference category (i.e., all other graduates from catalonia’s public universities excluding the ub). in contrast, the graduates form the rest of places show a lower valuation. it is perhaps worth stressing that when graduates are asked to value the education they received at university, they might understand this in a more global sense and so assess many elements including their own performance, the expertise of teachers, their family’s socioeconomic situation at that time, etc., whereas when asked to value the applicability of this knowledge to their current job, the focus is much more specific. in this sense, when responding to the more general question the current job and all its characteristics are irrelevant in the graduates’ valuation of the training received. fachelli & montolio (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc& tec. sci. vol. 2 nº 2 (2015): 14-37 | 37 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 construction and evaluation of scenarios as a learning strategy through modelling-simulation j. l. peñaloza figueroa*, c. vargas perez** * department of statistics and operational research ii av. filipinas s/n, universidad complutense de madrid, 28003 madrid, spain. jluispf@ucm.es; phone: + 34 913946746 ** department of applied economics iv av. universitaria s/n. universidad complutense de madrid, 28040 madrid, spain. cvargas@ucm.es received: 2014-02-26; accepted: 2014-03-22 abstract from a systemic perspective and in the context of an increasing generalization in the use of new technologies and the change in the educational paradigm -which emphasizes guided and autonomous learning-, the learning strategy should be routed to join logical reasoning and instrumental skills (software). the inclusion of new computing and communication resources to the learning process turns them into teaching tools, which makes it possible to organize the teaching and learning process in a different way. the design of these new scenarios of study has important implications to the way information is processed, to the different levels of learning (descriptive, explanatory or analytical ones) and to the way knowledge is acquired and evaluated. in addition, that design takes into account the greater student-content, student-student and student-teacher interactivities, always emphasizing guided independent learning. thus, the construction, analysis and evaluation of scenarios through models and simulation are the strategy that best suits the current learning style followed by students. keywords new technologies, paradigm, logical reasoning, instrumental skills, scenarios, interactivity, modelling and simulation peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 40 mailto:cvargas@ucm.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 1. introduction in a context of growing demand for software that can help make sense of increasingly larger and more complex sets of data, where internet connectivity and other communication media also grow simultaneously, allowing us to interact and learn from the massive data flow, the student´s ways of thinking and the teaching approaches are altered and tend to change for more dynamic structures. that is, the ways to extend background of new knowledge and to live with the data change, as well as the ways to support new technologies to overcome the old forms of social interaction and business. the idea is to make the students learn how to scientifically approach the process of modelling and evaluation of scenarios through simulation, in addition to developing intelligent views to communicate their results significantly. from this perspective, modelling and simulation have a dual purpose. first, guided learning of a phenomenon or situation through the reformulation of the mental model. second, autonomous learning as part of the natural process of change in mental images in interaction with the phenomenon under study. the purpose is to use the experience that students have outside the classroom, where they solve real problems of complex and ambiguous nature using cognitive and intuitive strategies and tools (krajcik, j. et al., 2000). hence, the task of the teacher is to show the student how to think and calculate, relying on the potential interactivity of new technologies interfaces as tools for amplification of logical reasoning, in order to lessen the cognitive burden required for the studies (bouciguez and abásolo, 2010). 2. contextualization of the relationship between new technologies and learning the huge development of digital environments and media has driven the proliferation of devices used by people to consume content at home, at work, at school and in the activities of every day. not long ago students and workers relied solely on their desktop computer for their online connection. now, an increasing number of people are likely to peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 41 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 access a wide variety of digital content through a variety of devices on a daily basis. with laptops, smartphones, tablets and other connected devices, people have become digital omnivores; not only by the media they consume, but also in the way they consume it. in this world where continuous connectivity between people, organizations, and between people and organizations prevail, the challenge is to ensure that student is able to develop its common sense1 and understand the relationships that exist between objects, not from their absolute position in space, but from their simultaneous interactions at a given time and space position. this space-time simultaneity has created an environment characterized by the availability of huge amounts of data, large number of variables and a wide variety of interrelationships that have led to the development of new knowledge and the performance of many complex tasks not achievable until a few decades ago. from a scientific perspective, the change has meant, first, the review and reflection on the knowledge acquired, the experiences and the operational skills, in order to implement increasingly complex operations. second, it has led to the diversification of technological designs aimed at creating platforms and digital media for autonomous guided learning, which has changed the way of understanding the functioning of complex actual systems (physics, chemistry, biology, economics, psychology, etc.). 3. current discussion and contribution to the state of the art the literature on ways to investigate current and future reality (prospective) shows that autonomous and self-guided learning has evolved from the development of quantitativeand-qualitative estimation and forecasting techniques towards handling situations involving uncertainty and multiple options, in addition to the various forms of data and information processing and their interpretation (medina, 2000; mujica, 1998; shoemaker, 1995; bertrand de jouvenel, 1967; kahn, 1967; gaston, 1957). 1 common sense is a representation of past experiences, and situations previously encountered and resolved. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 42 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 while this inquiry process has been developed in the context of a systemic view of the economy and society, the dominant emphasis has been on solving the problems associated with converting the models into machine language, and not on the design and construction of scenarios which are the basic elements of modelling and the subsequent simulation (guilhou & lagadec, 2002; prigogyne, 1977). particularly when a wide variety of multimedia devices exist in the market, offering the students opportunities to do things that would be otherwise impossible in the "traditional classroom" environments (schneps, 2014, harvard college observatory). from this perspective, the contribution of this paper is to understand the process of inquiry as a way of articulating the logical and instrumental reasoning, based on an iterative approach: student-teacher, student-content, student-student and studentcomputer, in order to improve the management of changes in students learning strategies without compromising scientific rigor in capturing specific ways to use the functionality of the system as well as capturing the different application scenarios. in order words, it emphasizes aspects of model designing and the use of the multimedia simulation tools already available. 4. change in the educational paradigm the premise accepted as the basis of education has gone from lecture based learning to another focused on the learner, grounded on the acquisition of vertical and transversal skills that strengthen the student´s ability of independent and guided learning. in this paradigm change, the idea is to abandon the search for absolute laws and theories in favour of finding approximate laws and theories, as truths are only in relation to the structure of dominant knowledge and methodology in use at that time, thus leaving open the possibility of being overcome in the future (popper, 1979; lakatos, 1978). learning centred on the "learner" and grounded in the idea of generating knowledge as approximations, by articulating the logical reasoning and instrumental skills (software), peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 43 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 including an assessment aimed at determining how to apply theoretical concepts and how to use operational and technological techniques, endorses systemic thinking. the fact that all advance in knowledge involves a reorganization of cognitive structures as a result of tests applied to statements taken as true so far, means the delimitation of its content, application and generalization (lakatos, 1978). this learning path enables understanding and consolidation of concepts and methods which are part of laws or theories used in the study of phenomena, and allows the subsequent model enactment (simulation) to explain the underlying feedback on learning and to interact with the various possible scenarios of reality. hence the theoretical thinking nestled along this discussion is systemic, and its enactment through simulation is understood as a paradigm of thought, made evident through certain structure of own language or conventions (andrade et al., 2001). the use of simulation as part of the learning process is due to the fact that humans better perceive the changes in situations represented in either temporal or spatial motion. so, if a student cannot decipher the exact mathematical formulas included in the models, then observing the relationship diagram and/or graphic, e.g. about climate, in motion, could enable him to predict certain events much faster than with other types of analysis2 . 4.1. systemic perspective on learning the difficulties that are generated in the process of knowledge acquisition (learning) are part of complex cognitive networks and of the "appropriation" of knowledge3 , which not only hinder, but also feedback, the learning process. the obstacles become evident in wrong answers due to the failed attempt to adapt knowledge, previously acquired, to new situations (sacas, 1997, werner et at, 1995), while feedback becomes evident in how the 2 today, models of weather forecasts tend to seek a balance between graphics of rain clouds and moving snow with maps which use numerical coordinates and numerical time tables of events (tables of coordinates of clouds and rain). 3 appropriating something and inserting it into the personal background. this is what significant learning does, producing useful long-life knowledge, in addition to transferring it to other fields of knowledge, to change reality learning creatively. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 44 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 student learns and appropriates knowledge, following the scheme: observation-intuitionreasoning-interpretation-observation. in this context, a concept that addresses the parts and the connections between the parts, to study the whole, will help to understand the complexity of the diversity of combinations of relations of one part to the other. it will also be useful to overcome obstacles and conceptual confusions that lead to stagnation and decline, by explicitly considering this feedback present in the acquisition of scientific knowledge. today, training in the development of conceptual and formal models and the use of digital simulators, are only included in masters and phd programs but are not in the curricula of university degrees. this situation has not been substantially modified so far, probably because of the level of knowledge required for its implementation in bachelor's degrees, particularly those abilities and skills related to formal modelling and coding (or machine language) so that they can be interpreted by a computer (evans, and fashan, 1993). advances in the development of "friendly" simulation programs based on the usercomputer-user interaction, open the door for the inclusion of methodological developments in the modelling of systems and/or scenarios, in the curricula of university degrees. this fact is reinforced by the wide range of simulation tools for academic and business fields, such as vensim, ithink/stella, anaylogic, evolution, powersim, smile, arena, promodel, siman, simular, among others. 4.2. new technology resources as teaching tools a to some extent new technologies are part of the daily activities of students and gradually install new practices and learning styles. in a context of continuous reduction in the costs of data collection and the generalization of digital storage, the relevance of the introduction of these new computing and communication resources lies in the actions or practices that are enabled by this technology itself, and the stimulus to organize the teaching and learning process in a different way, by using technology as a teaching tool. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 45 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 the growth of the "information technology" has created new types of data and data structures, which are faced by students in their personal, academic and social life, including techniques to collect their personal information, as well as their social or cultural group data, with the goal of monitoring the day-to-day life of these individuals or groups (facebook, twitter, linkedin, etc.). this makes today’s students live with the data and have a direct, first-hand experience. this experience also covers the use of search engines like google, yahoo, msn, gmail, bing, rtbot, duckduckgo, yacy, hakia, open directory project (odp), accoona, yandex, gennio, ask jeeves, apocalx, yippy, ixquick, kartoo, carrot, etc, technical (graphical, statistical, simulation, etc.) software, digital databases (big data, data mining, etc.), tools that could be used for academic learning and leisure; for example, in the preparation of the bachelor or master degree thesis (including topics like creating online companies, customer management, video game design, web positioning, dynamic marketing, ipad as a learning tool, etc.) . while the conversion of these resources in teaching tools is neither immediate nor automatic, its academic use is untapped, for example the possibility of studying the events with the whole population and not just a sample. the digital environment created by the it opens the real possibility of restructuring classroom teaching and the development of autonomous and self-guided learning styles. for example, using the internet to browse for data, bibliographical sources, procedures, estimates or opinions is a step towards this goal. however, the main activity in this environment is still leisure related. 5. how to organize the teaching-learning process differently our proposal to organize teaching and learning in a different way is based on significant changes in the way the student processes data and information to turn it into knowledge. thus, if the data filing can be accomplished in multiple ways, then the data only express peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 46 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 one part of reality, and they do not express reality itself. the information in turn can be understood as the meaning that the data acquires as a result of an intervention, i.e., a conscious and intentional action. the knowledge resulting from this interaction can be seen as an active and interactive understanding of reality, an activity that is not devoid of an axiological characterization and which has got historical and social dimensions. one implication of this metamorphosis is the outlining of the multidimensional interdependence relationships based on the spatial and temporal information feedback, making use of a calibrated modelling process at the different levels of learning (descriptive, explanatory and analytical). this transformation involves making explicit the connections of any type of intervention (by students or teachers) on databases and selected data sets, with the goal of getting information and then achieving new knowledge. in fact, accessing the former (data) is a necessary but not sufficient condition for the second (information), and accessing the second is a necessary, but not sufficient, condition for the third (knowledge), and so on, until an equilibrium where knowledge is consolidated and incorporated into the background of the students is reached. the pedagogical strength of this approach comes from the multi-perspective connections that students develop about the changes in the events or phenomena of interest. these multiple interconnections are specified in the modelling process, and can be understood as the different ways of seeing reality. for example, when watching a video, at standard speed, students can perceive the spatial-temporal nature of the changes in their global context and identify the complexity of the relationships between the changes produced as well as the limitations to carry out the measurements. in contrast, when watching it at slower pace they perceive other aspects of the problem that are not visible with the previous method. the scenario philosophy tells us that the investigation of the nature and the implications of the different perspectives or alternatives considered lets us visualize and identify the evolution of "future" behaviour of the phenomenon under study, considering different working hypothesis. thus, for example, the technique of scenario modelling can be a peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 47 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 useful tool to assess the management and application of concepts in defining variables, interrelationships, conceptual and mathematical formalization of behavioural relationships, as well as to explain the nature of the underlying feedback process, which affects the way knowledge is accumulated and the way prior knowledge is modified. it is also a good way to motivate overcoming superficiality and the prevailing relativism (the everything-is-allowed attitude) as it requires theoretical reflection, understanding of the dynamics of the problem and accuracy in the application of formalization, instrumental and operational techniques. 5.1. the pedagogical role of modelling th the pedagogical role of modelling refers to the competence that the student has to perform autonomously and conscious, throughout the modelling process in a given context (blomhᴓj, 2004). this is possible by linking the modelling process to the teaching-learning process, suggesting problems about different situations, for example, situations that are written literally, so that the student must read, analyse, interpret and model the overall situation in order to define a particular solution. logically, if a student faces a problem, in most cases he must go over the theory and investigate the nature of the problem in order to have conceptual elements that allow him to approach and begin the process of modelling. for example, the functions of real variables can be used as models of natural phenomena and problematic situations related to economics or statistics. in particular, exponential and logarithmic functions can be used, among many applications, to describe population growth, radioactive decay, compound interest or logarithmic adjustments and scales. pedagogical modelization is a cross competence in almost all disciplines, and its goal is the application of this method to the analysis of certain problem-situations, where a direct solution is not possible but it is necessary to propose alternative solutions (scenarios), based on the formulation of working hypotheses. the system modelling process begins by identifying the relationships and interrelationships between elements of the system peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 48 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 under study, and then it continues with the construction of the conceptual model for the behaviour relationships, which is nothing but an equivalent and logical representation of the real system, and as such constitutes a simplified abstraction of it. next it is translated into a model suitable for implementation on a computer, thus promoting the understanding of the underlying concepts, the control of variables, the formulation of conjectures and testing the student´s mental models. for example, stewart (2007) proposed modelization as a teaching and learning strategy in the field of mathematics to study calculus and differential equations. 5.2. the pedagogical role of simulation the pedagogical role of simulation has much in common with experimentation and/or "direct" experience with the problem under study; its value is that it allows assessing events which are often impossible or difficult to study in a laboratory, as in happens in many areas of knowledge, as they facilitate the dynamic representation of the system operation, and allows the visualization of the problem in its evolution and interaction. in academia, the interactive power of interface simulation allows a high level of interaction of students and instructors with the phenomena of interest, many of them with specific purposes associated with certain specific characteristics, such as interactive graphics environments and instructions for studying different types of scenarios (waiting lines, manufacturing processes, etc.). therefore, the simulations enable the student to extend and expand his cognitive process, helping him to address complex and ambiguous problems, give meaning to scientific knowledge, and to diversify his instrumental skills, especially those related to scientific and technological codes management. 6. construction, analysis and assessment of scenarios the purpose of the study of scenarios is not to predict future events, but to study, understand and create alternatives today, leading to achieve the desired objectives and peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 49 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 minimize the effect of the unwanted ones. hence, the claim of the modeller and the simulator is not to forecast, but to consider the many changing dynamics of chaotic emerging events, and to define and delimit the possible scenarios that are derived from them. in a complex world these characteristics imply that it is impossible to predict all possible states. in response, current scientific developments suggest to explore the multiple scenarios and possible trajectories, based on a set of assumptions that affect the scenarios. during the study and scientific learning of the different dimensions of the real world, it is usually not possible to find an analytical solution, so we turn to modelling and simulation as an alternative. modelization involves the formulation of conjectures or working hypotheses that determine the behavioural relationships and the interrelationships between system elements (logical, functional, and/or mathematical relationships) that take the form of a conceptual and/or formal model, although given the complexity of relationships in the real world, the "future" can only be explored. methodologies for scenario creation and simulation try to explore possible alternatives. although it is possible to compare certain aspects of the physical system with its formal or mathematical description, it will never be possible to establish an exact connection between them. the idealization of reality assumes that the observed data were generated according to certain logical and well-defined mathematical rules. from this perspective, the construction of "scenarios" is a conceptualization process, going from the possible scenarios, to the likely ones and then to the desired ones. the possible scenarios are all the conceivable ones, the likely scenarios are those with some chance of occurrence and the desired ones will be all those scenarios that can occur and are preferred over the others. therefore, the acceptance of a simulated model should be guided by its "usefulness" instead of its "veracity", both in its formal and its fictional description. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 50 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 7. the process of modelization and modelling creation our proposal aims to improve teaching and ensure that students acquire and manage certain concepts, develop procedural and instrumental skills, and above all not to perform activities of just copying the teacher’s instructions and explanations but to think, question, deduce and infer from them. the design of a learning environment based on conceptual and formal (mathematical) system modelling activities is the best way to improve the objectives pursued and from that to establish scenarios under certain hypotheses that will be analysed and evaluated through simulation. the procedure will be to follow the models or the different activities suggested by the teacher, with his help. then such assistance will be gradually reduced, guiding students to the systematic use of the content used by the teacher to ensure a shift of the interpersonal and collective competences towards the intrapersonal ones, so that the student is able to use them independently. the way to achieve this transfer includes 5 phases (law and kelton, 1999. chap. 5). phase 1. problem identification phase 2. dynamic hypothesis formulation phase 3. model formulation phase 4. validation of the model phase 5. scenario building phase 6. design, development and assessment of policies phase 1. problem identification. the early identification of critical factors in the process allows the definition of the relevant variables, parameters, and internal and external constants. in this process, among other things, dynamic behaviour, the agents involved in the system, feedback processes, and especially the behaviour of the environment are peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 51 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 discussed (gomez and dyner, nd). for example, in the normal foresight processes, after identifying the relevant factors, we proceed to weigh these factors considering their importance. the idea is that it is the model itself which determines which factors have the greatest impact. phase 2. dynamic hypothesis formulation. a dynamic hypothesis or conjecture is understood as the foundation on which the behaviour of a variable rests, sustained on a reasoning based on the feedback structure of the model that contains it, and on the model simulation using the appropriate software (steels et al, 2011). there are two possible modes for building dynamic hypotheses: estimative and explanatory. in the estimative mode a possible behaviour of the variable of interest arises from the structure of the model and a first dynamic hypothesis is proposed; then the behaviour proposed and the initial hypotheses formulated will be tested using the simulator. in the explanatory mode we start from an already known behaviour of the variable of interest and through structural analysis an initial hypothesis is proposed which is then tested and adjusted with simulation experiments. once this dynamic hypothesis is created, it is used for the definition of policies (design of a comprehensive and consistent set of structural changes) to promote the desired behaviour on the variable of interest. phase 3. model formulation. the development and elaboration of a model is the starting point for really testing the student´s or the teacher´s understanding of the problem. models promote system understanding ("insights") to infer and not necessarily calculate its behaviour and finally intervene on it. in this process, each variable must correspond to a concept in the real world and each behavioural equation must have dimensional consistency. in more advanced applications, the use of models that describe the relationships between the variables of the system in terms of mathematical expressions as differential equations or finite difference equationsin more detail will be necessary. these models can be characterized on the basis of certain features (continuous or discrete in time, with centred or distributed parameters, deterministic or random, linear or nonlinear, etc.). peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 52 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 the possibility of practical modelling testing using computers and construction and simulation software allows the student to continuously acquire conceptual and procedural knowledge. among the activities developed in the modelling process we can find: the design of conceptual and numerical schemes, scenarios simulation and sensitivity analysis, among others. (hestenes, 1987). modelization software has a relevant role in developing the student skills and abilities in the application of logical reasoning, in the conceptual understanding of the problem and in assembling relations and interrelations in a model (andaloro et al, 1991). simulation programs allow us to understand the dynamics of the interventions on the problem by creating different scenarios associated to the change of the working hypothesis. this form of computer supported learning is now developing in university education (de jong et al., 1992), providing a means to increase motivation and the range of topics and experiences that the student receives. generally, models and reality are related through two basic processes: abstraction and interpretation. hence, the first step in the modelling process is the abstraction of reality or situation under study. for that purpose it will be necessary to establish certain hypothesis, define variables, parameters and behavioural equations and finally develop appropriate mathematical operations to solve the problem. the second step will be to simplify the mathematical tools used. the results derived from the model should allow interventions on the real world. the next step will be to collect data of the effects in the model and compare them to the effects of interventions in the real world; if they don´t match, the data collected will be useful to modify the hypothesis; on the contrary, if the consequences of the intervention match those of the model, then the assumptions made are valid and so are the variables defined and the pre-set parameters. phase 4. validation of the model. the validation process consists of checking that the model meets the design requirements for which it was developed. it seeks to assess whether the model behaves according to the parameters set in its design. at this stage the differences between the operation of the model and the real system are also assessed by simulation. the most common ways to validate a model are: peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 53 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 1. the expert opinion. 2. the accuracy in the reproduction of the historical data. 3. the quality of the scenarios for the analysis. 4. model and simulation failure check this process of validation of the model runs on both the time domain and the spatial domain, or on both of them, including its assessment in extreme conditions, expressed as values of the scenario variables that will never be seen in the real world, for example, a null gdp, zero inflation, 100% birth or mortality rates, among others. under such conditions, the response of the model should not violate the laws (whether physical, economic, logical, etc.) that contextualize the problem. and it should be checked whether there is a relation between the variables that make up the model, the correspondence with reality of the concepts used in the model and its spatial and dimensional consistency. the other side of the validation process is studying sensitivity, which consists of determining the magnitude or degree of impact of changes in a domain or parameter on the relevant variables of the model. for example, we can use methods based on the linearization of variables to assess the sensitivity of these to changes in the cycles domain (forrester, 2011). that sensitivity is calculated from the coefficients eigenvalues, representing each variable as a linear combination of the others. these values express the rate of change in a variable as a result of changes in the range where model cycles are valid. phase 5. scenario building. a scenario is a method to guide the explanation of a system’s behaviour from the dynamics of its underlying structure. in the words of kahn (1967), they are descriptions of the possible "future", in which attention is paid to the causal processes and the decision points. according to kahn, the scenarios correspond to peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 54 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 two fundamental questions: how does a hypothetical situation in the "future" happen, step by step? and what alternatives exist for the different actors at the decision moment to prevent, divert or facilitate the process? a scenario can thus be defined as a set consisting of the description of a "future" situation and the trajectory of events which allow the situation to go from home to the "future" situation. the scenarios represent the symbiosis of modelization and simulation, built to test hypotheses or develop decisions structures, in different environments, so that when the decision maker faces the materialization of one of those "futures", he will be able to make decisions or actions to overcome the problem under study. it is a testing ground for ideas and a stimulus for new developments. therefore it is useful to organize guided learning, as a scenario can tell the story of the possible interventions and its consequences. this process of scenarios building has its corollary in simulation. simulation helps to investigate sets of working hypothesis, using models or "observations", which show the nature of the changes in the structure or in the scenarios parameters. through experimentation it seeks to identify changes in the pattern of the variables of interest behaviour to assess whether these feedback structures are dominant or not at some specific stage. the results of the experiments allow obtaining information that supports, contradicts or complements the initial dynamic hypothesis proposed. phase 6. design, development and assessment of policies. in the first place, the policymaking process involves the reclassification of exogenous variables, into scenario variables, i.e. variables which cannot be modified by the decision maker (student, teacher, client, etc.) and which contextualize the environment in which the system is immersed. and in the second place, despite being exogenous to the model, into decision variables which are subject to change by the user. experiments can be directed towards the development of tests, such as the deactivation of a cycle or a parameter modification. to deactivate a cycle one must proceed to temporarily override one of the existing causal relationships, in order to temporarily disable its feedback process and to examine its effect on the behaviour of the variable of peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 55 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 interest. attention should be paid to prevent this link to be part of another feedback structure to only assess the dominance of the cycle to be deactivated. changing parameters and auxiliary variables directly affects the behaviour cycles; thus the idea is to assess its dominance in certain stages of the variable of interest. 8. limitations of this type of studies and contribution to the learning practice obviously, the formulation of any proposal of autonomous and guided learning has certain limitations arising from the fact that the design of a model involves a structure with certain restrictions:  scenarios with missing information  scenarios with misinformation  scenarios with ambiguities  scenarios with contradictions  fully or partially overlapping scenarios these limitations are a consequence of the difficulties in defining the scenarios, visualizing the relationships and inter-relationships directly, identifying the situations that have arisen and finding the points of division and articulation of relationships, including delays in the impact of actions and the positive and negative feedback loops. unlike other proposals that only help to establish requirements concerning the studentmachine interface (hadad, gds, 2007; breitman, 1998, sutcliffe (1997), neglecting other aspects of the design and application domain, our proposal includes additional operations, also based on relationships, which are defined so that a hierarchical architecture of scenarios can be built in addition to investigating the evolution of these scenarios through simulation. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 56 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 this learning strategy is reinforced by the findings of the new studies by the harvardsmithsonian center for astrophysics (cfa) under matthew h. schneps leadership (harvard college observatory). the experimental evidence presented shows that the use of new technologies in the classroom provides significant learning gains among students who used the simulation applications of modern communication devices. for example, it is shown that students understand the unimaginable emptiness of space more effectively when they use the ipads (or other tablets) to explore 3-d simulations of the universe, compared to traditional teaching methods. 9. conclusions despite the obvious limitations of the human cognitive apparatus, models can be formally evaluated and they allow reaching more reliable conclusions about the system’s behaviour and the effectiveness of certain policies. the development of methods to explain the connection between behaviour and structure has strengthened the educational role of the modelization process as well as that of simulation using application software. studies on modelization as a teaching tool show that the "educational modelization" constraints are due not only to the weaknesses in the students´ scientific training but also to the teachers’ ignorance about the modelization process in both learning and scientific research. in educational terms, the perception of an endogenous explanation of a system’s behaviour, from its interdependence and feedback structure, promotes and encourages the development of the student´s systemic thinking. the use of new technologies as a simulation tool reinforces the change in the teachinglearning paradigm in the classroom. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 57 http://www.google.es/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=1&cad=rja&uact=8&ved=0ccaqfjaa&url=http%3a%2f%2fwww.cfa.harvard.edu%2f&ei=oezwu8tykuwc0awylohada&v6u=https%3a%2f%2fs-v6exp1-ds.metric.gstatic.com%2fgen_204%3fip%3d88.6.199.189%26ts%3d1408298042475208%26auth%3dyedmfpjoilim3dqjp76njhez7czca4m6%26rndm%3d0.4993944708005303&v6s=2&v6t=10909&usg=afqjcnerdgygrxz2pbpsy4_81ht1mztnsq&sig2=d1vhay0nwglpvt2hmsajla&bvm=bv.73231344,d.zgu http://www.google.es/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=1&cad=rja&uact=8&ved=0ccaqfjaa&url=http%3a%2f%2fwww.cfa.harvard.edu%2f&ei=oezwu8tykuwc0awylohada&v6u=https%3a%2f%2fs-v6exp1-ds.metric.gstatic.com%2fgen_204%3fip%3d88.6.199.189%26ts%3d1408298042475208%26auth%3dyedmfpjoilim3dqjp76njhez7czca4m6%26rndm%3d0.4993944708005303&v6s=2&v6t=10909&usg=afqjcnerdgygrxz2pbpsy4_81ht1mztnsq&sig2=d1vhay0nwglpvt2hmsajla&bvm=bv.73231344,d.zgu multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 the application of this learning strategy to a particular case will be part of future research on the limitations and progress in the use of modelling and simulation as teaching tools. 10. references acero, v., et. al. (2011). ¿cualitativo o cuantitativo? esa no es la cuestión: un método para el desarrollo de hipótesis dinámicas. ix congreso latinoamericano de dinámica de sistemas and ii congreso brasileño de dinámica de sistemas. andaloro, g., et al (1991). modelling in physics teaching: the role of computer simulation. int. j. science education, no. 13(3), pp. 243-254. andrade, h., et. al. (2009). framework para el desarrollo de ambientes software de aprendizaje y toma de decisiones con modelos en dinámica de sistemas. xiii congreso de informática en la educación. inforedu 2009. habana (cuba). bertrand de jouvenel (1967). the art of conjecture. editor basic books. michigan university. blomhφj, m. (2004). modelización matemática: una teoría para la práctica. revista de matemáticas vol. 23. 2. pp. 20-35, universidad nacional de córdoba. bouciguez, m.j. et. al. (2010). ambientes virtuales altamente interactivos para el aprendizaje de ciencias en contextos educativos. wicc 2010 – xii workshop de investigación en ciencias de la computación. braun, w. (2002). the system archetypes. the systems modeling workbook, w. braun, ed. coyle, g. (2000). qualitative and quantitative modelling in system dynamics: some research questions. system dynamics review, vol. 16, pp. 225-244. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 58 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 forrester, nathan (1983). eigenvalue analysis of dominant feedback loops. 1st international conference of the system dynamics society, chestnut hill, united states. gilbert, j.k. (2004). models and modelling: routes to more authentic science education. international journal of science and mathematics education, 2(2), pp.115-130. godet, m. (1991). from anticipation to action. a handbook of strategic prospective. unesco publishing. spanish version: de la anticipación a la acción. manual de prospectiva estratégica. barcelona. editorial marcombo s.a. spain. 1993. godet, m. and durance, p. (2011). la prospectiva estratégica: para las empresas y los territorios. dunod and unesco 2011. gómez, d., et. al. (2009). definición sistémica y simulación dinámica de escenarios: aplicación a la economía colombiana 2002-2007. resource document. http://www.ecsim.org/ vista/archivos/metodologia%20y%20escenarios%20colombia.pdf. guilhou, x. and lagadec, p. (2002). la fin du risque zero. eyrolles, editions d'organisation. france. hadad, g. d. s. (2007). uso de escenarios en la derivación de software. phd. dissertation. facultad de ciencias exactas. universidad nacional de la plata. argentina (november). hestenes, d (1987). how the brain works: the next great scientific revolution. in: c.r. smith and g.j. erickson (eds.), maximum entropy and bayesian spectral analysis and estimation problems (reidel: dordrecht/boston). p. 173–205. huang, j. et al. (2010). an extension of loop deactivation in the behavioural method. 28th international conference of the system dynamics society, seoul, korea. kahn, h. and wiener, a. (1967). the year 2000: a framework for speculation on the next thirty-three years. macmillan. isbn 0-02-560440-6. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 59 http://www.eyrolles.com/accueil/editeur/6/eyrolles.php http://www.eyrolles.com/accueil/editeur/18/editions-d-organisation.php http://www.eyrolles.com/accueil/editeur/18/editions-d-organisation.php multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 krajcik, j., et. al. (2000). un andamiaje de herramientas tecnológicas para promover la enseñanza y el aprendizaje de ciencias. in c. dede (comp.). aprendiendo con tecnologías. editorial paidós. buenos aires, argentina. lakatos, i. (1976). proofs and refutations. cambridge: cambridge university press. isbn 0-521-29038-4 lakatos, i. (1978). the methodology of scientific research programmes. philosophical papers volume 1. cambridge: cambridge university press. law, a. m. and w. d. kelton. (1999). simulation modeling and analysis. third edition, mcgraw-hill, new york (december). medina vásquez, j. and ortegón, e. (1997). prospectiva: construcción social del futuro, instituto latinoamericano de planificación económica y social (ilpes). universidad del valle, cali, colombia. peñaloza figueroa j.l. and vargas pérez, c. (2006). (http://orcid.org/0000-0001-91097247 and http://orcid.org/0000-0002-2905-4569) ¿qué debe cambiar en el aprendizaje de la estadística en las ciencias del comportamiento? xiv taller y ii encuentro internacional de la asociación española de profesores universitarios de matemáticas para la economía y la empresa. asepuma. peñaloza figueroa j.l. and vargas pérez, c. (2009). (http://orcid.org/0000-0001-91097247 and http://orcid.org/0000-0002-2905-4569). acquisition of skills and supervised and unsupervised learning of inferential statistics. 1st international conference on education and new learning technologies. edulearn09 proceedings http://library.iated.org/publications/edulearn09. barcelona, spain. 6-8 july. http://library.iated.org/related/penalozafigueroa2009acq. publisher: iated. isbn: 97884-612-9801-3 / issn: 2340-1117. peñaloza figueroa j.l. and vargas pérez, c. (2010) (http://orcid.org/0000-0001-91097247 and http://orcid.org/0000-0002-2905-4569). numeric indicators for competencebased evaluation in statistical learning. 4th international technology, education and peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 60 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0002-2905-4569 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0002-2905-4569 http://library.iated.org/publications/edulearn09 http://library.iated.org/related/penalozafigueroa2009acq http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0002-2905-4569 http://library.iated.org/view/penalozafigueroa2010num http://library.iated.org/view/penalozafigueroa2010num multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 development conference. inted2010 proceedings. http://library.iated.org/publications/inted2010. valencia, spain. 8-10 march, 2010. http://library.iated.org/view/penalozafigueroa2010num. publisher: iated. isbn: 97884-613-5538-9 / issn: 2340-1079. peñaloza figueroa j.l. and vargas pérez, c. (2013). (http://orcid.org/0000-0001-91097247 and http://orcid.org/0000-0002-2905-4569). modeling and simulation workshops as a learning tool and conceptual and relational consolidation. 7th international technology, education and development conference. inted2013 proceedings. http://library.iated.org/publications/inted2013. http://library.iated.org/?search_text=publication%3ainted2013&adv_title=&rpp=25&a dv_authors=pe%c3%91aloza&adv_keywords=&orderby=page&refined_text=. valencia, spain. 4-5 march, 2013. http://library.iated.org/view/penalozafigueroa2013mod. publisher: iated. isbn: 978-84-616-2661-8 / issn: 2340-1079. prigogyne, ll. (1997). el fin de las certidumbres, taurus. madrid. popper, k r, (1972). objective knowledge: an evolutionary approach, oxford-clarendon press. popper, k.(1934). the logic of scientific discovery. new york: routledge, 2009. isbn 10 0-415-27843-0. popper, k. (1934). la lógica de la investigación científica. translated by víctor sánchez de zavala (1ª edición). madrid: editorial tecnos, 1962. isbn 84-309-0711-4. schoemaker. paul j.h. (1995), “scenario planning: a tool for strategic thinking”, in: sloan management review, winter sterman, j. (2000). business dynamics. systems thinking and modeling for a complex world. boston, ma, usa. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 61 http://library.iated.org/publications/inted2010 http://library.iated.org/publications/inted2010 http://library.iated.org/view/penalozafigueroa2010num http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0002-2905-4569 http://library.iated.org/view/penalozafigueroa2013mod http://library.iated.org/view/penalozafigueroa2013mod http://library.iated.org/publications/inted2013 http://library.iated.org/?search_text=publication%3ainted2013&adv_title=&rpp=25&adv_authors=pe%c3%91aloza&adv_keywords=&orderby=page&refined_text http://library.iated.org/?search_text=publication%3ainted2013&adv_title=&rpp=25&adv_authors=pe%c3%91aloza&adv_keywords=&orderby=page&refined_text http://library.iated.org/view/penalozafigueroa2013mod multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2245 social and technological sciences issn: 2341-2593 stewart, james, lothar redlin. y saleem, watson (2007). precálculo, matemáticas para el cálculo. 5th. ed., cengage learning. vargas pérez, c. and peñaloza figueroa j.l. (2013). (http://orcid.org/0000-0001-91097247 and http://orcid.org/0000-0002-2905-4569). modeling and simulation workshops as a learning tool and conceptual and relational consolidation. 7th international technology, education and development conference. inted2013 proceedings. http://library.iated.org/publications/inted2013. http://library.iated.org/?search_text=publication%3ainted2013&adv_title=&rpp=25&a dv_authors=pe%c3%91aloza&adv_keywords=&orderby=page&refined_text=. valencia, spain. 4-5 march, 2013. http://library.iated.org/view/penalozafigueroa2013mod. publisher: iated. isbn: 978-84-616-2661-8 / issn: 2340-1079. villa o., et. al. (2007). el proceso de modelación matemática en las aulas escolares. a propósito de los 10 años de su inclusión en los lineamientos curriculares colombianos. octavo encuentro colombiano de matemática educativa. resource document. grupo de investigación en educación matemática e historia (udea, eat), university of antioquia. peñaloza and vargas (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 40-62 | 62 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0001-9109-7247 http://orcid.org/0000-0002-2905-4569 http://library.iated.org/view/penalozafigueroa2013mod http://library.iated.org/view/penalozafigueroa2013mod http://library.iated.org/publications/inted2013 http://library.iated.org/?search_text=publication%3ainted2013&adv_title=&rpp=25&adv_authors=pe%c3%91aloza&adv_keywords=&orderby=page&refined_text http://library.iated.org/?search_text=publication%3ainted2013&adv_title=&rpp=25&adv_authors=pe%c3%91aloza&adv_keywords=&orderby=page&refined_text http://library.iated.org/view/penalozafigueroa2013mod multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 26 modeling of impact dynamics and application in public security education rodríguez-millán, marcos1*, marco, miguel1, loya, josé antonio 2, moure, fernando3, miguélez, maría henar 1 1department of mechanical engineering, university carlos iii of madrid, avda. de la universidad 30, 28911 leganés, madrid, spain 2department of continuum mechanics and structural analysis, university carlos iii of madrid, avda. de la universidad 30, 28911 leganés, madrid, spain 3centro universitario de la guardia civil, avda. de la princesa s/n, aranjuez, madrid * corresponding author: email: : mrmillan@ing.uc3m.es; phone number: +34916245860 received: 2015-03-24; accepted: 2015-08-21 abstract this work presents a methodology for interfacing with the students in order to study impact problems. the results are derived from the bachelor thesis developed at the end of the fourth course in the security engineering degree. the knowledge regarding security topic and the use of new analysis tools (commercial finite element solver) have proved to be extremely useful for future guardia civil officers. keywords security engineering degree; impact; teaching methodologies; bachelor thesis multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 27 1. introduction the threats to the public security and premises security are on rise because of increasing terrorism and violence. in this sense, the protection of civil infrastructures and critical industrial facilities are topics of increasing relevance to defence agencies and governments. three key applications of armor systems may be considered in the security industry: (1) personal protection, including body armor and helmets, (2) vehicle armor, and (3) transparent armor. for each of these applications, specific requirements play an important role in the armor design and thus the ultimate choice of protection materials ensure that these systems are as lightweight as possible (report, 2011a). impact dynamics is strongly related with physical security of persons and infrastructure. the analysis of impact processes is required for evaluation of threats, performance of protections and other important issues for law enforcement members. the governing variables of the impact problem: target and projectile characteristics (geometrical and mechanical) and actual impact conditions (initial impact velocity). the election of protective material is a key factor in armor design. for instance, the largest part of protection materials in the automotive sector is metallic, principally steel and aluminum alloys (report, 2011a; wilson et al., 1988). however, many hybrid systems are currently using in this field, which combine metals, polymers and plastics, with or without reinforcements (mosse et al., 2006). on the other hand, the impact resistance of high-strength fabrics makes them desirable in applications such as protective clothing for military and law enforcement personnel. in this regard, aramid fibers as kevlar (dupont) and twaron (teijin) or ultra-heavy molecular weight polyethylene (uhmwpe), such as spectra or dyneema has been until now the most often used (tabiei and nilakantan, 2008). the phenomenon of the process of perforation is often analyzed on terms of ballistic limit. the minimum velocity for a complete perforation of the target is called the ballistic multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 28 limit vb. at higher impact velocity (v0>vb), the velocity after perforation is called residual velocity vr. the ballistic curve is described by initial impact velocity and residual impact which provides the essential information to enable the design of an efficient passive safety structure to resist an high velocity projectile of low mass penetrating a target. generally, the information of impact process is obtained by experimental tests; however; there are numerous parameters to analyze and consequently, an in-depth experimental study of the effect of each of these parameters would be extremely time consuming and expensive. the most common way to solve these inconveniences is to use numerical simulations because they provide a rapid and less expensive way to evaluate new design ideas. in the particular case of impact, it is easily possible to define a set of initial and boundary conditions. the mechanical behavior of materials must be defined in order to simulate impact problems. in addition to defining an adequate finite element mesh, an important aspect of conducting successful penetration simulations is the use of adequate material failure models. the knowledge about impact process, described previously, is key importance in the development of future guardia civil officers learning. they could be drawn into such departments. for instance: forensic science has needed to be adapted to new challenges in different areas in order to cover all aspects detected in crime scene such as understanding of projectile direction. other example is traumatic injuries resulting from vehicular accidents or assaults; has been recently reported in the literature (ni et al., 2013; marco et al, 2015). this work focuses on the bachelor thesis of students developed at the end of the fourth course in the security engineering degree, trying to go further in the understanding related to security, different industries and scientific field using a finite element code. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 29 the next section shows a brief description about the relation between university carlos iii of madrid and guardia civil university centre. the next sections show different bachelor thesis which were developed for students: the influence of shape projectile in the design of personal protection or the influence of layout of polycarbonate-aluminum in the design of crashworthiness structures, among others. these studies serve to link public, industrial and security interests and awake the interest and curiosity for research in the future guardia civil officers. 2. the institutions: guardia civil and university carlos iii. the guardia civil is a spanish national military law enforcement institution under the home affairs ministry, focused on the protection of the free exercise of rights and freedoms and to ensure public safety. since its foundation in 1844, it has been participating in the resolution of the main security issues affecting spain as a state both nationally and internationally. currently it is one of the most respected law enforcement agencies in the world owing strong international projection. the institution is the final user of advanced technology for security. the university carlos iii of madrid (uc3m) was established by an act of the spanish parliament on 5 may 1989, within the framework of the university reform act of 1983 (boe nº108, 1989). from the outset it was intended to be a relatively small, innovative, public university, providing teaching of the highest quality and focused primarily on research. both engineering and social science studies are offered at this university. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 30 3. methodology for interfacing with the students. the students used knowledge learned in different subjects of the security engineering degree. lightweight protection for mobile systems and strength of materials are two subjects focus on impact mechanics. students learn the principles of strength of materials such as stress, strain, inertial moment, yield stress, among others. in lightweight protection for mobile systems, students acquire the basic knowledge for understanding the keys to design and analyse impact processes. simple analytical and numerical models are explained. these approaches are developed for different materials and ranges of velocities according to needs of future guardia civil officers. once they have passed these subjects, students have a main idea in order to choose between different bachelor theses offered by experts in impact phenomenal of university carlos iii of madrid. the main drawback of development these bachelor theses is the interaction between students of guardia civil university centre (gcuc) and experts of university carlos iii of madrid (uc3m). the students, as future officers, have certain obligations subject to guardia civil institution. a methodology developed in order to solve these details as is shown in figure 1. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 31 figure 1. schedule of interactions between guardia civil university centre (gcuc) and experts of university carlos iii of madrid (uc3m). three different ways have been thought to get a good interaction between students of gcuc and experts of uc3m: (1) emails and video conferences, (2) the experts visit guardia civil university centre and (3) the students visit university carlos iii of madrid. use of email and video-conference tools is most common method of communication due to its ease-of-use and quickly response. they usually use this method during the development of their bachelor theses. however, to hold a meeting of supervisor and students in the first steps of their bachelor theses development are needed. students must understand the different parts of their study: aims, the significance of their work/research to gcuc, original contributions… their supervisors help students to find these goals. in addition, an advanced knowledge to using commercial software of finite element method (fem) is needed in order to bachelor theses about impact problems get gcuc uc3m questions, doubts,… answers, suggestions,… initial indications fem training course ( 12h aprox.) specific problems, rehearsals before their presentations solve problems, help their presentations initial indications fem training course ( 12h aprox.) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 32 successful. for this reason, a fem training course focusing on impact problems is developed for students and carried out in gcuc installations. it is conducted in 12 hours. after this course, students must work in their researches but they are always guided by their supervisors. however, the analysis of impact process is a complex problem when difficulties arise, it is necessary to organise a meeting. these meetings are usually in the installations of university carlos iii of madrid. finally, some examination drills are carried out in order to students gain self-confidence. 4. -the bachelor thesis and results given the recent rise in terrorism, civil and international conflicts, the number of people afflicted with war-related traumatic injuries is set to increase. the improvement of personal protections under ballistic and explosive threats is of great interest to guardia civil. ergonomics, lightweight and security are the main requirements that personal protections must comply. figure 2 shows a schedule for design new armour/protections. bachelor theses are developed using numerical models due to the high required cost need for performing experimental tests. the numerical models are validated with data from literature. the layout work is similar to shown in figure 2. firstly, it is important to classify the type of protection: combat helmet, body protection, energy absorption structural element. the following is to choose material/s for the protection. then, the development of numerical model which is adjusted to actual prototypes. once numerical model is validated with data from literature, numerous analyses may be carried out in order to go further the knowledge for different frameworks: geometry of projectiles, incidence angle, initial impact velocities… multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 33 figure 2. schedule for armor development (report, 2011b). different cases performed by the students of the ccuc developed during 2013/2014 are described below. 4.1: numerical analysis of ballistic limit of single and layered steel plates. steels have been used widely used in armour designs for vehicles due to be less constrained in thickness. this allows using single and layered aluminium target plates with or without spacing, figure 4. the idea of using layered plates instead of a single one in order to increase the ballistic perforation resistance is not new, and the effect of using targets made up of several thinner plates has been investigated in the literature for a long time (corran et al., 1983; britain, 1998; dey et al., 2007; deng et al., 2013). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 34 figure 3. example of perforation of separated thick target impacted by a blunt projectile. [11] in this bachelor thesis, a numerical analysis of different parameters of impact problems was investigated: material behaviour, incidence’s angle, thickness and air gap separation. armox t500 and aisi 4340 were selected due to its different stiffness and they are widely used in the security industry. both materials show a high strength, however armox t500 shows low strain hardening and limited ductility. this fact is not clear in order to predict the ballistic behaviour on perforation plates (rodríguez-millán et al, 2014). the plates was impacted by using ballistic impacts of a full-metal jacketed (fmj) bullet because it is currently one the most munition used for instance in “kalashnikov ak-47”. abaqus/explicit finite element code is used to simulate the perforation process. the thermoviscoplastic material behavior of the plates and fmj bullet were defined using the johnson-cook model (johnson and cook, 1983). the different studies carried out in this bachelor thesis are describing bellow. i) influence of material behaviour and thickness of plate on perpendicular impact test. this analysis was carried out in order to find the ballistic limit varying the thickness of plates for an initial impact v0≈ 750 m/s. aisi 4340 is a steel less strength than armox 500t and this was reflected in figure 4. the thickness required to prevent the perforation multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 35 in aisi 4340 plate was more than three times than armox 500t. the low strain hardening and limited ductility of armox t500 may be the reason. figure 4. residual velocity versus thickness of plate for armox 500t and aisi 4340. ii) influence of angle of obliquity on impact tests. the perforation resistance of armors impacted at certain obliquity is common described by the equivalent protection factor (epf), defined as the ratio of the areal density providing protection against oblique impact to the real density providing protection against normal impact. the epf is commonly plotted against the angle of obliquity of attack as is shown in the figure 5. this analysis revealed the reduction in armor thickness requires an increase in the incidence’s angle of impact. 0 100 200 300 400 500 600 700 800 0 5 10 15 20 r e si d u a l v e lo ci ty , m /s thickness of plate, mm velocidad residual (armox) velocidad residual (aisi) armox 500t aisi 4340 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 36 figure 5. epf versus obliquity impact for armox 500t. iii) influence of layout in separated layered plates for a given initial impact velocity. an analysis of layout of layered plates for armox 500t and aisi 4340 was studied for normal and obliquity impact. the thicknesses were 4 mm and 3.2 mm for armox 500t and aisi 4340 respectively. the air gap between both plates was 10 mm for two incidence angles (900 and 600) because this distance may be considered inside the vehicles. figure 6 shows the final stage of the perforation process for the different cases studied. this study revealed that armox-asis 4340 is the best layout configuration in terms energy absorption. thus, the more ductile material (asis 4340) is better to place in the back of the layout in the design of absorption energy structures. 0 0,2 0,4 0,6 0,8 1 1,2 0 10 20 30 40 50 60 70 e p f incidence's angle (0) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 37 figure 6. the final stage of the perforation process for: a) normal impact and armox 500t-aisi 4340 configuration of layout, b) normal impact and aisi 4340-armox 500t configuration c) obliquity impact and armox 500t-aisi 4340 configuration and d) obliquity impact and aisi 4340-armox 500t configuration. 4.2: impact behaviour of welded metal shields: numerical study armour protection structures are susceptible to failures along welds and joints, and simulations of these events are not yet reliable. although, this is a complex problem, a numerical analysis has been developed in this bachelor thesis in order to study the influence of obliquity in the impact process. the impact tests were analysed using the a) b) c) d) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 38 explicit solver of the finite element code abaqus. the target (aisi 4340) and fmj bullet were modelled using the johnson–cook constitutive relation and fracture criterion. figure 7. example of impact process on welded plates. figure 7 shows a sequence of images during the impact process in the weld zone. some of the most relevant results developed in the bachelor thesis are in figure 8. the residual velocity decreases as incidence angle increased up to 60º. this fact is associated to an increase of effective thickness which is varied with incidence angle. however, ricochet phenomenon is revealed beyond 60º, in other words, the interaction between projectile and surface is lower and therefore, the projectile does not get perforate the plate. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 39 figure 7. residual velocity versus incidence angle on welded plates. 4.3: numerical investigation on the impact behaviour of multi-layered composite plates of polycarbonate and aluminium. in this bachelor thesis, a numerical model is developed in order to study the influence of layout in multi-layered composite under impact loading. sandwich material systems – as special hybrid materialsare made by plates of polycarbonate and aluminium. they can combine the advantages of miscellaneous materials (e.g. low density, high bending resistance, energy absorption, high load-capacity at low weight) with each other (librescu and hause, 2000). three-layered example of metal/polymer/metal sandwich systems is hylite (carradò et al., 2011). hylite is an aluminium/polypropylene/aluminium sms with thicknesses 0.2/0.8/0.2 mm which was introduced into the automotive market through the audi a2 as is shown in the figure 8. r e si d u a l v e lo c it y , m /s incidence angle, (º) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 40 figure 8. location of sandwich material systems (hylite) in audi a2 (display.3acomposites, 2015) six different layout of polycarbonate (pc)-aluminum alloy (aa) plates were analysed: (i) pc-aa-pc-aa, (ii) pc-aa-aa-pc, (iii) pc-pc-aa-aa, (iv) aa-aa-pc-pc, (v) aa-pc-aa-pc and (vi) aa-pc-pc-aa. abaqus/explicit finite element code is used to simulate the perforation process. the thermoviscoplastic material behavior of the plates was defined using the johnson-cook model (johnson and cook, 1983). the projectile is defined by an analytical rigid body since experimental tests revealed no plastic deformation on the projectile-surface after impact. this definition allows reducing the computational time required for the simulations. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 41 figure 9. comparison between experimental test and numerical model in terms of residual velocity-initial impact velocity. the numerical simulations were compared to experimental data developed by uc3m as is shown in the figures 9-10. figure 9 shows a comparison between numerical results and experiments in terms of ballistic curve vr –v0. a good agreement is observed between numerical simulations and experiments. in addition to the ballistic curve, local deformation was estimated, figure 10. petalling mechanism was revealed for all plates. figure 10. comparison of final stage of the perforation process for the pc-aa-aa-pc configuration between experimental tests and numerical simulations. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 42 from analysis developed in this bachelor thesis some conclusions are revealed. aa-pcpc-aa is the best layout configuration in terms energy absorption. thus, the more ductile material (pc) is better to place in the middle of the layout in the design of absorption energy structures. johnson-cook model has proved insufficient in order to simulate the thermoviscoplastic behaviour of polycarbonate plate. 4.4: numerical study of different aramid fibers under normal impact process. composites have become increasingly important in defence and security industries in the last years. the use of these materials in combat helmets, body protections and combat vehicles requires an exhaustive analysis of their behaviour in order to satisfy the safety requirements. one of the main challenges of modern personal protection is the optimization for energy absorption. in this sense, personal protections are usually based on fibre reinforced polymer composites, especially kevlar and twaron fibres due to its high stiffness, light weight and high energy absorption capacity. in this bachelor thesis, a comparison between different aramid fibers was developed using abaqus/explicit finite element code to simulate the normal perforation process. the modelling of composite material in a code of finite element is hard task. however, possible simplifications of mechanical behaviour of composites may be carried out by the development of shell models. the parameters for modelling of aramid fibers were found in the literature (gower et al., 2008; talebi et al, 2009). figure 11 shows the residual velocity versus initial impact velocity for three different aramid fibers: kevlar 29, kevlar 129 and twaron. this study revealed that twaron has more energy absorption capacity for the used boundary conditions. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 43 figure 10. residual velocity versus initial impact velocity for three different aramid fibers. 5. conclusions the experience of development of bachelor thesis structure, evaluation and topics has constituted a challenge for both institutions guardia civil and university carlos iii of madrid. the perception of students and advisors has been highly positive. students have the opportunity to develop real applications of the knowledge and abilities acquired during their degree. the contact with operative units of guardia civil seems crucial to achieve this objective, proposing to impulse some specific topics to be rapidly adapted in their units. ackowledgements the authors acknowledge the financial support for the work to the ministry of economy and competitiveness of spain under the project dpi2011-25999. 0 5 10 15 20 25 200 250 300 350 r si d u a l v e lo ci ty , m /s initial impact velocity, m/s kevlar k29 kevlar k129 twaron multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 44 6. references boe nº 108, may 6 1989, creación universidad carlos iii de madrid (p. 13323). http://www.boe.es/boe/dias/1989/05/06/pdfs/a13323-13323.pdf britain, g. (1998). on the ballistic resistance of multilayered targets with air gaps. international journal of solids and structures. 35(23): 3097–3103. carradò a., faerber j., niemeyer s., ziegmann g., palkowski h. (2011). metal/polymer/metal hybrid systems: towards potential formability applications, composite structures, 93 (2): 715-721. corran, r. s. j., shadbolt, p. j., & ruiz, c. (1983) impact loading of platesan experimental investigation. international journal of impact engineering, 1(l): 3–22. deng, y., zhang, w., & cao, z. (2013) experimental investigation on the ballistic resistance of monolithic and multi-layered plates against ogival-nosed rigid projectiles impact. materials and design, 2013;44, 228–239. dey, s., børvik, t., teng, x., wierzbicki, t., & hopperstad, o. s. (2007). on the ballistic resistance of double-layered steel plates: an experimental and numerical investigation. international journal of solids and structures, 2007; 44(20), 6701–6723. display.3acomposites (2015). http://www.display.3acomposites.com/es/productos/hylite /caracteristicas.html accessed march 2015. gower, h. l., cronin, d. s., plumtree, a.(2008) ballistic impact response of laminated composite panels. international journal of impact engineering. 35(9): 1000–1008. johnson g.r., cook w.h. (1983), a constitutive model and data for metals subjected to large strains, high strain rates and high temperatures, proceedings of 7th international symposium on ballistics, 541–547. http://www.display.3acomposites.com/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 45 librescu l, hause t. (2000) recent developments in the modelling and behaviour of advanced sandwich constructions: a survey. compos struct. 48:1–17. marco m, rodríguez-millán m., santiuste c., giner e., miguélez m.h.. (2015) a review on recent advances in numerical modelling of bone cutting, journal of the mechanical behavior of biomedical materials. 44: 179-201. mosse l, compston p, cantwell wj, cardew-hall m, kalyanasundaram s. (2006) stamp forming of polypropylene based fibre-metal laminates: the effect of process variables on formability. j mater process technol. 172(2):163–8. ní a, cassidy m., curtis m., destrade m., gilchrist m. d. (2013), a combined experimental and numerical study of stab-penetration forces, forensic science international 233:7–13. report (2011a). opportunities in protection materials science and technology for future army committee on opportunities in protection materials science and technology for future army applications national materials advisory board and board on army science and technology division on engineering and physical sciences. report (2011b). opportunities in protection materials science and technology for future army applications. national academy of sciences. rodríguez-millán, m., vaz-romero, a., rusinek, a., rodríguez-martínez, j. a., & arias, a. (2014). experimental study on the perforation process of 5754-h111 and 6082-t6 aluminium plates subjected to normal impact by conical, hemispherical and blunt projectiles. experimental mechanics, 54(5), 729–742. tabiei, a., nilakantan, g. (2008) ballistic impact of dry woven fabric composites: a review. applied mechanics reviews. 61(1), 010801. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3693 social and technological sciences eissn: 2341-2593 rodríguez-millán et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 26-46 | 46 talebi h., wong s.v., hamouda a.m.s. (2009) finite element evaluation of projectile nose angle effects in ballistic perforation of high strength fabric, composite structures. 87(4):314-320 wilson d.v (1988). aluminium versus steel in the family car the formability factor. journal of mechwork technology. 16:257–277 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 60 tutoring social sciences learning prades plaza, sara universitat jaume i email: sprades@uji.es av. vicent sos baynat s/n 12071 phone: +34964729948 abstract this paper seeks to vindicate the role of mentoring in teaching, having proven to be a tremendous tool for effective communication and personal relationship with individual students. exhibited an innovative educational experience conducted with students of degree in early childhood education from the universitat jaume i, trying to promote the continuous character of the orientation process from the first years of training to adult life and employment, including the importance of enhancing the relation between the different educational levels. we also wanted to expose that much more than the task of transmitting information and knowledge, mentoring is essential in all educational models that are based on self-learning new trends, including the bologna process. therefore, we believe that mentoring is the crux of a new learning model based on personal and professional autonomy. this approach to work is based on a philosophy of autonomous learning that aims to meet the need for cognitive maturation of university students and improve their educational guidance. with this experience teaching innovation we have sought to improve our academic methodology, giving a strong weight to the tutoring, both onsite and virtual. we also have tried to encourage teamwork in groups and independent learning. thanks to these improvements, we have continuously evaluated the students, while students self evaluate their learning process. in summary, the proposal of educational innovation that we have carried out has shown the need to promote a system of mentoring to advise, guide and support students in their learning process. keywords tutoring, mentoring, self-learning, teaching task, innovation in education, students’ affairs. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 61 introduction one of the problems that face our students, especially in the early years of the degree, is disorientation, typical of those who enter a new educational level, with different teaching methods than what they have known until that time. therefore, the main objective of the experience presented here is to improve the educational guidance of teacher students. that objective has been achieved through mentoring, a teaching strategy that we consider underused by students and lecturers. the choice of this problem is justified mainly for two reasons. on the one hand, many professors consider mentoring as a supplement to their activity, that it isn’t valued and well paid. the current situation caused by the economic crisis and budget cuts affecting the spanish university is causing in a major sector of lecturers a strong reluctance to carry out tutoring, considering it as an unpaid function. in addition, this perception deepens in existing models of accreditation and promotion of university professors, taking into account the teaching and research tasks, but not efforts devoted to mentoring. as for the students, one of the issues that they appreciate of their lecturers, as is clear from several surveys, is the role of motivation and guidance, both personal and professional or academic. they value that lecturers worry to promote high expectations among students and that they be able to empathize with them to treat their problems in an appropriate way. with these premises, we started our experience to demonstrate that mentoring should be within the teaching, the most powerful tool of communication and personal relationship between individual students and lecturers. this is to promote the continuous character of the orientation process, from the first year of university until adult and working life, and enhancing the importance of traffic between the different educational levels. we also would like to expose that mentoring serves for reporting and transmitting knowledge, but also promotes self-teaching, in which new trends of teaching innovation are based, including the bologna process. at last, we try to contribute that lecturer understands this new professional status of their teaching role, who has become a guide that accompanies the student, who is the true protagonist of the teaching-learning process (álvarez y gonzález: 2005, p. 2). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 62 1. mentoring: an essential tool in the teaching-learning process in the literature on teaching methods and strategies, we can find numerous definitions of mentoring which are determined by the concept of education and guidance that each author has (arbizu, lobato y castillo: 2005, p. 5 et seq.). in our case, we believe that guidance is a formative intervention and a teaching task in the student's learning process. some of the issues that promotes the new european higher education area are selflearning and personal sufficiency, independent problem solving and team-working in university students. through mentoring, both individually and in groups, these goals can be achieved successfully. in addition, you can set aside the transfer of knowledge from the lecturer, rote learning and the control of it with exams because the mentoring develops the basic skill of learning to learn. we understand that mentoring is a university student right and a duty of the professor, but beyond this we consider the condition of mentoring as a space where dialogue can resolve students and lecturer’s doubts in the learning process. so, it should serve to optimize student learning and lecturer teaching. we believe it should be an opportunity for the academic and scientific dialogue on the subject that links students and lecturer, on their learning objectives, contents, assessment criteria and references, among other issues. tutoring may be in person or online, which enables a closer relationship between both sides, although it supposes a greater impersonality because they are not together physically. virtual tutoring solves problems of students that, due to their shyness, would have never asked and solves specific issues that arise during the learning task. also, this type of tutoring saves waiting time because the lecturer can attend student’s doubts at all times (garcía valcárcel: 2008, p. 9). in either of the two varieties, we believe that mentoring should be a space of recommendations not only academicals but also personals and professionals. in the same way, we believe that to carry out this advising task, lecturers should be trained in terms of orientation, because they have to supervise the student, to accompany him and guide him beyond the classroom in order to combat school failure. (garcía nieto: 2008, p. 193). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 63 therefore, one of the key aspects of the work of teaching that is too often neglected is communication with their students to advise and guide them through the tutorials. we understand that today the main job of the teacher is to guide students in the teachinglearning process in which students are the protagonists. thus, the lecturer helps students to plan their academic development, professional and social, in the early years of the degree, when students are disoriented because of changes of this new educational level, and in last years of the degree, when the immediate laboral future generates uncertainties of all kinds. that is why the lecturer must be a motivator professional, plus a good counsellor, concerned about the learning of their students. similarly, the lecturer must be a professional reference for students, with that purpose he will need to have extensive knowledge of the degree to help the student to choose for electives, in order to assist him in career opportunities that exist in the profession for which students are being trained. the lecturer also has to be experienced in the use of management and advisory services offered by the university. by contrast, for the lecturer this closer relationship with students is an enormous gratification, because he can learn more about the characteristics of their students. at the same time, through this close relationship that promotes mentoring, lecturers help to develop the work of the university institution, committed to the society in which it is inserted. however, the need for advice of the student is different in every moment of the educational process, so the lecturer must make dissimilar recommendations for each level. as for the new students, our experience as high school teacher shows that lots of students are enrolled to many college degrees by chance and not by vocation. each time fewer students, at the end of the last year before the university, have clear ideas about which university degrees choose. the marks obtained in the entrance examinations to the university, the proximity or distance of the centre where a certain degree is offered from their home and employment expectations are often decisive in their choice. therefore, when they reach the university, the number of students disoriented with respect to higher education is huge. this situation sometimes causes the abandonment of the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 64 studies program and a high degree of dissatisfaction. in addition, that causes a lack of clarity about what are they learning, that in many cases, lasts throughout the degree. in the same way, high school students are used to having a tutor assigned by the school with whom he has one hour per week for advice. in this session, students receive guidance from the tutor or from the psychologist on study skills, college tours, choices they have when they finish their educational stage, administrative procedures to claim notes or to enrol at the university and other topics. the tutor is who solves any questions that students may have since their entry into the education system. by accessing to university, this figure disappears for the first time in student's life, they do not have this reference person that advises, accompanies and guides. by the time they start higher education, when they start a new grade where choices are greater than at any previous stage of education, students are alone and confused. therefore, we understand the importance of the tutor's role in the transition between high school and higher education. in this case, tutoring may also be conducted by students of final year of the degree, who could introduce tutored students in the degree and in the university system. moreover, our experience as a tutor of internships in the last year of the degree of master has made us understand that the needs of support in recent years aren't in a lower grade. at that time, students face, often for the first time, with the workplace and with typical difficulties and obstacles of a professional without experience. in this case, the problem is not a lack of motivation, but the shortcomings that they have related to their future employment. that is the reason because they also appreciate the advice of the lecturer, who facilitates the transition from university to working life. as students work outside the university in recent years of career, it should be noted that in this case the advice is shared with a tutor in the workplace, a professional who helps and guides the student from the workplace in which he aspires to be inserted. that supervisor is coordinated with the academic tutor of the university, which controls the process and acts as a bridge between the supervisor and the supervised. in the same way, the professor who supervises the degree final project is an excellent figure to carry out this task to guide and help the student to decide if he continues with a multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 65 more specialized training or if he already goes to laboral world. but to fulfil this function is not sufficient with the willingness of the lecturer, but the tutor has to be formed to select the suitable methods and strategies to advise adequately the student. only by forming the lecturer, he will understand that tutoring is another tool to improve learning, which has contents and a planning of activities that will contribute to the development of the required skills. so, how we have tried to expose, we believe that mentoring is essential to educate university students, in order to be well trained and aware of what is their tasks at the university. 2. development of the experience for the reasons set out before, we decided to carry out an innovative educational project during the 2013-2014 academic year with our students in the second year of the early childhood teacher degree. during the first week of classes, we introduced our students the tutorial plan, consisting of a series of meetings that had to keep students and lecturer. then, we spent a poll our students with three objectives: knowing about them, creating a climate of confidence in the classroom and make them share the learning process of the teaching profession. among other things, we asked for personal data, including the birthplace, residence, associative activities or sports performed regularly in order to better understand our students. similarly, with the same goal, they were asked what they would like to learn in the subject that at that time began. at the same questionnaire, they were asked about the opinion of the universitat jaume i and about the teaching they were receiving in their degree. regarding previous ideas about social sciences, in the questionnaire students were asked to draw a map of the world, noting ecuador and parallel 0º. also, we asked students to quote five geographic names and five countries or cities. as historical contents, we demanded to write the name of the three most admired persons of the past in chronological multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 66 order and some information about them. then, we asked to do the same with three historical facts. when we had finished to analyse the data from the surveys, we began with the tutorial calendar, understood as a personalized meeting scheduled in the syllabus. the interview allowed us to cater, facilitate and guide each student in his or her learning process. from each meeting, the lecturer filled a monitoring form where was reflected if the student arrived in the expected date and time and each topic talked about. the high percentage of attendance indicates the interest of students in tutoring. also, the students prepared a diary where they reflected their daily process of learning and other topics of the subject matters addressed in individual tutoring. this paper was delivered by the students at the end of the year and thanks to the comments reflected in it, we could value the utility that tutoring had in their learning process. simultaneously, we asked the students to be grouped into sets of approximately ten students. once clusters were done, we asked them to choose a representative, who should meet with the lecturer once a month. these students were chosen from volunteers by their peers, this is the reason because were learners committed to their function, assumed as an own decision. these representatives helped the lecturer in the transmission of the information needed to pass the subject and helped her to coordinate the different working groups of the class. then, we exposed the purpose of mentoring meetings with the representatives of groups and we established the calendar of meetings. in addition to the specific issues to the field of study, we were going to talk about various topics such as public speaking strategies, issues that promote college success, self-esteem and optimism when facing academic tasks or exams, etc. these mentoring meeting set out the need to develop skills to learn through observation, investigation, analysis, deduction, discovery, classification and finally reflection. at the meetings, the lecturer tried to flee from an expository teaching and to promote an open space for dialogue between students and lecturers. (rodríguez espinar: 2004). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 67 also, thanks to these meetings, the lecturer was able to carry out a careful control over the distribution of tasks between the members of each group, of problems that each group had to confront, and of the development of activities that they had to make for learning. in the same way, we resolved doubts and directed the work of the students. from each meeting, a student wrote a page to compile the discussed proposals in order to solve unexpected or negative results that were taking place throughout the year. after each session, the students attending the group tutorials carried out meetings with their work group. peer tutoring has developed and implemented successfully in the englishspeaking world in order to mitigate the fact that the information the lecturer wants to convey to their students doesn’t reach all students by the enormous amount of learners who attend the subject. through these peer mentoring, we seek to achieve autonomy and independent study and to take decisions according to their personal and professional interests. also, we attempted to foster teamwork and personal involvement of each of the students in their own teaching and learning process. at the same time, after each tutorial group, the lecturer thought about how students were doing their tasks, looked for alternatives to activities that weren't successful and attractive to students, analysed what were the problems for students and the causes thereof. that is, took out a self-evaluation process of teaching. at the same time, the lecturer also offered tutorials through traditional or virtual personal encounter, usually after the request of students, when they had any question on the subject of study. these tutorials also allowed dialogue and mutual understanding between lecturer and student. 3. project’s results the impact analysis aims to determine the effects of the implementation of the action plan tutorial among our students. for this purpose, we have examined both positive and negative consequences it has had on the beneficiaries and students' opinions about the usefulness of the scheduled tutorials. to verify the results, we have compared the academic marks of the students who participated in this tutorial plan with those in the previous year who had not multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 68 done it and who had only received private tutoring when they asked for it to the lecturer. in addition, participating students were surveyed about their satisfaction with the mentoring program, if it had aroused the interest, the relationship with the lecturer, and finally the utility of the same. as mentioned above, to carry out our objectives, we have raised the combination of individual and group tutorials. the individual tutorials have given excellent results, as they offer the possibility of a teaching-learning process more adapted to the needs of each individual student. regarding the group tutorials, they have also had notable results, as they have supported the commitment of the students with their learning, decision-making and awareness of themselves regarding each stage of the educational process. most students were satisfied with peer tutoring because they thought that this kind of tool was very complete to solve their needs. however, they recognized the work of the lecturer in terms of advice on individual tutorials. in general, according to their statements, both kind of tutoring satisfied them, because they felt more guided in the teaching and learning process than they felt in other subjects in which they received only an individual tutoring by the lecturer. therefore, we conclude that the peer tutoring system combined with personal interviews has more benefits than traditional mentoring between lecturer and student. regarding the student-tutors, who participated as these voluntarily, they showed at the end of the experience the convenience of an academic gratification for the work done. it should be noted in this regard, that our students want to become teachers and, therefore, it was not difficult to find students willing to supervise others, because of the vocational component for their studies. perhaps, if this experience would has been conducted with students from another degree, would has been more difficult to find students who will show interest in participating. about the participant lecturer, it should be noted that this educational innovative experience has been extremely rewarding for her, to see first hand what students think of her teaching style, what their expectations are and where are the difficulties of the subject taught. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 69 to improve this proposal, we will consider in the future the coordination with other professors of the area of social sciences teaching, so that they can work with this method. the analysis of references, reflections joint study about academic performance of our students and the training that we carry out will affect the quality of our teaching, our preparation as lecturers and, ultimately, achieve the best academic results in our students. in this regard, it would be desirable in order to carry out a comprehensive mentoring of students regarding academic, personal and professional matters, a maximum of ten students tutored for each lecturer. if this premise would be possible, the tutor would be able to advise the student comprehensively in academic skills, identify potential learning difficulties and help the student to plan his or her studies and electives. it should be noted that among the main difficulties encountered is the excess of students we had to attend, the lack of time to do it and the mentality prevailing in the university culture that leads to consider mentoring as a voluntary activity in the process of teaching and student learning in higher education. so, with this teaching innovation experience, we have sought to improve our academic methodology, giving a strong weight to the tutoring, both onsite and virtual. we also have tried to encourage teamwork in groups and independent learning. thanks to these improvements, we have continuously evaluated the students, while students self evaluate their learning process. to conclude, i must say that the proposal of educational innovation that we have carried out has addressed the need to promote a system of tutorials to assist students in their learning process. this approach is based on the work of independent learning, which aims is to meet the need for cognitive maturation of the college student. 4. references álvarez pérez, p., gonzález afonso, m. (2005). la tutoría académica en la enseñanza superior: una estrategia docente ante el nuevo reto de la convergencia europea. reifop, 8 (4), 1-4. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3758 social and technological sciences eissn: 2341-2593 prades plaza (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 60-70 | 70 arbizu, f., lobato, c., castillo, l. (2005). algunos modelos de abordaje de la tutoría universitaria. revista de psicodidáctica, 10 (1), 7-22. garcía nieto, n. (2008) la función tutorial de la universidad en el actual contexto de la educación superior. revista interuniversitaria de formación del profesorado, 22 (1), 2148. garcía valcárcel, a. (2008). la tutoría en la enseñanza universitaria y la contribución de las tic para su mejora. relieve, 14 (2), 1-14. rodríguez espinar, s. (2004). manual de tutoría universitaria. recursos para la acción. barcelona: octaedro. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 100 take care of well-being: how facilitators and engagement predict performance of university students martínez, isabel m.1*, peñalver, jonatan1, meneghel, isabella1. (universitat jaume i de castelló, wont research team) *corresponding author: universitat jaume i, departamento de psicología evolutiva, educativa, social y metodología. av. de vicent sos baynat, s/n, 12071, castellón de la plana: castellón (spain).email: imartine@uji.es. phone: +34 964 729 585 received: 2015-02-12; accepted: 2016-01-15 abstract the interest in developing a high quality educational system requires constant research of the variables involved in the teaching-learning process. among these variables, social and academic facilitators are important because there is empirical evidence about their positive relationship with engagement, commitment, selfefficacy, happiness and satisfaction in the academic context. moreover, the psychological well-being of university students (i.e., engagement) showed to positively affect future academic success. in line, the aim of this study is twofold. first, the relevance of social and university academic facilitators was analyzed depending on the faculty of belonging. second, the effect of social (e.g., good relationship with classmates) and academic facilitators (e.g., updated website with new information and easily accessible) as well as academic engagement on academic performance (i.e., gpa) was tested. the sample consisted of 965 university students. the anovas’ results showed the existence of statistically significant differences in social and university academic facilitators among the different faculties. regression analyses demonstrated that social (but not university’s) facilitators and academic engagement were positively related to academic performance. additionally, the interaction between social facilitators and academic engagement was positively related to academic performance. the effect was also significant when controlling for gender and faculty. the identification of different facilitators allows to develop different activities depending on the faculty, as well as leading to the optimization of teaching-learning process. moreover, academic facilitators do not affect academic performance. from a practical view, it means that specific interventions can be implemented during the course so that students’ social facilitators and engagement increase. keywords university students, well-being, academic engagement, academic facilitators, social facilitators. mailto:solares@uji.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 101 1. introduction improving education systems is an important goal in all countries, due to the particular importance of education in personal and professional development of individuals. specifically, the university systems are of unique interest because, through university, education is trying to develop the skills that will ensure future job success between current students. therefore, it is important to optimize the teaching-learning process through the study of the variables that have an impact on this process. the nature of the university setting focuses on long periods of learning and evaluation creates an environment where students are faced with ongoing challenges and pressures to meet deadlines and maintain high performance. research has shown high levels of stress in university students but little research has focused on the link between the academic environment of university students, their levels of well-being and academic performance. then, the aim of this study is twofold. first, analyze the facilitators perceived by students: social facilitators (e.g., good relationship with classmates) and academic facilitators (e.g., updated website with new information and easily accessible) depending on the faculty of belonging. we hope that facilitators are different in each faculty. second, test the effect of social and academic’s facilitators, as well as academic engagement, on academic performance (i.e., gpa). 1.1 facilitators and academic performance facilitators are defined “as the aspects of the situation that may promote performance or one’s ability to optimally perform one’s job (or study)” (salanova et al. 2010, pp 55). following job demands-resources (jd-r) model (bakker and demerouti 2007), facilitators are similar to resources (e.g., autonomy, supportive team climate), but they are more specific for the educational situation (salanova et al. 2010). in the teaching-learning process, facilitators refer to factors related to the university or academic facilitators (e.g., services, availability of scholarships), to teachers and fellow students or social facilitators (e.g., solidarity, comradeship, social support), and to themselves or personal facilitators (e.g., motivation multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 102 towards studies) (salanova et al. 2005). previous research showed differences in perceived facilitators for students from different faculties (salanova et al. 2005; peñalver et al. 2013, martínez et al. 2014). we therefore expect that: hypothesis 1: facilitators (i.e., social and academic) will show differences depending on the faculty of belonging. facilitators have been shown to play role in burnout, engagement, satisfaction, performance, self-efficacy and commitment (salanova et al. 2005; salanova et al. 2010). specifically, in the academic context a greater perception of facilitators was found to be related to lower levels of cynicism, and greater levels of happiness, commitment with the university, efficacy in studies and satisfaction. in addition, they are related to smaller tendency to abandon studies and higher levels of vigor and dedication (salanova et al. 2005). although in the preview literature only personal facilitators have been found to correlate positively and significantly with academic performance (salanova et al. 2010), recent research shows the importation of variables associated with student social support and positive relationship for increase academic performance (meneghel et el. 2015). in this sense, we consider that facilitators act as resources: perception of high levels of social and academic facilitators help to address the academic demands and affect students’ performance. thus, students use facilitators to solve problems and difficult situations related to their studies and get a good performance. we therefore expect that: hypothesis 2: facilitators (i.e., social and academic) will be positively associated with academic performance. 1.2 academic engagement and academic performance academic engagement may be considered as an indicator of well-being in academic context, and it is defined as a positive psychological state characterized by vigor, dedication and absorption toward education (schaufeli et al. 2002). previous research showed that engagement is positively related to satisfaction (i.e., with professors, with studies, with faculty, with university), happiness, and commitment with the university (salanova et al. 2005), multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 103 organizational facilitators, personal facilitators, social facilitators (salanova et al. 2010), academic resilience and academic self-efficacy (martínez et al. 2014). as discussed above, engagement is identified with a positive motivational state by which performance enhancing mechanisms are activated. specific to the university setting, in their study among university students from spain, portugal, and the netherlands, schaufeli and colleagues (2002) found that academic engagement was positively related to academic performance (measured as the number of passed exams relative to the total number of exams done). more recently, bakker and colleagues (2015) showed the relevance of student engagement as a psychological process in education that facilitates performance in a diary study. in fact, engaged students are intrinsically motivated to advance in learning, attend classes, and participate in activities related with their study. they are usually curious persons, ask questions, and enjoy challenges related with their learning. vigorous, absorbed and dedicated students are energetically immersed in their studies, which makes them successful as well (salanova et al. 2010).we therefore expect that: hypothesis 3: academic engagement will be positively associated with academic performance. previous literature showed that there is a linear relationship between facilitators, engagement and performance (salanova et al. 2010), in the sense that academic engagement mediates the relationship between facilitators and future performance (i.e., gpa). chambel and curral (2005) showed that student’s well-being also mediated the relationship between control and performance. then, well-being is a variable than can explain significant relationships enhancing or reducing the effect of the facilitators on performance. however, there is reason to believe that reciprocal interactions among supportive context, with high levels of academic and social facilitators, and student engagement are also relevant for academic outcomes (furrer et al. 2006). in this study, we propose that academic engagement acts as a moderator of the relationship between the perception of facilitators and academic performance, in the sense that when academic engagement is high, facilitators have a stronger relationship with academic performance. we therefore expect that: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 104 hypothesis 4: academic engagement will moderate the relationship between perception of facilitators and academic performance such that when academic engagement is high, facilitators have a stronger relationship with academic performance. figure 1. the proposed model of the study. 2. method 2.1 sample and procedure a stratified sample of 965 students (59.3% female) was drawn from undergraduate students of jaume i university in castellón (spain). the students belonged faculty of human and social sciences (fhss, 37.7%), faculty of health sciences (fhs, 16.8%), faculty of law and economics (fle, 22%) and school of technology and experimental sciences (stes, 23.5%). regarding the course they were doing, 31.5% were in the first year, 39.8% in the second, 21.1% in the third, 6.7% in the fourth, and 1% were in their fifth year. to conduct the study, the research team designed a questionnaire containing the study variables. this questionnaire (which required 20 minutes to administer) was completed by students in the classroom. participants completed the questionnaire by answering questions regarding their academic activity. participation was voluntary, and confidentiality was guaranteed to all respondents. facilitators academic performance academic engagement multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 105 2.2 measures facilitators. an inventory was composed of two sets of items: social facilitators (five items) and academic facilitators (eight items). item examples include: indicate how important these facilitators in your role as a student are: “updated website with new information and easily accessible (e.g., institutional mail etc.)” and “good relationships with professors” respectively. all items are scored on a 5-point importance rating scale ranging from 0 (not important at all) to 4 (very important). academic engagement. engagement was measured by short versions of the utrecht work engagement scale (schaufeli et al. 2006), which was previously validated for students (schaufeli et al. 2002). the vigor, dedication, and absorption dimensions were each measured by three items (item examples include “when i’m doing my work as a student, i feel bursting with energy.”, “my studies inspires me”, and “i am immersed in my studies”). all items were rated on a seven-point likert scale that ranged from 0 (never) to 6 (every day). academic performance. performance was assessed by the gpa given from the university data, which ranges from 5 to 10. the gpa obtained, comprised from the beginning of the students career to the second evaluation of the current semester. the gpa was obtained 4 or 5 months after the questionnaire was filled. the participant’s permission was obtained for receiving their gpa from the university database. control variables. previous researches show the effect of socio-demographic variables (martinez el al. 2014). therefore we consider gender and faculty of belonging as control variables. 2.3 data analyses in order to achieve our objectives, descriptive analyses and internal consistence of scales were computed. also, correlations of all the variables were calculated. afterwards, analysis of variance anova, and post hoc tests (dms) were performed to show differences related with gender and facilitator perceptions between faculties. finally hierarchical multiple regression analyses were conducted using the method of successive steps (cohen and cohen 1983), using multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 106 as dependent variable academic performance. the independent variables entered in the equations in three successive steps (aiken and west 1991). in the first step, gender and faculty were introduced. in the following step, academic facilitators, social facilitators and academic engagement were introduced. finally, in the third step, we included the two-way interactions of the independent variables (academics facilitators x engagement and social facilitators x engagement). 3. results 3.1 descriptive analyses table 1 shows means, standard deviations and correlations. the coefficient (cronbach’s alpha) of academic engagement was α = .87. for facilitators values of cronbach’s  were not computed because instead of referring to an underlying latent factor both are sets of basically independent environmental or social factors. correlations were significant and in the expected direction. as expected, academic facilitators are significantly positively correlated with academic engagement; however they do not correlate with the academic performance. on the contrary, social facilitators are significantly positively correlated with both academic engagement and academic performance. also as expected, the interrelations among academic engagement and academic performance are positive. table 1. means, standard deviations and correlations for the study variables (n = 965) variable m sd 1 2 3 1 academic facilitators 3.48 0.51 2 social facilitators 3.04 0.56 .55** 3 academic engagement 3.69 0.95 .18** .26** 4 academic performance 7.00 0.85 ns .18** .21** **p < .01 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 107 the results of anova show significant differences in facilitators when we consider the faculty of belonging: perceptions of academic facilitators (f= 2.79; p < .05); social facilitators (f= 15.71; p < .001); and engagement (f= 10.31; p < .001). the faculty of human and social sciences show highest level of both facilitators (figure 2). the faculty of law and economics shows the lowest levels of social facilitators and the school of technology and experimental sciences show lowest levels of academic facilitators. significant differences were found also when the gender is considered. women perceive more academic and social facilitators (f= 52.72; p < .001 and f= 26.13; p < .001 respectively). also, women present significantly higher engagement levels (f= 17.81; p < .001). then, we consider faculty and gender as control variables in regression analyses. figure 2. academic and social facilitator in four faculties: fhss, fhs, fle and stes 3.2 regression analyses overall, we found substantial support for our hypotheses. results of regression analyses show an increase of variance explained in the successive steps, showing the importance of considering step 3 to evaluate the interaction effect. regarding the results, considering academic performance as a dependent variable, we observe a main positive effect of faculty (β 1 1,5 2 2,5 3 3,5 4 fhss fhs fle stes academic facilitators social facilitators multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 108 = .26, p < .001), social facilitators (β =.55, p < .001) and engagement (β = .70, p <.005) on academic performance (table 2). these results support h2 and h3. moreover, results showed a significant interaction effect of social facilitator and engagement (β = -.90, p < .001), which provides partially support h3. the significant interaction effect of social facilitators and academic engagement is graphically represented in figure 3, following the method recommended by aiken and west (1991) and jaccard, turrisi and wan (1990). values of the moderator were chosen 1 sd below and above the mean. entering these values in the regression equation generated simple regression lines. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 109 table 2. hierarchical multiple regression analysis of gender, faculty, academic and social facilitators and academic engagement on academic performance (n= 965) variables β r2 r 2change step 1 .10 .10*** gender .07* faculty .29*** step 2 .14 .04*** gender .05 faculty .26*** academic facilitators -.05 social facilitators .11** academic engagement .15*** step 3 .16 .02*** gender .04 faculty .26*** academic facilitators -,10 social facilitators .55*** academic engagement .70** acad. facilitators x acad. engagement .13 soc. facilitators x acad. engagement -.90*** multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 110 figure 3. two-way interactions effect of social facilitators and academic engagement on academic performance when social facilitators’ level is low, students have low levels of academic performance. however, when social facilitators are high, the effect on academic performance is different in relation to the academic engagement level. the results show higher levels of academic performance when academic engagement is high (1 sd above the mean score) in both cases, for high and low levels of social facilitators. also, for students with low level of social facilitators (1 sd below the mean score), results show that when their level of academic engagement is higher, their level of academic performance increases. a different picture is showed for students with low level of social facilitators (1 sd below the mean score) and low level of academic engagement. moreover when student show high levels of academic engagement, the effect of low social facilitators is damped and the effect of high facilitator is enhanced. 1,00 1,50 2,00 2,50 3,00 3,50 4,00 4,50 5,00 5,50 low social facilitators high social facilitators high academic engagement low academic engagement multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 111 4. discussion the interest in developing a high quality educational system requires constant research of the variables involved in the teaching-learning process. among these variables, social and university’s facilitators are particularly important because there is empirical evidence about their positive relationship with engagement, commitment, self-efficacy, happiness and satisfaction in academic context (salanova et al. 2005). thus, the first aim of this study was to analyze the relevance of social and university’s facilitator depending of the faculty of belonging. as expected (hypothesis 1) we found significant differences between faculties, that is: in the faculty of human and social sciences there are highest levels of both facilitators (i.e., social and academic), whereas the lowest levels of social facilitators are in the faculty of law and economics and the lowest levels of academic facilitators are in the school of technology and experimental sciences. then, our results showed how social (but not academic) facilitators are antecedents of academic performance, supporting hypothesis 2. including academic engagement as an indicator of psychological well-being of university students, we also confirm that it is an antecedent of academic performance over time, supporting hypothesis 3. finally, the test of the interaction effect between facilitators and engagement showed that only the interaction between social facilitators and academic engagement is significant, whereas the interaction between academic facilitators and academic engagement isn’t, partially supporting hypothesis 4. 4.1 theoretical contributions a lot of studies have analyzed the teaching-learning process focusing on the internal variables of the process (e.g., curriculum, didactic strategies) and there are fewer studies that focused on psychosocial and academic variables (e.g., facilitators, obstacles). to identify these psychosocial and academic variables permits to comprehend the teaching-learning process from a broader point of view. furthermore, to understand the effect of these variables on the well-being and performance of students allows carrying out interventions directed to improve this process. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 112 the present study represents a step forward with respect to previous research into academic context in several ways. first, we could conclude that there are different kinds of facilitators (i.e., social and academic) which are identified as such by most students and that there are differences in their levels in terms of the different faculties. although this result has clear and useful implications at practical level, it is also theoretically relevant. in fact, on one hand this result showed which specific facilitators could be provided to help students during their courses, and on the other, future research can study the psychological differences that characterized students depending on faculty belonging, which affects the perception of academic and social facilitators. these results updated previous findings about relevant facilitators and their implications on well-being and performance (salanova et al. 2005; salanova et al. 2010). these findings must be constantly revised and actualized. in fact, the academic context is suffering continuous changes, as curriculum goals that have become more academic and skill-oriented (shoshani and steinmetz 2014), the bologna treaty about the european higher education area, and new challenges in the use of information and communication technology. in this sense, to identify specific facilitators relevant in each context is fundamental. second, whereas previous studies found that engagement fully mediated the impact of facilitators on future academic performance (salanova et al. 2010), in this study we test their interaction, examining how academic engagement moderates the relationship between social facilitators and performance. taking separately, our results indicate that student with higher levels of social facilitators tend to achieve better performance, and the same occur for students with higher levels of engagement. however, when these antecedents are taken together, results showed that students attain better performance when social facilitators and academic engagement were high. this is relevant because, as suggested by furrer and colleagues (2006), we confirmed that there is an interaction effect between contextual variables and engagement to performance. finally, the results indicate that academic facilitators, which are more related to the physical context or university infrastructure, do not affect academic performance. in this sense, and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 113 contrary to our expectations, these kinds of facilitators are not relevant for students to achieve better performance, or at least not as important as the social ones. 4.2 implications for practice the findings of this study reveal that students’ social facilitators and academic engagement are key factors in achieving academic performance. in terms of practical implications, we conclude with a few suggestions. first, from an educational point of view, specific interventions can be implemented during the course so that students’ social facilitators and engagement increase. as fredricks and colleagues (2004) suggested, most educational interventions are aimed at increasing achievement without taking into account the importance of students’ engagement. however, effective interventions should attend to this important variable, focusing on its antecedents to help its development. in this sense, efforts from teachers and family (which are social facilitators as well) may help students to improve their engagement (bakker et al. 2015). for example, it is possible to reduce student’s fear of failure through positive reinforcement and frequent feedback (caraway et al. 2003), which also influence positive emotions that are relevant for academic engagement (reschly et al. 2008). second, this study focuses on issues that directly affect the quality of learning and psychological well-being of university students, as facilitators are. in this sense, taking into account facilitator by faculty of belonging was a key issue in proposing and carrying out the improvement of educational quality at a practical level. it is important to note that social facilitators, and not academic, are relevant in this sense. thus interventions may be especially directed toward enhance relationship with teachers and classmates, in order to assist the development of social support ties. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 114 4.3 limitations and future directions this study has several limitations which highlight important avenues for future research. with the exception of academic performance rates (i.e., gpa), perhaps the clearest limitation is the use of self-reported data, which increases the risk of common method bias (podsakoff et al. 2003). we tried to minimize such errors by differentiating the response scales for each of these variables, as suggested in podsakoff and colleagues (2012). another limitation of the present study is that data are cross-sectional, with the exception of the gpa, which is collected at a different point in time (i.e., between 4 and 5 months later). although sem analysis gives some information about the possible direction of the relationships, cross-sectional study designs do not allow one to draw firm conclusions regarding the causal ordering among the variables studied (i.e., moderation effect). clearly there is a need for longitudinal studies that may allow stronger causal inferences to be made about the relationship between facilitators, engagement, and performance. in addition, although the sample of the study came from four different faculties and a number of degrees, our results are based on a sample from the same university. thus, the results need to be replicated in different universities in order to allow our findings to be generalized to broader academic contexts. acknowledgements this research was supported by a grant from consellería de educación generalitat valenciana – programa prometeo (prometeo/2013/025). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 115 5. references aiken, l. s. and west, s. g. (1991). multiple regression: testing and interpreting interactions. sage, newbury park, ca. bakker, a. b. and demerouti, e. (2007). the job demands-resources model: state of the art. journal of managerial psychology, 22, 309-328. bakker, a. b., sanz vergel, a. i. and kuntze, j. (2015). student engagement and performance: a weekly diary study on the role of openness. motivation and emotion, 39(1), 49–62. caraway, k., tucker, c., reinke, w. and hall, c. (2003). self-efficacy, goal orientation, and fear of failure as predictors of school engagement in high school students. psychology in schools, 40, 417–427. chambel, m. j. and curral, l. (2005). stress in academic life: characteristics as predictors of student well-being and performance. applied psychology: an international review, 54, 135147. cohen, j. and cohen, p. (1983). applied multiple regression/correlation analysis for behavioral sciences (2nd ed.). hillsdale, nj: lawrence erlbaum. fredericks, j. a., blumenfeld, p. c. and paris, a. h. (2004). school engagement: potential of the concept, state of the evidence. review of educational research, 74, 59–109. furrer, c., skinner, e. a., marchand, g. c. and kindermann, t. a. (2006, march). engagement vs. disaffection as central constructs in the dynamics of motivational development. presented at the biennial meeting of the society for research on adolescence, san francisco, ca. jaccard, j., turrisi, r. and wan, c. (1990). interaction effects in multiple regression. sage, newbury park, ca. martínez, i. m. and salanova, m. s. (2003). niveles de burnout y engagement en estudiantes universitarios: relación con el desempeño y desarrollo profesional [burnout and engagement levels in university students: relation with performance and professional development]. revista multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 116 de educación, 330, 361-371. martínez, i. m., meneghel, i. and peñalver, j. (2014, october). bienestar, desempeño y género en estudiantes de la universitat jaume i [well-being, performance and gender in studens of jaume i university]. presented at the i jornadas nacionales de salud, emoción y género, castellón de la plana, spain. meneghel, i., martínez, i. m., salanova, m. and de witte, h. (2015). how to promote academic satisfaction and performance: building academic resilience through coping strategies. manuscript submitted for publication. peñalver, j., rodríguez, r. l., meneghel, i. and martínez, i. m. (2013, october). facilitadores académicos y satisfacción percibida en estudiantes universitarios [academic facilitators and perceived satisfaction in university students]. presented at the xx reunión anual de la s.e.a.s. v jornadas "emociones y bienestar”, madrid, spain. podsakoff, p. m., mackenzie, s. b., jeong-yeon, l. and podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. podsakoff, p. m., mackenzie, s. b. and podsakoff, n. p. (2012). sources of method bias in social science research and recommendations on how to control it. annual review of psychology, 63(1), 539-569. reschly, a. l., huebner, e. s., appleton, j. j. and antaramian, s. (2008). engagement as flourishing: the contribution of positive emotions and coping to adolescents' engagement at school and with learning. psychology in the schools, 45, 419–431. salanova, m., martínez, i. m., bresó, e., llorens, s. and grau, r. (2005). bienestar psicológico en estudiantes universitarios, facilitadores y obstaculizadores del desempeño académico [psychological well-being in university students, facilitators and obstacles of academic performance]. anales de psicología, 21(1), 170-180. salanova, m., schaufeli, w. b., martínez, i. m. and bresó, e. (2010). how obstacles and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3751 social and technological sciences eissn: 2341-2593 martínez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 100-117 | 117 facilitators predict academic performance: the mediating role of study burnout and engagement. anxiety, stress & coping, 23(1), 53-70. schaufeli, w. b., bakker, a.b. and salanova, m. (2006). the measurement of work engagement with a short questionnaire: a cross-national study. educational and psychological measurement, 66, 701-716. schaufeli, w. b., martínez, i. m., pinto, a., salanova, m. and bakker, a. (2002). burnout and engagement in university students: a cross-national study. journal of cross-cultural psychology, 33(5), 464. schaufeli, w. b., salanova, m., gonzález-romá, v. and bakker, a. (2002). the measurement of burnout and engagement: a confirmatory factor analytic approach. journal of happiness studies, 3, 71-92. shoshani, a. and steinmetz, s. (2014). positive psychology at school: a school-based intervention to promote adolescents’ mental health and well-being. journal of happiness studies, 15(6), 1289-1311. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 acquisition of transversal skills through pbl: a study of the perceptions of the students and teachers in materials science courses in engineering moliner, m. lidón1*, guraya, teresa2, lopez-crespo, pablo3, royo, marta4, gamezperez, josé5, segarra, mercè6, cabedo, luis5 1departamento de educación, castellón, spain. 2department of mining and metallurgical engineering and materials science, university of the basque country upv/ehu, bilbao, spain. 3department of civil and materials engineering, university of malaga, malaga, spain. 4departamento de ingeniería mecánica y construcción, universitat jaume i, castellón, spain. 5departamento de ingeniería de sistemas industriales y diseño (esid), universitat jaume i, castellón, spain. 6departamento de ciencia de los materiales e ingeniería metalúrgica. facultad de química. universitat de barcelona. barcelona, españa. *corresponding author: mmoliner@uji.es phone:+34 964 729806 received: 2015-03-15; accepted: 2015-05-03 abstract the main goal of this work is describing the experience in using project based learning (pbl) methodology in materials science courses in several engineering degrees. the courses were taught simultaneously in four different spanish universities. in addition, the impact of this proposal on the acquisition of transversal skills through the perceptions of students and teachers is assessed. in order to do so, voices of over 54 students and 5 lecturers involved in the work have been gathered and studied descriptively. discussion groups and self-reports have been collected and subsequently analysed. the analysis of the data collected by these two instruments has been coded and categorized into general dimensions. the results show how, moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 121 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 through the pbl, self-learning is encouraged and the development of transversal skills such as teamwork are enhanced. meanwhile, the lecturers consider pbl an appropriate methodology for materials science courses; however its implementation requires a higher work load. the faculty acknowledged pbl as a very interesting tool to work and evaluate transversal competences. hence, the pbl methodology stands as one of the most effective methods that meet the demands of the 21st century and enables students to acquire not only content based knowledge, but also other abilities. keywords active learning methods; teaching methodologies; european higher education area 1. introduction competence can be defined as «the ability to tackle a range of similar situations in an efficient manner. this is done by combining consciously, quickly and creatively a number of cognitive skills; knowledge, capacities, micro-competences, information management, attitudes, ways of perception and evaluation» (perrenoud, 2001:9). for engineering degrees, in addition to theoretical and practical skills acquired during the degree, a number of other skills should be grasped, including social skills and independent working. such a comprehensive education will make it possible to develop successfully a professional career. these abilities are named cross curricular competences because they do not belong to one particular subject or even a degree, but make the performance and success to increase manifolds. other examples of such cross curricular competences are time management, efficient communication, public speaking, independent information searching skills, leadership and problem solving. as a consequence, the latest changes in the higher education curriculum promoted by the european higher education area (ehea) march in this direction (carreras, 2008). nevertheless, these changes are still hotly debated because of the difficulties of integrating them. moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 122 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 pbl learning has been demonstrated to be among the most effective teaching tools and assessment methods towards this new goal. in addition, pbl also deals with a number of other issues introduced by the new education system. for example, it helps distributing the time devoted by students for the different modules, both in the classroom and outside the classroom, in accordance to the credit system ects. in addition pbl contributes to the developing of cross curricular skills during the degree (carreras, 2008). the evaluation of such cross curricular skills in traditional teaching methods is a difficult task. however, pbl allows one to incorporate easily the evaluation of such cross curricular skills (valero, 2011; valero 2007). the pbl is a model of learning in which students raise, implement and evaluate projects that have an application in the real world. this method uses problems as a starting point for the acquisition and integration of new knowledge (barrows, 1986). as other methodologies, it requires a series of well-structured and planned steps set by the teacher. moust, bouhuijs and schmidt (2007) and schmidt (1983) proposed seven steps to implement the resolution of the project: 1. clarification of concepts and terms that appear in the proposal project from the dialogue between group members. 2. first tentative definition of the problem. after steps 3 and 4 this step can be repeated if considered necessary. 3. analysis of the problem from the contributions of all group members through brainstorming. 4. development of a systematic summary with several explanations to the analysis of the previous step. once generated the greatest number of ideas about the problem, the group tries to systematize and organize them, highlighting the possible associations among them. 5. set up of learning objectives and common decision on aspects of the problem, which are to be investigated and understood. moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 123 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 6. search for more information, individually. 7. synthesis of collected information and writing a report on the acquired knowledge. these phases point out the many cognitive processes and competencies required to the students. given the generic nature of pbl activities, the students improve a wide range of cross curricular skills such as problem solving, decision making, teamwork and communication (de miguel, 2005). thus, knowledge is gained while they learn to learn in a progressively independent way as well as they learn to apply that knowledge in solving various problems similar to those they face in the performance of different facets of this work: working in teams under supervision, being progressively autonomous, identifying learning goals, managing time effectively, identifying which aspects of the problem can be ignored or need to explore more deeply and investigating on their own, thus directing their own learning. through this process, they benefit from the participating peers, which provide the necessary contrast to their inquiries and ways of understanding what they are studying (vizcarro and juárez, 2009). one of the strengths that characterize the pbl is its application to different educational levels as well as its versatility to be able to deal with various areas of knowledge. the literature review offers examples of these benefits. reeves and laffey (1999) used pbl in an introduction to engineering lecture and found an increase in the students' problemsolving skills. however, its implementation in the materials science knowledge area is not very common, being that one of the most traditional engineering subjects and where the introduction of non-traditional teaching methods such as pbl seems to be more difficult. in this sense, jonassen and kanna (jonassen and khanna, 2011) have analysed the feasibility of introducing a similar methodology (pbl) in a subject in the field of materials science for mechanical engineers at the university of missouri and have found that this methodology is very difficult to implement unless this introduction is carried out in curricular way. thus, their findings show that the introduction of this methodology in only one subject (for this type of degrees) requires great effort from both parties involved: teachers and students, whereas if the methodology is implemented at the level moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 124 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 of curriculum for the whole degree, it is much more effective than the traditional learning methodologies. the main objectives of this work are two. firstly, to describe a pbl methodology developed simultaneously in materials science course of four spanish universities. secondly, to assessing how the new pbl methodology helps the students to develop cross curricular skills. this will be done by analysing the experiences of both students and lecturers. 2. proposed methodology: integration of different teaching methods 2.1. didactic methods the experience described here is part of idm@ti (acronym in spanish for network of educational innovation in materials science). idm@ti is formed by lecturers from seven spanish universities. within this network, it was decided to implement the pbl simultaneously on their respective courses, with the aim of analysing if it was possible to apply it in subjects in this field, regardless the specific degree in which the course is given or any other particular situation of the university. the participating universities were university of the basque country (hereinafter upv/ehu), jaume i university (hereinafter uji), universitat de barcelona (hereinafter ub) and university of malaga (hereinafter uma). the activity was developed with students from the universities enrolled in different degrees and studying courses related to materials science. all degrees were in the field of engineering. as a common element to all projects, the methodology is structured in three sections defined in the documentation provided to the students: -the project and its objectives. -a closed proposal with minimal requirements within and outside the classroom. moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 125 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 -the exact details of the evaluation methodology. with such common criteria, each participating university adapted the methodology to their needs and abilities. the main focus of each pbl project was to study the design of a commercial product. once the current design was thoroughly analysed from a materials science view point, the product was redesigned, with the aim of improving a type of properties. the choice of the product to be studied was based on the following criteria: to be affordable for the development of the activity; to involve a wide range of teaching materials of interest; and to be disassembled easily so that each component can be analysed separately. the idea of the pbl is to reproduce as closely as possible the conditions under which engineers work in companies. the following generic skills have been identified to be needed when working for a company: 1. own work management 2. summarising and managing information 3. decision making 4. problem solving 5. information searching skills 6. technical communication skills (oral and written) 7. critical awareness when reasoning 8. adaptation to new situations 9. creativity 10. entrepreneurial skills 11. awareness about quality 12. project design and management 13. team working 14. negotiating work load 15. discussing, listening and finding agreements moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 126 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 16. team working towards enhanced learning pbl was designed in a way such that these skills are practised. thus pbl allows us to expose the engineering student to conditions as similar as possible to conditions in engineering jobs. cross curricular skills were evaluated regularly through rubrics. a number of rubrics were designed to assess tasks such as minutes of meetings, participation in wikis, ict use, content and presentation videos, reports, etc. 2.2 study description 2.2.1 methodology this works is based on a descriptive method. the methods is based on understanding the perceptions of lecturers and students about how cross curricular skills are trained with pbl activities (hernández, fernández y baptista, 2001). perceptions were studied by surveys based both on multiple choice questions and on open questions. 2.2.2. participants the target population of this study was the students and lecturers of 4 different universities. students who were involved in pbl learning were asked to answer a survey. the final sample was 54 students and 5 lecturers from the different universities taking part in the study. 2.2.3. tools for gathering information qualitative information was collected through a survey with open questions about how cross curricular objectives were met with pbl. students and lecturers were asked to think about how the project has influenced them. this qualitative information was complemented with another survey based on multiple choice questions. only students were asked to answer this second survey. 17 questions were asked about in this second survey. answers were given following a liker type scale moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 127 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 (1. strongly disagree; 2. disagree; 3. uncertain; 4. agree; 5. strongly agree). the survey included a number of questions about how the project helped them in acquiring cross curricular skills. 2.2.4. analysing the results the survey was created through google drive platform that allowed delivery to all participants, students and lecturers. statistical treatment of data was done with the help of software spss (ibm corp. released 2010. ibm spss statistics for windows, version 19.0. armonk, ny: ibm corp.). the study was based on frequency analysis of the multiple choice questions. open questions were evaluated by examining their content. 3. discussion of the proposal and results 3.1. qualitative analysis first, the surveys answered by lecturers are summarised and discussed and then the surveys answered by the students are shown. lecturers believe that pbl helped developing skills such as independent working, information searching techniques, presenting results, team working, leadership, entrepreneurial skills and creativity. a gradual increase in independent working for problem solving and decision making has been observed through the course. initially, student groups required often supervising and assistance. the authors believe this initial behaviour was due to the novel methodology employed in the course, where no previous outline of tasks is given. however, it was observed that as the course progressed, the meetings between the student groups and the lecturer were used more and more as an opportunity to share information and verify that work was done is the right direction. students developed information searching techniques by looking up and exploring different sources, such as patents, scientific papers, text books, internet website, etc. the moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 128 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 results were presented periodically, both through oral presentations and written essays. this helped developing communicative skills that are key once the students graduate. team work skills were also trained, since the vast majority of activities were developed in groups. a range of experiences were found in the different universities. some lecturers believe that team working has been smooth and minor problems that appeared were easily settled as time went by. however, when the duration of the project was short, it was often found that problems were not settled. these experiences often happened when students did not seek for help from the lecturer when the problem arises. in general, pbl help student to establish deadlines with team mates, negotiate working load, share knowledge and communicate via tools such as email, dropbox and whatsapp. one member of the team normally took the leader position. this involved distributing tasks, team coordination and cheering up the group. when the leader was acknowledged by the other members, conflict resolution tended to be easier and faster. in one of the pbl experiences, it was found that entrepreneurial skills and creativity were also strengthen, since the student has to combine a wide range of knowledge learned during the degree to solve problems similar to those met in real working life. compared to traditional teaching methodologies based on exams and lab essays, cross curricular skills are promoted much more in pbl based methodologies. in addition, standard evaluation tests do not allow assessing such skills. cross curricular skills can be easily evaluated through rubrics. a number of rubrics were designed to assess oral presentations, essay writing and team working. the results of the student survey show clearly that students believe that self-learning is the skill mostly fostered through pbl activities. the fact that they had to "sort out they own problems" made them responsible of their own learning. it was also very useful to understand the practical use of many subjects covered in the materials science course. it can also be observed that students believe pbl has improved them in dealing with team working. they also have concerns related to the different levels of commitment by the team members. nevertheless, these differences are overall rated as positive by the moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 129 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 students. for the sake of fairness, they believe peer to peer evaluation would be very advantageous. this would allow them to distinguish between students with different levels of involvement on the project. students believed these situations were partially the lecturer's fault and suggest there should exist a procedure to avoid such inequalities in terms of involvement. students believe that experience based learning, unlike rote learning, is more long-lasting: "other methods might be able to cover larger course contents, but will soon forget about them". students also believe that they have faced a nearly real working experience which will help them being more efficient in the future when they get a job. in summary, they believe they have learned a greater deal of useful contents and skills somehow related to materials science because they have applied them to a practical situation, they had the possibility to link ideas leant in different course and they spent a long time working on the course. 3.2. quantitative analysis quantitative analysis was done based on the multiple choice survey. students were asked to assess how the project has helped them to acquire each skill. the results are summarised in tables 1 to 5 and figure 1. table 1. survey results for cross curricular skills (i) skill 1. own work management 2. summarising and managing information 3. decision making 4. problem solving answers freq. % freq. % freq. % freq. % 1.strongly disagree 0 0 0 0 0 0 0 0 2. disagree 5 9,3 3 5,6 0 0 1 1,9 3. uncertain 18 33,3 15 27,8 8 14,8 9 16,7 4. agree 20 37 26 48,1 26 48,1 30 55,6 5. strongly agree 11 20,4 10 18,5 20 37 14 25,9 total 54 100 54 100 54 100 54 100 table 1 shows that 57.8% of students agrees or strongly agrees with respect own work management. 33.3% of students believe they have learned something and 9.3% have learned little. 48.1% of students believe the project has helped them much in learning how to summarise and manage the information, 18.5% of the sample thinks the project moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 130 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 has helped them very much, 27.8% thinks the project has helped them somewhat and only 5.6% thinks the project has helped them little. answers related to decision making are in the very much, much, somewhat and little ranges, with values 37%, 48.1% and 14.8% respectively. 25.9% of the sample believes pbl has helped them very much to learn skills related to problem solving, 55.6% believes it has helped them much, 16.7% believes it has helped them something and one person believes it has helped him/her little. table 2. survey results for cross curricular skills (ii) skill 5. information searching skills 6. technical communication skills (oral and written) 7. critical awareness when reasoning 8. adaptation to new situations answers freq. % freq. % freq. % freq. % 1.strongly disagree 0 0 0 0 0 0 0 0 2. disagree 2 3,7 4 7,4 0 0 1 1,9 3. uncertain 12 22,2 13 24,1 12 22,2 14 25,9 4. agree 29 53,7 23 42,6 31 57,4 24 44,4 5. strongly agree 11 20,4 14 25,9 11 20,4 15 27,8 total 54 100 54 100 54 100 54 100 table 2 shows that 20.4% of the students believe the methodology allowed them to improve very much their searching skills, 53.7% much, 22.2% somewhat and only 3.7% believes it has helped them little. as regards the sixth skill, 25.9% believes they have trained and improved very much technical communication skills, 42.6% much, 24.1% somewhat and 7.4% little. additionally, 20.4% of the students points out that pbl has helped them very much to be critically aware when reasoning, 57% much and 22.2% somewhat. regarding the following skill, 27.8% believes that the methodology has helped them very much in getting adapted to new situations, 44.4% much, 25.9% somewhat and only one person little. table 3. survey results for cross curricular skills (iii) skill 9. creativity 10. entrepreneurial skills 11. awareness about quality 12. project design and management answers freq. % freq. % freq. % freq. % 1.strongly disagree 0 0 0 0 1 1,9 1 1,9 2. disagree 1 1,9 1 1,9 0 0 0 0 3. uncertain 12 22,2 7 13 7 13 11 20,4 4. agree 26 48,1 32 59,3 31 57,4 24 44,4 5. strongly agree 15 27,8 14 25,9 15 27,8 18 33,3 moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 131 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 total 54 100 54 100 54 100 54 100 table 4. survey results for cross curricular skills (iv) skill 13. responsibility towards learning answers freq. % 1.strongly disagree 0 0 2. disagree 1 1,9 3. uncertain 8 14,8 4. agree 30 55,6 5. strongly agree 15 27,8 total 54 100 in the question related to the creativity, 27.8% believes they have very much seized this skill, 48.1% much, 22.2% somewhat and 1.9% little. it can also be seen that 25.9% believes they have worked very much entrepreneurial abilities through pbl, 59.3% much, 13% somewhat and 1.9% little. when the students were asked about how pbl has helped them to be aware about the quality, 27.8% thinks it has helped them very much, 57.4% much, 13% something and only one student believes it has helped him/her little. table 3 also shows that 33.3% of students believes pbl has helped them very much to improve their project design and management skills, 44.4% much, 20.4% somewhat and 1.9% little. it can also be seen that 27.8% of students see pbl as a very good tool to be responsible of their own learning, 55.6% thinks it is good, 14.8% thinks it is moderately useful and 1.9% thinks is not very useful. the students perception of skills related to team working skills are summarised in table 5. table 5. survey results for team working related skills skill 14. team working 15. negotiating work load 16. discussing, listening and finding agreements 17. team working towards enhanced learning answers freq. % freq. % freq. % freq. % 1.strongly disagree 1 1,9 0 0 1 1,9 1 1,9 2. disagree 2 3,7 3 5,6 2 3,7 2 3,7 3. uncertain 11 20,4 10 18,5 10 18,5 13 24,1 4. agree 24 44,4 29 53,7 32 59,3 24 44,4 5. strongly agree 16 29,6 12 22,2 9 16,7 14 25,9 total 54 100 54 100 54 100 54 100 moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 132 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 table 5 shows that 29.6% of the samples believes that pbl has improved very much their abilities towards team working, 44.4% much, 20.4% somewhat and 5.6% of the students believes the methodology is little or not useful at all to improve team working skills. 22.2% thinks they have learnt very much to negotiate working load with their peers, 53.7% much, 18.5% little and 5.6% little. additionally, 16.7% of the students believe the project has helped them very much is discussing, listening and finding agreements, 59.3% much, 18.5% somewhat, 3.7% little and 1.9% nothing at all. finally, 25.9% of the students worked in teams very much towards an enhanced learning, 44.4% much, 24.1% somewhat, 3.7% little and 1.9% nothing at all. figure i. survey results moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 133 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 4. discussion the results clearly show that cross curricular skills were trained and improved through pbl methodology. very good agreement between lecturers and students was observed in self learning and team working skills. table 4 shows that 83.4% of students have been responsible for their own learning, in agreement with previous works where this skill was also evaluated, kay et. al (2000), chin &chia (2010) o sungur, tekkaya & geban (2010). this suggests that students are quite involved in building up their knowledge based on previous experiences and learning. such knowledge can be used and applied in a wide range of situations. according to the survey, other skills closely related to self-learning also achieved good scoring. skills such as summarising and managing information, information searching skills and decision making scored more than 60%, 75% and more than 80% respectively. this latter point is key in the current society with increasing complexity and an ever higher number of challenges and demands. one of the factors responsible for the increasing complexity of the society is the development of new icts along with new systems of communication and information sources. this fact makes necessary a change in the role of the teachers with respect to the traditional one (marcelo, 2001). thus, the job and main tasks of the lecturers are renovated and they become helpers towards the learning process of the student. the use of pbl requires a change in the main functions of the lecturers but also a change on evaluation methods. traditional pedagogic methodologies cannot be used to evaluate cross curricular skills and a new evaluation system is required. this new system must be more dynamic and much included the knowledge, the know-how and how to behave (alsina, 2011). the ideal method for evaluation requires the student to assimilate knowledge, skills and attitudes. such method leads to facing reasonably complex tasks, generally involving a range of different subjects (monereo, 2009:92). thus, pbl is a methodology that promotes an educational evaluation aiming at improving the learning process. such evaluation can be done through learning rubrics. these tools, together with other tools suggested by the students, moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 134 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 such as self-assessment and peer assessment, can help the lecturers to improve this skillbased approach. regarding team working skills, surveys reveal that 75% of students understand that pbl has helped them to strengthen this skill and other related skills such as negotiating, discussing, listening and finding agreements with their peers. nevertheless, students have experienced a number of difficulties often appearing when team working. most of these problems can be solved with self-responsibility. each team member must take responsibility for the tasks and goals that were assigned to him/her. committing oneself for different tasks is the best way to produce good group results. this degree to which members of the group are mutually dependent on each other, named interdependence, increases the motivation and thus the performance, both from the group as a whole and of each team member. prieto (2007) points out that self-responsibility means that each member should contribute somehow towards the learning process and the success of the group. in addition, self-responsibility also means that each member should be able to demonstrate his/her skill to his/her peers. consequently, it is very advisable to spend some time at the beginning of the project to raise awareness about team working and make commitment to complete different jobs defined by the group. such approach can be enhanced by involving all team members in leading the group. this can be done by exchange periodically the functions within the group. 5. conclusions the current work explains in detail a number of advantages and disadvantages of pbl. the authors believe the experiences herein described can be very useful to other lecturers that are considering introducing pbl in their courses and also to lecturers that have already put pbl into practice. moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 135 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 overall good agreement was observed on how students and lecturers perceive pbl activities. most activities are rated as very positive, both by the students and the lecturers. we have demonstrated that pbl is a very powerful tool to improve and evaluate cross curricular skills in materials science courses of engineering degrees. 6. acknowledgements authors wish to thank the vicerrectorado de estudiantes, ocupación e innovación educativa from universitat jaume i de castellón for the financial support through the project (pie 2907/14). additionally the authors thank the vicerrectorado de innovación docente from university of the basque country upv/ehu the financial support. 7. references alsina, j., (coord.). (2011). evaluación por competencias en la universidad: las competencias transversales.barcelona: octaedro. barrows, h.s. (1986). a taxonomy of problem-based learning methods. medical education, 20(6), 481–486. http://dx.doi.org/10.1111/j.1365-2923.1986.tb01386.x j. carreras, (2008). cuadernos de docencia universitaria 05, ediciones octaedro, barcelona chin, c. & chia, l. (2010). implementing project work in biology though problem-based learning. journal of biological education, 38(2), 69-75. http://dx.doi.org/10.1080/00219266.2004.9655904 hernández, c., fernández, c. y baptista, p (2001). metodología de la investigación. méxico: mcgraw hill. moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 136 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1111/j.1365-2923.1986.tb01386.x http://dx.doi.org/10.1080/00219266.2004.9655904 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 jonassen, d. h., s.,khanna k., implementing problem based learning in materials science,proceedings of the 2011 asee annual conference and exposition, vancouver, 2011. kay, j., barg, m., fekete, a., greening, t., hollands, o., kingstone, crawford, k. (2010). problem-based learning for foundation computer sciencie course. computer sciencie education, 10(2), 109-128. http://dx.doi.org/10.1076/0899-3408(200008)10:2;1c;ft109 marcelo, c. (2001). aprender a enseñar para la sociedad del conocimiento. revista complutense de educación, 12(2), 531-593. de miguel, m. (coord.). (2005). metodologías de enseñanza para el desarrollo de competencias. orientaciones para el profesorado universitario ante el espacio europeo de educación superior. madrid: alianza. monereo, c. [coord.] (2009). pisa como excusa. repensar la evaluación para cambiar la enseñanza. barcelona: graó. moust, j.h.c., bouhuijs, p.a.j., schmidt, h.g. (2007). el aprendizaje basado en proyectos: guía del estudiante. cuenca: ediciones de la uclm. perrenoud, ph. (2001). la formación de los docentes en el siglo xxi. revista de tecnología educativa, 2001(3), 503-523. santiago de chile: oei-credi prieto, l. (2007). el aprendizaje cooperativo. madrid: ppc. reeves, t y laffey, j. (1999). design, assessment, and evaluation of a problem-based learning environment in undergraduate engineering.higher education research and development, 18(2), 219-232. http://dx.doi.org/10.1080/0729436990180205 sungur, s., tekkaya,c., y geban, o. (2010). improving achievement through problembased learning.journal of biological education, 40(4), 155-160. http://dx.doi.org/10.1080/00219266.2006.9656037 schmidt, h.g. (1983). problem-based learning: rationale and description. medical education, 17, 11-16 http://dx.doi.org/10.1111/j.1365-2923.1983.tb01086.x moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 137 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1076/0899-3408(200008)10:2;1-c;ft109 http://dx.doi.org/10.1076/0899-3408(200008)10:2;1-c;ft109 http://dx.doi.org/10.1080/0729436990180205 http://dx.doi.org/10.1080/00219266.2006.9656037 http://dx.doi.org/10.1111/j.1365-2923.1983.tb01086.x multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3896 social and technological sciences eissn: 2341-2593 m. valero (2007). las dificultades que tienes cuando haces pbl, la educación superior hacia la convergencia europea: modelos basados en el aprendizaje, universidad de mondragón, capítulo 8. m. valero, j. garcía, (2011). jornadas de enseñanza universitaria de la informática (jenui) cómo empezar fácil con pbl, sevilla. vizcarro, c. y juárez, e. (2009).¿qué es y cómo funciona el aprendizaje basado en proyectos?. la metodología del aprendizaje basado en proyectos. consultado en fecha 03/03/2015. http://www.ub.edu/dikasteia/libro_murcia.pdf moliner et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 121-138 | 138 http://polipapers.upv.es/index.php/muse 6. acknowledgements multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 49 circumstances and conditions in the accreditation process of university degrees at the albacete school of industrial engineering (spain) v. miguel-eguía, m. garcía-teruel, j. a. martínez-martínez, f. garcía-sevilla escuela de ingenieros industriales. universidad de castilla-la mancha, avenida de españa s/n, campus universitario, 02006 albacete, spain * corresponding author: email: valentin.miguel@uclm.es; phone: + 34 967599200 received: 2016-05-12; accepted: 2016-08-11 abstract this work considers some of the main aspects involved during the period that elapsed between the validation of our degrees to the present time, which the school of industrial engineers of albacete are currently under-going. one of the most disturbing factors has been teaching staff, who have been subjected to spain’s economic circumstances. other considerations, related to some difficulties of the re-accreditation process, have also been taken into account. finally, we point out that although new learning methodologies were expected, the scope and the way quality systems operate have arrived somewhat late. keywords accreditation, quality guarantee, quality systems http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 50 1. introduction as we all know, the creation of the european higher education area (ehea) has meant a complete transformation in european convergence in this area, and has led to recent diverse bibliography that has analysed these aspects, basically in relation to the role of european interaction in the convergence process to determine the importance of mobility programmes and the role of research in the european convergence process (teichler, 2006). another outstanding aspect about which more forums and bibliography have been generated in recent years has been the role of new teaching methodologies in the process. for instance, the reference work (benito & cruz, 2006) can be consulted in many citations on scholar google. the evaluation system, which has been contemplated in particular, has also given way to a large body of literature on continuous evaluation processes and evaluation by competences. these last two circumstances even call into question the concept of releasing professorship, which has been overestimated in the past (vidal, 2012). after having necessarily referred to the indicated aspects, determining the importance of a system which guarantees degrees is essential, this being the backbone of european convergence that forms part of mutually recognising the competences that students acquire, as set out by the press release made by european ministers about the agreements reached during the bergen conference held in may 2005 (bergen, 2005). the text below stands out: “we adopt standards and guidelines to ensure a quality ehea, as proposed by enqa. we are committed to introduce the proposed peers evaluation model of national quality agencies by respecting commonly accepted criteria and patterns. we take the principle of a european quality agencies register based on national reviews. we request its practical application to be undertaken by enqa in cooperation with eua, eurashe and esib, which will send a report through the follow-up group. we emphasise the importance of cooperation among nationally recognised agencies in order to enhance mutual recognition from decisions made about quality accreditation or guarantee.” http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 51 the above paragraph indicates not only the existence of a process to set up national and supranational quality guarantee agencies, but also the need for coordinating shared criteria and for a european quality agencies register. the importance of cooperation between nationally recognised agencies complements this register, and is a main objective in europe, and also in spain between aneca (the spanish quality and accreditation evaluation agency) and spanish autonomous agencies. to further complement this register, cooperation among nationally recognised agencies is relevant to enhance mutual recognition from decisions made about quality accreditation and guarantee. these national agencies not only evaluate by examining the peculiarities of each system, but also work as a network and establish mutual recognition processes. this objective has been set in europe, and also in spain between aneca and the spanish autonomous agencies (marcellán, 2005). an interesting description of the european accreditation system before the 2005 bergen conference is found in the work by michavila and zamorano (michavila & zamorano, 2007). certainly during the verifying process of bachelor memories imparted in the industrial engineers school of albacete, we all considered the importance of the re-accreditation of degrees after the 4-year period, although other factors also came into play, which probably did not allow us to pay all our attention to this process; among them, we list the role of the new evaluation methodologies and systems. having verified these specifications, the centre’s internal quality system was gradually shaped by naming course and degree coordinators, and by setting up an internal quality guarantee committee, which is beginning to operate with the preparation of the first degrees follow-up report. some authors stress the fact that this aspect has been incorporated later in spain than in other european countries (galán-palomares, 2008). royal decree 1393/2007, of 29 october, sets out the legal framework to organise official university teaching by defining the process to follow up degrees and by allocating aneca and the spanish autonomous evaluation agencies their respective competences (rd 1393/2007). to comply with this royal decree, in 2009 (aneca, 2010) aneca defined the reference framework to follow up the http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 52 bachelor and master degrees included in the register of universities, centres and degrees (ruct). this it did by collaborating with the agencies that form part of the spanish university quality agencies network (reacu), and it did so by taking the basic criteria as a basis to follow up the official degrees defined by these evaluation agencies. in 2010, aneca (aneca, 2011) offered the possibility of participating in a pilot scheme to follow up the undergraduate and master degrees already implemented in the spanish autonomous communities that did not have an evaluation agency (asturias, cantabria, castilla-la mancha, extremadura, la rioja, navarre and murcia), and to universities that answered to the spanish ministry of education and concordat universities. this process continued until the so-called monitor programme came into being in 2011, in which aneca ran the aforementioned pilot scheme by taking 59 bachelor and 34 master degrees as a reference, of which respectively 7 and 6 corresponded to the area of engineering and architecture (aneca, 2012). it can thus be concluded that learning methodologies and evaluation by competences were dealt with by pilot schemes well before post-bologne plans were implemented into the school of industrial engineers of albacete, while awareness and the definition of quality systems came later. in the school of industrial engineers of albacete, verification reports were prepared and presented to aneca before a monitor programme outline was clearly defined by the agency. very little was known about the dimensions and the way to prepare the re-accreditation self-reports we were recently involved in, nor about the process which we shall have to undertake in the near future. while the written reports were being prepared, nobody was able to foresee the vast effect that the economic crisis we are still going through would have, which has greatly conditioned the development expected to take place in the years following the degrees verification process. http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 53 this work presents what the teachers belonging to the school of industrial engineers of albacete have progressed, as far as academic aspects are concerned, the way the economy has evolved, indicators, and aspects related to the difficulty of managing the reaccreditation process owing to some changes in criteria made by aneca during the period involved. 2. socio-economic factors to consider at the school of industrial engineers of albacete when the verification reports were being prepared, the socio-economic situation of spain, and of the spanish autonomous community of castilla-la mancha, allowed some more or less optimistic extrapolations to be made in not only staff matters, but also in relation to general laboratory and teaching means. as we all know, this situation was compromised very shortly afterwards by the severe economic crisis of spain, which is still ongoing. the impossible situation of organising permanent job posts and the application of a 10% replacement rate have generally brought the development of spanish universities to a standstill, which is particularly marked in our university, the uclm. the law passed in 2011 on implementing a replacement rate below 10% has been the strictest of spain’s recent history (see table 1) (montesinos et al., 2014). other factors that have affected our school of industrial engineering are considered and listed below. although technical industrial engineering studies have been taught since 1978 at what was then the polytechnic university school of albacete, in april 2008 the school of industrial engineers was created to accommodate studies related to the industrial field. at that time, such studies, plus computer engineering studies, were taught in the same centre. this had its peculiarity because it affected the way the personnel of each centre were assigned. basically a division took place by implementing an internal teaching load organisation plan for both types of school, which considered the reality of that time. the change of syllabi in the already existing higher school of computer studies and the school http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 54 of industrial engineering evolved differently compared to their initial starting points. this led to certain asymmetries in the personnel of each school. during the period when postbologne degrees were taught, some members of staff were re-assigned. however, the authors believe that this situation was strongly conditioned by the division into certain subjects that took place. table 1. main measures that have affected jobs in public administrations (pa) since 1993 (montesinos et al. 2014) year source description no. of new posts in public administration 1993-1994 gpe (general plan of employment) exclusively limited to those considered essencial 1995-1996 gpe below the staff members replacement rate 1997 gpe below the 25% staff members replacement rate 1998-1999 gpe 2000-2001 gpe 2002 gpe 2003 gpe below the staff members replacement rate 2004 gpe 2005-2006 gpe 2007 gpe 2008 gpe 2009 gpe below the 30% staff members replacement rate 2010 gpe below the 15% staff members replacement rate 2011 gpe below the 10% staff members replacement rate 2012 “real decreto ley” 20/2011 and gpe no new staff members are admitted 2013 gpe 2014 gpe new part-time staff members and interim civil servant job posts 1998-2002 gpe no part-time personnel can be contracted, nor can interim civil servants be assigned, save in exceptional, urgent and necessary cases 2003 gpe 2004 gpe 2005-2011 gpe 2012-2014 gpe http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 55 in our school, three pre-bologne technical industrial engineering bachelor degrees of three year-duration were taught, which were transformed into their respective post-bologne four year-degrees. afterwards, the master degree of university industrial engineering came into being. both situations implied growth, which was initially compensated for the bachelor degrees, and also by re-adapting teachers to cover the master degree. with this new structure, and after creating today’s centre, studies considerably increased as the number of students went from 680 to 1,050, mainly because the bacherlor degrees lasted longer. the teacher profiles of our school during the pre-bologne process resulted from the development of both these studies and the centres where they were taught. the most important milestones were not only the way the syllabi evolved, but also the incorporationelimination of new studies from this branch, or others, in the centre. the most outstanding consequence was that certain knowledge areas were lacking: manufacturing processes engineering, fluids mechanics and projects engineering. the usual teaching of these areas was taught from other areas. some knowledge areas were poorly justified, such as signal theory, and were marginalised as far as number of staff members were concerned, of which there was only one for the above area. another point we stress was the incorporation of the chemical engineering area, which did not appear within the pre-bologne framework. finally during the indicated period, nine full-time teachers retired. 3. progress of the teachers who taught degrees over the 4-year period covering 2010/2011-2013/2014 table 2 provides the figures which indicate the progress made by the teachers of each bachelor degree during the period these degrees were accredited. as we can see, the number of teachers was cut by about 12%. the greatest loss affected part-time lecturers, which the uclm tightened by means of budgetary cuts due to the economic crisis. this loss has been partly compensated by reorganising members of staff and by increasing the teaching http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 56 workload of some part-time lecturers who currently remain. even though the analysis has been done by taking the academic year when the written reports were forwarded to aneca and academic year 2013/14 as references, if we bear in mind the forecasts made for academic year 2015/16, the situation will improve compared to that indicated. as we can see in table 2, permanent staff members exceed 80%, which is an adequate percentage according to an objective criterion, especially when we consider the number of retirements that have taken during the study period and the difficulty in replacing them. table 2. progress made by the centre’s teachers per category and degree mechanical engineer. bachelor electric engineering. bachelor electronic & automat. engineering bachelor category academ year 2010/11 academ. year 2013/14 academ. year 2010/11 academ . year 2013/14 academ. year 2010/11 academ. year 2013/14 full professor 1 5 1 4 2 5 university lecturer 13 15 12 13 11 13 bachelor lecturer 25 18 30 18 27 20 contracted phd lecturer 4 4 3 4 4 5 assistant phd lecturer 2 2 1 3 4 2 assistant lecturer 0 0 1 0 0 0 partial-time lecturer 17 11 15 14 15 9 total 62 55 63 56 63 54 3.1 teacher status during the years that the re-accreditation period covered, the number of 6-year periods increased considerably in the centre compared with the number during the pre-bologne period; an increase of more than 10% has been recorded. another aspect related with tenured lecturers is that quite a high percentage of teachers are accredited as assistant phd lecturers and contracted phd lecturers, which implies the http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 57 possibility of replacing permanent staff members in the near future, especially when the spanish ministry of education, culture and sports has announced an increase to 50% in the replacement rate. 4. economic figures related to managing teaching means the budget that the various uclm centres (around 40 in all) receive come mainly from the two different uclm funds: funds that correspond to the centre’s usual operating costs (general budget –gb–). these funds are estimated basically according to number of students and to the mark related to the studies experimentalism. it is worth indicating that of all the centres that comprise the uclm, our school alternates between being in first and second place as far as the amount received is concerned (it competes with the framework centre of the school of law and social sciences in the city of ciudad real, with over 2,000 students). evidently we are first on the list of technical/scientific centres. variable funds, depending on the objectives met, which form part of a programme contract (pc) agreed on with the regional government of castilla-la mancha. in this case, the parameters used to decide on these funds follow coefficients, which are considerably more heterogeneous, and even the quantities allocated to each centre depend on other qualitative considerations (not quantitative ones as in the case of general funds). in any case, our centre occupies the first five to eight positions of the total number. since academic year 2010-2011, in which post-bologne bachelor degrees began (this was done through immersion in our centre; that is, all years at the same time), the changes made to the funds received by the industrial engineers school of albacete (eiiab) are illustrated in figure 1 (quantities are provided in calendar years, and not in academic years). http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 58 figure 1. changes in the funds received (per calendar year) these changes reflect the impact of the economic crisis (2012 in particular) and gradual recovery (still not completely reached). this fact has negatively affected the investments made in acquiring teaching and research material, mainly in contracting teaching staff, as we previously mentioned. regarding the funds received through the pc, table 3 briefly summarises the basic points considered. each centre decides what points to increase (that fall within preset intervals), and by what percentage, according to its most immediate objectives. when budgets lowered, the whole centre’s budget was centralised. previously to the crisis effects, almost 70% was shared among the various areas so they could operate individually. what this centralised management has accomplished is to optimise expenses and to guarantee that teaching is not affected. at the same time, the shares decided on in the past excessively fragmented the budget, which meant that some laboratories did not receive enough funds which, in turn, led to substantial shortages. the generic computer rooms suffered a similar situation. indeed, despite the increase in financing being considerable, http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 59 the centre’s board decided to continue with this budget policy as it proved much more efficient than the former model. table 3. criteria considered in the programming contract training if-1 number of new students if-2 mean mark for new students to start university studies if-3 drop-out rate if-4 rate of students on exchange programmes if-5 rate of oversees students on exchange programmes if-6 practical placements rate if-7 academic efficiency rate if-8 academic staff’s pedagogic training rate if-9 student evaluation of the training they receive research and generating technology ii-1 rate of phd teachers ii-2 rate of awarded research 6-year periods ii-3 rate of research activity results ii-4 participation rate in regional research projects ii-5 participation rate in national and international research projects ii-6 participation rate of principle investigators in public research projects ii-7 rate of financing obtained from public calls ii-8 rate of financing obtained in the contracts set out in art. 83 of the lou law in relation to one’s environment iv-1 continuous training rate iv-2 rate of cultural events iv-3 rate of bibliographic investments iv-4 tele-teaching rate iv-5 rate of mastering foreign languages iv-6 participation rate in training plants iv-7 occupational hazards rate 5. variation in the criteria used by aneca during the re-accreditation process prior to the re-accreditation process that we are currently going through, it was necessary to make the verification reports “official” by introducing them into a telematic application that aneca provided universities with. this has been a difficult process as it was http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 60 necessary to adapt these previously approved reports to an indicated application. in fact, the process has become an exercise to amend reports, whose most relevant points are indicated below. 5.1 evaluation of proposals to amend syllabi regarding the evaluation of the proposal to amend the syllabi of the university industrial engineering degrees that our university teaches, the uclm forwarded to aneca the proposals of three degrees on three different dates: -bachelor degree in industrial electronic and automatic engineering. the proposal was sent in september 2014. -bachelor degree in mechanical engineering. the proposal was sent in november 2014. -bachelor degree in electrical engineering. the proposal was sent in march 2014. in its evaluation reports, aneca stressed the obligation to rectify any aspects related to criterion 4 (student access and admission). these reports highlighted two very relevant points: recognition of official non-university higher education credits; recognition of credits for students over the age of 40 with work or professional experience. 5.2. recognition of official non-university higher education credits on 31 october 2014, aneca issued its evaluation report on the proposal, sent in september 2014, of amending the syllabus of the uclm bacherlor degree in industrial electronic and automatic engineering. in its report, aneca requested, among other aspects that had to be necessarily rectified, the explicit conclusion of the amendments made as to the recognition of the official non-university higher education qualifications not included in the previously verified report. it also added that in order to evaluate the suitability of these recognitions, knowledge and results of learning, a comparative table http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 61 must be provided that corresponds to the subject matters of at least one non-university higher education degree, whose competences could be recognised in this degree. this request was related to the fact that the previous version of the verification report of these studies, which had been favourably evaluated in 2010, did not include any agreement about explicitly recognising advanced vocational training students since such recognition was not legally acknowledged until 2011, when royal decree 16/18/2011 was published on 14 november on recognising studies in the higher education domain (boe no.302, of 16 december 2011, sec.1, pp 137575-137588). regardless of the conflictive opinion that such recognition sparked (many teachers believe that this situation should not take place as they are qualifications with clearly different academic competences), some university studies, for which fewer students have registered in recent years, have seen in their recognitions a way to attract students from non-university higher education. thus, logically, and as the royal decree (rd) sets out, this type of agreements signed between a university and education administrations “must be informed to the spanish ministry of education and be officially published”. consequently, aneca must ensure that such recognitions are made in accordance with that set out in this rd. hence at that time, the request it asked of us to evaluate the suitability of the recognitions that the uclm signed with the regional ministry of education as to its degree in industrial electronic and automatic engineering is logical. about such recognitions, the question is: at a later date, did aneca stick to the same criterion adopted in the evaluation it issued about the proposal to amend the other two syllabi that our university forwarded it? the answer if affirmative, be it with a slightly differentiating nuance, as a similar text to that previously indicated was sent to us for both the reports about amending the syllabi of the degree in mechanical engineering and the degree in electrical engineering. however, it was necessary to attach an annexe to the latter, which listed the university teachings of this degree and of the advanced vocational training qualification for which recognitions could take place. http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 62 5.3. recognitions of credits for the over 40s with work and professional experience another aspect that aneca outlined to be rectified in its evaluation reports was related with the recognitions of credits obtained through professional experience of people aged over 40. that set out in the previous point can also be cited here because in the previous version of the verification report to amend these studies, evaluated as favourable in 2010, no explicit criterion was included to recognise people with work experience and professional experience. such recognitions are controlled through organic law 8/2013, of 9 december, on the improvement of quality education (lomce), which modified the requirements to gain access and be admitted to official degree teachings, and also specifically by rd 412/2014, of 6 june, which included the basic regulations of the procedures by which admission to university degree teachings is accomplished (boe no. 138, sec. i, pp. 4330743323, of 7 june 2014). so it is logical that aneca pays special attention to this point since the legislation that affects it has been recently modified. yet we wondered if the identical criteria were maintained on this point in the evaluation of the three university degrees forwarded to it. in this case the answer is negative since the criterion about this point clearly varied de-pending on the time each report was evaluated: the evaluation report about the degree in industrial electronic and automatic engineering, which was the first one sent, mentioned nothing about this matter. with the report of the degree in electrical engineering, sent several months later, aneca requested that the report of the verified syllabus included the accreditation criteria, as well as the area where the work and professional experience was acquired in relation to both these degrees. 6. conclusions what comes over from this work is that the centre’s background, along with the economic situation while the accreditation process of the degrees taught at eiiab was underway, have implied difficult situations for maintaining members of staff and teaching resources. http://dx.doi.org/10.4995/muse.2016.4547 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 63 yet efficient work has been done despite these difficulties, and these aspects have been overcome. finally, it is absolutely necessary to plan personnel means in order to achieve successive re-accreditation processes. although the changing framework introduced into the process, performed to make reports official by applying the corresponding telematic application, has allowed the rectification of some minor errors in the previously approved verification reports, it has introduced absurd asymmetries into the management that is common to all degrees. so after having accomplished the re-accreditation process, the intense task of rewriting records has to be faced. based on experience in implementing plans, correcting criteria to recognise not only the different credits in each degree, but also the reconsideration of evaluation systems and competences assigned to each subject matter, will be essential in order to better adapt to these issues. 5. references aneca (2010), memoria de actividades 09, aneca pp. 94. aneca (2011). memoria de actividades 10, aneca pp. 84 aneca (2012). memoria de actividades 11, aneca pp. 85 benito, a., cruz, a. (2006). nuevas claves para la docencia universitaria en el espacio europea de educación superior. ed. narcea madrid 137 pp. comunicado de la conferencia de ministros europeos responsables de educación superior. bergen, 19-20 de mayo de 2005. www.ond.vlaanderen.be/hogeronderwijs/bologna/links/language/2005_bergen_communi que_spanish.pdf galán-palomares, f.m. (2008) la participación estudiantil en los sistemas de garantía de la calidad, compromiso de mejora. univest08, girona, june pp. 11. http://dx.doi.org/10.4995/muse.2016.4547 http://www.ond.vlaanderen.be/hogeronderwijs/bologna/links/language/2005_bergen_communique_spanish.pdf http://www.ond.vlaanderen.be/hogeronderwijs/bologna/links/language/2005_bergen_communique_spanish.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4547 social and technological sciences eissn: 2341-2593 miguel-eguía et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 49-64 | 64 marcellán, f. criterios de garantía de calidad en educación superior: praxis europea. educatió (2005) 23 pp. 15-32. michavila, f., zamorano, s. (2007). la acreditación en el espacio europeo de educación superior. https://upcommons.upc.edu/revistes/bitstream/2099/7526/1/16_246-259.pdf montesinos, a. pérez, j.j., ramos, r. (2014). el empleo de las administraciones públicas en españa: caracterización y evolución durante la crisis, documentos ocasionales nº 1402, madrid, banco de españa. real decreto 1393/2007 de 29 de octubre. http://www.boe.es/boe/dias/2007/10/30/pdfs/a44037-44048.pdf teichler, u. el espacio europeo de educación superior. visiones y realidades de un proceso deseable de convergencia. revista española de educación comparada, 12 (2006), pp. 37-79. vidal, c. el espacio europeo de educación superior y su implantación en las universidades españolas. revista catalana de dret públic, núm. 44, 2012, p. 253-283. http://dx.doi.org/10.4995/muse.2016.4547 https://upcommons.upc.edu/revistes/bitstream/2099/7526/1/16_246-259.pdf http://www.boe.es/boe/dias/2007/10/30/pdfs/a44037-44048.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 1 an overview of the management consulting firms of the spanish company in china: company profile and success and failure factors a. niñerola*, f. campa-planas, a.b. hernández-lara, m.v. sánchez-rebull business management department. universidad rovira y virgili av.universidad 1, 43204 reus, spain. * corresponding author: email: angels.ninerola@urv.cat; phone: + 34 977759865 abstract this article explores the vision of management consulting firms (mcf) specialized in internationalization processes of spanish companies in china. the main objective is to shed light on a process full of uncertainty as is international expansion into emerging markets on the point of view of the advisors of that process. first, it is analyzed whether there is a spanish company profile that is internationalized toward china in terms of sector of activity, size and experience in international business. and second, with their client’s experiences, success and failure factors in the internationalization process are studied. to achieve those objectives in depth interviews were held with partners and managers of six management consulting firms with spanish clients in their portfolio. as a result it is concluded that the sectors where the spanish company has invested are several, the size of the company is important to undertake an implementation process in china and its international experience as well. eight key factors were found relevant to invest in china successfully. five factors are related with their resources (humans and financials) and the other three are related with their strategy. the main reasons of failure are also discussed. the article provides an outside perspective on the reasons why an internationalization process is successful and other fails. keywords management consulting firms; internationalization; foreign direct investment; company profile; chinese market; key success factors multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 2 1. introduction in a global economy, many companies seek to extend their domestic market by finding new customers and profitability. the removal of trade barriers, greater geographic mobility, both of products and of people, and the reduction of transport costs, means that many companies consider expansion as a way of benefiting from higher production and sales volumes. on the one hand, internationalization allows them to remain competitive and becomes a strategic option for the company (kyläheiko et al., 2011; sapienza et al., 2006), but on the other, it generates high uncertainty and risk, this is why it is a widely studied area. this eagerness to compete leads many companies to choose to invest in emerging markets because they offer greater business opportunities (enderwick, 2009; sakarya et al., 2007). usually these countries are characterized by having many natural resources, a developing infrastructure and an expanding economy, making them attractive for foreign investment (chen, 1996; wang and guo, 2011). within the emerging market category, china is the largest recipient of foreign direct investment (fdi) worldwide (the worldbank, 2013), initially because its production costs were low and, latterly, due its domestic market potential, especially in the east and south (cui and liu, 2000). although the trend in spanish business has been to move to places like south america or europe (galán and gonzález-benito, 2001) because they are closer culturally (johanson and vahlne, 1977; johanson and wiedersheim-paul, 1975) and physically, for the above reasons, more and more companies are choosing china as an investment destination (moyano vázquez, 2009). our study focuses on this asian country for two reasons, its current international importance, and because, as a non-traditional destination for spanish business investment, it is under-researched. the literature has studied the internationalization process, from a qualitative, quantitative and theoretical methodological perspective, for the industrial and service sectors and in multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 3 many countries and regions, but always from the point of view of the company that is going international. in this study, we analyze spanish companies that are investing in china. due to the country's complexity, many companies require the services of a management consulting firm (mcf) during deployment. since international consultants have an objective and experienced view of the internationalization process of spanish companies in china, the present article focuses on their opinions. some previous studies make a connection between the internationalization process and the mcfs, taking the latter as the analysis unit, rather than the companies they advise. some authors have identified the entry barriers that mcfs face in their own internationalization process (freeman and sandwell, 2008), other have looked into their motives in internationalizing (glückler, 2006; poulfelt et al., 2014), but none have investigated their opinions on this means of business growth for their clients. mcfs have multiple views of this process, many of their customers have succeeded but many also have failed, so the opinions of mcfs may be of value, but have not previously been studied. thus, the aim of this paper is to explore the realities facing spanish companies in china, from the point of view of their internationalization advisers. the first research question is whether there is a type or profile, as defined by terms of sector of activity, size or international experience, for a company that goes international in the asian market. the second question is whether consultants experienced in advising companies in china have found factors that influence the success or failure of the process. the data for answering these questions are drawn from in-depth interviews with partners or directors of these specialized consulting firms. the remainder of this paper is organized as follows: in section two we will review, firstly, the literature on profiles of companies that have gone international, their size, sector and international experience and secondly, internationalization success and failure factors. then, in section three, we explain the methodology followed during the study, continuing multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 4 in section four by analyzing the case studies that arise. finally, we discuss the results and present the main business implications. 2. theoretical background traditional theories on internationalization suggest that companies go abroad not only because they have competitive advantages, whether in production, technology or other areas (hymer, 1976; kindleberger, 1969), but also because going abroad implies lowers costs (buckley and casson, 1976; caves, 1996; dunning, 1988). although it is true that in the early stages of the opening of the market in china, when they first began to receive substantial direct investment from foreign companies, these were motivated by competitive cost advantages (boisot and meyer, 2008), nevertheless this paradigm has now changed remarkably and other factors, such as the potential of the chinese domestic market, make investment attractive (chen and yeh, 2012). so, current internationalization theories, for example, network theory (hosseini and dadfar, 2012), better explain the motivations of companies in internationalizing than do traditional theories. this is because the decision is based, not only on the productive competitive advantage of the company, but also depends greatly on the strategic alliances developed by the company. in china, these alliances give added value for competing and succeeding in the market (gao et al., 2012; tung and worm, 2001). it is evident that companies internationalize because there is some advantage in so doing, whether this be in production or in relationship terms; but what are the characteristics of a company that undertakes this internationalization process? much of the literature has focused on the size of companies and the international experience of their executives for analyzing their propensity to undertake business beyond their borders. whether the company size is a relevant characteristic of the firm that decides to invest abroad is a controversial issue. in regard to direct investment, because of the resource multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 5 needs, the bigger the company, the greater its propensity to internationalize (lin, 2010; todo, 2011). although there are authors such as reynolds (1997) who claim that small-tomedium enterprises (smes) are growing in the international market mainly in sectors where economies of scale are not necessary, others like reuber and fischer (1997) assert that smes can cope with the internationalization process and overcome their lack of resources using other entry mechanisms such as networking (chetty and blankenburg holm, 2000). in the export field, though, there are more authors who have shown that an sme can be successful (dean et al., 2000) and some of these even being major exporters, according to wagner (1995). in contrast, the literature on top management's international experience is in agreement that, the greater the management experience and training, the greater the propensity to internationalize ((moreira et al., 2013; nielsen and nielsen, 2011; reuber and fischer, 1997; tihanyi et al., 2000). it is this management team experience that allows companies with less resources to internationalize, by using this intangible resource for finding less capital-intensive methods, such as strategic alliances with partners (reuber and fischer, 1997). using the views of the mcfs interviewed in the study, we aim to compare, on the one hand, whether a company needs to be big to successfully invest in china or whether there are also successful spanish smes and, on the other hand, if the international experience of these companies has been crucial during the deployment process. the literature on success and failure factors for internationalization is sparse. only a few authors, such as brenes et al. (2008), yang and lee (2002) and zeng et al. (2009) have explicitly studied this topic. of these three works, the first focuses on latin america and the other two on china. all three papers suggest that the management team's involvement and commitment are key success factors. yang and lee (2002) add technological capability, good definition of objectives and the relationship with the authorities as three key factors multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 6 and zeng et al. (2009) believe that that it is marketing that has the greatest effect on the company's performance in china. few studies attempt to identify success factors for a company's deployment in a foreign market, most authors focus on a specific factor and its effect on the internationalization process. for example, cremer and ramasamy (2009) focus on the role of the management team as a determinant of success, kyläheiko et al. (2011) and liu and wu (2011) on the firm's technological capability, and authors like luo (1997) or tsang (1998) analyze whether it is institutional influence networks (guanxi) that determine company success in china. consequently, additional empirical evidence on the factors that influence the success of the internationalization process in general and in emerging markets like china in particular, is needed. 3. methodology we use a case study-based methodology to answer the questions in the previous sections. an exploratory and descriptive study (yin, 1981, 1984), based on six interviews with partners and executive managers of consulting firms, was conducted to give a broad perspective of the internationalization of their clients in china. although the qualitative nature of our work makes it difficult to generalize the results, the fact that six cases are studied, taking into account that each of these implicitly includes many customer experiences, leads us to believe it to be representative of the number of spanish companies that are currently developing direct investment activities in china. eisenhardt (1989) supports the number of cases studied in this article because, although he notes that there is no ideal number of cases to be studied in order to understand a phenomenon, he observes that analyzing less than four is not optimal for generalizing the results, while more than ten involves an amount of information that is difficult to handle. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 7 our research was conducted during the period 2012–2014. in-depth interviews of more than one hour with partners or managers of six consultancy firms with spanish clients and with a presence in china were conducted – three by telephone and three in person. before the interviews, the study objectives were explained to the respondents and the confidentiality of their personal data guaranteed, hence we have encoded the names of the consultancies and the respondents names do not appear. each interview was taped for later transcription and analysis in order to ensure verbatim and accurate information gathering. responses were encoded using a computer program to establish patterns or similarities. it should be noted that, in this sample, five consultancies had a long-standing international presence, totaling more than thirty years advising companies, and the other one was recently established exclusively to advise spanish companies in china (see table 1). table 1. characteristics of the management consulting firms. year of foundation mcf’s headquarters number of interviewees interviewee interviewee’s location mcf1 1941 spain 2 former managing partner and actual managing partner of shanghai office china mcf2 1979 hong kong 1 business development manager germany mcf3 2013 spain 1 managing director spain mcf4 1845 united states 1 partner of shanghai office and head of commercial due diligence for mainland china china mcf5 1963 united kingdom 2 partners of barcelona office. tax manager and legal manager spain mcf6 1964 spain 1 managing director spain all the mcf participants in the study provide legal and tax advice for the deployment and further development of overseas activities. mcf2 offers additional recruitment services and mcf3 offers translation services and chinese search engine / social network ranking optimization (seo). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 8 the mcfs have different types of client. regarding size of companies they advise, mcf1 and mcf4 have a portfolio of medium and large enterprises, the first basically of spanish origin, and the second more international. mcf2 and mcf5 also have customers from different countries, but these are usually smaller. finally, mcf3 is dedicated to advising local small businesses. spanish companies of all types are represented in the sample. 4. analysis of cases this section is divided into two subsections which address the questions posed in the introduction. firstly, the mcf views are analyzed to determine whether, based on their experiences with clients, they detect a categorization for spanish companies investing in china. then, in the second subsection, a list of factors, that the mcfs perceive as necessary for these investments to be successful, is set out. 4.1. spanish company profile in china following claver et al. (2005) and moyano vázquez, (2009), we consider company profiles on the basis of three factors: sector of activity, size and experience in international business. in regard to the sectorial activity of spanish enterprises located in china, according to spanish government ministry of economy and competitiveness sources (http.://datainvex.comercio.es), these investments were concentrated in twenty sectors (2012 data). the most important of these (72% weighting) was the financial services sector (sector 64), through the investments made by major spanish banks over the years concerned. eliminating this sector because it distorts the view of the rest, we found that manufacture of vehicles and trade (retail and wholesale), represent 20%, 17% and 10% respectively. these are followed by other manufacturing (sectors 21, 22, 23, 24, 25, 27 and 28) and the food and chemicals (see table 2). table 2. characteristics of the management consulting firms. sector 2012 2011 var. incl.6 4 excl. 64 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 9 10 manufacture of food products 89.360 56.000 59,6% 1,7% 6,3% 13 manufacture of textiles 12.110 11.950 1,3% 0,2% 0,9% 20 manufacture of chemicals and chemical products 88.110 20.140 337,5% 1,7% 6,2% 21 manufacture of basic pharmaceutical products and pharmaceutical preparations 44.770 45.220 -1,0% 0,9% 3,2% 22 manufacture of rubber and plastic products 14.340 14.180 1,1% 0,3% 1,0% 23 manufacture of other non-metallic mineral products 88.410 109.590 -19,3% 1,7% 6,3% 24 manufacture of basic metals 60.970 55.560 9,7% 1,2% 4,3% 25 manufacture of fabricated metal products, except machinery and equipment 122.870 110.480 11,2% 2,4% 8,7% 27 manufacture of electrical equipment 56.080 52.420 7,0% 1,1% 4,0% 28 manufacture of machinery and equipment n.e.c 32.730 22.560 45,1% 0,6% 2,3% 29 manufacture of motor vehicles, trailers and semi-trailers 290.430 271.370 7,0% 5,7% 20,6% 32 other manufacturing 7.200 6.970 3,3% 0,1% 0,5% 35 electricity, gas, steam and air conditioning supply 20.030 18.530 8,1% 0,4% 1,4% 41 construction of buildings 20.920 22.400 -6,6% 0,4% 1,5% 42 civil engineering 2.750 2.700 1,9% 0,1% 0,2% 43 specialized construction activities 48.410 38.680 25,2% 0,9% 3,4% 46 wholesale trade, except of motor vehicles and motorcycles 149.670 115.650 29,4% 2,9% 10,6% 47 retail trade, except of motor vehicles and motorcycles 243.490 173.990 39,9% 4,8% 17,2% 52 warehousing and support activities for transportation 14.190 16.340 -13,2% 0,3% 1,0% 62 computer programming, consultancy and related activities 4.680 4.360 7,3% 0,1% 0,3% 64 financial service activities, except insurance and pension funding 3.699.820 3.271.080 13,1% 72,4% 66 activities auxiliary to financial services and insurance activities 850 700 21,4% 0,0% 0,1% 69 legal and accounting activities -3.980 -3.060 30,1% -0,1% -0,3% 70 activities of head offices; management consultancy activities 50 0 0,0% 0,0% 71 architectural and engineering activities; technical testing and analysis 1.760 0 0,0% 0,1% source: datainvex multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 10 thus, spanish companies are investing in many sectors (moyano vázquez, 2009), and the consultants also note this. according to mcf1, "being china the type of market it is, businesses of all kinds go there". concentrating on growth sectors, chemicals and food are two of the main ones. the mcf2 respondent says: "the kind of spanish company that now goes to china is a company that sells products, mainly food (wines, oils, etc.) but also fashion. then medicalpharmaceutical-chemical companies are also deployed there. and the industrial sector remains important." in relation to size, as being one of the aspects that define the profile of a company investing in china, we found disagreements, both in the literature and in the views of the mcfs. the actual managing partner of mcf1s shanghai office explained that, in his opinion, the typical pattern of internationalization of spanish companies in latin america had been different to that in china. “in the americas, the big companies came first and then the medium-sized ones followed. in china, however, medium-sized companies were the pioneers, the first to move but, in recent years, it is clear that increasingly it is large companies that are setting up shop". today, when the chinese market continues to be very competitive, the consultancies believe that the size of the company that is internationalizing is important because deployment takes a long time, is complicated and requires a level of monitoring and control, which is difficult to achieve without the requisite resources. this does not mean that small and medium companies cannot sell in china. they can export (dean et al., 2000), but direct investment is more complicated, according to mcf1. nevertheless, it is not only size that is important; in this regard the respondents' views are nuanced. based on their experience, mcf4 and mcf5 believe that, depending on the sector and the activity undertaken, size need not be exclusionary. from mcf2, comes the claim that a company's level of expertise is as important as its size: "investment in china, is linked a little to company size but, if you offer a truly specialized product with high added-value or multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 11 differentiated from the competitors, there's no need be a large company at all, as long as you have the financial resources for the investment." the mcfs have found that most companies investing in china are companies that have previously invested, mainly in europe, south america or north africa, so their managers have some experience in international business. china is not usually the first internationalization destination for these companies, they initially invested in markets with more cultural affinity or geographically closer (galán and gonzález-benito, 2001). despite having experience in international markets, because of this asian country's complexity there are many companies that use consultants to carry out the process and, for those that do not use them, it is because they are very large and have a specialized department in their company to carry out these market studies (mcf5). although it is difficult to define a profile of the company that invests in china, we can conclude that its size is important, although there are exceptions, and that this is not usually its first business internationalization destination, it will already have some foreign experience. in relation to the sector of activity, spanish business has invested in several sectors, notably financial. but if we look at which are currently growing with a significant percentage of the total, then chemicals, food and manufacturing, as well as trade, are the dominant investments. 4.2. key success factors in internationalizing to china any internationalization strategy carries many risks for the company (figueira-de-lemos et al., 2011; forlani et al., 2008), even more so if the destination country is as different from spain as is china. for a spanish company, a market like china is not only geographically distant, but poses a major cultural challenge. the psychological distance defined by johanson and wiedersheim-paul (1975), culture, politics, language etc. causes many companies to fail in china. for this reason, some authors (yang and lee, 2002; zeng et al., 2009) have wondered whether there are factors to consider when investing in this country with certain minimum safeguards for success. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 12 this specific question was asked to the mcfs during the interviews and a total of eight items were to be taken into account for survival in china, "survival " understood as achieving results enabling the company to continue this international project (van praag, 2003). not all the mcfs reported the same success factors, but there was broad agreement (see table 3). table 3. key success factors according to management consulting firms mcf1 mcf2 mcf3 mcf4 mcf5 mcf6 commitment and resources x x x x x x top management involvement x x x x x human resources x x x x x a good partner x x x x x x network relationships x x x x differentiation x x x adapt the strategy x x x x x protect technology x x x these factors fall into two groups. the first group consists of the five factors, relating to company resources and relationships, the second, the remaining three factors related to its strategy and products or services. 4.2.1 key success factors related to resources and networks to get good results, all the mcfs consider that the company's level of commitment to the international project is essential. this commitment is reflected in the investment of adequate resources, as much financial as in time and people, since this is a long-term investment and hence not immediately profitable (cui and liu, 2000; lu and beamish, 2001). mcf1: "currently, for china, the level of commitment required is equal to or greater than for spain, [...] so... you have to go to china with a firm commitment, if you go halfway, it almost certainly will go wrong" multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 13 mcf5: "you have to commit the necessary resources and create a structure... sending a couple of expats to look for business does not work." mcf6: "it is a long process and therefore needs resources" this commitment requires the close involvement of top company management as it is an investment that will not pay off immediately, so the consultancies believe that the project is unsustainable without full management support. "going to china is a strategic decision; therefore, senior management must support the project" (mcf5), "and motivate other staff" (mcf6). the business development manager of mcf2 adds that "this does not mean that senior management be in charge of the project, because the decision may come from the cfo, or the export manager. but if this person has not been able to convince the top company management, it makes no sense. and if the company goes ahead without having convinced senior management, it will probably fail. " the third factor to consider is the company's human resources in the country of investment. the mcfs remarked on the importance, on the one hand, of sending top professionals from spain (as did zeng et al. (2009) in his study of business success factors in china) and, on the other, of finding qualified staff in the country. the literature also stresses the importance of this factor for project success (yang and lee, 2002). throughout the interviews, the mcfs explained this as follows: mcf1: "the company did not send a second or third level professional, it sent a first class professional, gave them authority, gave them skills, were there during those five or six years and so we can see that before they had nothing, and now they have an office with 25 professionals. why? because of the people, i.e. the resources, and the wherewithal the company gave him. we have other firms that have not succeeded in china. why? because they sent a lawyer, like he was renting there, then he came, was part-time, hired two chinese and then turned to go, so the continuity, the intensity of the project..." mcf4: "having a capable leadership, with talent, who can bring the project to a successful conclusion. do not send grade c management, rather those of higher quality. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 14 mcf6: "have qualified local staff who speak english" in a country like china, having such a big cultural difference with spain, many companies use a partner, whether an equity partner or a business associate, to develop their activities. this is one of the most studied areas in terms of internationalization processes in china. some authors point out the advantages of using a partner to enter the chinese market. lu and beamish (2001) justify the need to take advantage of a partner's knowledge, if they themselves do not know the market. the right partner can provide strategic, organizational and financial advantages (luo, 1998) since they speed up market entry (glaister and wang, 1993). however, there are potential drawbacks to consider when investing with a partner. according to hu and chen (1996), the partner's level of commitment can be a problem for proper company functioning and zutshi and tan (2009) point to trust as the main drawback of joint ventures in china. all the mcf respondents stress the importance of choosing a good partner. a partner can help companies at the time of their entry into the country, because they know the workings of the market. however, from past experience, some mcfs recommend caution. mcf1: "i would not say that you should avoid a partner because there's always something... but i think before going into china with a chinese partner, do a very careful analysis of what the chinese partner gives you, and what risks there are in having a chinese partner. and if what they bring is really greater than the risk, then it makes sense" mcf2: "the only advice i can give because we do it, is to carry out a due diligence on your partner; even if you have a good relationship." mcf3: "have a chinese partner or a couple of chinese workers who can advise you at all levels, since the person here does not have to know chinese or how everything works" mcf4: "many companies change their partner and not just for cultural reasons. the right partner means that, it's the right match from a business point of view as well as from a relationship point of view. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 15 mcf5: "the partner is essential in helping you understand how business culture works in china, because they have different ways of working to western ones and it costs to close deals." mcf6: "it is important to work with a partner because when you open a branch the aim is that this cultural approach leads to a commercial approach." it could be argued that it is important to have a good relationship with the partner in china, but also in any other country. however, having good relations with the institutions or authorities, in china known as guanxi, is vital in this asian country and can cause project success or failure (gao et al., 2012; luo, 1997; tsang, 1998). having good contacts (social networks), means it is possible to overcome the language and bureaucracy barriers that companies encounter in china (freeman and sandwell, 2008; nair and stafford, 1998). mcf4, 5 and 6 indicate that the government role in china is more than a regulatory issue; hence it is important to have good relations, especially locally. consequently, the person in the company who deals with the administration must be able to relate well to the authorities. the ceo of mcf3 notes that it is very difficult to have guanxi if you are not chinese, so it is desirable that this person be a local. tung and worm (2001) also regard that using an expatriate it is difficult to establish a strong relationship with the authorities. 4.2.2 key success factors related to business strategy moving on to factors related to the company's strategy in china, in three of the six interviews, differentiation is designated a success strategy. wigley et al. (2005), also consider that this factor is critical for success at the international level. the consultants remark that, for a business to succeed in china, it "must be able to offer a product with differentiated know-how" (mcf2), "if you're not a little different, you can't do anything" (mcf3). internationalization requires that each company make country-specific strategy changes, because each deployment project has specific characteristics. legislation may be different, also customer taste, market entry mechanisms: distribution channels, advertising, etc. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 16 (pedersen and pedersen, 2004). so, a company's internationalization strategy is affected by the investment destination. especially in china, whose differences with the spanish market are so pronounced, five of the six consultants say that adapting the strategy is imperative. it is essential for a successful business to adapt procedures to chinese culture at all levels (yang and lee, 2002). mcf2: "for example, sometimes the spanish company has a product with high addedvalue in spain and sells products in spain, but to have something to offer in china, it must become a service provider. it must offer a service, not a product. so it must restructure its business strategy, this step is complicated for companies to do. " mcf4: "know the market and adapt your strategy. take into account that china is very big and that trying to encompass it all and treat it all the same is a mistake. you should see it like a europe." mcf6: "we must adapt the strategy because there are cultural differences to be taken into account (...)" the last factor mentioned by three of the respondents, is the protection of company technology or know-how (pagnattaro, 2012). the consultants stress that this is an issue of great concern to companies investing in the country and although, according to mcf5, the protection of intellectual property in china has improved, it still remains an unresolved issue. the above eight factors are those that the mcfs considered relevant to success in china. not taking them into account may lead to business failure there. the managers who responded to the interview noted that many businesses fail because they do not devote sufficient resources, financial and human, to the project. that is to say, they do not meet the first three requirements. "one mistake spanish companies, unlike german or british ones, make is to think in the short term, about short-term costs rather than long-term profits" (mcf4). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 17 the second reason for failure relates to the chinese partner (hu and chen, 1996). many joint ventures have not been successful in china for two reasons. the first lies in sociocultural differences with your partner, resulting in you having distinct ways of working and understanding business. so, as far as possible, the mcfs recommend always holding majority ownership to take the decisions. the second problem with a partner, linked to the last success factor, is that of having to share information, leaving the company's intellectual property unprotected and making it easier for the partner to set himself up to take advantage of this know-how. 5. conclusions the internationalization of businesses is a much-investigated area in business literature, but not from the point of view taken in this article. the mcfs vision in regard to the internationalization process of their clients is one of the main contributions of this study. through interviews with the partners and managers of international consulting firms, we gathered the opinion of the consultants for spanish companies in china. the profile of companies that go international, and the key factors for their success, are the two questions that this study has answered. company size is considered relevant to the company internationalization process, both in the literature and also in the opinion of mcfs. in this regard, the level of resources needed by a company deciding to make direct investment abroad makes it difficult for an sme to achieve, although there may be exceptions: the product or service it offers can have the requisite degree of specialization, and it may have adequate financial resources. there is consensus in the literature and among the respondents in terms to their management international experience. the spanish companies that are going to china are experienced in international business, and it is usually not their first cross-border venture. normally, because of their geographical and cultural proximity, they have previously invested in europe, south america or north africa. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 18 as for business sector, there are twenty where spanish business has invested, notably financial services, manufacture of vehicles and both wholesale and retail business. should also be stressed the strong growth of chemicals and, to a lesser extent, food. our conclusion, therefore, is that spanish companies do not invest in a given sector in china, they is represented in many areas. on the other hand, they are experienced companies, since they have already invested in other markets, and are of large to medium size – rarely small businesses. the differences between spanish and chinese market have resulted in some deployments failing. through the mcfs experience, we identified eight factors to consider for internationalization success. five of them concern company resources and contacts or networks, the other three relate to business strategy. in the first group of factors, the mcfs considered that adequate resource investment (time, money and people), management involvement, human resource quality in china, choice of a good partner and having good contacts are key factors for a successful deployment. in the second group, differentiation of product or service, knowing how to adapt the company strategy to the chinese market and devoting efforts to technology protection seem, in the consultants' opinion, essential. faced with the many uncertainties inherent in company internationalization, this advice from mcfs experienced in international markets can provide guidance to companies who are considering deploying in china and help them seize the opportunities offered by one of the largest markets in the world. 6. references boisot, m., and meyer, m. w. (2008). which way through the open door? reflections on the internationalization of chinese firms. management and organization review, 4(3), 349–365. http://doi.org/10.1111/j.1740-8784.2008.00116.x brenes, e. r., mena, m., and molina, g. e. (2008). key success factors for strategy multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 19 implementation in latin america. journal of business research, 61(6), 590–598. http://doi.org/10.1016/j.jbusres.2007.06.033 buckley, p. j., and casson, m. c. (1976). the future of the multinational enterprise (p. 132). new york: holmes & meier. caves, r. . (1996). multinational enterprise and economic analysis (2nd edition, p. 322). cambridge univertisty press. chen, c.-h. (1996). regional determinants of foreign direct investment in mainland china. journal of economic studies, 23(2), 18–30. http://doi.org/10.1108/01443589610109649 chen, c.-i., and yeh, c.-h. (2012). re-examining location antecedents and pace of foreign direct investment: evidence from taiwanese investments in china. journal of business research, 65(8), 1171–1178. http://doi.org/10.1016/j.jbusres.2011.07.032 chetty, s., and blankenburg holm, d. (2000). internationalisation of small to mediumsized manufacturing firms: a network approach. international business review, 9(1), 77– 93. http://doi.org/10.1016/s0969-5931(99)00030-x claver, e., quer, d., and molina, j. f. (2005). inversión directa en el exterior : evidencia empírica de las empresas españolas en china. revista española de dirección y economía de la empresa, 14, 25–42. cremer, r. d., and ramasamy, b. (2009). engaging china: strategies for the small internationalizing firm. journal of business strategy, 30(6), 15–26. http://doi.org/10.1108/02756660911003086 cui, g., and liu, q. (2000). regional market segments of china: opportunities and barriers in a big emerging market. journal of consumer marketing, 17(1), 55–72. http://doi.org/10.1108/07363760010309546 dean, d. l., mengüç, b., and myers, c. p. (2000). revisiting firm characteristics, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 20 strategy, and export performance relationship: a survey of the literature and an investigation of new zealand small manufacturing firms. industrial marketing management, 29, 461–477. http://doi.org/10.1016/s0019-8501(99)00085-1 dunning, j.. (1988). the eclectic paradigm of international production: a restatement and some possible extensions. journal of international business studies, (19), 1–31. eisenhardt, k. m. (1989). building theories from case study research. academy of management review, 14(4), 532–550. http://doi.org/10.2307/258557 enderwick, p. (2009). large emerging markets (lems) and international strategy. international marketing review, 26(1), 7–16. http://doi.org/10.1108/02651330910933177 figueira-de-lemos, f., johanson, j., and vahlne, j. e. (2011). risk management in the internationalization process of the firm: a note on the uppsala model. journal of world business, 46(2), 143–153. http://doi.org/10.1016/j.jwb.2010.05.008 forlani, d., parthasarathy, m., and keaveney, s. m. (2008). managerial risk perceptions of international entry-mode strategies: the interaction effect of control and capability. international marketing review, 25(3), 292–311. http://doi.org/10.1108/ 02651330810877225 freeman, s., and sandwell, m. (2008). professional service firms entering emerging markets: the role of network relationships. journal of services marketing, 22(3), 198–212. http://doi.org/10.1108/08876040810871165 galán, j. i., and gonzález-benito, j. (2001). determinant factors of foreign direct investment : some empirical evidence. european business review, 13(5), 269–278. gao, h., knight, j. g., and ballantyne, d. (2012). guanxi as a gateway in chinese-western business relationships. journal of business & industrial marketing, 27(6), 456–467. http://doi.org/10.1108/08858621211251460 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 21 glaister, k. w., and wang, y. (1993). uk joint ventures in china: motivation and partner selection. marketing intelligence & planning, 11(2), 9–15. http://doi.org/ 10.1108/02634509310027855 glückler, j. (2006). a relational assessment of international market entry in management consulting. journal of economic geography, 6, 369–393. http://doi.org/10.1093/jeg/lbi016 hosseini, m., and dadfar, h. (2012). network-based theories and internationalization of firms : applications to empirical studies. the business & management review, 3(1), 182– 191. hu, m. y., and chen, h. (1996). an empirical analysis of factors explaining foreign joint venture performance in china. journal of business research, 35(2), 165–173. http://doi.org/10.1016/0148-2963(95)00087-9 hymer, s. h. (1976). the international operations of national firms: a study of direct foreign investment (pp. 139–155). mlt press. johanson, j., and vahlne, j.-e. (1977). the internationalization process of the firm a model of knowledge development and increasing foreign market commitments. journal of international business studies, 8(1), 23–32. johanson, j., and wiedersheim-paul, f. (1975). the internationalization of the firm four swedish cases. the journal of management studies, 12(3), 305–323. kindleberger, c. p. (1969). american business abroad. the international executive, 11(2), 11–12. kyläheiko, k., jantunen, a., puumalainen, k., saarenketo, s., and tuppura, a. (2011). innovation and internationalization as growth strategies: the role of technological capabilities and appropriability. international business review, 20(5), 508–520. http://doi.org/10.1016/j.ibusrev.2010.09.004 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 22 lin, f.-j. (2010). the determinants of foreign direct investment in china: the case of taiwanese firms in the it industry. journal of business research, 63(5), 479–485. http://doi.org/10.1016/j.jbusres.2009.04.005 liu, x., and wu, x. (2011). technology embeddedness, innovation differentiation strategies and firm performance: evidence from chinese firms. innovation: management, policy & practice, 13(1), 20–35. lu, j. w., and beamish, p. w. (2001). the internationalization and performance of smes. strategic management journal, 22(6-7), 565–586. http://doi.org/10.1002/smj.184 luo, y. (1997). does guanxi influence firm performance. asia pacific journal of management, 14, 1–16. luo, y. (1998). joint venture success in china : how should we select a good. journal of world business, 33(2), 145–166. moreira, m. r. ., maia, m. a. ., sousa, p. s. ., and meneses, r. f. c. (2013). factors influencing the internationalization of services firms: the case of design, engineering and architecture consulting firms. lecture notes in business information processing, 143, 246– 262. http://doi.org/10.1007/978-3-642-36356-6_18 moyano vázquez, g. (2009). presencia empresarial española en china. boletín económico del ice, (2972), 119–129. nair, a. s., and stafford, e. r. (1998). strategic alliances in china: negotiating the barriers. long range planning, 31(1), 139–146. http://doi.org/10.1016/s0024-6301(97)00100-3 nielsen, b. b., and nielsen, s. (2011). the role of top management team international orientation in international strategic decision-making: the choice of foreign entry mode. journal of world business, 46(2), 185–193. http://doi.org/10.1016/j.jwb.2010.05.003 pagnattaro, m. a. (2012). preventing know-how from walking out the door in china: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 23 protection of trade secrets. business horizons, 55(4), 329–337. http://doi.org/10.1016/ j.bushor.2012.02.003 pedersen, t., and pedersen, b. (2004). learning about foreign markets: are entrant firms exposed to a “‘shock effect’”? journal of international marketing, 12(1), 103–123. poulfelt, f., smith, i. s., and christensen, k. g. (2014). when west meets east – the case of a scandinavian consulting firm ’ s entry in india. european j. international management, 8(4), 373–391. http://doi.org/10.1504/ejim.2014.062964 reuber, a. r., and fischer, e. (1997). the influence of the management team’s international experience on the internationalization behaviors of smes. journal of international business studies, 28(october 1996), 807–825. reynolds, p. d. (1997). new and small firms in expanding markets. small business economics, 9, 79–84. sakarya, s., eckman, m., and hyllegard, k. h. (2007). market selection for international expansion: assessing opportunities in emerging markets. international marketing review, 24(2), 208–238. http://doi.org/10.1108/02651330710741820 sapienza, h., autio, e., george, g., and zahra, s. (2006). a capabilities perspective on the effects of early internationalization on firm survival and growth. academy of management review, 31(4), 914–933. the worldbank. (2013). foreign direct invesment, net inflows. retrieved january 28, 2015, from http://data.worldbank.org/indicator/bx.klt.dinv.cd.wd?order=wbapi_data_ value_2013+wbapi_data_value+wbapi_data_value-last&sort=desc tihanyi, l., ellstrand, a. e., daily, c. m., and dalton, d. r. (2000). composition of the top management team and firm international diversification. journal of management, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 24 26(6), 1157–1177. http://doi.org/10.1177/014920630002600605 todo, y. (2011). quantitative evaluation of the determinants of export and fdi: firmlevel evidence from japan. world economy, 34, 355–381. http://doi.org/10.1111/j.14679701.2011.01331.x tsang, e. w. . (1998). can guanxi be a source of sustained competitive advantage for doing business in china? the academy of management executive (1993-2005), 12(2), 64–73. tung, r. l., and worm, v. (2001). network capitalism: the role of human resources in penetrating the china market. the international journal of human resource management, 12(4), 517–534. http://doi.org/10.1080/713769653 van praag, c. m. (2003). business survival and success of young small business owners. small business economics, 21(1), 1–17. http://doi.org/10.1023/a:1024453200297 wagner, j. (1995). exports, firm size, and firm dynamics. small business economics, 7(1), 29–39. wang, h., and guo, k. (2011). an empirical study on factors influencing foreign direct investment to china. in service systems and service management (icsssm). wigley, s. m., moore, c. m., and birtwistle, g. (2005). product and brand: critical success factors in the internationalisation of a fashion retailer. international journal of retail & distribution management, 33(7), 531–544. http://doi.org/10.1108/09590550510605596 yang, j., and lee, h. (2002). identifying key factors for successful joint venture in china. industrial management & data systems, 102(2), 98–109. http://doi.org/10.1108/ 02635570210419645 yin, r. k. (1981). the case study crisis: some answers. administrative science quarterly, 26(1), 58–65. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3711 social and technological sciences eissn: 2341-2593 niñerola et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 1-25 | 25 yin, r. k. (1984). case study research: design and methods (applied social research methods) (p. 171). sage publications. zeng, s., xie, x. ., tam, c. ., and wan, t. . (2009). relationships between business factors and performance in internationalization: an empirical study in china. management decision, 47(2), 308–329. http://doi.org/10.1108/00251740910938939 zutshi, r. k., and tan, w. l. (2009). impact of culture on “ partner selection criteria ” in east asian international joint ventures. international entrepreneurship and management journal, 5, 369–393. http://doi.org/10.1007/s11365-008-0076-1 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 15 matlab as a tool as analysis and problem solving competency development in chemical engineering degree using matlab maría-fernanda lópez-pérez, s.c. cardona, j. lora, a. abad departamento de ingeniería química y nuclear. universitat politècnica de valència (upv). plaça ferràndiz i carbonell, s/n 03801 alcoy, alicante (spain). * corresponding author: email: malope1@iqn.upv.es; phone: + 34 966528586 received: 2016-05-06; accepted: 2016-08-18 abstract analysis and solving problems is a chemical engineering student capability. in order to develop this ability, activities that encompass problem-solving by students may involve problems in real-world settings. in chemical engineering degree, matlab is a numerical software package that helps in the process of designing, evaluating and implementing a strategy to answer an open-ended question or achieve a desired goal. in this context, matlab is software used in process simulation. several lectures of escuela politécnica superior d’alcoi presented an innovation and improvement educational research project (pime) in order to used matlab, like coordination teaching tool between some subjects. the principal purpose of this work is the students improvement using, as has been mentioned previously, matlab in a problem-based learning methodology. this methodology allows a more effective coordination in the degree. the present paper presents a realworld problem and the common elements of most problem-solving contexts and how is designed to function across all disciplines. keywords analysis and solving problems, problem-solving learning, matlab, coordination http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 16 1. introduction due to the profound change that must take place in learning in the university context, the most important step is education-based training and evaluation for competencies. the upv, in line with the bologna process has launched a number of strategies for evaluating them, starting with generic competencies, linked to numerous subjects of university degrees (v. yepes, 2014). the generic competencies that have defined the upv are: • understanding and integration. • implementation and practical thinking. • analysis and problem solving. • innovation, creativity and entrepreneurship. • design and project. • teamwork and leadership. • ethical, environmental and professional responsibility. • effective communication. • critical thinking. in the present case, the generic competency to be treated in this work is the competence of analysis and problem solving. according to the upv, this competition can be defined as "analyze and solve problems effectively, where significant elements are identified and defined”. three levels of complexity can be established: first demonstration proficiency, problem solving using knowledge learned in the classroom or in books. a second level, which is developing its own criteria in order to solve the problems, using reflection and experience. and a third, more developed, when the student is able to develop and propose solutions in unusual and unfamiliar topics (a villa, 2007). in the degrees, the required http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 17 levels would be the first two, the first level for years 1st and 2nd and the second level of development would be taken into 3rd 4th. the next level would correspond to master studies. the subjects need a coordination following eees framework (c. armengol, 2009). this coordination must be addressed both horizontally and vertically level, focusing on coordination of content and evaluation of specific and generic subject competencies of the same course or specialty, and ensure the correct distribution of resources (d. cazorla, 2010; jf morales, 2013). this requires a lot of dedication on the part of academic committees of degree and this complex work must be well planned, so that the creation of tools to facilitate the coordination work will be needed (y. garcia, 2013). in the case of engineering, the use of mathematical models that describe the behavior of processes is an important part of the basic knowledge that a student must possess (s.c. cardona, 2014). in engineering, the use of mathematical software for student training (j.m. gonzalvez, 2013) is necessary and essential. because the market there are many software for solving mathematical models, a wide dispersion of software for the same purpose appears, a student manages different software for each subject to solve problems in various areas, but actually use the same numerical method (e.g., solving ordinary differential equations). one consequence of using two or more software for the same purpose during the degree is that the student doesn’t dominate any of them, because the time is limited. this lack of teacher coordination in the use of software could cause great uncertainty to students about the purpose of the mathematical tools and that reduce time for learning the contents of each subject. therefore, the use of the same general purpose mathematical software can be a tool of coordination and integration of the contents of many of the degree subjects and this tool could enhance a working method based on problem-examples (maria-fernanda lopezperez, 2012.2013), to strengthen the general competency develop. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 18 a project for innovation and educational improvement of several teachers has been raised for real coordination in the degree of chemical engineering, and enhance analysis and problem solving competency. the purpose of the project entitled "using matlab as a teaching strategy and horizontal and vertical coordination between subjects chemical engineering degree" is the use of this software as a teaching tool that serves as a link between subjects. to improve coordination between the subjects and pose a teaching methodology based on problem solving is the main objective. 2. proposed methodology: integration of different teaching methods in this project, working method based on problem-examples that reinforce contents that are important in the application of chemical engineering, using matlab as an integration tool is the main objective. for this, several tasks were developed: 1. involve chemical engineering degree lectures using single mathematical software in their subjects in the complex mathematical calculations. 2. convert the student in an advanced user in matlab. 3. develop a learning methodology based on transversal problems that allows students to achieve the analysis and problem solving competency. 4. promote coordination of the contents of different subjects, both horizontally and vertically in the different courses. the coordination provides greater cohesion to the degree. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 19 3. discussion of the proposal and results 3.1 participation of interested lecturers the next step was to involve different lectures who teach at the chemical engineering degree to participate in this coordination project using matlab. this mathematical software would be used in problem solving and also should coordinate the subject contents by solving a complex problem in order to the competency. the subjects were a total of eight (table 1), from the first year to fourth, where the required level of knowledge and the mathematical complexity increased. table 1. subjects into coordination project using matlab. subject character year semester maths ii basic formation 1 b calculation methods obligatory 2 a chemical kinetics obligatory 2 b mass transfer obligatory 2 b chemical reactors obligatory 3 a experimental chemical engineering ii obligatory 3 a analysis and simulation of processes obligatory 3 b biological wastewater treatment elective 4 b therefore, all subjects were adapted for the teachers. the teacher used matlab to solve problems. because some teachers did not have the sufficient level mathematical software, several workshops were conducted. three workshops were supported by the instituto en ciencias de la educación (ice) and were taught by a degree of chemical engineering, teacher with high knowledge in matlab. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 20 3.2 students as advanced users in matlab. in some subjects, during the degree, (experimental engineering i, ii, and iii) students have planned matlab workshops. in these workshops, a manual are given to students, with the most representative matlab functions in order to be used in solving the problems of the course. these workshops offer the possibility that students learn matlab along the degree, which facilitates students reach educational competencies. therefore, the basic contents assimilation in the subjects of the degree is improved. also, better cohesion in the development of the title is achieved. finally, students will learn, along the title, an advanced level in numerical/symbolic software, which will provide added value the degree; professional market appreciates this knowledge. 3.3 analysis and problems solving competency development all tasks that have been discussed above must be reflected in the contents of the degree subjects. chemical engineering examples or problems will increase difficulty while students advance in their studies to achieve the competencies. therefore, for the development of analysis and problems solving competency, problembased learning methodology were used. this type of learning has the highest "learn to solve problems by solving problems". these problems must be appropriate to the level of the course, and should career of management statements, should be motivating to the formation and development of concepts is facilitated. students perceived cohesion of content and true coordination when the concepts are related to real examples along the courses, students could also understand the concepts in a more effective way, increasing their motivation to the subjects. first, a complex problem that is resolved in semester b during 4th course was chosen. that problem was the design of a sequencing batch reactor sbr. this problem is very complex, since the design of such reactors implies knowing: http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 21  bacteria that remove contaminants, respiration and kinetics of microorganism growth.  kinetic of biodegradable organic matter.  oxygen transfer process from the gas phase to the liquid therefore, in reactor design the student must possess concepts studied in many of the above subjects. this subjects and concepts coordination is very difficult. the problem statement might be: “in a milk company in ---town, sbr reactor is going to be installed for removing pollutants that discharged into the wastewater sewer. the discharge flow is 200 l/d and the control parameters discharge must be below discharge limits. design the reactor (volume) and operational parameters”. as you can see the statement is completely open, so that each teacher can adapt it to your needs and purposes. to better understand how the problem arises, we will be developing subject by subject, what are the problems associated with the design of the sbr and that can be resolved in each of them, their complexity and how to relate with others. remember that for 1st and 2nd course, analysis and problem solving competency include identify and analyze a problem to generate alternative solutions. for 3rd and 4th course the level of results required increases, in which the student must use the experience and judgment to analyze the causes of a problem and build a more efficient and effective solution. in the first course students will deal with concepts that should have to design the bioreactor. the subject math ii; as is well known is a discipline that sometimes is a stumbling block for students. math is difficult, but also students do not see the useful in the real live. in the present case, solving systems of differential equations are needed for the development of http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 22 sbr reactor design, because it is a batch process where the variables change over time. one of the thematic units in math ii is ordinary differential equations and their analytically solving. when the math concepts have been explained, the problem can be presented to the student. but, this problem should be presented as a real situation, the teacher explain the importance of wastewater treatment and the used reactors for this purpose. the lectures teach wastewater problem can be the math lecture or a person familiar with wastewater matter. some videos of biological reactor operation could be presented. the video length would be a maximum of 15 minutes, because the subject can’t be invalidated. at this point the teacher will give the bacteria growth differential equation, oxygen and pollutant variation differential equations. such processes can be modeled and are ordinary differential equations which can be solved analytically (laplace). at the first the teacher uses only a variable, it can bacteria, for later, offers the students a more complex problem , which varies not only bacteria variable, but also the substrate and oxygen, then the teacher presents the complete system of ordinary differential equations (eq. system 1).   02 11 1 1max 1 1 1 * 2 02 11 1 1max 011 11 1 1max 1 1 cc(0) 1 · )0( )( )0( 1 )(             endototalh s o hhendohh s h h s rox sk s y y cckla dt cd xxrox sk s t dt dx ssx sk s y t dt ds  (sist. eq. 1) where s1 is substrate, xh are heterotrophic bacteria and c is oxygen concentration. all constants numeric value is provided, because the aim objective is the system resolution and not the physical concepts. when students solve the problem on the paper manually, the problem should be solved with matlab, with the relevant functions, dsolve or ode. the teacher has seen a very simple resolution and a real application of the subject. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 23 the next subject is calculation methods in chemical engineering (2nd year, semester a), where equations and systems of ordinary differential equations are solved numerically by the methods of euler, runge-kutta and multistep. in this point, both subject, math ii and calculation methods are related. the same problem can solve with matlab, now numerically, once students have learned to solve systems manually. it is interesting that students learn to do with matlab and can see the savings in time. in these two subjects they have learned concepts and have got a way of solving systems with matlab. students have also seen the usefulness of the concepts learned. in this moment, students have seen the mathematical part of the sbr reactor design, so that engineering concepts can be introduced. in order to make a mathematical model, where there are chemical reactions, the reaction rate has to be studied to obtain a kinetic equation where relates the reaction rate with operational conditions and composition. -ra=[k(t)][f(ca,cb,…) (eq. 2) –ra is reaction rate (mol/l.·h), k is a constant (is temperature function) and ca, cb, reactive concentration. calculation of the kinetic constant and the reaction order are described in this subject. matlab is used in this subject to solve these problems, using interpolation as polyfit or linearizing. students have experimental data to kinetic equation resolve, e.g. organic matter reduction for bacteria action. there are several models, such as: the first order model monod. specific rate constant (usually expressed by k) is obtained in first order kinetics, while for the model monod, three constants have to be evaluated: the affinity constant (ks), the specific rate (μmax/y) and finally, the maximum growth constant (μ). in this part, the teacher would give students experimental data (substrate concentration versus time), and students could use the equations presented above to obtained these constants using matlab. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 24 limiting sbr operational parameters is oxygen transfer because the most treatments are aerobic. therefore, students have to know the chemical reaction mass transfer concepts. this knowledge is introduced in mass transfer subject (2nd year, semester b). gas diffusion into a liquid in irreversible reaction models are explained in this subject. the model describes the diffusion through the film and can be obtained from oxygen matter conservation equation. the resulting equation has more than one independent variable and a differential equation in partial derivatives appears. in this part, this subject relates to the kinetic subject. when teachers, responsible for the two subjects are coordinated, can relate both subject, giving cohesion to the degree contents; thus students relates concepts that otherwise notices as separate issues. now, this equation have to be solved by numerical method, based on finite differences, therefore, this part of the problem is also related to calculation methods subject, so the teacher must be coordinated and should teach similar examples. naturally, the numerical resolution method is not explained in the subject, but matlab is used to solve the problem. when teacher coordination exists, students have worked with numerical methods in order to solve similar problems and have the matlab codes (remember students have a matlab manual that can be used along the degree). no time is lost mathematical problem solving. when the coordination does not exist, students does not relate both subject, and most of times must remember the concepts again, and starts with a new matlab program. in this step, much time is wasted in mathematical operative methods, and the problem is not focused on the part of chemical engineering concepts and gas-liquid diffusion. the student loses time and effort to solve the problem mathematically and neglects the chemical issue. in this subject, the teacher should transmit the importance of the oxygen diffusion processes into a biological reactor, the oxygen concentration profiles in the liquid film for different times to reach steady state and as the oxygen amount that is transferred per unit of time and area. thus, the student can understand the importance of the oxygen that microorganisms need for their metabolism and catabolism. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 25 in these two years (1st and 2nd year) basics concepts in the reactor design with chemical reactor, kinetics and mass transfer have been taught, with mathematical resolution using matlab. a basic matlab program is made as templates in order to students solve similar problems. in the third year, the problem is focused on sbr reactor design and simulation process. the reactor design problem is continued in chemical reactor subject (3rd year, semester a). in this subject, the concept of reactor size calculation is introduced from a kinetic equation. when the kinetic constants and reaction order is known, the reactor size can be calculated. the teacher provides experimental data and students have to calculate the time using interpolations and integers. these calculations are very simple with matlab, and have seen in math i. students only focuses on reactor design concepts in the sbr example, student calculates the decay time of the substrate and after sbr volume. this subject is taught in the same time that experimental chemical engineering ii (3er year, semester a), and the two first class sessions are dedicated to second matlab workshop. students learn new concepts, matlab functions, and new methods for using in the subject problem solving. in this subject, the concepts studied in chemical reactors subject is design a biological reactor with experimental data. students make experiments to determine cod variation (chemical oxygen demand) in an aerobic reactor. in this way, students become familiar with a real process in a laboratory scale reactor. but at this point, two subjects are being offered at the same time (semester a) and the laboratory practice may make before theoretical session. this is a problem that should be studied. in the laboratory practice and chemical reactors subject, the same matlab tools and functions are used. besides, transfer mass laboratory experimental are made in experimental engineering laboratory practices, where oxygen transfer gas-liquid coefficients are calculated through matlab parameter fit. it is very important that students see a connection between all subjects and a relationship with their professional life in the future. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 26 in 3th year, semester b, students are faced with the subject analysis and simulation process (3rd year, semester b). bioprocess technology also is offer in this semester, and student is introduced in biochemistry process, microorganisms kinetic (monod) that knows from chemical kinetic subject. in analysis and simulation process, students work with appropriate mathematical model to a hybrid respirometer. although this example is not a sbr reactor, the respirometer is a biological reactor works in batch. six nonlinear ordinary differential equations describe this model, and are coupled together. oxygen, biomass and substrate concentration are model dependent variables and the time is the independent variable. an additional difficulty in this mathematical model is how the substrate injection in the aerated reactor is modeled. this injection is usually done in the pulse form, and can be repeated over time. when the mathematical model is raised and the biokinetic parameters are known, the biological system can be studied through variable data or operation conditions modification:  change recirculation flow between tanks  change oxygen transfer rate  change substrate amount in the injection, time injection the results of the simulation allow a success in experimental design, because this experiments last long and a make a prediction of the system behavior is very interested. matlab is naturally the software used to solve the problems but simulink help (matlab environment) in the simulation problems. in this semester, the mathematical operations are more difficult than other years and thus students’ knowledge about the matlab has been incorporating during all the previous years. thereby, students only spend time in problem solving. the teacher saves re-explain the mathematical calculations. and finally, students arrives to biological treatments subject (4th year, semester b). in the 4th year, semester a, chemical engineering degree has a problem, because there is not http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 27 any subject related with the sbr reactor. this weakness in the coordination should be remedied because students quickly forget the mathematical functions, subjects… in biological treatment subject, sbr concept and biological wastewater treatments are learned. the mathematical models are more complete than previous (9 differential equations). the model adds all previously knowledge acquired (kinetic constants, matter transfer processes, simulation concepts…). the biological meaning is given in this subject. now, student is able to solve the sbr problem, is able to designs a sbr reactor and is able to optimizes the operational parameters. student is able to use company data provided, and is able to look for the information he needs. the main aim, general competency, is reached: student uses the experience and criteria to analyze a problem causes and build an effective solution. 4. conclusions in this paper we have presented a methodology of problem-based learning for the chemical engineering degree to enhance general competency, “analysis and problem solving”. several subjects have been coordinate using matlab as a link. the first step is to start with a problem that can be developed throughout the subjects of the degree. the main objective is more and more subjects are incorporated and develop a common project all of them. http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 28 5. references armengol asparó, c. et al (2009). “la coordinación académica en la universidad. estratégias para una educación de calidad”. revista electrónica interuniversitaria de formación del profesorado, 12 (2), 121-144. cardona, s.c., abad, a. (2014) aplicación de matlab a la resolución de balances de materia y energía, salvador. ii congreso de innovación docente en ingeniería química (ii cidiq). isbn 978-84-370-9271-3 cazorla d., maciá m., puerta j.m., serrano r., rojo t, (2010), xvi jornadas de enseñanza univer-sitaria de la informática garcía cid, y (2013) una experiencia de coordinación docente en la enseñanza universitaria xxi jornadas asepuma – ix encuentro internacional anales de asepuma nº 21: 516 gozalvez, j.m. , lora, j. (2013) cálculo de operaciones de separación con mathcad isbn 978-84-9048-094-6 editorial universitat politècnica de valencia lópez-pérez, maría-fernanda, abad.,a (2012) how promote scientific competency in experimentation subjects in chemical engineering degree, i international symposium on innovation and quality in engineering education isbn. 978-84-695-3891-3 lópez-pérez, maría-fernanda, abad., a. (2013) módulos de aprendizaje y abp, como pilares centrales de la metodología docente activa, en la planificación de una asignatura a implantar, xxi congreso universitario de innovación educativa en las enseñanzas técnicas (cuieet) isbn. 978-84-608-1217-3 morales j.f., peña l.m. (2013) propuesta metodológica para la enseñanza del cálculo en ingeniería, basada en la modelación matemática., vii cibem http://dx.doi.org/10.4995/muse.2016.4623 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4623 social and technological sciences eissn: 2341-2593 lópez-pérez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 15-29 | 29 villa, a., poblete, m., aprendizaje basado en competencias (2007) ediciones mensajero. i.s.b.n. 978-84-271-2833-0 pp. 327 yepes piqueras, v. (2014) http://excelcon.blogs.upv.es/2014/10/01/cuales-son-lascompetencias-transversales-de-la-upv/ http://dx.doi.org/10.4995/muse.2016.4623 http://excelcon.blogs.upv.es/2014/10/01/cuales-son-las-competencias-transversales-de-la-upv/ http://excelcon.blogs.upv.es/2014/10/01/cuales-son-las-competencias-transversales-de-la-upv/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 118 problem-based learning on cell biology and ecophysiology using integrated laboratory and computational activities c. sousa faculdade de ciências, universidade do porto, portugal e-mail: up199502480@fc.up.pt received: 2015-02-12; accepted: 2016-01-13 abstract since all the known biological systems require water for their basic biochemical processes, one can find several osmoregulation mechanisms on living organisms for adaptation to related environmental challenges. osmosis is a cellular mechanism of water movement across membranes which is known to be present throughout the tree of life and occurs by either diffusion across the membrane bilayer or by a faster movement mediated by transmembrane channel proteins, called aquaporins. the expression of aquaporins is regulated at, the cellular level, by environment conditions such as hydric stress, therefore allowing the adaptation of organisms to increase salinity in soils, water deprivation and increase beverage intake. osmosis and diffusion concepts have been described to be difficult to learn, so, in order to promote meaningful learning, we used a problem-based learning approach that integrates a laboratory activity and a computer simulation model of osmosis and a two phase conceptual mapping. we observed that high school students developed adequate laboratory skills and were able to communicate their results as text and using scientific drawings; and the learning environment was adequate. therefore we presented a successful implementation case of integrated pbl, in a public portuguese school, that may constitute an example to facilitate the implementation of active inquiry strategies by other teachers, as well as the basis for future research. keywords problem-based learning; experimental activities; computer simulation; osmosis. mailto:up199502480@fc.up.pt multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 119 1. introduction water is a condition sine qua non of life, however 97.5% of the total earth’s water content is found in the oceans and global warming and the consequent increase in sea level and soil salinization will diminish the productivity of farmlands throughout the world (ivits et al., 2013). at the cellular level, the influx (out flux) of substances into (out of) a cell or an organelle can occur either: (i) according to physical and chemical properties by a passive process from an area of high concentration to an area of low concentration or (ii) by an active process, consuming energy (atp molecules), occurring against the concentration gradient and involving specific mediators on the cell membrane. the mechanism of water movement across membranes, from the side with lower solute concentration to the side with higher osmolality, is called osmosis and it is known to be present throughout the tree of life (verkman, 2011), since this cellular mechanism presents some evaluative advantages such as not requiring consumption of any atp molecule. this transmembrane movement occurs by either diffusion across the membrane bilayer or by a faster movement mediated by transmembrane channel proteins, called aquaporins aqps and were the focus of a nobel prize (agre, 2003). cells in hypotonic environment would increase in volume and evenly burst, hence the osmoregulation at the cellular level is an important process that is known to be mediated by vacuoles. hence this osmolality problem is solved by living organisms by various mechanisms that continuously and actively pump ions out. plant, algae, fungi and some bacterial cells have an additional barrier the cell wall that avoids the variation on total cell volume and the burst of the cells by water influx. however, turgidity is a necessary condition for plants to maintain their firmness or turgor. it has been described the expression of aqps in all organisms in their cellular membranes (ishibashi et al., 2011), with the main role of water transportation, which constitute an evidence of evolution by adaptation of organisms to the environmental challenges related to water-based life. this cellular membrane permeability is selective, since water (h2o) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 120 crosses through the membranes, while the hydronium ion (h3o +) does not permeate these proteins, so this distinction is essential to life (agre, 2003). a limiting factor for plant growth and development is the hydric stress in hypertonic environments, such as due to water deprivation and increased soil salinity, that if continued and/or at high levels will cause plants to wilt and eventually shrivel and die. plants have evolved several biochemical and molecular mechanisms to cope with these types of abiotic stress, such aqps expression, therefore one can find some species and cultivars with higher drought and salt tolerance. recently, unesco created the centre for membrane science and technology, that is responsible for the development of desalination processes based on reverse osmosis, during which seawater is forced through a semipermeable membrane that separates salt from water, so this substantiate the importance of the study of transmembrane transport. this theme constitutes an excellent example for students to learn about the existence of interactions between science, technology, society and environment (stse education) since the development of the technological applications of reverse osmosis can be used at a large-scale to produce drinkable water from sea water, with potential economic and life expectancy benefits in several developing countries. socio-scientific issues are also included such as the development of transgenic overexpressing aqps plants that could be used in agriculture to increase yields and the development of drugs that modulate aqp expression and function can be applicable to medicine. while pbl exists in a variety of forms, depending on the discipline and the goals of the curriculum, it tends to include features such as learner autonomy, active learning, cooperation and collaboration, authentic activities and reflection and transfer (ertmer & simons, 2006). osmosis and diffusion concepts have been described to be difficult to learn, so, in order to promote meaningful learning, we used a problem-based learning approach that integrates a two phase conceptual mapping and experimental activities, such as a laboratory activity, testing the effects of osmolarity in plants and a computer simulation model of osmosis (kottonau, 2011; lindgren et al., 2009). the computer simulation was suggested in order promote inquiry learning since it allows students to specify a few multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 121 parameters (e.g. concentration inside and outside the virtual membrane) and then observe the outcome (lindgren & schwartz, 2009). learning initiation occured by introducing a problematic situation that included a core question, consisting in a conceptual change eliciting question, designed to trigger learning. other essential questions of different types were also included, such as conceptual change and motivation types (yip, 2004) and the complementary experimental activities were designed to facilitate learning by developing students observational and laboratory skills. 2. materials and methods 2.1 participants study participants were students attending the tenth grade at a public school (high school, with age from 14 to 16 years, and 15 years as median age) that willing to participate in the study had the corresponding informed consent signed by the parent/person responsible for education (n = 18). the author participated in the instruction by teaching the corresponding curricular unit to all the students at a portuguese public school (oporto city, portugal). 2.2 educational intervention the results presented were obtained during a one-week period of an action research project on pbl, during the academic year 2012/2013. the pbl methodology used included an ill-structured problem with adequate guidance by the author/tutor according with age and unfamiliarity with pbl of the students. hence the author/tutor promoted the discussion and the questioning by the students using an electronic presentation in each lab-class (≤ 14 students). the open-ended question proposed to students for this class was ”is it beneficial for the organisms that life is based on the existence of water?”, and the students discussed, in small groups (≤ 4 students), and proposed new questions and possible solutions to this problem. the theme is included in the discipline biology and geology in the curricular unit on the diversity of mechanisms for obtaining nutrients used by different organisms. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 122 on the first day the duration of the activities was 135 minutes and then 45 minutes at the end of the week for discussion and elaboration of the post-conceptual map. 2.3 experimental activities 2.3.1 laboratorial experimental activities in this study we used pelargonium x hortorum, red flowers cultivar, native of south africa, that is commonly used as an ornamental plant in portugal. the experimental setup consisted in three groups of plants maintained in mineral soilcontaining vases watered, every other day, during one week, with solutions with different salinity, containing different concentration of minerals (table 1). table 1. culture conditions for maintenance of each group of plants pelargonium x hortorum. group a group b group c mineral soil + mineral h2o (h2o total mineralisation: 47.7mg/l) mineral soil + 12% nacl (in deionized h2o) mineral soil + deionized h2o note: deionized water was obtained by the students using the equipment available in the physics-chemistry lab of the school; as well as the 12% nacl solution (using nacl, sigma). commercially available water was used for plants belonging to group a. students observed the macroscopic morphology of plants of groups a and b and were asked to predict the morphology of the group c of plants. for the microscopic observations, we used petals of the flowers of this plant since they present hydrophilic pigments in vacuoles that are responsible for the red colour of the petals. so, students, in small groups (≤4 students), prepared samples by peeling a piece of the superior epidermis of petals of plants of group a and placed two of these samples in two conditions, in slides containing one drop of either 12% nacl (sample a1) or mineral h2o; then observed both under the microscope and illustrated a few representative cells multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 123 observed in each slide. the media of sample a1 was then carefully replaced by deionized h2o and observed under the optical microscope and illustrated (sample a2). 2.3.2 computational experimental activity in the second part of the lab report students were asked to answer the question “what happens to water molecules when cells are in an isotonic medium?”, so in order to observe what happens at the cellular level in plant cells under an isotonic environment, students designed and performed a virtual experiment using the computational simulator of osmosis available online “netlogo osmosis simulator” (kottonau, 2011). 2. 4 instruments a heurist diagram including the laboratorial procedure was provided to the students and they were asked to complete and present it, at the end of the class, as the lab report to be evaluated by the teacher/author. students, in small groups, were asked to elaborate a conceptual map on transmembrane transport. then, after referring the concept of aquaporins, a conceptual map, with a defined skeleton structure, was provided to the students and they were asked to complete it with the missing concepts and liaison terms (fig. 1). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 124 figure 1. portion of the conceptual map on transmembrane movements (referring only to biomolecules, ions and h2o) completed by high school students. the problematic situation, with the central question “is it beneficial for the organisms that life is based on the existence of water?”, as well as the corresponding pbl worksheet, in which students were asked to complete the list of facts and questions related to the topic and to define an experimental question, were provided to students as hands-outs and were discussed in small group in class. the lab report based on the heurist diagram that students were asked to complete, and that also included the request to illustrate their observations under the microscope, using the adequate magnification, including the arrangement of a few number of cells within the tissue; and to answer some open-ended questions as well as to write a small text on science, technology, society and environment interactions about osmosis. to assess the impact of our strategy on learning and long-term retention and ability to apply knowledge in an integrated way, students’ scores obtained in specific questions on the exam at the end of the unit were analyzed. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 125 at the end of the class, students were asked to respond to a questionnaire in order to evaluate their perceptions about the pbl environment in class, regarding some dimensions, according to senocak (2009), such as: teacher support (the extent to which the teacher acts a facilitator or metacognitive coach and supports several of the goals of pbl), student interaction and collaboration (the extent to which problems act as stimulus and pbl starts with an ill-structured problem to be solved) and the quality of the problem (extent to which problems act as stimulus and pbl starts with an ill-structured problem to be solved). students were asked to respond a small questionnaire about their self-perceptions of the contribute of each activity in their learning. 3. results and discussion conceptual mapping has been describing as an useful strategy for learning and assessment (schönborn & anderson, 2008), so we used it to assess the preconceptions of students about osmosis before the last round of instruction: 70% of students classified osmosis as a non-mediated type of transport and 70% of the students classified it as a passive type of transport. at the end of the class, after discussion about aqps, all the students were able to correctly complete the conceptual map in figure 2, corresponding to meaningful relationships between concepts. since non-integrated classes have been described not bridge the gap between what students learn during pbl classes and the adequate laboratory skills, we used an integrated pbl approach (azer et al., 2013) by designing a problematic situation, or problem, and scaffold strategies that promote the development of the desired capabilities. upon discussion of the problem in small groups, 83% of the students were able to define an adequate experimental question as “what are the effects of increasing salinity at the cellular level?”, and “what are the differences at the cellular levels of plants maintained in normal soils and in soils with higher salinity?”. all the groups of students completed correctly the list of facts and proposed several adequate and interesting questions, of high order, some of which were selected for further multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 126 study, such as: “in what way is water essential to life?”, “what happens at the cellular level responsible for human death by over digestion of water?”, “why is the distribution of water in the planet non uniform?” and “is it possible that drinkable water ends in the near future?”. at the end of the class, all the students were able to propose a possible solution to the central question, referring to the benefits and the challenges of water to known living organisms. more than 80% of the students were able to predict that plants of group c would be similar to the plants of group a and that their cells should be turgid, and they considered that observation alone, at the macroscopic level, of this process is not enough, and that further observation at microscopic level are necessary (tomažič & vidic, 2011). by asking students to make scientific (anatomic) drawings it is possible to promote the development of several skills including the abilities to interpret complex information and to distinguish among similar structures, as well as communication skills in a visual form. students developed the adequate laboratory skills since they were able to select the adequate magnification and field of view for drawing and most students were able to illustrate some details of individual cells and represent form, proportion, and spatial relationships accurately (table 2). table 2. drawings by students of microscopic observations of pelargonium sp. (petals of the flowers). student 1 student 2 student 3 student 4 a1 a2 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 127 note: drawing were made upon observation under microscope using a 40x objective. cells of sample a1 were maintained in slide media containing 12% nacl, and this media was then replaced by deionized h2o (condition a2). by analyzing the students’ drawings (table 2) we were able to observe some misconceptions, such as 35% of students that do not distinguished the vacuole from the cytoplasm (e.g. student 1 in table 2). in the second session we used these misconceptions as an opportunity to discuss with students the plant cell structure and functions of the vacuole and tonoplast, as suggested by others (dicarlo, 2006), and asked students to redraw and correct their legends. upon the replacement of slide media containing 12% nacl (sample a1, table 2) by deionized h2o (sample a2, table 2), all students were able to observe an increase in the vacuolar volume in most cells, and elaborated the conclusion that the vacuole and cytoplasm shrinkage observed is a reversible process, under these conditions. more than 80% of students were able to quantify the vacuole volume in both samples, completing correctly the graph provided in the corresponding question of the lab report. osmosis and diffusion concepts have been described to be difficult to learn, may be due to the fact that these processes involve invisible particles and constitute abstract ideas (fisher et al., 2011). however, all the groups of students were able to perform the experiment using the computer simulation by adjusting the same concentration inside and outside the virtual membrane and all students concluded that water molecules are constantly going inside and outside the cell, as a dynamic equilibrium, maintaining the cell and vacuole volume. the students’ laboratory skills were assessed by the tutor/author, by observing during the class the complying with safety rules and laboratory manuals, communication skills and productive working skills, and the results were: excellent for 32% of students, good for 47%, and satisfactory for 21% of the students. 50% of the students were able to write a small text, as asked in the final question of the lab report, about osmosis referring the interactions science-technology-society-environment in a future scenario of increase of sea level by climatic changes. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 128 the lab activity was successful since we observed positive classification of the lab report of all students, and 37% had an excellent grade (corresponding to a classification of 85% or higher) and 26% were classified with good (≥ 75% and < 85%) and 37% with average (< 75% and ≥ 50%). regarding long-term retention, 78% of the students were successful (≥50% score) in recalling and applying knowledge, while 22% of the students obtained an insufficient score in the questions about osmosis in the exam at the end of the unit. the analysis of the responses to the questionnaire that evaluates the perceptions of the students about the pbl process shows that the learning environment was adequate and successful, since their responses, given in a five-point format ranging from always to never (5-always, 1-never), corresponded to positive results (≥ 3) in 83 to 100% of students. 83-100% of students in the teacher support dimension, 88-96% of students considered the good quality of the problem given and 100% of the students evaluated positively the dimension of student interaction and collaboration. analyzing the responses to the questionnaire the average of the total score obtained by each student was 89 ± 16, which is in accordance with the interval defined previously by others corresponding to a minimum score of 23 and a maximum score of 115 (senocak, 2009). the analysis of the responses of students of self-perception of contributes to learning showed that students considered as corresponding to the higher contribute: the powerpoint presentation and the hands-out containing the problematic situation (47% of the students), experimental work (41%) and computer simulation (12%). the themes included in this teaching-learning strategy are relevant since united nations defined the theme water for life for the decade 2005-2015 and the year 2015 as the international year of soils. therefore, it is important to use this strategy in secondary education in order to increase the awareness about the scarcity of potable water, due to the expected rising of sea level, caused by climatic changes and the consequent increase in salinity of soils as well as contamination of underground potable water (sousa, 2014; sousa, 2015). since this constitutes a global problem, it is useful to discuss the causes and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 129 consequences, as well as ways of reducing the negative impacts, such as the technological development of reverse osmosis processes for desalinization of sea water (unesco, 2014). 4. conclusions since the aims of concept mapping and those of pbl are strategies that naturally complement each other in promoting meaningful learning (addae et al., 2012), in our strategy we used pbl combined with two mapping phases. we also included some experimental activities, such as laboratory activities and computational simulations that were integrated in the context of implementation of pbl in a k–12 environment. our results showed that we were able to encourage students’ creativity and inquiry that are considered by several authors as essential for biology learning (dicarlo, 2006), as well as critical thinking and questioning, as referred by students in their satisfactory feedback about the process, at the end of the corresponding unit. as observed by the satisfactory performance in the exam and lab report we also promoted the meaningful learning with long-term retention, and the ability to interpret laboratory results and its communication using drawings, graphically and in writing. in conclusion, the application of the designed problem in high school promoted a multidisciplinary discussion and constitutes a good example of stse education and socioscientific issues discussion about the causes and consequences of environmental changes and ways of reducing their negative impacts (ashraf, 2013; unesco, 2014). therefore we presented a successful implementation case of integrated pbl in a public portuguese school that may constitute an example to facilitate the implementation of active inquiry strategies by other teachers, as well as the basis for future research. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 130 acknowledgments the author is grateful to the faculty administration for providing funds for this study, and specially to the students that participated in the study, and to the escola secundária aurélia de sousa (porto, portugal) for the conditions to conduct the project. references addae, j. i., wilson, j. i. & carrington, c. (2012). students’ perception of a modified form of pbl using concept mapping. med. teach, 34, e756–e762. agre, p. (2003). nobel lecture: aquaporin water channels. nobelprize.org. http://www.nobelprize.org/nobel_prizes/chemistry/laureates/2003/agre-lecture.html. ashraf, s. s. (2013). raising environmental awareness through applied biochemistry laboratory experiments, biochem. mol. biol. educ., 41(5), 341–347. azer, s. a., hasanato, r., al-nassar, s., somily, a. & alsaadi, m. m. (2013). introducing integrated laboratory classes in a pbl curriculum: impact on student’s learning and satisfaction, bmc medical education, 13, 71. dicarlo, s. e. (2006). cell biology should be taught as science is practised, nat. rev. mol. cell. biol, 7(4), 290-6. ertmer, p. a. & simons, k. d. (2006). jumping the pbl implementation hurdle: supporting the efforts of k–12 teachers. interdisciplinary journal of problem-based learning, 1(1), 40-54. fisher, k. m., williams, k. s. & lineback, j. e. (2011). osmosis and diffusion conceptual assessment, cbe life sciences education, 10, 418–429. ishibashi, k., kondo, s., hara, s. & morishita, y. (2011). the evolutionary aspects of aquaporin family. am j physiol regul integr comp physiol., 300, r566–r576. http://www.nobelprize.org/nobel_prizes/chemistry/laureates/2003/agre-lecture.html multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 131 ivits, e., cherlet, m., tóth, t., lewińska, k.e. & tóth, g. (2013). characterisation of productivity limitation of salt-affected lands in different climatic regions of europe using remote sensing derived productivity indicators. land degradation and development, 24 (5), 438-452. kottonau, j. (2011). an interactive computer model for improved student understanding of random particle motion and osmosis. journal of chemical education, 88(6), 772-775. http://lsvr12.kanti-frauenfeld.ch/koj/java/osmosis_fast.html). lindgren, r. & schwartz, d. l. (2009). spatial learning and computer simulations in science. international journal of science education, 31(3), 419-438. schönborn, k. j. & anderson, t. r. (2008) bridging the educational research-teaching practice gap, conceptual understanding, part 2: assessing and developing student knowledge, biochem. mol. biol. educ., 36(5), 372–379. senocak, e. (2009). development of an instrument for assessing undergraduate science students’ perceptions: the problem-based learning environment inventory. j sci educ technol, 18, 560–569. sousa, c. (2014). potencialidades de atividades experimentais laboratorial e computacional integradas em aulas investigativas. in livro de resumos do ii encontro internacional da casa das ciências (isbn 978‐ 989‐ 98309‐ 1‐ 2). p60‐ 61. sousa, c. (2015). experimental activities-integrated problem-based learning in biology classes promoting ssi and stse discussion. new perspectives in science education conference proceedings. italy: libreriauniversitaria.it (in press). tomažič, i. & vidic, t. (2011). future science teachers’’ understandings of diffusion and osmosis concepts, journal of biological education, doi 10.1080/00219266.2011.617765. unesco (2014), unesco center for membrane science and technology website, university of new south wales: http://www.membrane.unsw.edu.au/. verkman, a. s. (2011) aquaporins at a glance, j. cell science 124, 2107-2112. http://lsvr12.kanti-frauenfeld.ch/koj/java/osmosis_fast.html http://www.membrane.unsw.edu.au/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3763 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 118-132 | 132 yip, d. y. (2004). questioning skills for conceptual change in science instruction, j. biol. educ., 38(2), 76. microsoft word 1_ballester_et_al ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 1 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 implementation of the balanced scorecard in the hotel sector through transformational leadership and empowerment ballester-miquel, josé carlos1*, pérez-ruiz, pilar 2, hernández-gadea, javier 3 palacios-marqués, daniel 4 1* esic business & marketing school josecarlos.ballester@esic.edu (corresponding author) 2 esic business & marketing school pilar.perezruiz@esic.edu 3 esic business & marketing school javier.hernandez@esic.edu 4 department of business aministration. universitat politècnica de valència, camino de vera s/n, 46022 valencia, spain. dapamar@doe.upv.es received: 2016-03-12; accepted: 2016-10-20 abstract one of the current problems of the entrepreneurial world is the successful implementation of the balanced scorecard (bsc), which is why it is interesting to study the variables that can influence its application in business. the main purpose of this paper is to establish a theoretical model through the literature review, that should consider the relationship between the transformational leadership among company staff (avolio, bass and jung, 1999) and the empowerment of the rest of employees (thomas and velthouse, 1990 spritzer, 1995) upon the achievement of an increased capacity for organizational learning in the implementation of the bsc in companies (easterby-smith & araujo,1999) from the following perspectives: financial, customer, business process, learning and growth in major companies in the hotel sector, specifically in four and five-star hotels. http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 mailto:josecarlos.ballester@esic.edu mailto:pilar.perezruiz@esic.edu mailto:javier.hernandez@esic.edu mailto:dapamar@doe.upv.es ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 2 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 the balanced scorecard (bsc) is a strategic management model (norton & kaplan, 1992) that allows the measurement and control of the tangible and intangible objectives based on the company´s mission and vision statements. the implementation of bsc has a great influence on the predisposition, training and motivation in middle as well as executive management through empowerment and leadership, hence the importance of the analysis of these variables by means of the bsc. keywords transformational leadership, empowerment, balanced scorecard introduction the balanced scorecard (bsc) is a very useful tool for the corporate strategic management as it coordinates the objectives derived from the mission statement with the active participation of employees through performance indicators. traditionally, companies used to analyze only the financial part as an indicator of business value and made no evaluation of other intangible objectives that defined the orientation of the company. in the early 90s norton and kaplan conducted a study on important indicators in the case of approximately a dozen companies to finally reach an agreement on the bsc. it is important to measure the level of customer care in the hotel sector, as well as that of internal functioning in processes and the business involvement from the point of view of employee collaboration in the achievement the objectives; and this is the reason why the application of the bsc in the hotel sector is particularly relevant. to successfully implement the bsc, a company should rely on a pool of employees with strong learning abilities and possibilities of personal development in the decision making of the company, power and control over aspects of their professional development, as well as on mechanisms of communication and transmission of information which can be freely and swiftly consulted, as these three aspects define empowerment. if, in addition, company leaders act with high ethical standards, inspiring teamwork, encouraging innovation and creativity, without losing sight of the needs and individual differences as indicated by transformational leadership, it can be affirmed that the bsc can be implemented in the company and great results can be obtained out of it. in short, we try to present a theoretical model that in case of empirical validation would serve to establish the initial criteria of company analysis in order to infer the best http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 3 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 implementation of the balanced scorecard system. this could be achieved by measuring the organizational learning capacity taking into account the level of leadership and empowerment in the company and thus infer the level of success in the implementation of the bsc. 1. transformational leadership leadership is defined as the process of directing and influencing work activities carried out by members of a group and is recognized as a key component that influences the overall effectiveness of the company. the overall concept of this variable is total leadership that avolio and bass (2004) include in the models integrated under the "new leadership” paradigm which is the sum of the transformational, transactional and laissez faire variables. bass` significant contribution is the possibility of measuring transformational leadership, considering that it is the natural evolution of charisma and it could not be measured until that moment. in summary, burns (1978) identified two types of leadership based on the leader follower figure with transactional and transformational interactions. the most traditional transaction creates a leadership that involves an exchange relationship between leaders and followers, whereas transformational leadership is based on leaders changing their follower’s values, beliefs and needs, leading to superior performance in organizations in a time of renewal and change. focusing on the topic in discussion, piccolo and colquitt (2006) found that transformational leadership is positively related to the perceived levels of the top five work characteristics (variety, identity, significance, autonomy and feedback), which were connected to the intrinsic motivation of the objective and commitment. intrinsic motivation is related to both task performance and organizational citizenship behaviour. according to james macgregor burns, transformational leadership is a process in which the leader and the followers help each other to further raise the level of morale and motivation of a human working group. http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 4 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 therefore, transformational leadership is defined as a style that helps to change the morale, ideals, interests and values of the followers and subordinates of a company and to rise above their own interests and thus leaders motivate them to better perform pieterse, knippenberg, schippers and stam (2010). according to conger (1999) these leaders inspire trust, respect and admiration, promote problem solving and deal with each worker individually. transformational leadership by bass (1985) and bass and avolio (1994) involves performance above previously set expectations. the above mentioned transformational dimensions are: a) the idealized influence, resembling charisma, that produces a model to be followed by identification (attribute) and is identified in the way of acting of the leader (behaviour), b) the inspirational motivation which is based on the transmission of the vision of the future, c) the intellectual stimulation with a continuous approach to problems and assumptions and d) the individualized consideration that reflects the leader`s individual behaviour towards their followers. therefore, sosik (2002) found that transformational leadership is positively related to empowerment, group cohesion and group effectiveness. continuing with the independent variables, it has been found that transformational leadership and psychological empowerment (schein, 1973) support an increase in the productivity, performance and learning capacity of the company, generating a commitment which, at the same time, produces benefits. adapting the concept to the tourist sector, it has been mentioned above that according to dominguez (2009) in order that companies in the sector could gain a competitive advantage, they should perform differently compared to their competition, and so that this happens, there should be an ideal environment in the organization and a management team that know how to coordinate and direct their workers. at this point we introduce the concept of transformational leadership as a key concept in high and middle management in the actions to be performed within the company (rodriguez and castillo, 2009). 2. empowerment another variable to consider is the concept of empowerment that is emerging in recent years as a concept with major implications for both organizational effectiveness and occupational health, since it mentions the sense of control and power that individuals have in a work context. http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 5 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 although the term organizational empowerment may have been used by some authors before the decade of the nineties, it was not sooner than that time when it gained strength, especially with the works of thomas and velthouse (1990) about psychological empowerment, barley and kunda (1992) on the theory of satisfaction of people at work or perez (1989) on the improvement of the quality of working life. analyzing each of the above, rosenberg (1979) and perez (1989) considered that the existence of empowerment in the company must be linked to a good quality working life, and mentioned some of the factors that influence it such as the increase of the loyalty and motivation of workers in the tasks they perform. barley and kunda (1992) have conducted several studies which directly connect people´s satisfaction with the tasks they perform at work, economic performance and customer satisfaction. in this sense, spreitzer (1995) changed the existing paradigm of empowerment and proposed two perspectives: the relational / personal perspective and the psychological one. the individual perspective considers concepts dealing with the person, with self sufficiency, power and the competition perceived by the individual. on the other hand, the psychological perspective takes into account intrapersonal aspects, however, the components of interaction with other people in the company and the exhibited behaviour are also considered as important (zimmerman, 1995). other authors have also dealt with dimensions of psychological empowerment over the years, such as thomas and velthouse, (1990), describing empowerment as the increased intrinsic motivation during the performance of one´s function based on four concepts: a) significance of task, defined as the moment when an individual perceives a connection between their work and their own personal standards; b) competence, which it is the belief that he / she has in their abilities to develop their activity; c) impact, which is the amount of influence that a person feels he / she gets from certain work results; and, finally, d) self determination, defined as an individual's perception of choice related to the tasks he / she commits to. subsequently, other authors have made changes to these dimensions, as menon and borg (1995) who modified three of them: perceived control, perceived competence and goal internalization. this regrouping is done after researching different studies where it can be observed that most of the items saturate the dimension of "impact" which, subsequently, defines this new scale of three and where perceived control is the sum of impact plus choice. http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 6 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 more current authors like matthews, diaz and cole (2003) have considered the concept on 3 levels: dynamical structure, control of decisions and information flow, or like chritens (2012) who takes into consideration four essential aspects: emotional or intrapersonal, cognitive, behavioural and interpersonal. analyzing the concept of empowerment in the hotel sector, authors like sasser and arbeit (1976) started talking about the concept of empowerment as an ideal environment for the creation of corporate culture adapted to the needs of the people who form it. in this regard, several studies conducted by ulmer (1987) identified that for several years salary had no longer been the only main variable to motivate workers in hotel chains nationwide. therefore, worker satisfaction is considered an essential feature in the tourist sector in order to ensure quality service to customers, considering that workers have a direct contact with them and that customer service is a fundamental feature of company evaluation in the service sector. this should be achieved through the business philosophy of the company and carried out by its management so that there is a global integration of departments (foster and cadogan, 2000; dominguez, 2009). currently, the hotel industry has great importance in the spanish gdp and there is a strong competition in the market, which is why hotels should obtain a competitive advantage over their competition, and one of lescano’s (2011) ideas is precisely the organizational empowerment meant to create added value to customers with the offered service. empowerment has been considered as a predominant variable in companies that wish to have a capacity of organizational learning superior to that of other companies, since, according to studies of chiva, alegre and lapiedra, (2007), employee involvement and motivation are a prerequisite to learn inside the company. http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 7 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 3. organisational learning capability (olc) once the previous two variables have been analyzed, organizational learning is perceived to be a process through which organizations learn (chiva and alegre, 2008). the learning capacity of an organization is certainly a competitive advantage that lead to greater and better organizational performance, which is a peculiarity of each company. in the process of analyzing learning abilities, it is interesting to observe the development of the one-loop, double-loop and deutero learning (argyris and schon, 1996), where the concept of one-loop action-result represents the company where workers work under the transactional leadership in which success is rewarded and errors are penalized, their direct causes being analyzed, while the double-loop learning allows questioning why certain actions are done, going back as far as its causes, which is more in line with the transformational leadership and reaches the highest level of teaching deutero-learning where the company learn how to learn. there are studies that describe three levels of learning: an individual, group and organizational one, with a cycle of the knowledge determined through the three levels, and that establish four dimensions of learning: intuition, interpretation, integration and institutionalization (crossan, lane and white, 1999). other authors maintain the three levels of learning, but establish three dimensions to it within the organization: acquisition, transfer and integration (jerez, and lapiedra alegre, 2007). respectively, there is no general consensus about the aspects that influence the learning capacity within organizations, the altruistic leadership style having a positive influence (atwood, mora and kaplan, 2010) on it. transformational leadership styles promote individual and group learning, because they promote risk-taking, fallibility and dialogue as a way to obtain learning (berson, 2006). also, the olc construct is based on the context of learning and has five dimensions: experimentation, risk investment, interaction with the business environment, communication and participatory decision-making (chiva, and lapiedra alegre, 2007). on the other hand, a factor not proposed in this study but favourably associated to the learning capacity in business is innovation. a recent study of the critical factors of brand innovation identifies it as an intermediary positive factor in the learning capacity of an organization (nguyen, yu xiaoyu et al. 2016). http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2017.6364 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 n° 1 (2017): 1-15 1 8 http://polipapers.upv.es/index.php/muse/ the hotel industry, specifically high-end hotels, should devote great efforts within their organizations to the learning of their customers' needs, which is why the investment in tct facilitates workers to interact with customers more efficiently. (law and jogaratnam, 2005) tn this sense, we reinforce the benefits of having a good capacity for learning at an organizational level in order to better coordinate goals and to measure the results of our corporate mission statement through the implementation of the bsc. the balanced scorecard also allows us to unify operational criteria in the case of hotel chains where the systems established in the bsc have to be replicated. 4. balanced scorecard the balanced scorecard is a strategic management system developed by robert s. kaplan and david p. norton (1992) which is based on the mission and vision of the company. tt suggests establishing an alignment of employees' goals with those of the company, based on the development of four perspectives: the learning and growth perspective and the business process perspective as internal variables, and the customer and financial perspectives as external variables. for its successful implementation in the company, we start from the correct sequential application of the perspectives ranging from internal to external ones. figure 1. sequential development of the bsc perspectives learning and growth business process customer financial source: own elaboration the bsc implementation requires developing objectives for each of the prospects in such a way that the strategy of the company is made measurable through monitoring indicators that measure these objectives. the objectives of the learning and growth perspective (lgp) lie at the basis of the other three perspectives so that they function as inducers of the results of these perspectives. according to norton and kaplan the categories of variables that belong to http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 9 http://polipapers.upv.es/index.php/muse/ this approach are: employees´ capabilities, the capacity of the information systems and empowerment as delegation of power. performance indicators in lgp seek to strengthen individual as well as teamwork performance by setting different goals in each case. most companies decide to start the implementation of the bsc through their employees` knowledge and acceptance of the benefits that entails the whole company working in the same direction, meaning that every member, regardless of their rank, should be aware of the mission and vision statements and the objectives and perspectives set to achieve the expected goals (norreklit and schoenfel, 1998). the main idea is to consider the bsc as a management system that connects the strategy of the organization with the specific tasks of individuals and teams, namely with measurable key perfomance indicators (kpi) (voelpel et al. 2006). the way to understand the whole bsc is based on the strategic map in which all perspectives are related in a cause effect chain with their objectives and indicators. in the hotel sector the bsc is an important tool for the coordination of different stakeholders that the strategic function of the company aims at, thus giving meaning and continuity to intangible factors. 5. theoretical approach transformational leadership facilitates organizational learning, senge (1990) and promotes the creation of organizational knowledge (nonaka and takeuchi 1995). a positive evolution between transformational leadership and learning capacity was obtained as a result of a survey of 408 spanish companies in the three sectors, where podsakoff`s scale of leadership measurement was used, podsakoff et al (1996), verifying the goodness of transformational leadership in the generation of creativity in organizations, a basic factor in order to achieve high learning capacity. garcia, romerosa and llorens (2005). based on the development of these studies we can establish a first hypothesis that links transformational leadership and learning in organizations. http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 10 http://polipapers.upv.es/index.php/muse/ hypothesis 1: hotel organizations with a greater degree of development of transformational leadership show an increased learning capacity. the notion of learning capacity is linked to empowerment as a dimension of the measurement scale, goh and richards (1997), goh (2003). in the study by mishra and bhaskar (2010), about the question “empowerment: a necessary attribute of learning organization?” the authors conclude that there is a very weak relationship between these two variables in the business environment in india. however, in the study by khalil et al (2011) it was found that there is a positive relationship between the role of self-efficacy and self-determination of psychological empowerment and organizational learning. a positive relationship is established among three of four dimensions of psychological empowerment, namely meaning, competence and impact related to innovation capability, ertürk (2011). in short, it seems significant that studies should find a positive relationship between psychological empowerment and learning organization and thus we set the following hypothesis: hypothesis 2: in hotel organizations with a high degree of empowerment the learning capacity is improved. the third hypothesis we propose on a theoretical level in this study consists of verifying the positive effect organizational learning capacity has on organizations that decide to implement the bsc. the two perspectives about the way organizations learn, developed only empirically so far, are those that include companies that learn and those that manage knowledge, fernandez millan and salgueiro (2006). the implementation of bsc in companies requires the organization of some of the following characteristics: understanding of the objectives, acceptance of change, proactivity in its implementation, and maximum performance of both individual and, especially, group skills. the main characteristics that define the learning capacity of an organization (céspedes, jerez and valle, 2005), are: http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 11 http://polipapers.upv.es/index.php/muse/ a) commitment to learning b) system perspective c) openness and experimentation d) transfer of knowledge based on the concurrence of common features needed for the development of organizational learning capacity and the implementation of the balanced scorecard we establish the following hypothesis: hypothesis 3: hotel organizations with a greater orientation towards learning show a higher degree of efficiency in the adaptation of the balanced scorecard. figure 2. proposed theoretical model. source: own elaboration 6. conclusions once the proposed fieldwork is done, we should develop – if the results of the relations raised in the hypotheses are confirmed – a predictive methodology to assess the efficiency and effectiveness of the implementation of the balanced scorecard based on learning capacity, which at the same time will be conditioned by empowerment and transformational leadership. http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 12 http://polipapers.upv.es/index.php/muse/ we could establish some items that assess the previously mentioned constructs, which would allow to predict the success of the implementation of the balanced scorecard. the implementation of the bsc would also serve to control the initial constructs before the implantation is carried out, irrespective of the sector we address, but especially in the hotel sector, where personal relationships with customers have an exceptional impact on the final result. after that, the obtained feedback on learning capacity with an improved bsc could be studied. http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 13 http://polipapers.upv.es/index.php/muse/ 7. references a. srinivasa rao waheed kareem abdul, (2015),"impacto de liderazgo transformacional en el rendimiento del equipo: un estudio empírico en emiratos árabes unidos", la medición de la excelencia empresarial, vol. 19 iss 4 págs. 30 – 56. bass, b. y avolio, b. (1990) “the implications of transactional and transformational leadership for individual, team, and organizational development.” research in organizational change and development, vol. 4, 1990, pp. 231-272. bass, b. y avolio, b. (2000) “full range leadership development: manual for multifactor leadership questionnaire.” mindgarden, california, 2000. blackburn, r., kovalainen, a. (2009). “researching small firms and entrepreneurship: past, present and future“international journal of management reviews, núm. 11, 127-148. bustinza, o. f., molina, l. m., arias-aranda, d. (2010). “organizational learning and performance: relationship between the dynamic and the operational capabilities of the firm. afr. j. bus. manage.” journal business management, núm.4, 4067-4078. byham, t. (2011). “la nueva realidad del liderazgo de los líderes en mandos medios.” rh magazine, 2(6), 24-28. carolyn timms p., brough deborah g., (2012)," burned but connected: the co existence of psychological burnout and participation ", oficial de la administración educativa, vol. 50 iss 3 pp. 327 – 345. céspedes lorente, j.j., jérez gómez, p., valle cabrera r. (2005), “las prácticas de recursos humanos de alto rendimiento y la capacidad de aprendizaje organizativo” cuadernos de economía y dirección de la empresa. núm 24. chenevert, d. y m. tremblay, (2009) “empoderamiento y prácticas de compensación sobre el rendimiento de los recursos humanos. las empresas canadienses.” revista internacional de gestión de recursos humanos, 20, 738-770. fernández mesa, a., alegre vidal j., chiva gómez, r. (2012) “orientación emprendedora, capacidad de aprendizaje organizativo y desempeño innovador” journal of techonology management & innovation. vol.7 no.2 santiago jul. 2012. hall, m. (2008). “el efecto global de los sistemas de medición del rendimiento en función de claridad, empowerment psicológico y desempeño gerencial, contable, organizaciones y la sociedad,” 33, 141-163. hater, j. y bass, b. (1988) “superior’s evaluations and subordinate perceptions of transformational and transactional leadership.” journal of applied psychology, vol. 73, 1988, pp. 695-702. http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 14 http://polipapers.upv.es/index.php/muse/ infante, j.c. infante, f.j. martínez, m. garcía. (2016). evolución curricular en si/ti en el sector hotelero español: estudio longitudinal y comparativo al sector empresarial español, international journal of scientific management and tourism, vol.2, 1, pp 109-118. jimenéz jimenéz d., sanz valle r., (2006), “innovación, aprendizaje organizativo y resultados empresariales” cuadernos de economía y dirección de la empresa. núm. 29. kaplan robert s., norton david p. (1992) “the balanced scorecard, measures that drive performance” harvard business review, july–august 2005. kaplan, r.s. y d. p. norton. (2001). “la organización centrada en la estrategia balanced scorecard: cómo las empresas prosperen en el nuevo ambiente de negocios”, boston, ma: harvard business school press. lillis, a. m. (2010), “el papel de la medición y la evaluación en la construcción de capacidades organizativas y de rendimiento, la contabilidad, las organizaciones y la sociedad,” 35, 689. mathieu, j.e. y zajac, d.m. (1990), "una revisión y meta-análisis de los antecedentes, correlatos y consecuencias del compromiso organizacional", boletín psicológicas, vol. 108, nº 2, págs. 171-194. mendoza, j., y flores, r. (2006).” empowerment en grupos de trabajo en organizaciones mexicanas: motivación, trabajo en equipo y desempeño”. 9(4), 391-399. calderón molina, m., hurtado gonzález j., palacios florencio b., galán gonzález j., (2014),"la adopción del balanced scorecard hace aumentar los niveles de clima organizacional, compromiso de los empleados, la satisfacción en el trabajo y dedicación de trabajo?", una decisión de la administración vol. 52 iss 5 pp. 983 – 1010. millissa f.y. cheung, chi-sum wong, (2011),"el liderazgo transformacional, líder de apoyo, y el empleado de la creatividad", liderazgo &amp; desarrollo de organización oficial, vol. 32 iss 7 pp. 656-672. moreno meza, r. j.; espíritu olmos, r. (2014). el empowerment, el liderazgo y la innovación en empresas de servicios, como una ventaja competitiva de las medianas empresas del municipio de tecomán, colima, méxico. en desarrollo gerencial revista de la facultad de ciencias económicas, administrativas y contables, vol. , pp.72-97. nurdan özaralli, (2003),"efectos de liderazgo transformacional en la potenciación y la efectividad del equipo", el liderazgo y el desarrollo de la organización oficial, vol. 24 iss 6 págs. 335 – 344. osorno balbín, a., oquendo gómez, e. m., monsalve trujillo, i. i. & martínez gómez, j. (enero-junio, 2016). gestión del conocimiento, innovación para el crecimiento empresarial. revista science of human action, 1(1), 104-116. palomo vadillo, m. t. (2010). “liderazgo y motivación de equipos de trabajo. “editorial esic. españa. http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6364 eissn: 2341-2593 ballester-miquel et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 1-15 | 15 http://polipapers.upv.es/index.php/muse/ pastor tejedor, j., (2015) “on the ultimate goal of management in spanish hospitals” departamento de economía y dirección de empresas, centro politécnico superior de la universidad de zaragoza, zaragoza, españa. pearce, j. l. (1993). “hacia un comportamiento organizacional de obreros contratados: su implicación psicológica y los efectos sobre el empleado co-trabajadores,” academy of management journal, 36, 1082-1096. peralta, s. (2010) “aplicación del mlq a formadores de rrhh: un estudio empírico” cuadernos de estudios empresariales 2010, vol. 20 127-144. roca puig, v.; escrig tena, a.b. y bou llusar, j.c. (2002): “compromiso con los empleados y estrategia competitiva: un análisis intersectorial de su repercusión sobre los resultados”, cuadernos de economía y dirección de la empresa, 12, pp. 267-289. segovia romo, a., mendoza gómez j., (2013) “el desempeño del empleado de nivel medio en relación al empowerment psicológico y al liderazgo transformacional” xviii congreso internacional de contaduría, administración e informática. shelley d. dionne, francisco j. yammarino, leanne e. atwater, william d. spangler, (2004),"el liderazgo transformacional y el rendimiento del equipo", el diario del cambio organizativo, gestión, vol. 17 iss 2 pp. 177-193. shrivastava, p. (1983): “a typology of organizational learning systems”, journal of management studies, 20(1), pp. 8-28. spreitzer, g.m. (1995). “psychological empowerment in the workplace: dimensions, measurement and validation. academy of management journal,” 38(5), 1442-1465. teresa z. taylor joseph psotka peter legree, (2015),"relaciones entre aplicaciones de conocimientos tácitos y/transaccional líder transformacional", estilos de liderazgo y desarrollo de la organización oficial, vol. 36 iss 2 pp. 120 – 136. torres petit, e., gutiérrez gonzález, l., (2007) “liderazgo con empowerment: promotor de la innovación” universidad del zulia. facultad experimental de ciencias. departamento de ciencias humanas. unidad académica estudios del desarrollo apartado 526. http://dx.doi.org/10.4995/muse.2017.6364 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 133 accreditation of spanish engineering programs, first experiences. the case of the terrassa school of engineering mª dolores álvarez*, mireia mata, javier cañavate, joaquim marqués, carme espot, santiago forcada, jordi voltas, núria garrido, jordi sellarès, alfred gil terrassa school of engineering, colom,1 08222 terrassa, barcelona spain * corresponding author: email: sots-qualitat@eet.upc.edu; phone: + 34 937398200 received: 2015-7-10; accepted: 2016-1-3 abstract the implementation of the european space for higher education has entailed new requirements for spanish higher education programs. regulations (rd 1393, 2007) stablish that university programs, in order to have official validity, must be submitted to an external evaluation process before their official implementation, denominated validation, and to an ex-post process or accreditation. terrassa school of engineering (eet) was one of the first schools in spain to adapt to the european space for higher education, in the academic period 2009-10 and then, one of the first university institutions submitted to an accreditation process. in this communication, the important role of the internal quality assurance system in the assessment of the school’s programs is exposed as well as the approach followed in the key steps of the process: accreditation keywords internal quality assurance system, programme assessment, accreditation, european higher education area multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 134 1. introduction in recent years, due to the implementation of the european higher education area (ehea), university studies in spain have undergone great changes. the spanish legislation (rd 1393, 2007) states that the official university degrees must undergo a process of external evaluation in order to be officially valid. these external evaluation processes include an ex ante assessment, which is called validation and an ex post assessment process, which is accreditation. between these two stages, there is an annual monitoring of all university programs, which may include corrections or modifications, aimed at improving aspects under consideration. thus, the validation of a degree, its annual monitoring and implementation, the introduction of possible modifications and the accreditation are assessment processes that nowadays are an important part of the life cycle of a university degree. the internal quality assurance system of every school (iqas), as a relevant part in the process of developing the curricula, plays a key role throughout the lifecycle of programs. it enables improvement in evaluation, lecturers and lecturing quality, external traineeships and international mobility, as well as in analysing stakeholders satisfaction and employability of graduates. the implementation of the iqas allows detecting necessary modifications and opportunities, planning actions of improvement and measuring results in relation to the actions carried out, which results in better quality of the university system. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 135 figure 1 life cycle of a recognized university degree (gpaq, 2015) in this document, the important role that the iqas of terrassa school of engineering (eet) is exposed. the approach to the evaluation of the different programs, the new experiences related to the process and the methodology used in monitoring, and the final accreditation of the implemented degrees will be described and discussed in order to provide relevant information about this important improvement in the spanish university system. the spanish national agency of evaluation and accreditation (aneca) validates the proposal of new study programs through the process called verifica. the accreditation request is processed after 6 years of monitoring the institution, where the programs have been implemented. the process includes an inspection visit at the terrassa school of engineering in 2015, carried out by an external evaluation committee (eec) designated by the agency. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 136 2. terrassa school of engineering. study programmes. terrassa school of engineering (eet) is a higher education school belonging to polytechnic university of catalonia (upc). during its more than 100-year history, eet is specialized in engineering education, and is well known for its prestige and quality. eet is located on the campus of the polytechnic university of catalonia (upc) in terrassa sharing services and resources with other schools. the campus includes higher education, research centres and other facilities, occupying 72,000 m2 in which are located 4 schools, 23 departments, an institute of textile research intexter, the catalonian centre for plastic, 37 research groups, 5,500 students, 400 teachers and 250 professionals and researchers. every day, over 6,000 people work and study in this campus, making it an important economical centre of the city. starting the process in 2009-10, the terrassa school of engineering was one of the first schools to adapt its studies to european higher education area (ehea). this adjustment included simplification of the syllabi and development of common itineraries between similar programs, improving the efficiency of the implementation of the new curricula. the different programs currently taught at the eet-upc are on in table 1. the term industrial engineering, in spain encompasses all competences related to mechanical, electrical, electronical, textile, chemical and industrial design engineering. the eet-upc in addition offers the students the possibility of obtaining a double degree. this double programme requires that students enrol for one extra year of additional subjects (66 ects credits). once these subjects are accomplished, the students obtain two diplomas awarded by the upc. the additional year ensures the achievement of competencies corresponding to both degrees. (see table 2) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 137 table 1 academic programs offered by eet-upc. area obtained degree industrial engineering degree in industrial design and product development engineering degree in electrical engineering degree in industrial electronics and automation engineering degree in mechanical engineering degree in chemical engineering degree in textile engineering telecommunications engineering degree in audio-visual systems engineering master (90ects) master in fibrous materials technology (textile, paper, graphic) table 2 academic programs offered by eet-upc. double degrees first degree second degree (double degree) degree in electrical engineering degree in mechanical engineering degree in industrial electronics and automation engineering degree in industrial electronics and automation engineering degree in electrical engineering degree in mechanical engineering degree in mechanical engineering degree in electrical engineering degree in industrial electronics and automation engineering multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 138 first degree second degree (double degree) degree in chemical engineering degree in textile engineering degree in industrial design and product development engineering degree in chemical engineering degree in mechanical engineering degree in textile engineering degree in textile engineering degree in mechanical engineering degree in chemical engineering degree in industrial design and product development engineering degree in industrial design and product development engineering degree in mechanical engineering degree in textile engineering 3. the importance of the internal quality assurance system in the life cycle of a university degree. monitoring process. as already mentioned above, programs annual monitoring is compulsory during the period going from the validation of the study programs to their accreditation. according to the quality assurance agency (aqu), the monitoring has two main objectives: constitute a useful tool for managing the school allowing assessment of the academic contents, development through the analysis of data and indicators, and producing, when necessary, improvement proposals oriented to correct any observed deviations in the ordinary development from the stated syllabi. produce a source of useful evidences for degree accreditation. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 139 as specified in the “bachelor and master degree programs monitoring guide" published by the agency (aqu, 2013), the annual monitoring reports of the eet-upc degrees focused in four different dimensions: public information on the operational development of academic program. public information on the indicators. the analysis of academic programme and improvement actions. the adequacy of the internal quality assurance system (iqas) for monitoring the university degree. the university, upc through its planning, evaluation and quality office (gpaq) has developed a computer application called sat where different centres must submit annually their reports. the office after reviewing these reports forward them to the accreditation agency aqu. the deep discussion on these four dimensions is not easy. in this sense, the iqas implemented in the eet-upc, ensures collecting relevant information and data in order to provide an efficient management of the whole results corresponding to academic programs, which facilitates the monitoring process and modification in degrees, ensuring continuous improvement through objective data analysis. the iqas eet-upc is divided in different processes. each process has a responsible person that will assure that the process is properly operating, and that will be regularly reviewing it in order to determine the need for modifications. in addition, monitoring indicators characterize each process. (see figure 2) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 140 figure 2 importance of the iqas in the monitoring process measurement, analysis and temporal evolution of these indicators is a task assigned to the responsible for the process. he, in collaboration with a commission, generates an annual report (ar) that is included in the annual report of the school. this report is annually approved by the board of the school and is a public document. the reports generally include the following sections: aiming goals: the specific objectives are specified in relation to the process. actions: actions taken to achieve the specified objectives. results: indicators / results and their evolution. assessment: achievements assessment and improvement proposal multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 141 4. the accreditation process. the eet-upc experience in order to apply for accreditation, the eet-upc, developed a preparatory report for this purpose (self-assessment report) that was elaborated following a template provided by aqu. figure 3 shows the process. as in the case of the annual monitoring reports, the upc has developed a software application that allows filling the different sections of the template that generates evidences by linking relevant documents. the application is available in a workspace to members of the internal evaluation committee (iec) and a section for the technical review of the selfassessment report that the gpaq performs. several standards are discussed in the report, most of them are related to the school in general, but some of them are specific for each study program. any statement or justification included in the analysis must be supported by relevant documentation (evidence). an external team of auditors (external panel), appointed by aqu analyses the report, reviews the evidence and pays a visit to the school. for one or more days, the external panel holds meetings with several groups of stakeholders, visits the facilities and analyses in situ, if further documentation is needed. the panel elaborates a report that is delivered to aqu for final approval. the self-assessment report for accreditation considers six standards: the quality of the study program, the relevance of public information, the effectiveness of iqas, the adequacy of the faculties to the study program, the effectiveness of the learning support systems and the quality of the results of the programs. the implementation of the iqas has been fundamental in order to analyse the 6 standards, the evidences and the improvement proposals included in the report. as explained in previous section, since the implementation of iqas, the school gather: historical values of indicators related to the quality processes of the school. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 142 the annual reports provided by the responsible for the processes including objectives, actions and results. depending on the achievements, an evaluation is issued. the annual monitoring reports submitted to the aqu containing detailed proposals for improvement and, in some cases, proposals for some modifications in the study programs. furthermore, and linked to each process constituting iqas of the school, a documentary check of system own evidences is available. often, these evidences are included in the report. given the previous analysis and documents generated internally along the life cycle of the degrees, the final presentation of the self-assessment report has been much more agile and detailed, producing a report of excellent quality. 4.1. self-assessment report. internal organization figure 3 shows the flowchart for the process leading to the elaboration of the selfassessment report required for accrediting the degree programs at the eet-upc. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 143 figure 3 elaboration of the self-assessment report the first step was the creation of a self-assessment commission (sac) whose composition and functions were approved by eet-upc. table 3 shows the composition and functions of sac. as shown in the flowchart (see figure 3), the sac was responsible for the elaboration of the report which, once reviewed by the gpaq and made available to the public, was approved by the board of the centre. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 144 table 3. composition and functions of the self-assessment commission (sac) the self-assessment commission decided the appointment of four members to start setting out a draft document (see table 3). the first draft was reviewed and modified by the rest of the members in the commission. this working method enabled the possibility to all the individuals involved to contribute with data and expertise, to the analysis. the report is public (spanish only) and available to all groups (internal and external) through the website of the school (eet 2015). the external team of auditors, in turn, issued a report reviewing the evidences available, and requesting some clarifications and additional information regarding some specific evidences. stamen description responsibilities for the accreditation number faculties coordinators of the study programme assessment, review/editing of the report, evidences 7 administrative staff head of administration editing, technical consulting 6 responsible for the direction area technical consulting, evidences academic management technical consulting, evidences it manager technical consulting, evidences logistics and services technical consulting, evidences direction assistant technical consulting, evidences direction director editing of the report, evidences 3 vice director for quality editing of the report, evidences vice director for academic organization editing of the report, evidences students students representatives assessment, review/editing of the report 4 enterprise companies assessment, review/editing of the report 1 total 21 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 145 4.2 contents of the self-assessment report as stated above the report for accreditation included the analysis of the 6 quality standards criteria set by aqu: • standard 1: programme quality (analysis at school level): the programme's design (competences profile and structure of the curriculum) is updated according to the requirements of the discipline and it meets the academic required level according to qfehea (meces) in spain. • standard 2: relevance of public information (analysis at school level): the institution properly informs all stakeholders of the programme characteristics and the management processes necessary for quality assurance. • standard 3: efficacy of the programme internal quality assurance system (analysis at school level): the institution has a formal internal quality assurance system that assures the quality efficiently and the continuous improvement of the program. • standard 4: adequacy of faculties for the programme (analysis at school level): faculties are enough in number and their profile fits academic needs according the quality criteria. • standard 5: effectiveness of learning support systems (analysis at school level): the institution has adequate and efficient guidance services and resources for student learning. • standard 6: quality of programme learning outcomes (analysis at specific degree level): learning and assessment activities are consistent with the programme competences profile. the outcomes of these processes are adequate in terms of both academic achievements, which correspond to the programme's level as of the qf-ehea in spain, and the academic and employment indicators. as previously stated, each quality standard is associated to evidences that support the analysis. from the analysis, several proposals of improvement are included in the report. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 146 at table 4 we highlight some contents of the report related to the standards. table 4 relevant aspects of the accreditation report standard important contents standard 1: programme quality analysis on student’s profile. academic coordination standard 2: relevance of public information information resources and their adequacy to the different interest groups standard 3: efficacy of the programme's internal quality assurance system importance of sgic in the lifecycle of the degrees mechanisms for information compilation about the learning outcomes, results and satisfaction standard 4: adequacy of faculties analysis of the faculties’ adequacy, curriculum, experience, research. analysis of the degree of satisfaction of students regarding faculties analysis of the satisfaction of upc with with regard to faculties. mechanisms for assigning subjects to faculties. standard 5: efficiency of learning support systems explanation and analysis of the professional and academic counselling provided by the school. description and analysis of learning resources available to the students and pedagogic approaches. standard 6: quality of programme (learning) outcomes 4 subjects of every degree programme were selected and studied specifically including achievement, evaluation system, competences provided, learning methodology. results of the evaluation of every subject of the syllabi. final thesis, issues, typology, results. traineeship possibilities, companies and placements relevant indicators of the development of the students. employment data analysis. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 147 4.3 visit of the external evaluation committee the upc gpaq informed us on the composition of the external team of auditors, and proposed a schedule for the visit to the school. all panel members were external to our institution and were selected according to a specific profile and requirements set by aqu. the external committee consisted of a president, two academic members, a representative from a company, a student representative and a secretary. table 5 shows the schedule for the visit. this programme was agreed on the proposal of the committee. the visit schedule shows that an important point is the focus on interviewing representatives of different agents present at the educational institution: responsible, teachers, students, graduates and employers. apart from visiting the facilities, the external team of auditors objective is to gather opinions and comments from several groups, assessing their satisfaction with the services and academic programs of the school. table 5 programme of the visit of the external team of auditors may 13th of 2015 timetable activity 8:3010:30 external team of auditors previous work (documentation review) 10:3010:45 self-assessment team reception by the management team 10:4511:30 interview with the management team and the self-assessment team 11:3011:45 break 11:4512:30 interview with initial common phase industrial engineering students, and initial phase of bachelor’s degree in audio-visual systems engineering students. 12:3013:15 interview with students of 3rd and 4th bd in audio-visual systems engineering bd in mechanical engineering bd in electrical engineering bd in industrial electronics and automatic control engineering multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 148 may 13th of 2015 timetable activity 13:1514:00 interview with students of 3rd and 4th bd in industrial design and product development engineering bd in textile technology and design engineering bd in chemical engineering 14:0015:00 lunch 15:0016:00 external team of auditors work 16:0016:45 graduates interview 16:4517:30 employers interview may, 14th of 2015 timetable activity 8:30-9:15 interview with initial common phase industrial engineering teachers and initial phase of bachelor’s degree in audio-visual systems engineering teachers. 9:1510:00 teachers staff interview (not included in previous audience) 10:0011:00 space and facilities visit 11:0011:30 break 11:3012:15 public audience/ panel´s work 12:1513:00 2nd interview with the management team and the self-assessment team 13:0013:45 preparation of conclusions 13:4514:15 preliminary conclusions and farewell 14:1515:15 lunch 15:1517:15 external team of auditors work/ start preparing the external evaluation report the school management team proposed the members for each group, and vice director for quality organized the meetings. table 6 shows the selection criteria for the representatives of each group. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 149 the purpose of these meetings was to inform about: the accreditation process and its importance. members who were part of the external team of auditors. the programme of the visit. possible issues at hearings. the questions listed in the "guide to the accreditation of recognized bachelor and master’s degree programs” version 1.0 "published by the aqu (aqu, 2013). table 6 criteria for the selection of the representatives of each group at the audience group criteria number students 1st and 2ond year (2 for degree minimum) students representatives 10 students 3rd and 4th year (2 for degree minimum) must have participated in international or company placement programs must be doing final thesis 17 faculty 1st and 2ond year (2 for degree minimum) coordinator of compulsory subjects representatives of the departments involved in the degree 8 faculty 3rd and 4th year (2 for degree minimum) lecturers assigned to the departments involved in the degree wide age range 13 graduated (2 for degree minimum) graduates with double degree master students working students 9 employers companies that provide placements for students companies that participate in grants, agreements, etc. 7 total 64 including the self-assessment team, members of the management team and the participants in the public audience, more than 90 people met the external team auditors. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 150 the visit of the facilities was programmed to show common spaces (classrooms, study rooms, computer rooms, library ...) and teaching laboratories. the coordinators provided the committee with a dossier describing the labs that included the following data: use of the laboratory in the degree. name of the coordinator of the degree. name of the facility. capacity (number of workplaces). subjects that make use of the facility. photographs, name, description and special features of the most relevant equipment. before concluding the visit, the external panel presented preliminary findings to the management. at this moment, we are expecting the external evaluation report, but the eac anticipated that they will propose the accreditation of every degrees at the time they highlighted a number of good practices related to various standards. they highlighted specifically some aspects as good relationship with industrial environment, learning support systems; tutoring and counselling, internationalization, library resources and digital campus, also praised the evaluation system for transversal competences and and labour market indicators. 5. conclusions the implementation of iqas at eet-upc has been effective for continuous improvement and quality assurance for the programme degree offered. iqas has enabled the monitoring and the appropriate modification of academic programs through the validation and accreditation thereof. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4683 social and technological sciences eissn: 2341-2593 mª dolores álvarez et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 133-151 | 151 the accreditation process of the degree programs offered by the eet-upc has been carried out on a normal course. we emphasize the high participation and collaboration shown by the various stakeholders in this process. the strong relationship with the different interest groups has facilitated the external team auditors to have a first-hand perception of stakeholder’s opinions and considerations, beyond the specific figures and indicators that appear on the self for accreditation. 6. references aqu. 2013. agencia per la qualitat de sistema universitari de catalunya. guia per al seguiment de les titulacions oficials de grau i màster. versió 1.0. 2013. eet. 2015. escuela de ingenieria de terrassa. eet-upc. [en línea] 04 de junio de 2015. [citado el: 04 de junio de 2015.] http://www.eet.upc.edu/intranet/direccio/autoinformeacreditacio-final. gpaq. 2015. gabinet de planificació, avaluació i qualitat upc. [en línea] 2015. [citado el: 04 de junio de 2015.] http://www.upc.edu/gpaq. rd 1393. 2007. real decreto 1393/2007, de 29 de octubre, por el que se establece la ordenación de las enseñanzas universitarias oficiales. 2007. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 exploring innovative and creative ways of teaching c. franco*, c. gillanders dpto. de didáctica de la expresión musical, plástica y corporal. universidade de santiago de compostela, av. xoán xxiii, s/n, 13782 santiago de compostela, spain. * corresponding author: email: carmen.franco@usc.es; phone: + 34 881812015 received: 2013-12-15; accepted: 2014-08-04 abstract with the intention of providing a starting point to reflect on both aspects in which the subject teaching innovation and introduction to educational research in arts of the master’s degree in teaching in secondary school breaks up and merges, we created a situation, understood as a learning environment, that seeks to show us how we react before a situation different from the expected one. for this reason, we played with the idea of a classroom in construction, something that still has to be finished off. we created confusion and surprise, but slowly a discussion started. and from the reflections raised, the exchange of opinions and relating the assignment to other subjects of the master’s degree, in particular with learning and teaching of art, we began to go more deeply into the meaning of various concepts related to teaching and research by means of projects carried out in groups and from different approaches. the purpose was to offer a space to experience through action, to get to know our previous ideas and how to approach the different research paradigms. keywords innovation; teaching; projects, reflection; action; research franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 53 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 1. introduction as from the academic year 2009/2010 the new postgraduate degree in teacher training in compulsory secondary education and post-compulsory schooling, vocational training and language teaching begins in spain. its main aim is to “offer future teachers the pedagogical and didactic training required by the current regulations”1. to end these studies -students must have a bachelor’s degree to be able to take this coursestudents must present a final master’s dissertation which can be considered as “an academic work that summarises all the competences developed during the master’s degree and in which all the learning training modules take part –specially the practicumand as a research related to teacher practices in the corresponding specialization”2. in the subject teaching innovation and introduction to educational research in arts we address the different research paradigms and we expect the students to get to know the necessary tools to become teachers-artists-researchers. in many cases, it is the first contact that they have with educational research. students have different backgrounds, they come from the faculty of fine arts, architecture, art history or the music conservatoires. in relation to students with a musical background, we notice that, except for some specialisations such as musicology or music pedagogy, there doesn’t exist a research tradition or the need to submit written papers during their studies. for this reason, as from the academic year 2011/12 we included in the programme of the subject as a compulsory requirement that students take the informational competences course offered by the general library of the university. in this course different themes necessary to achieve any written paper are dealt with such as the tracing of information in the internet, bibliography search, search engines and databases, how to quote, apa or cide rules and some electronic resources (google docs, blogs, wikis, etc.). in this way, the formal aspects of the written works 1 subject’s programme: http://www.usc.es/es/centros/cc_educacion/materia.html?materia=76701&ano=64 2 subject’s programme: http://www.usc.es/es/centros/cc_educacion/materia.html?materia=76701&ano=64 franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 54 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 that are required during the postgraduate degree, specially the final master’s dissertation, are covered. 2. the theoretical background the reasons which took us to begin this subject with a performance-related experience have their roots in a series of schools of thought that come in their origin from the philosophy coined by john dewey (1859-1952). we pretended to create a situation that contributed to the breaking-up of the preconcepts that a priori our students have because, in our point of view, for future teachers most important of all is reflection directed to drawing up questions such as: what type of teacher do i want to be? what relationships do i want to establish in my classroom? what kind of learning will i propose to my students? etc. the attitude with which students come into a classroom is, very often, an attitude of wait, a passive attitude. they come in waiting to receive, accustomed to the active role of teachers. their mission usually is to listen to what they are told in an oral way. the fact of finding an upset classroom aims to surprise and confront our students with a different situation that leads to changing the attitudes they normally adopt: come in, look for a seat, sit down, wait, listen, look at the whiteboard, answer if they are asked, etc. we wanted to initiate them, as future teachers, into a different dynamic that receives nourishment from surprise and that is the creator of enriching experiences through reflection. as villar angulo (1994) states based on his study of dewey and goodman, reflection starts from an initial point of uncertainty, doubt or perplexity, looking for the solving of curricular problems or the search for materials that can help solve the matter. in this sense, we can talk of an intuitive thinking of teachers that directs their personal and particular conceptions to the resolution of an aim, which in our case is a possible contribution to the qualitative improvement of arts education in secondary school. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 55 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 dewey (1949, 1971) understood education as an anthropological-social process by which man develops, in interaction with others, through the reflective action of cultural experiences. the basis of his educational position rests on two basic pillars: the meaningful experience and the experimental verification. being more specific, in his classic work how we think (1910) dewey argues that, before a situation that entails a difficulty, people can adopt three possible positions: avoid it, imagine that one can control it and, lastly, really deal with the situation. if one adopts this last solution, one normally begins to reflect. nevertheless, in spite of all the time that has passed since the statement of these ideas, already beginning the second decade of the xxi century, we find that the revolution that was expected in education during the last century has yet to come. we are a lot of teachers that believe that education is at a standstill and continues in a paradigm that should already have been overcome. teachers such as us, who work in the initial training of future teachers, have the responsibility of addressing teacher practices that aim at the improvement of education in this century. many are the authors (acaso, 2013; agra, 2005) that pursue the creation of a new model, of innovative practices that contribute to the germ of a new didactic approach. as maría acaso (2012) points out, when a teacher decides to open the door of his/her classroom he/she is already establishing small changes, invisible pedagogies that have the power to transform habits and attitudes far from negligible. whilst everything is in transformation, the educational world stays the same, anchorage; maría acaso (2013) invites us to start the reduvolution. combining the words revolution and evolution she encourages us to carry out a real transformation of the educational spaces and she does this by proposing a road map that comprises several phases: • to accept that what we expect to teach is not what students learn. • to walk towards a change in power relationships. • to live the classroom, occupying it. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 56 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 • to move from appearance towards experience. • to give priority to research and not to evaluation. the final part of the experience belongs to a collaborative type of work in which activities were proposed to be resolved in groups integrated by students with different backgrounds (faculty of fine arts, design, music conservatoire, architecture, communication, etc.); students had to develop their own ideas and organize them around a common theme (from different perspectives) to achieve, at the end of the assignment, the union of all these parts as a whole. all things considered, with this experience, and taking into account the contributions of the a/r/tography, we aim to transform knowledge through action by means of reflection and continuous analysis that allows us to comprehend the process and meanings (irwin and springgay, 2008). 3. the experience: the process’ beginning as a starting point strategy of this subject, we created a situation to observe the reaction of students when entering our classroom before a situation different from the expected one. for this reason, we played with the idea of something in construction: tables badly placed, chairs on tables, white paper covering chairs, tables and parts of the classroom... and in this setting some videos were projected that, in some cases, talked about classrooms of the past (different furniture, different arrangements, yesterday’s resources...) and, on the other hand, classrooms of the present (more open, with contemporary furniture, modern technological resources...). what meeting points exist? what were the differences? one video projected diverse musical experiences, another talked about different artistic experiences, collaborative and interdisciplinary. and all this in an atmosphere that made use of darkness and surprise. this situation, understood as a learning environment, hoped to promote reflection about what we lived, about what we experienced and about our own personal history. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 57 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 well then, before this strategy students stayed without a reaction, still, in a waiting attitude, in silence, watching the videos, looking between the tables for somewhere to sit down... what does this attitude mean? what do they think? is it that as students we always wait for someone to tell us what we must do? they were confused, surprised, without knowing what to do and that worried them. we must, as future teachers, with the aim of changing the things that we don’t like, reflect about them and discuss them. 4. following sessions a game was suggested: draw a teacher and draw a classroom. what ideas come to our head? what comes up? how do we represent it, how do we visualise it? the initial purpose was to get to know the previous ideas with which we face these concepts, and discover our own preconceptions and prejudices. a series of drawings were done about these aspects, using quick and spontaneous sketches that, afterwards, were commented on by all of us. in this game, broadly speaking, more negative than positive aspects emerged. the first, always associated with the teaching staff or their own institution. and the second ones, related to the coexistence with other youngsters. in the first group, with relation to the perception students have with the teaching staff, undesirable features in teachers were highlighted, such as: coldness in their behaviour, strictness and inflexibility in their attitudes, absence of understanding students’ points of view, the confirmation of non fluent communication, etc. the adjectives used when talking about them were very clarifying: students focused on features that pointed at an unfavourable perception of the activity. thus, although in some cases the infinite patience of some teachers was stressed, in general teachers’ monotony, couldn’t-care-less attitude, their tendency to give lectures and discursive classes and their easy annoyance, stress, boredom, etc. was highlighted. they even mentioned what seemed, at a first glance, as unconnected features with the activity emphasizing the fact that some teachers walk about the classroom with a stoop or franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 58 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 that they often use similar clothes, as in uniform, perhaps wanting to outline the rigidity of their behaviour that also affects the way they present themselves to others. these negative opinions extended, as well, to physical spaces, that were described as ugly places, without personality and aesthetics. with regards to the positive opinions, as we mentioned before, the students related them to friendship in the classroom mentioning various key ideas related to youth, opportunities, entertainment and the need of new challenges. the game, at first intended only for the students of the subject, grew and they saw the need to know what other students of the postgraduate degree thought, students from other modules and with an initial training in biology, history, chemistry, languages, etc. to progress in this knowledge, an artistic intervention was proposed in the lobby of the faculty; it is a busy hall: as an entrance, as an exit, as a place to walk through when going to a classroom... a structure is constructed that invites students to leave their thoughts, their opinions, they are invited to write or draw on a post-it. the ephemeral installation what do you think of secondary teachers? how do you see the secondary classroom? is placed during three days in the hallway and afterwards it is removed so as to gather all the data. this action is open to anyone who wishes to take part and, in consequence, not only students leave their reflections but teaching staff also take part leaving comments about the art installation. during the following classes students received information about different issues related to innovation and research by means of conferences with various professionals. we thought it would be also very interesting to interview teachers on active service so that they could offer us their point of view. we asked students to interview different teachers. in groups they had to design a protocol for the interviews. an adviser from the training and resources centre of la coruña came one session and shared with us the life-long training for teachers and the different programmes and innovation projects promoted by the xunta de galicia (proxecta plan, abalar project, contract-programme, etc.). during franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 59 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 his presentation a lot of questions arose such as what do we understand by innovation? are all the plans/programmes that the xunta de galicia organizes and offers innovative? to complete this overview we proposed students to look into the matter and, as well, we uploaded files to the virtual classroom, documents that offered more information on the subject (agra pardiñas, 2005; eisner, 2004; oriol de alarcón, 2004). ex-students of the postgraduate degree that are now working as secondary teachers also came to one of the sessions to share with us their experiences in the classrooms. the last training session, before commencing another group project, was an approach to research in arts education and an approach to the process of research. 5. work group we suggested students work in groups with the aim of going in depth and experiencing through action the different research paradigms. as the total number of students was seventeen, four groups were formed, groups that had to go in depth into the research paradigms: quantitative, qualitative, action research and arts-based research. various texts were uploaded to the virtual classroom. students had to look for examples of research or thesis that could help them understand in a better way the methodology they had to search for. in this way, we could treat the aspects of a same proposal from different and enriching approaches. what problems or issues could be inquired into from these points of view? so as to have a real basis and avoid a simple abstract exercise, the artistic intervention we mentioned before what do you think of secondary teachers? was used as a starting point. 5.1 quantitative paradigm the group that worked from this paradigm took the decision to quantify the information there was of the classroom in all the post-its that were gathered up during the art franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 60 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 installation. on the basis of this information, they decided to make the object of the study all the data focused on the classrooms. the material they worked on was eighty one documents, some graphic and others written. of these, only twelve documents clearly relate to the description of the classrooms. the principal references that students find are thoroughly quantified: references to materials: 2; existence of a platform: 1; masterly aspect of the classrooms: 3; distribution of students in twos: 1; individual tables: 5; austere furniture: 7; lack of ergonomics: 2; etc. as from the total of documents gathered (81) only in twelve references are made to the classrooms, and because the group thinks the educational space is a decisive element in education, they decided to pass a survey with 26 questions about the secondary education classrooms amongst the students of the master’s degree, classrooms that had been part of their school training period. later on they made a series of diagrams from the results obtained when employing this methodology. there are diagrams about the profile of the interviewed, about the rural or urban location of the centres, about the type of illumination, about the comfort, about the colours used in the classrooms, about the layout of tables, etc. the result of their research is a written paper that contains references to the quantitative paradigm in the theoretical framework, followed by the presentation of the study: data analysis, graphics and statistics were related, reflections, etc. 5.2 qualitative paradigm in relation to the group that worked from the qualitative paradigm, they decided to focus on the profile of the secondary teacher. for this reason, following the guidelines set out by different authors, they prepared an open questionnaire for secondary students and an interview with teachers in active service with the aim of obtaining data from which they could extract conclusions. to encourage youngsters and make them feel comfortable when answering the survey they focused the questions on their ‘favourite teacher’. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 61 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 our students of the master’s degree knew that, with this proposal, they were following a mixed design. the data interpretation was of a qualitative nature due to the open nature of the questions asked. they also took into account the context in which the interviews were undertaken. they were also conscious of the different voices, the shades and the expressiveness present in the answers of the interviewed people. after the analysis phase students arrived at some conclusions that we summarize as follows: the contemporary teacher possesses quite a high adaptability to the environment, uses his/her creativity when preparing his/her activities and classes and transmits confidence to his/her students. students support the fact that teachers prepare their sessions very well, explain and resolve the doubts students have with their opinions. this circumstance clearly indicates that secondary school teachers, with regard to knowledge transfer, are responsible before their students. all the data supports the fact that students show interest in the subject of their favourite teacher, and, as well, everybody agrees that they nearly always have tasks to do. furthermore, the contemporary teacher communicates without any serious problems of authority management with his/her students. after finishing this research, our students in an act of great responsibility, concluded that the approach of the survey about the favourite teacher could have invalidated the answers because these could be influenced by the type of rapport or affective relationship that a priori they had with this teacher and that, at first, it is not comparable to other teachers. this group also presented a written document that gathered all the research that they shared with the rest of the classroom. 5.3 action-research another of the groups based their work on the action-research methodology. they also started their proposal from the profile of the secondary school teacher as seen from the students of the master’s degree although, in this case, they added all the information each franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 62 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 one of them had written in their classroom diaries during their stay at schools (practicum 1). all the schools had very similar idiosyncrasies: public schools in an urban location. to limit the subject, they based the proposal on the motivation or lack of motivation that exists in the secondary school classrooms and how to deal with it. the initial idea was to raise a series of strategies that can help improve the motivational situation in the classroom through a hyphothesis of action generating a self-reflective spiral that allows, if necessary, to begin the process again. as future teachers, it worried them how to establish the dynamics to facilitate a reflective practice in their profession. the temporary limitations of the research made it impossible for a complete implementation of the whole process: observation, planning, action and reflection (always as a spiral process). for this reason, the project, at this specific point, was not completed. it was a theoretical proposal that must be completed with the next phases of reflection, conclusion and, if necessary, the preparation of new proposals to continue the action-research process. the part that was undertaken comprised the following actions: a) the establishment of the common problems related to motivation in secondary school classroom that could be inferred from the data collected from the diaries; b) the description and analysis of these problems to discover the possible causes; c) the construction of a theoretical framework that explains the psychological process of motivation in learning; d) the reflection about how, when and why we must motivate; and e) the preparation of a series of proposals that promote motivation in the learning process of students. amongst the proposals that were established we can mention that they found essential the implementation of creative activities adapted to the students’ interests, because in this way it is possible to improve the teaching-learning process and, thus, keep a high level of intrinsic motivation in the classroom. they also found important the preparation of a participative and collaborative project that could help a continuous exchange of information and experiences between the teachers of the school. as a summary, the aim franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 63 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 was to apply the concept of teacher-researcher that reflects about his/her own practice. as happened in the other groups, a detailed paper was written in which the two paradigms, action-research and a case study were presented. 5.4 arts based research the group that worked with this approach greatly enjoyed the experience during the development of the proposal because, due to their background in fine arts and music, they felt very comfortable working with an artistic product. starting from the same initial proposal, they decided to undertake an artistic intervention in a school similar to the one they had done at the beginning of the course. the first difficulty they came across was to find a school that would allow them to use a busy location in the school grounds. the nature of the question beneath the artistic action (how do you see your teacher?) complicated the matter. against all prediction, one of the members of the group obtained the authorization of a secondary school in ferrol (a coruña) where she had stayed during her training period (practicum 1). the following phase consisted in creating an artistic device that made possible the obtaining of active answers from the secondary school students that took part in the enquiry. after some deliberations an idea arose to make a kind of ballot box covered with paintings, cinema images, music scores, cds. it had to function as a ballot box and for this reason an opening was designed on the top of the box where students could leave their ideas and answers to the question how do you see your teacher? this question was arranged in big letters on top of a comic strip. the ballot box, the sign with the question, together with some papers and pencils, were deposited on a table in the hall of the school for one week. this school is a public school located in one of the neighbourhood of the built-up area of the city. teachers and students were told the nature of the project, and they were invited to take part in a voluntarily and franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 64 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 anonymous manner. the first day some photos were taken of the reaction of students when finding the ballot-box. during this time a superficial visual follow-up was undertaken from which the following data was obtained: the appearance of the installation in the hall of the school did not cause a lot of expectation and the lack of interest was most notorious; the involvement of students was very reduced, very few students took part in the proposed game and gave their opinion. all in all, during the follow-up, our students counted between 20 and 30 paper slips. on the day arranged for the installation collection with the opinions of the teenagers our students are warned by the director that there are only 5 paper slips. this circumstance makes our students ask themselves what has happened? simply that one part of the teaching staff felt attacked by the opinions of the secondary students and practised a kind of censorship, eliminating the paper slips that were not favourable to them. although they didn’t see these paper slips, according to the director, these were insults and disqualifications towards the teachers. amongst the conclusions that this group draw from this action after the analysis of what had happened we can mention the following: • why don’t the students take part in a proposal of this kind? the reasons can be various, such as: they are not used to being asked their opinion; they don’t think it will be possible (as it happened in part); they don’t see reasons to justify their participation; even being anonymous, they are afraid of reprisals. • it would have been understood if the teaching staff of the school had not given their approval to the artistic intervention but, once approved, it would have been responsible to have admitted the criticism with an open spirit. all teachers know that working with teenagers she/he is exposed to the fiercest criticism; these exist even if there is no feasible possibility of making them explicit. the coherent attitude is to get to know them and reflect on them: have they a real basis? do all my students have this opinion or only a few? what can i do to improve? etc. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 65 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 • although the project was not completed, it did reflect the existence of students that are not happy with their teachers. the image of the teacher is damaged and ignoring this is not a solution to the problem that this brief intervention has shown. • the development of similar projects in which the students’ voices can be made visible can allow a lot of information to an open teacher, ready to reflect about her/his educational practice. it can also promote a critical spirit in students, a critical spirit that goes beyond the discrediting and that favours the interpersonal relationships that take place in an educational centre. in view of this experience, we can affirm that, despite the difficulties mentioned above regarding the implementation of the proposal, this research approach provides very interesting results. this group also wrote their reflections. each group worked in an autonomous way. for this reason a global presentation in the classroom was necessary so that we could all participate in the results of the research. students were in charge of the presentations of the different paradigms and the different experiences were narrated in first person. the time spent on the different tasks went by in the classroom of the subject excepting those detailed moments mentioned before (artistic installation, questionnaires, etc.). the last week of class before the students went to the training at schools (practicum), each group presented their research to the rest of the class and we uploaded them to the virtual classroom of the subject. 6. final conclusions in general, we think the experience was very positive as can be seen by the reasons we set out below: • the methodological approach to the subject has been liberating inside a system that not always awards freedom of expression and development to students as we franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 66 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 can see from the following comments: “sincerely, at first i found the subject a bit chaotic, but the fact of learning by doing has been a very satisfactory experience, and in my case, much more meaningful. to overcome the fear of liberty of proposing our own tasks, and undertaking them correctly and in a respectable way, has also improved our self-esteem” (d.v., 2012) or “...it has awakened a great interest in me, both by the tasks proposed as well as the treatment given by teachers and classmates, and i would have liked this experience to have been longer...” (d.v., 2012). • the introduction of changes in education as the ones discussed in the theoretical background, are beneficial both for students and for teachers as observed in the case we present. reflection, not only of students but of teachers as well, lets us confirm the importance of providing spaces of reflection and discussion. • the approach of the subject (performance-related activities, reflection spaces, opportunity to express oneself, undertaking of research) allows us to confirm that our students, as future teachers, would not like to reproduce the teachers’ image they keep in their memory. thus, the fact of reflecting on the secondary school teachers’ figure helps them make up the teacher they would like to be. • the practical approach to the subject, with the direct participation of students in the construction of ideas and the undertaking of research, promoted the motivation of the students of the master’s degree who expected lectures. • the use of artistic strategies, such as the intervention in the hall of the faculty or the situation created at the beginning of the subject, predisposes students to work in a relaxed, cooperative and united manner, as would be done by an artistic group who share a common goal. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 67 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 • if students’ voices are heard, students take a leading role in their own learning. this idea can be implemented in any subject because it breaks with school traditions and routine. finally we would like to thank all our students of the subject teaching innovation and introduction to educational research in arts during the academic year 2012/2013 that made this experience possible. and our special thanks to our colleague m.j.agra for sharing with us her ideas. 5. references acaso lópez-bosch, m. (2012). pedagogías invisibles: el espacio del aula como discurso. madrid, los libros de la catarata. acaso lópez-bosch, m. (2013). reduvolution: hacer la revolución en la educación. barcelona, paidós. agra pardiñas, m.j. (2005). el vuelo de la mariposa: la investigación artístico-narrativa como herramienta de formación, in r. marín viadel (ed.), investigación en educación artística: temas, métodos y técnicas de indagación sobre el aprendizaje y la enseñanza de las artes y culturas visuales. granada, editorial universidad de granada. dewey, j. (1910). how we think. resource document. kansas city public library. https://archive.org/details/howwethink000838mbp dewey, j. (1949). el arte como experiencia. méxico, fondo de cultura económica. dewey, j. (1971). democracia y educación. buenos aires, losada. eisner, e. (2004). el arte y la creación de la mente. barcelona, paidós arte y educación. irwin, r. & springgay, s. (2008). a/r/tography as practice-based research, in springgay, s. et al (eds), being with a/r/tography. rotterdam, sense publishers. franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 68 https://archive.org/details/howwethink000838mbp multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2228 social and technological sciences eissn: 2341-2593 oriol de alarcón, n. (2004). metodología cuantitativa y cualitativa en la investigación sobre la formación inicial del profesorado de educación musical para primaria. aplicación a la formación instrumental. revista electrónica complutense de investigación en educación musical, i (3), 1-63. villar angulo, l. & vicente rodríguez, p. (1994). enseñanza reflexiva para centros educativos. barcelona, ppu franco and gillanders (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 53-69 | 69 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 156 corporate social networks applied in the classroom de juan-jordán, hugo1, guijarro-garcía, maría, guardiola-contreras, javier marketing department1, esic business and marketing school *corresponding author: avda. de blasco ibáñez, 55, valencia (spain), hugo.dejuan@esic.edu, +34 627468823 received: 2016-01-21; accepted: 2016-05-22 abstract the impact of online social networks has been extensive because of the new way they enable not only in terms of the relation, communication and collaboration among people, but also between people and businesses. so much so, their use is already habitual within organizations, known as corporate social networks, in order to achieve the same benefits. the present study aims to analyze the advantages these corporate social networks have in the classroom seen as a micro-organization where a group of students interact, work and collaborate during a master´s or postgraduate course. to support this research, during 2015 a corporate social network (yammer) has been introduced to 5 groups of students of various master´s in the prestigious business school esic. the feedback obtained from those students and some examples of classroom dynamics prove the usefulness and great value of a corporate social network in postgraduate classes, although some common difficulties and considerations raised by the students themselves have to be taken into account in order to manage its optimal adoption in class. this study also tries to propose some guidelines and best practices obtained as a result of the experience of use and the adoption of social networks in class in order to improve the learning process and innovate in the methodology applied to education. keywords online social networks, corporate social network, innovation, education, master´s degree, education http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 157 1 introduction with the rapid evolution caused by ict (information technology) nowadays the teacher has ceased to be a mere speaker or imparter of classes to become a facilitator of learning for their students adell (1997) cebrián and rios (2000), aguaded and cabero (2002), cabero and cervera (2002), cabero (2007), while students have become participants on the classroom stage. this participation has a very relevant component of social interaction inside and outside the classroom. authors like cook & steinert (2013) note that online courses are much more likely to succeed, among other aspects, when collaboration, cooperative work and social interaction of students are encouraged in a seminar. it has also been shown that for the success of moocs (massive open online courses) students require socialization (coughlan, 2014), and authors like (gašević, et al., 2015) highlight the importance and advantages of socialization in education. given the importance of social interaction in education, the ict that support this process nowadays have a greater social orientation (skrypnyk, et al., 2015). on the other hand, web 2.0 and social software technologies enable two-way communication, sharing, extracting and organizing knowledge, as well as building social relationships, all this having great potential for the educational world (anderson, 2008). therefore, it seems coherent to try to boost students` need for social interaction by implementing social software technologies in the classroom. the present study analyzes the experience of using an online social network in the classroom to research on the advantages of web 2.0 and the social technologies linked to class dynamics in several postgraduate courses. this will allow us to identify not only the advantages of a social network like yammer in the classroom, from the perspective of the student and the teacher, but also the difficulties of its adoption and use. the experiment also reveals a set of uses and good practices of social networks in the classroom, and eases the way for whoever wishes to implement this educational innovation in their educational environment. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 158 2 online social networks and the classroom dale dougherty, vice-president of o'reilly media coined the term web 2.0 (o'reilly, 2007) at the web 2.0 conference of o'reilly media and medialive international in 2004, and as the result of this technological evolution and its uses a new era began in the world of the internet. according to erragcha and romdhane (2014) web 2.0 rests on 5 pillars: (a) participation, where all users can express their opinion and they actually express it, (b) opening, based on collaboration and information exchange, (c) conversation and dialogue among different users, (d) community or groups of people with common interests, (e) interconnection and links to other sites, resources or people. examples of web 2.0 are communities, services and web applications, video hosting services, wikis, blogs, mashups, folksonomies and, undoubtedly, the most important applications of web 2.0: online social networks (pallis et al., 2011). online forums, the predecessors of online social networks, have been essential in online education and still are for moocs or "massive open online courses" (joks et al., 2015). technology for learning can be divided into three broad categories: i) information technologies that assist delivery and access to information ii) interactive and communication technologies that mediate the relationship of exchange with the user and iii) social software that sustain group activities such as decision making, planning or training activities of higher order (anderson, 2008; hüulsmann, 2004). social networks, by their nature are found in ii) and iii) of these categories. as the concept of web 2.0 has been maturing, the world of education has been increasingly adopting blogs, wikis, syndication through rss and social bookmarking as teaching tools. while this approach gives students the opportunity to experience the most collaborative and social learning, the weakly connected toolset has sometimes led institutions to frustration as learner data are distributed in a wide range of disconnected tools (siemens, et al., 2015). http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 159 when online social networks are designed and used to give support and catalyze collaboration and sharing of content among members directly linked to the organization or related to it (like the ecosystem existing in a classroom) are called “enterprise social networks” or esn in the web 2.0 culture (back, gronau and tochtermann, 2009). according to riemer and tavakoli, (2013) the term enterprise social networking (esn) refers to the application of social media platforms that facilitate short messages and establish social connections within an organization (or classroom). esn enables organizations to create spaces where people can connect, communicate, collaborate and exchange information. according to treem and leonardi (2012) what makes esn unique and potentially transformative within an organization is that it allows users to exchange messages with other users, post messages to a large or unknown audience, list and select other members in order to form groups or teams, and publish documents, images, videos etc. that can be searched, sorted and found by others. but, most importantly, it can be recorded, stored and made available to other co-workers or classmates, as well as seen at any time. li (2010) summarizes the concept of esn as any kind of technology that allows certain users to create and modify content such as corporate wikis, business blogs, corporate instant messaging, etc. leonardi, huysman and steinfeld (2013) define esn as web-based platforms that allow employees or students of the organization to (1) communicate messages to certain peers or send messages to everyone in the organization; (2) explicitly indicate or implicitly disclose certain co-workers as communication partners; (3) publish, edit, and organize texts and files relative to themselves or other people; and (4) viewing messages, connections, texts and files that are communicated, published, edited and sorted by anyone in the organization at any time of choice. according to leonardi et al., (2013) esn allows visibility and persistence in communicative actions, and expands the range of people, networks and texts that employees can learn from in the entire organization, which thereby increases their social learning opportunities. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 160 3 testing the introduction of corporate social networks in the classroom in response to the previous literature and research, it makes much sense to try to introduce a corporate social network or esn in the dynamics of a postgraduate class, meaning that a class is an organization in itself that socially relates and interacts not only during in-class sessions, but also outside them, anytime and anywhere. the experiment was tested in 5 different groups that were studying a master´s degree in the valencia campus (spain) of esic business school marketing & business school in 2015. a corporate social network was specifically set up for each of the five groups during different seminars dealing with topics related to digital economy during 1 month approximately. the corporate social network was implemented using the yammer platform (a microsoft office365 pay-per-use software service) that was already available in the cloud infrastructure at esic business & marketing school. the network was named after the edition of the master´s degree that the group was studying and was minimally set up with the esic corporate identity, as well as the basic network information, descriptions and tags. in addition, a specific group for the seminar to be taught was created in the network and another group for the use and support of the yammer network. students were invited via email to be part of the network accessing it from the invitation they had received by email, while those who had failed to do so were helped to register in the first face-toface session. besides helping them to access the network, about 30 minutes were allotted to help them to understand and manage this social network. students had full powers to set up and complete their profile on the network, create new groups, write posts, comment on other participants’ posts or mark them with "likes", post announcements, resources, links, files, videos, documents, etc. in short, they were not network administrators, but had all the authorization of a user with full power. during the seminar for which the yammer network had been created and where 30 minutes were allotted for its explanation to the students, the teacher also encouraged students to use the network as any other available tool in the dynamics of the in-class seminar. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 161 two months after the end of the seminar where the corporate social network was presented to each group, students were asked to fill out a survey to get their feedback on the platform and its use in the classroom dynamics, as well as on its transfer to the workplace. they were asked to rate the following eight issues on the likert scale of 5: 1. do you know what a corporate social network (esn) is? 2. do you know the value, features and potentiality of yammer? 3. do you know the value, features and potentiality of yammer to boost learning and training? 4. what is the contribution of yammer in a seminar, course or master´s degree? 5. how do you assess your experience with yammer in the classroom? 6. what influence do you think the following variables might have on the failure of yammer in classroom? 7. what do you think would help yammer become a more useful tool in the classroom? 8. do you think you will use yammer in business or for other professional purposes in the future? the students` responses were collected with the online survey software surveymonkey.com the teacher who had used yammer in this experiment replied on the uses, advantages and difficulties of yammer in a postgraduate class via personal interview. on the other hand, the authors of this study accessed the yammer networks that had been created for the 5 groups (only in the open groups) and thoroughly analysed students` and teachers` uses and consumption of these networks. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 162 4 obtained results 4.1 usefulness and value of the corporate social network in the classroom there were repeated dynamics of use of the corporate social network during the different class sessions with a great pedagogical interest for students, as exemplified with real cases that can be summarized as follows: 1. sharing knowledge between students and teacher via posts and comments to the posts. 2. sharing resources, events and recommendations on educational improvements between students and teacher 3. notifying students 4. sharing and spreading considerations and advice to students 5. group exercise correction and feedback to all students 6. facilitating student interaction during students` presentations 7. facilitating classroom presentations 8. sending congratulations to students and teacher 9. giving social support to conversations among students 10. sharing links and resources for tasks with the class 11. tagging and classifying knowledge 12. inventorying resources and links in groups 13. setting up open in-class groups and private groups for student projects 14. recording and sharing videos with the class http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 163 4.2 student feedback the survey was responded by 37 students with an average age of 33 years with ages ranging from 24 to 49 years. of all respondents only one used the yammer tool regularly, 6 of them were already familiar with this social network but unaware of their use in education, as for the rest, this was a first experience of use, and therefore, the analysis of the survey will give us comprehensive information on their perception of use of the network in an educational context. out of the 37 students, 20 have a good opinion (table x) about the tool but claim that it could be better exploited; as any new technology, its use has a learning curve, after which its achievable performance improves. there is a group of 7 people that considers the experience as average and only one person as bad. table x. evaluation of the experience of using yammer in the classroom none, as i have not tried or used it in any way (1) bad, because i have not seen much meaning or value to it in the classroom (2) average, because it makes sense, but i have not made the most of it (3) good, i think it has added value, although it could have been used better (4) very good, because it has added value to the class and i will make much more use of it in the future (5) 0 1 13 20 3 as about its different contributions to the lesson (table y) the evidence says that students see much potential to the tool in terms of sharing all kinds of resources: files, documents, links, news and photos. in addition, students welcome the fact that the tool can be used to create workgroups and also view favourably the way in which the tool can contribute to collaborative learning although it has not been used much for this purpose. regarding the more social part and the relationship with other people, they acknowledge its potential usefulness, but have not implemented it during these sessions. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 164 table y. contribution of yammer to a seminar or course it contributes nothing new, because i have not used it (1) yes, it might add value, but i have barely used it only for this purpose (2) yes, it adds value, although we have use it little for this purpose (3) yes, it is valuable and we have used it for this purpose, although we could have made more use of it (4) yes, it undoubtedly adds a lot of value for this purpose, and we have used it (5) sharing files and group work documents 0 6 9 17 6 creating groups for business cases, projects, exercises, class dynamics, etc. 0 5 11 17 5 sharing links and valuable resources 1 6 8 22 2 sharing news and events 1 10 12 14 2 sharing pictures and interesting group moments 1 11 14 10 3 debates and group thinking 0 11 20 7 1 congratulating classmates, giving "likes" to others, creating a good group dynamics 2 13 13 7 4 better knowing peers and teachers in a ubiquitous and timeless way 3 15 14 6 1 on the possible variables of failure of a tool like yammer in the classroom (table z) students respond that they do not think they might feel uncomfortable with the use of social technology and there is no doubt about the value it adds to the lesson. the effort to maintain a social network and the workload can definitely be a problem and a source of the tool`s failure. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 165 table z. in what way do you think the failure of yammer in the classroom can affect the following variables? i strongly disagree. (1) i do not think so, but it might be the case sometimes. (2) undecided. it may happen sometimes. (3) yes, it is likely. (4) yes, it is absolutely true. (5) the student is uncomfortable or does not see the potential of social technologies 9 10 14 4 1 yammer cannot add value in the classroom, because it is meant for other purposes 12 14 7 5 0 the effort to maintain the yammer network active does not compensate for the obtained value 6 8 12 12 0 we have much workload to deal with secondary issues instead of key processes in the classroom 1 8 10 16 3 neither students nor teachers know how to use yammer to create value in the classroom 14 9 9 5 0 finally, feedback meant to improve the use of yammer in the classroom is requested (table zz). students ask for more extensive and specific training on the tool as, for example, a specialized seminar on the matter. they also suggest that a person should be designated to motivate the entire group. they strongly disagree with turning the training on this tool into something assessable and, on the whole, with all teachers generalizing the use of this technology in the classroom. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 166 table zz. what do you think could make yammer a more useful tool in the classroom? strongly disagree (1) slightly agree (2) agree (3) strongly agree (4) absolutely (5) a specific seminar to learn how to make the most of it in class 5 6 14 8 5 someone with the role of a community manager to motivate the group daily 2 6 12 15 3 transform its use into an assessable case or academic subject 13 7 8 8 2 teachers should apply it in their every-day classes 1 5 13 10 9 more 2.0 vision and training in the classroom and in life in general. 1 6 12 10 9 4.3 teacher feedback it was easy to explain corporate social network to students and the concept was quickly understood, since they were already accustomed and comfortable using other social networks in other areas of their life (like facebook, for example). the advantages for the teacher during the seminars were several, as described in the following list: 1. the possibility to give feedback to different groups on cases, presentations or tasks, a feedback that is open and reusable in other groups. 2. progressively documenting references, useful links and resources as they appear during seminars or even outside of class time, once they have been consulted with the teacher or approved offline. 3. notifying students about the following lessons or any changes, replacing the email. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 167 4. avoiding copies and file transfers by usb or other exchange mechanisms, for example, presentations of cases discussed by the groups. 5. asking questions and encouraging discussion not only during face-to-face classes, but also during the whole course. 6. facilitating dynamism and interaction in class, especially in the cases where students can sometimes be more reluctant to interrupt or participate in class. the teacher positively assessed the contribution of yammer in the classroom and would recommend it to other teachers from the beginning and throughout the whole master´s degree, as well as during all its seminars. 5 conclusions this research establishes that the use of corporate social networks (in this case yammer) as a support in the communication and interaction between students and teachers in a postgraduate program is considered a good practice because it facilitates communication and social dialogue among all the members related to the class. the fact is that students’ activity in online social networks has been shown to have a positive effect on various educational achievements like creative potential (dawson, et al., 2011), the sense of belonging (dawson, et al., 2011) and academic achievement (gašević, et al., 2013), while the latter is transferable to the context of a social network in the very classroom. it is also a good support for knowledge management and a useful tool for many class dynamics (as listed among the real cases under study). the majority of students believe their experience with yammer in the classroom has been good and believe it has added value, although it could have been put to better use. the uses they most welcome, among the many they carried out and the ones available, were: "sharing files and work documents in the group," "creating groups for cases, projects, exercises, class dynamics, etc.", "sharing links and valuable resources " and " sharing news and events." http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 168 however, according to students, this corporate social network competes in some way with the rest of the communication, interaction and knowledge management systems existing among students (ex .: email lists, whatsapp, dropbox, etc. ), and it takes up part of students` scarce time and attention to achieve the network´s highest level of performance. moreover, students claim that they have no time to deal with the corporate social network in class and this effort does not compensate them for the value they obtain from it. students` experience and feedback shows that it would be recommendable to run the platform from the beginning of the postgraduate program, together with some training and examples of good practices, so that students should know how to make the most of it with the least effort possible. it needs to be done at the beginning of the master’s degree to make sure that the platform meets students’ necessities and avoid any improvised solutions that would be inadequate, less complete and with less potential for a group of postgraduate students. students also welcome the idea of a person who should boost class content and conversation in the corporate social network; it is undoubtedly so because the more useful the network, the more value it contains (in the form of contributions to the content and to dialogues) and only by ensuring sufficient activity will it attract the interest and activity of other users. this information is complemented by the respondents’ suggestion that more vision and training in the web 2.0 culture are needed in the classroom and in life in general. it is also recommended that all teachers who teach postgraduate seminars as well as tutors of master’s dissertations should know how to leverage the platform inside and outside the classroom in an appropriate, coherent and complementary way to the rest of the solutions in the virtual campus they might use. from the same viewpoint, teachers have found it very useful as a support tool for their seminars in all cases. however, its performance would be maximized if students joined the seminar with good practices of use of the platform acquired beforehand and if all previous and subsequent teachers consistently took advantage of the platform and the dynamics that allow to use a corporate social network in the classroom. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 169 on the less positive side most students believe that "they doubt" that transferring the use of a corporate social network as the one used in class to their professional life would happen in the future. this is an opinion based on the insufficient attachment generated by the platform during the short period of study or by the previously mentioned deficiency in the web 2.0 culture ("the student is not comfortable or does not see the potential of social technologies"). from the above results and considerations the authors conclude that corporate social networks like yammer applied in the classroom are a very good platform for knowledge management, communication and interaction among all the actors involved in postgraduate education and strongly recommend it. however, proper student and teacher guidance and training is needed to motivate and maximize its use throughout the entire master’s degree and possibly after its completion. among the limitations of this experience the fact that it has been limited to a single master’s seminar and to only 5 groups could be emphasized. it is also a limitation the mediation of only one teacher during only one edition of each of the five master’s degrees in 2015. in future research the authors intend to carry out the study of this experience from the beginning of several master’s programs and continue it throughout their duration, even further on, during the stage of alumni in order to strengthen social links between former students, teachers and the very platform of the organization. it would also be advisable to extend the study of this experience to an undergraduate or college environment and compare the results with the ones obtained in this study. http://dx.doi.org/10.4995/muse.2016.6342 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 170 6 references adell, j. (1997). tendencias en educación en la sociedad de las tecnologías de la información. edutec. no. 7, sin: 1135-9250. aghaei, s., nematbakhsh, m. a. and khosravi farsani, h. (2012). evolution of the world wide web: from web 1.0 to web 4.0. international journal of web & semantic technology (ijwest), pp. vol.3, no.1. aguaded g., j. i. y cabero, a., j. (2002). educar en red. internet como recurso para la educación. madrid: ediciones aljibe. anderson, t. (2008). social software technologies in distance education: maximizing learning freedoms. en: international handbook of distance education. s.l.:merald group publishing, p. 167–184. back, a., gronau, n., tochtermann, k.: “web 2.0” in der unternehmenspraxis. oldenbourg, münchen (2009) berners-lee, t., cailliau, r., groff, j. and pollermann, b. (2010). world-wide web: the information universe. internet research, pp. vol. 20 iss: 4, pp.461 471. http://dx.doi.org/10.1108/10662241011059471 boyd, d. m. & ellison, n. b. (2007). social network sites: definition, history, and scholarship. journal of computer-mediated communication 13(1). http://dx.doi.org/10.1111/j.10836101.2007.00393.x cabero a., j. y cervera, m. g. (2002). materiales formativos multimedia en la red. guía práctica para su diseño. madrid: doblas. cabero a., j. (coord.) (2007). tecnología educativa. madrid: mcgraw-hill interamericana de españa, s.a.u. cebrián de la s., m. y ríos, j. (2000). nuevas tecnologías aplicadas a las didácticas especiales. madrid: ediciones pirámide. cook, d. & steinert, y. (2013). online learning for faculty development: a review of the literature. medical teacher, 35(11), p. 930 – 937. http://dx.doi.org/10.3109/0142159x.2013.827328 coughlan, s. (2014). the irresistible urge for students to talk. [en línea] available at: http://www.bbc.com/news/business-26925463[último acceso: 08 5 2016]. http://dx.doi.org/10.4995/muse.2016.6342 http://dx.doi.org/10.1108/10662241011059471 http://dx.doi.org/10.1111/j.1083-6101.2007.00393.x http://dx.doi.org/10.1111/j.1083-6101.2007.00393.x http://dx.doi.org/10.3109/0142159x.2013.827328 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 171 dans, e. & allen, d. (2001). b2b e-marketplaces: percepción de la propuesta de valor en un mercado incipiente. economía industrial, iv(340), pp. 101-108. dawson, s., tan, j. p.-l. and mcwilliam, e. (2011). measuring creative potential: using social network analysis to monitor a learners’ creative capacity. australasian journal of educational technology, 27(6), p. 924–942. http://dx.doi.org/10.14742/ajet.921 erragcha, n. and romdhane , r. (2014). social networks as marketing tools. journal of internet banking and commerce, 19(1). gašević, d., kovanović, v., joks imo vić, s. and siemens, g. (2015). where is research on massive open online courses headed? a data analysis of the mooc research initiative, alberta, canada: athabasca university. gaševic, d., zouaq, a. and janzen, r. (2013). “choose your classmates, your gpa is at stake!”: the association of cross-class social ties and academic performance.. american behavioral scientist,, 12 march, 57(10), p. 1460–1479. http://dx.doi.org/10.1177/0002764213479362 hüulsmann, t. (2004). the two-pronged attack on learner support: costs and the centrifugal forces of convergence. en: c. w. &. o. z. j.e. brindley, ed. learner support in open, distance and online learning environments. oldenburg: universitat oldenburg, p. 498–504. joks, s.; kovanović, v.; skrypnyk, o.; gašević, d. ; dawson, s.; siemens, g. (2015). the history and state of online learning, alberta, canada: athabasca university. leiner, b.; kahn, r.; postel, j.; cerf, v.; kleinrock, l.; roberts, l.; clark, d.; lynch, d.; wolff, s. (2009). a brief history of the internet. acm sigcomm computer communication review, pp. volume 39, number 5, 22-31. leonardi, p., huysman m., steinfield, ch. (2013). enterprise social media: definition, history, and prospects for the study of social technologies in organizations. journal of computer-mediated communication, 19, 1–19 http://dx.doi.org/10.1111/jcc4.12029 li, h.x. (2010), sns tide – embracing the new transformation of social network, posts and telecom press, beijing. o'reilly, t., 2007. “what is web 2.0. design patterns and business models for the next generation of software.” , s.l.: communications & strategies. http://dx.doi.org/10.4995/muse.2016.6342 http://dx.doi.org/10.14742/ajet.921 http://dx.doi.org/10.1177/0002764213479362 http://dx.doi.org/10.1111/jcc4.12029 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6342 social and technological sciences eissn: 2341-2593 de juan et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 156-172 | 172 pallis, g., zeinalipour-yazti, d. and dikaiakos, m. d. (2011). online social networks: status and trends. new directions in web data management 1, pp. sci 331, pp. 213–234. http://dx.doi.org/10.1007/978-3-642-17551-0_8 riemer., k, tavakoli., a (2013) "the role of groups as local context in large enterprise social networks: a case study of yammer at deloitte australia", business information systems working paper. ronald, b., kilduff, m. and tasselli, s. (2013). social network analysis: foundations and frontiers on advantage. annual review of psychology, 64(1), p. 527–547. http://dx.doi.org/10.1146/annurev-psych-113011-143828 siemens, g., gašević, d. and dawson, s. (2015). future technology infrastructures for learning, alberta, canada: athabasca university. skrypnyk, o.; joks, s.; kovanović, v.; dawson, s.; gašević, d.; siemens, g. (2015). the history and state of blended learning, alberta, canada: athabasca university. treem, j.w., & leonardi, p. (2012). socialmedia use in organizations: exploring the affordances of visibility, editability, persistence, and association. communication yearbook, 36, 143–189. walsh, j. p., & ungson, g. r. (1991). organizational memory. academy ofmanagement review, 16(1), 57–91. http://dx.doi.org/10.5465/amr.1991.4278992 http://dx.doi.org/10.4995/muse.2016.6342 http://dx.doi.org/10.1007/978-3-642-17551-0_8 http://dx.doi.org/10.1146/annurev-psych-113011-143828 http://dx.doi.org/10.5465/amr.1991.4278992 microsoft word 2_bares_et_al multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 16 using interactive response systems in economics: utility and factors influencing students’ attitudes bares lópez, lydia1*, fernández pérez, ana mª1, ferrándiz león, esther1, flores varo, mª esther1, león rodríguez, mª dolores1 department of economics, university of cadiz, avda. enrique villegas velez 2, 11002 cadiz, spain. * corresponding author: email: lydia.bares@uca.es; phone: + 34 956015472 received: 2016-03-09; accepted: 2016-11-12 abstract the european higher education area (ehea) involves changing traditional methods to promote innovative teaching experiences. this paper has two main aims: a) to show evidence of the use of interactive response systems (irs) to identify gaps in the understanding of the course contents and b) to investigate factors influencing students’ attitudes towards the use of irs. the experience was developed through a collective tutoring session in the subject of economics using irs. economics is a first-year subject in the degree of business administration and management offered by the university of cadiz, which includes contents of microeconomics and macroeconomics and uses economic models to explain the function of the economy and the behaviour of economic agents. results show that irs technique allows detecting gaps in learning and comprehension. from our econometric estimations, we also identify two strongly significant variables affecting students’ attitudes towards irs: gender and received explanations regarding the use of irs. variables such as first enrolment in the subject and the number of hours devoted to studying have a positive and significant effect on the attitude to irs, but at a lower level of significance (from 5% to 10%). keywords teaching innovation, new technologies, interactive response systems (irs) http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 17 1. introduction the transformations within current society require the training of professionals who can adapt to changes arising in a constantly evolving world. leadership and communication, problem solving, team working, negotiation skills are needed to become a part of teamwork. academia must accept this challenge and include the possibility in the teachinglearning process that students learn this set of transversal skills in line with the needs of society. the development of the european higher education area (ehea) and the implementation of the european credit transfer system (ects) involve a gradual change of the traditional teaching approach to promote innovative experiences in the learning-teaching process. new teaching methodologies involve a different perspective on learning, in which the merely expository teaching class must be changed towards new approaches, focusing on the interaction between lecturers and students with active participation on both sides. new technologies are called to play a role in this process, since they provide a wide range of possibilities and opportunities to improve teaching. interactive response systems (irs) are a prominent example of this. the irs is a variant of a traditional technique that can be performed by the well-known method of cards, where teachers raise a number of questions or items to students, who previously have been given a set of cards on hand to answer with different colours or distinctions (a, b, c, or another set of options). an irs is an electronic device connected to software that processes the student’s answer to a multiple-choice questionnaire immediately, graphically through diagrams. the responses are recorded individually and anonymously, allowing a feedback process for decision-making in the classroom. in this paper we present a case study based on the use of an irs to solve doubts and detect lack of understanding on the subject of economics in the bachelor´s degree in business administration and management at the university of cadiz. the aim of this paper is twofold: a) to show evidence about the usefulness of irs to detect doubts and lack of http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 18 understanding of the course contents and b) to provide evidence on the factors affecting a favourable attitude of the students towards the use of an irs for the detection of doubts and gaps. the paper is organised as follows. in section 2, we present the literature review. section 3 describes the methodology. section 4 shows an econometric model to identify the factors that explain a positive attitude of students towards the use of the irs to detect doubts and lacks in the learning of the subject. section 5 presents the conclusion and discussion of the paper. 2. literature review the implementation of new teaching methods that enhance students’ performance has been a major concern for academics in the last decades (butler 1992), as reflected in the use of new technologies for teaching (gül et al. 2010). although it has been applied to different academic disciplines such as psychology (morling et al. 2008; dallaire 2011; ludvigsen et al. 2015), health (schackow et al. 2004; hughes et al. 2011), sciences (kay and knaack 2009; moss and crowley, 2011) or accounting (carnaghan and webb 2007; camacho-miñano and del campo 2014), it cannot yet be considered as widely used. in this sense, emenike and holme (2012) argue, using a methodological analysis of binary data between users and non-users of the system for a sample carried out at faculties of chemistry, that this technology is still at an early user stage, in transition towards a more widespread use status taking the life cycle of technology adoption (technology adoption life cycle talc). in their work, fallon and forrest (2011) compare alternately the traditional use of cards and an electronic irs in a sample of 70 students in prior review sessions to assessment tests of the subject. in their research they obtained evidence of the degree of preference of students for the irs, considering the main advantage to be the possibility of preserving http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 19 anonymity of the responses. however, in line with other research, they did not find clear evidence to show significant advantage over results in the use of an irs (stowell and nelson 2007; lasry 2008). there are a broad number of contributions about the use of the irs in teaching. king and robinson (2009) review previous studies citing, among other, research focused on the use of the system and the evaluation of students’ perception about these kinds of experiences (van dijk et al. 2001; fies and marshall 2006; caldwell 2007; macgeorge et al. 2008). in their paper, king and robinson (2009) provide the results of an experiment carried out on a sample of 145 students of the second course, which show that the majority of students appreciate the advantages and usefulness of using an irs in the classroom. their results, in line with other studies (liu et al. 2003; siau et al. 2006), suggest the effectiveness of the system in terms of its ability to promote and encourage student participation in the classroom, showing a rate of 84.61% participation by an irs compared to 28.67% when the student must respond with an outstretched hand and 12.59% when a verbal response is requested on a voluntary basis. even students who did not regard the irs as particularly useful, reported greater participation when using it. however, in line with previous literature king and robinson (2009) found no correlation between the use of the system and the improvement of the results of students. additionally, anthis (2011) considers the need of separating the effects of using interactive systems itself versus the possibility of contemplating the same type of questions to the students, answered by an outstretched hand. in this sense, the system's effectiveness would show better academic results in the students who used the irs against those who participated in a traditional response system in the classroom. their results have shown no evidence to confirm this approach. meanwhile palmer et al. (2005) and gebru et al. (2012) confirm that the use of systems that allow feedback on the teaching-learning process (learning-teaching feedback) can increase student knowledge retention during the course, reducing their loss learning over time. according to the study of han and finkelstein (2013) students consider irs as a http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 20 useful tool for their engagement and learning. fallon and forrest (2011) discuss the advantages of the irs applied to review sessions prior to assessment tests of students. dallaire (2011), in line with previous studies that have investigated the direct results of the system of learning (kennedy and cutts 2005; stowell and nelson 2007; morling et al. 2008, poirier and feldman 2007), research on the effectiveness of the irs in a sample of 151 students. although the goal of their work is not to examine the improvement achieved by this system of learning, the results suggest that the degree of effectiveness is conditioned by the mode of use and the characteristics and student final marks. with respect to the drawbacks of implementing the irs in the teaching methodology, the main disadvantages are the training needed for setting up the software and for its use by teachers; the time spent on the acquisition of these skills, and the acquisition cost of the equipment. furthermore, it is a time-consuming activity because instructors should examine if this kind of experiences are properly awarded to students (white et al. 2011). in spite of the disadvantages, the results obtained about previous experiences in the literature and its widespread use, leads us to value their potential advantages over their disadvantages. we use the irs as a complementary teaching tool in the classroom during a review session before the exam. 3. methodology the implementation of an irs in teaching has been a commitment of the faculty of economics and business at the university of cadiz to the introduction of new communication technologies in the teaching-learning process. this tool has been purchased for use in the subjects taught in the faculty and allows students to use the personal response devices without having to pay any fee for their use. the next paragraphs show the methodology followed to carry out this experiment. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 21 3.1 target students this innovative teaching experience is applied to economics. economics is a first-year subject in the bachelor’s degree in business administration and management offered by the university of cadiz (spain). economics is taught during the first semester within the framework of economic theory in the basic training module. the aim of the subject is to provide a basic knowledge to enable students to understand microeconomic and macroeconomic issues from both a theoretical and practical point of view. in general, economic models are related to graphical and mathematical analysis which students often find hard to understand. in this sense, students' academic results are not satisfactory in terms of marks obtained in the subject, so the lecturers consider the use of alternative teaching methods over traditional attendance and assessment that assist improvement in student performance. 3.2 development of the experience the implementation of this innovative teaching experience followed four steps: 1. technical procedure development. the system needs to install the software able to recognize the signal from the irs in the control unit. 2. to design the items or questions for each topic in the course. these questions allow teachers to evaluate the level of academic achievement and learning by students. at the same time, these questions enable teachers to identify understanding gaps in the contents of the subject. to display the selected items we use the corresponding software presentations (the technology incorporated in the system supports powerpoint or similar). 3. review session. in this session, teachers pose a set of questions and students respond to them with their irs. students must select the correct answer by pressing an http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 22 alphanumeric code in their device. previously, students must have been identified in the system (newer versions of the system allow to use devices like pdas or mobile phones instead of traditional interactive remote control). for this, each remote control is given to a student, assigned a corresponding identifier. at the beginning of each item the teacher gives the students time to think. after that, the system is activated to capture the responses from each user. 4. the answers are instantaneously analysed by the system. the results are shown together for the whole group to preserve student anonymity. for each question the results are graphically available as a bar diagram, histograms, etc. teachers have the option of knowing the identity of each user, if they want it for the evaluation of the experience for the possible marks of the results although this is not part of the objective of this experience. by taking into account the results obtained by students, teachers can improve the teachinglearning process, providing additional explanations to the issues raised to clarify concepts, and give appropriate explanations to resolve detected gaps in understanding for the achievement of the proposed objective of this experience. following the division of the matter for the subject of economics the set of items is structured into two thematic sections corresponding to microeconomics and macroeconomics, of ten questions each, reflecting faithfully the structure of the teaching program of the course. this experience is an experimental activity that has been developed in four groups of students. the number of students who were voluntarily involved in this activity was 84. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 23 4. results table 1 shows the results of the test using the irs. the test consisted of asking students a set of 20 questions related to theory of macroeconomics and microeconomics, which is taught in the subject “economics”. the total number of items have been classified into two categories: type a) “study” (12 items) and type b) “comprehension” (8 items). the data in this table shows the percentage of right answers in each group of classes for each item. additionally, this table shows the average number of right answers in all items and its standard deviation as indicator of the degree of spread of the results. table 1. results of the experience by group of class. type of question item a b c d average (*) standard deviation study (type a) 1 97% 25% 82% 79% 71% 0.2277 2 40% 63% 91% 43% 59% 0.1763 3 90% 81% 82% 57% 78% 0.1020 4 17% 19% 36% 43% 29% 0.1095 5 43% 56% 27% 50% 44% 0.0891 6 63% 13% 64% 0% 35% 0.2861 7 45% 50% 73% 21% 47% 0.1396 8 73% 31% 91% 14% 52% 0.2952 9 100% 94% 100% 93% 97% 0.0334 10 100% 88% 100% 86% 93% 0.0669 11 100% 44% 55% 79% 69% 0.2006 12 18% 63% 0% 29% 27% 0.1822 comprehension (type b) 13 81% 75% 91% 100% 87% 0.0881 14 68% 25% 45% 79% 54% 0.1896 15 7% 0% 9% 7% 6% 0.0286 16 70% 69% 36% 79% 63% 0.1352 17 82% 25% 91% 43% 60% 0.2621 18 91% 13% 64% 7% 44% 0.3372 19 55% 56% 36% 14% 40% 0.1503 20 45% 13% 27% 50% 34% 0.1392 average 64% 45% 60% 49% 54% 0.0766 (*) percentage of right answers to each question as shown in table 1, from the twelve type a items related to the study, only questions 3, 9 and 10 have shown a high percentage of right answers (above 75% of total answers). the http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 24 share of correct answers for the rest of type a items is relatively low, particularly in items 4 and 12, which suggests a lack of dedication of students to study the subject. the results for type b items (comprehension) confirm low rate of success (below 75%). especially remarkable is the low percentage of right answers to item 15 (6%). the only exception is item 13, in which the rate of right answers is 87%. it is worth mentioning that this item refers to an issue taught in the first lesson of the subject. therefore, this result may suggest that students have just started to study the subject. this is confirmed by the overall average of right answers, that is 54%. our results show relatively small dispersion among groups. from the results, we conclude that the use of irs is a useful way to detect students’ lack of study and comprehension and other related issues. first, it has enabled teachers to detect that students have not prepared the subject sufficiently to pass the subject. this enables teachers to inform students of the need to increase their study of the subject. second, the results obtained imply the need to repeat this experience after students have devoted enough time to study, so they can detect genuine lack of comprehension by students. third, assuming reasonable results for type a items, the experience shows that using interactive response system enables the detection of lack of comprehension of the contents. fourth, the detailed analysis of the results obtained by group of class, so teachers can focus in the solving of these concerns. also, it has served to facilitate the participation of students in the review class and obtained immediate feedback. this is an important issue, because traditionally there has been extremely low participation of students in these sessions. 4.1 effort perception and usefulness of the experience according to students we surveyed students attending the collective session about two main issues: 1) the effort they had made to study the subject; and 2) the usefulness of the experience. for that purpose, students were asked to fill in a questionnaire with a set of questions in the likert http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 25 scale (ranging from 1 to 5 points, being 5 “a lot/very useful/strongly agree”). additionally, this scale was transformed into a binary variable that equals 1 when the answer is 4 or 5; and 0 in other case. the questionnaire was administered in two rounds: before and after the final exam. this enabled us to detect whether the results obtained in the exam shape students' perceptions. the number of attendees at the tutorial session was 84. 73 students completed the survey before the exam and 60 answered after the exam. note that filling the survey was not compulsory and thus, some students decided not to complete it. table 2 shows students' opinions about the usefulness of this experience to consolidate knowledge and to detect doubts in the subject. in general, the mark granted to most of the items is very high. this suggests that students appreciate the use of this tool to consolidate and detect doubts. however, our results suggest a different perception between the evaluation of the usefulness before and after the final exam of the subject. focusing on the percentage of students that consider this initiative as highly valuable, it dramatically decreases when students are surveyed after the final exam. this may indicate that students are open and motivated to the use of this teaching tools but their perception is moderated and conditioned by the mark obtained in the subject. the opinion of students about teacher explanations was very positive (84.9% rated this item 4 or 5). according to the results, 78% of students consider the irs useful or very useful to detect lack of comprehension and 75% of them consider that it facilitated the study of the subject. additionally, table 3 shows the rating of students to the number of hours devoted to studying the subject. interpretation of these results must bear in mind that this survey was carried out during the last week of class, very close to the date of the final exam. only 19.2% of students reported that they had devoted a high number of hours to study the subject before attending the tutorial class. when taking binomial values it arises that only 42.5% of students report having prepared the subject enough to participate in the tutorial class. taking into consideration that students’ perception about this question may be http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 26 overestimated, these results contribute to explain the small percentage of right answers showed in table 1, specially in type a items, confirming the usefulness of this experience to obtain information about the level of study of the subject by students. besides this, when questioned after having been informed about their final mark, 16.7% of students argued that they have devoted the maximum time possible to prepare this subject. this percentage increases to 40% when using binomial values. these results indicate that students’ opinion about their own effort is moderated when they are informed of their final mark. table 2. perception about the utility of the experience. likert binomial 1 2 3 4 5 tot 0 1 tot what is your opinion about teachers’ explanations of the use of sri? pretest posttest 1.4 0.0 4.1 8.3 9.6 28.3 37.0 36.7 47.9 26.7 100 100 15.1 36.6 84.9 63.4 100 100 what is your opinion about the utility of the tool to reinforce the contents taught in the subject? pretest posttest 0.0 0.0 4.1 5.0 20.5 23.3 31.5 41.7 43.9 30.0 100 100 24.6 28.3 75.4 71.7 100 100 what is your opinion about the utility of this experience to facilitate the preparation and study of the subject? pre-test posttest 0.0 0.0 5.5 6.7 19.2 23.3 39.7 38.3 35.6 31.7 100 100 24.7 30.0 75.3 70.0 100 100 what is your opinion about the utility of the tool to detect doubts and lacks in the subject? pre-evaluation posttest 5.5 0.0 2.7 8.3 13.7 25.0 32.9 31.7 45.2 35.0 100 100 21.9 33.3 78.1 66.7 100 100 number of observations: pre-test: 73 post-test: 60 table 3. perception about students effort. likert binomial 1 2 3 4 5 tot 0 1 tot how much have you studied the subject before attending the tutorial class? pre-evaluation post-evaluation 1.4 3.3 20.5 13.3 35.6 43.3 23.3 23.3 19.2 16.7 100 100 57.5 60.0 42.5 40.0 100 100 number of observations: pretest: 73 posttest: 60 http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 27 4.2. factors affecting students’ perception of the usefulness of the irs to analyse the factors shaping students opinion about the usefulness of the irs, we put forward a set of empirical models that are described in the following paragraphs. 4.2.1. model and variables the dependent variable measures the students' perceptions about the usefulness of the irs to resolve doubts and to identify gaps in the study of the subject. more specifically, we constructed a binary variable taking the value of 1 if student considers the use of the irs useful and the value of 0 otherwise. we use a binary logistic regression owing to the fact that the dependent variable in the model is a dummy variable resulting from the choice between two options (1 or 0). logit models use the standard logistic probability distribution and the interest lies primarily in the response probability, on the following form: p(y =1/x) = g(β0 + β1x1 + … +βkxk) = g(β0 +xβ). in the logit models, g is the logistic function; p(y =1/x) the probability of something happening given the values of explanatory variables, y is a dummy variable taking values 1 or 0; x is a vector of (k-1) independent variables and β is the k x1 vector parameters. we are primarily interested in explaining the effects of each explanatory variable on the probability that a student considers the irs useful to resolve and identify doubts. in logit models coefficients β are not, by themselves, especially useful because of the nonlinear nature of the logistic function. the parameters do not display directly the effects of each explanatory variable on the response probability but, knowing the sign of the parameters, we can determine whether the explanatory variable has a positive or negative effect on response probability, but not the magnitude of the effect (wooldridge, 2003; cabrer et al. 2001). our dependent variable is students’ perception of the usefulness of irs to detect doubts (doubts), which is a dummy variable that equals 1 when the student considers the irs as a useful or very useful tool and 0 otherwise. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 28 our explanatory variables are the following: gender is a dummy variable that equals 1 for males and 0 for females. firstenrol is a dummy variable that equals 1 for students who are enrolled in the subject for the first time and 0, otherwise. -pass is a dummy variable that equals 1 for students who have passed the subject and 0, otherwise. mark contains information about the mark obtained in the final exam. it ranges between 0 and 10. explan is a variable measured in the likert scale (from 1 to 5) that refers to the students’ evaluation of the explanations given by the teachers about the use of the irs. prestudy is a variable measured in the likert scale (from 1 to 5) that refers to students’ evaluation of the number of hours devoted to studying the subject before attending the tutorial class. study: is a variable measured in the likert scale (from 1 to 5) that refers to students’ evaluation of the number of hours devoted to study the subject. table 4 shows the descriptive statistics of these variables. table 4. descriptive statistics of the explanatory variables. average median maximum minimum standard deviation gender 0.50 0.50 1.00 0.00 0.50 firstenrol 0.88 1.00 1.00 0.00 0.32 pass 0.30 0.00 1.00 0.00 0.46 mark 3.78 3.59 9.60 0.00 2.33 explan 3.82 4.00 5.00 2.00 0.93 prestudy 3.37 3.00 5.00 1.00 1.02 study 3.77 4.00 5.00 2.00 0.99 no. observations 60 60 60 60 60 source: own elaboration http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 29 4.3. results of the models the main results stemming from the regression analysis are shown in table 5. model 1 has been estimated including all the explanatory variables. the main difference among regressions 1a and 1b is that regression 1a shows the results considering the variables explan, prestudy and study as likert type variables (from 1 to 5), while regression 1b takes the variables explan, prestudy and study as dummy variables (0 or 1). except the variable mark, which is a continuous variable, all explanatory variables are binary variables in both regressions. regressions 1c and 1d are exactly the same as 1a but we removed the variables pass and mark respectively, the reason being the high correlation among them. the results show that: the variables gender (gender) and explanations received about the use of the irs (explan) are relevant in all the models for 1% and 5% levels of significance. this result suggests that women are more likely to consider the irs useful than men. besides, it also highlights the importance of giving proper information and instructions about the use of this tool. the variable that refers to the time spent by students studying the subject (study) is relevant but considering a 10% level of significance. this indicates that students, who are more engaged to the study of the subject, are also more concerned about the usefulness of the irs. firstenrol is relevant for a 10% level of significance in all the models, except in model 1b. this suggests that students who are enrolled in the subject for the first time are more motivated by the innovation in teaching. mark, pass and prestudy are not relevant to explain the dependent variable. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 30 table 5. results of the estimation of logit models. mod 1a mod 1b mod 1c mod 1d mod 2 mod 3 mod 4 mod 5 constant -6.73*** -1.95 -6.46*** -6.42*** -5.70*** -4.98** -4.14** -4.83** gender -2.02** -2.49*** -1.92** -1.95** -1.81** -2.01*** -1.75** firstenrol 1.66* 1.55 1.62* 1.61* 1.22 1.66* 1.60* pass -0.73 -1.31 -0.14 0.27 -0.55 -1.23 -0.45 mark 0.15 0.24 0.03 -0.02 0.11 0.24 0.18 explan 1.08** 1.82** 1.11** 1.11** 1.02** 1.08** 0.98** prestudy 0.09 0.29 0.05 0.05 -0.04 -0.01 0.56 0.30 study 0.68* 1.66** 0.67* 0.71* 0.47 0.67* 0.57 log-likelihood -27.22 -27.46 -27.36 -27.36 -31.17 -28.71 -30.42 -30.42 mcfadden r-sq 0.29 0.28 0.28 0.28 0.18 0.25 0.20 0.20 correctly predicted cases 81.7% 78.3% 78.3% 81.7% 75% 75% 75% 75% lr test (chi-squared) 21.94 (0.0026) 21.46 (0.0032) 21.66 (0.0014) 21.67 (0.0014) 14.03 (0.0292) 18.97 (0.0042) 15.54 (0.0165) 15.54 (0.0165) * level of significance: 10% ** level of significance: 5% *** level of significance: 1% in order to test the robustness of our results we present the models 2, 3, 4 and 5 in table 5. these models sequentially remove the variables relevant in model 1. the robustness checks show that the results do not significantly change when the different relevant variables in the analysis are removed from the model. this confirms the reliability of our results. 5. discussion and conclusions this teaching initiative based on the use of the irs has been demonstrated to be useful to detect students' lack in the study and comprehension of the subject as shown by students' opinions (78% consider the irs useful or very useful to detect lacks of comprehension and 75% consider that it facilitated the study of the subject). therefore, it becomes a useful tool for teachers to improve the learning process in several ways. first, teachers can inform students about the need to increase their study of the subject in order to pass it. second, as teachers become aware of the concepts that students find difficult to understand, they can http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 31 plan their theoretical class in subsequent years to focus on clarifying them. third, the use of the irs improves the retrieval of information with respect to the traditional system of raising hands, because in the latter method, students are often reluctant to participate and it takes time to tabulate the data. assuming that the use of the irs is not a magical recipe to guarantee students' performance and that there are some technical difficulties in the use of this tool, we have nevertheless shown the usefulness of its use for students. the regression analysis of our data provides evidence on the factors affecting the attitude of students to the irs for detecting lacks and doubts in the subject. the estimated model has shown that gender and explanations received about the use and usefulness of the tool exerts a positive and significant effect. female students in general evaluated this item more positively than males. the rating of explanation received about the use of this experience has a positive effect on students’ attitude to the irs. lastly, if the level of significance is increased, the variables accounting for students who enrol for the first time in the subject and the number of hours devoted to study become relevant, indicating a positive effect on the students' attitude to the use of the irs. the main limitation of this initiative is that the irs has only been used in one session, but given the positive feedback obtained, teachers of the subject are considering its regular use throughout the course. future research in other subjects could contribute to greater depth of understanding of the reasons that create a positive attitude towards new teaching methodologies in general, and the irs in particular. there is also ground to examine the effect of these methodologies on students’ performance. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 32 6. references anthis, k. (2011). is it the clicker, or is it the question? untangling the effects of student response system use. teaching of psychology, 38(3), 189-193. butler, j. a. (1992). use of teaching methods within the lecture format. medical teacher, 14(1), 11-25. camacho-miñano, m. d. m. and del campo, c. (2014). useful interactive teaching tool for learning: clickers in higher education. interactive learning environments, 1-18. cabrer, b., sancho, a. and serrano, g. (2001). microeconometría y decisión. pirámide, madrid. caldwell, j. e. (2007). clickers in the large classroom: current research and bestpractice tips. life sciences education, 6(1), 9-20. carnaghan, c. and webb, a. (2007). investigating the effects of group response systems on student satisfaction, learning and engagement in acounting education. issues in accounting education, 22 (3), 391-409. dallaire, d. h. (2011). effective use of personal response “clicker” systems in psychology courses. teaching of psychology, 38(3), 199-204. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 33 emenike, m. e. and holme, t. a. (2012). classroom response systems have not “crossed the chasm”: estimating numbers of chemistry faculty who use clickers. journal of chemical education, 89, 465-469. fallon, m. and forrest, s. l. (2011). high-tech versus low-tech instructional strategies: a comparison of clickers and handheld response cards. teaching of psychology, 38(3), 194-198. fies, c. and marshall, j. (2006). classroom response systems: a review of the literature. journal of science education and technology, 15(1), 101-109. gebru, m. t., phelps, a. j. and wulfsberg, g. (2012). effect of clickers versus online homework on students´ long-term retention of general chemistry course material. chemistry education research and practice, 13, 325-329. gül, h., gül, s. s., kaya, e. and alican, a. (2010). main trends in the world of higher education, internationalization and institutional autonomy. procedia social and behavioral sciences, 9, 1878-1884. han, j. h. and finkelstein, a. (2013). understanding the effects of professors' pedagogical development with clicker assessment and feedback technologies and the impact on students' engagement and learning in higher education. computers & education, 65, 6476. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 34 hughes, c., roche, a. m., bywood, p. and trifonoff, a. (2011). audience-response devices (clickers): a discussion paper on their potential contribution to alcohol education in schools. health education journal, 72 (1), 47-55. kay, r. and knaack, l. (2009). exploring the use of audience response systems in secondary school science classrooms. journal of science education and technology, 18 (5), 382-392. kennedy, g. and cutts, q. (2005). the association between students’ use of electronic voting systems and their learning outcomes. journal of computer assisted learning, 21(4), 260-268. king, s. o. and robinson, c. l. (2009). `pretty lights´ and maths! increasing student engagement and enhancing learning through the use of electronic voting systems. computers & education, 53, 189-199. lasry, n. (2008). clickers or flashcards: is there really a difference?. the physics teacher, 46, 242-244. liu,t., liang, j., wang, h., chan, t. and wei, l. (2003). embedding educlick in classroom to enhance interaction. in lee, k., mitchell, k. (eds.) proceedings international conference computers in education (icce), 117-125. hong kong (china). ludvigsen, k., krumsvik, r. and furnes, b. (2015). creating formative feedback spaces in large lectures. computers & education, 88, 48-63. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 35 macgeorge, e. l., homan, s. r., dunning, j. b. (jr), elmore, d., bodie, g. d., evans, e., khichadia, s., lichti, s. m., feng, b. and geddes, b. (2008). student evaluation of audience response technology in large lecture classes. educational technology research and development, 52(2), 125-145. morling, b., mcauliffe, m., cohen, l. and dilorenzo, t. m. (2008). efficacy of “clickers” in large, introductory psychology classes. teaching of psychology, 35, 45-50. moss, k. and crowley, m. (2011). effective learning in science: the use of personal response systems with a wide range of audiences. computers & education, 56(1), 36-43. palmer, e. j., devitt, p. g., de young, n. and morris, d. (2005). assessment of an electronic voting system within the tutorial setting: a randomised controlled trial. bmc medical education, 5:24. poirier, c. r. and feldman, r. s. (2008). promoting active learning using individual response technology in large introductory psychology courses. teaching of psychology, 34, 194-196. schackow, t., milton, c., loya, l. and friedman, m. (2004). audience response systems: effect on learning in family medicine residents. family medicine, 36 496-504. siau, k.; sheng, h. and fui-hoon nah, f. (2006). use of a classroom respond system to enhance classroom interactivity. ieee transactions on education, 49(3), 398-403. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.5476 issn: 2341-2593 bares lópez et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 16-36 | 36 stowell, j. r. and nelson, j. m. (2007). benefits of electronic audience response systems on student participation, learning and emotion. teaching of psychology, 34, 253-258. van dijk, l. a., van den berg, g. c. and van keulen, h. (2001). interactive lectures in engineering education. european journal of engineering education, 26(1), 15-28. white, p., syncox, d. and alters, b. (2011). clicking for grades? really? investigating the use of clickers for awarding grade-points in post-secondary education. interactive learning environments, 19(5), 551-561. wooldridge, j.m. (2003). introductory econometrics, a modern approach. second ed. south-western college publishing, cincinnati, oh. http://dx.doi.org/10.4995/muse.2017.5476 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 1 applied mathematical problems in engineering llopis-albert, carlosa; palacios-marques, danielb a departamento de ingeniería mecánica y de materiales, universitat politècnica de valència, camí de vera s/n, spain, 46022, email: cllopisa@upvnet.upv.es b departamento de organización de empresas, universitat politècnica de valència, camí de vera s/n, spain, 46022, email: dapamar@doe.upv.es received: 2016-04-02; accepted: 2016-07-21 abstract there is a close relationship between engineering and mathematics, which has led to the development of new techniques in recent years. likewise the developments in technology and computers have led to new ways of teaching mathematics for engineering students and the use of modern techniques and methods. this research aims to provide insight on how to deal with mathematical problems for engineering students. this is performed by means of a fuzzy set/qualitative comparative analysis applied to conflict resolution of public participation projects in support to the eu water framework directive. keywords: fuzzy sets; qualitative comparative analysis; public participation project; conflict resolution; decision-making http://dx.doi.org/10.4995/muse.2016.6679 mailto:dapamar@doe.upv.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 2 1. introduction integrated water resources management entails technical, scientific, political, legislative, and organizational aspects of water system. water resources management suffers from continual and growing pressures, which derive from reasons such as human activity, population growth, living standards increase, land-use and climate changes, growing competition for water, and pollution from industrial, municipal, and agricultural sources. the eu water framework directive (wfd) establishes a framework for the protection of all water bodies by promoting sustainable water use based on long-term protection of water resources, and enacts to achieve good qualitative and quantitative status of all water bodies by 2021. the eu wfd also states that all members shall encourage the active involvement of stakeholders in the implementation of the directive and development of watershed management plans (ec, 2000). a way to deal with stakeholders' conflict resolution problems is by using a configurational comparative method. this is performed by means of a fuzzy set/qualitative comparative analysis, fsqca (ragin 2008), which overcomes some of the limitations of strictly qualitative or quantitative studies. this technique has been widely used in the literature to deal with qualitative comparative analysis (e.g., berbegalmirabent and llopis-albert, 2015). http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 3 2. public participation projects in the water framework directive stakeholders’ satisfaction depends on diverse factors, such as their heterogeneous interests, educational backgrounds, employment, knowledges, resources, experiences, places of provenance, levels of participation, etc. stakeholders are selected by considering all groups who in some way will be affected by the implementation of measures, which includes those who have interests, claims or rights (ethical or legal) to the benefits of the measures undertaken, are likely to bear its costs or adverse impacts whatever its overall worth. the stakeholders involved in the water decision-making of a ppp range from governments, water agencies authorities, environmental organizations, irrigation user communities, private firms, universities and research agencies, political parties, labor unions, experts, advisors, mass-media, citizens to international organizations. table 1 presents the factors or conditions that leads to the stakeholders' satisfaction in the decision-making process of a ppp for a watershed management, which cover environmental objectives pursued, the actual capacity of efficiently carrying out those objectives, the socio-economic development of the region, the level and mechanisms of stakeholders’ participation in the ppp, and the alternative policies and measures that should be implemented in the hydrological plans. http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 4 3. methodology this study uses actors belonging to different watersheds and countries to assess stakeholders' preferences or degrees of acceptance regarding the diverse factors or conditions that lead to their satisfaction. then this work deals with different watersheds realities and national legislations, and presents a general overview of the european stakeholders' satisfaction in the decision-making process of ppp for water resources management. the diversity of watersheds in terms of the management strategies they apply and their stakeholder engagement makes this work even more suitable for studying how combinations of conditions in the decision-making process can result in stakeholder satisfaction. eventually, we will identify what combinations of the considered conditions are necessary or sufficient to achieve the stakeholders’ satisfaction in ppp. the study is based on different european reports (eu, 2003; oecd 2015; oecd 2014), research papers dealing with these issues (e.g., verweij et al., 2013; srinivasan et al., 2012), online reports from webpages of european water agencies authorities, massmedia information, meetings, personal interviews, surveys, and expert judgment. data have been collected during long-time period of years, so that a longitudinal view on the course of the stakeholders’ preferences has been obtained. therefore, we are intended to identify which combinations of factors are sufficient to explain the outcome by means of a qualitative comparative analysis (qca) to http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 5 overcome some of the limitations of strictly qualitative or quantitative methods, and to more systematically analyze conjunctural causal patterns a configuration is a combination of factors -which is named as conditions in qca terminologythat is minimally necessary and/or sufficient for obtaining a specific outcome. these configurations consist of conditions or factors that can be positive, negative, or absent. conditions are sufficient and necessary only in combination with other conditions or which are only one alternative among others that only apply to some cases but not to others. contrary to qca, which can only analyze binary variables, fsqca overcomes this limitation by incorporating the possibility to examine varying levels of membership of cases to a particular set. first calibration procedure of outcomes and antecedent conditions into fuzzy sets is required, which categorizes meaningful groupings of cases (ragin, 2008). fuzzy values range from full membership (1) to full non-membership (0). a crossover point (0.5) represents neither in nor out of the set. second the truth table is constructed, which is a matrix space with 2k rows, where k is the number of antecedent conditions and represents all the logically possible combinations of causal conditions and sorts the cases according to these logically possible combinations. each column represents a condition, and each empirical case corresponds to a configuration depending on which antecedent conditions the case meets. third a reduction of the the number of rows in the truth table is carried out. we have used a version of the quine–mccluskey algorithm (quine, 1952), although several http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 6 algorithms can also minimize a truth table. this allows to obtain a set of combinations of causal conditions by using boolean algebra, where each combination is minimally sufficient to produce the outcome. the row reduction depends on two criteria: a) the coverage, which indicates the empirical relevance of a solution, that is., it measures the proportion of memberships in the outcome that is explained by the complete solution; b) the consistency, which quantifies the degree to which instances sharing similar conditions display the same outcome. on the other hand, the raw coverage indicates which share of the outcome is explained by a certain alternative configuration, while the unique coverage indicates which share of the outcome is exclusively explained by a certain alternative path. truth tables are analyzed by the fs/qca software (ragin 2008). 4. results and discussion the main aim is to determine which particular combinations of these conditions lead to stakeholders’ satisfaction by examining which combinations of the conditions are necessary or sufficient to achieve it. the degree of acceptance or preference regarding the different factors is analyzed by means of using a continuous fuzzy set. it is ranged from 0 to 1, i.e., from low degree of acceptance or agreement to high degree of acceptance or agreement. we have considered 7 factors, which comprises several sub-factors as shown in table 1. for each stakeholder and factor the aggregate final score is the arithmetic average of the fuzzy scores for each sub-factor. the calibration process has allowed http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 7 transforming the diversity of factors used in this work into fuzzy variables, so that they match or conform to external standards. fuzzy scores calibration is based on different reports, research papers, online reports from webpages of european water agencies authorities, mass-media information, meetings, personal interviews, surveys, and expert judgment. the truth table is obtained after several rounds of analyses and because there are 7 factors the dimensions are (27) rows and 7 columns, which entails 128 possible configurations. the matrix is checked for necessary conditions for the outcome and also for the negation of the factors indicated by the tilde (~) sign. a condition has been considered as necessary when its consistency score exceeds the threshold value of 0.9. results show that there are only 2 necessary conditions, which are the environmental objectives and the socio-economic development of the region. after the minimization process using the coverage and consistency values, the combinations of causal conditions are obtained, thus providing the combinations of factors that are minimally sufficient to produce the outcome (see table 2). in this table black circles () indicate the presence of a condition, white circles (⭕) denote its absence, and blank cells represent ambiguous conditions. results also suggest that: a) no unifying causal path explains the outcome; b) all configurations present acceptable consistency indices (<0.80); c) high raw coverage values are obtained; d) apart from the necessary conditions, the presence of policies (both control measures and technical measures) appear in most of the configurations, which shows that the outcome strongly http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 8 depends on the types of policies undertaken; e) the greater understanding of the problem (for instance, using a mathematical techniques like that her presented or models taking into account the key underlying biophysical processes, e.g., llopis-albert et al., 2014; 2015) the better management practices and consensus will be achieved among the different actors; f) better results are obtained if actors are involved at early stages, on account of they are less likely to obstruct decisions and more likely to support them; g) good outcomes in a ppp are also related to clear goals, strong control of time, organization and information. table 1. variables or factors considered in the fuzzy set/qualitative comparative analysis applied to ppp in support to the eu water framework directive. environmental interests – objectives pursued (1) good quantitative status of water bodies, both surface and groundwater good chemical status of water bodies good status of water dependent ecosystems low environmental impacts of future land-use land-cover changes and climate changes socio-economic interests objectives pursued (2)(3) operative efficiency (2) short realization time low implementation costs low maintenance, management and infrastructures construction costs socio-economic development of the region (agriculture, industry and tourism) (3) maximize water for agricultural and industrial use maximize water for tourism and urban use create employment, social equity increase future water demands level of stakeholders’ engagement in the water decisionmaking process (4) perceived obstacles to the integration of stakeholder engagement in water policies and practices (4.1) lack of political will and the shift of power lack of knowledge weak legal frameworks scant participation level perceived obstacles hindering the lack of clarity on the use of engagement processes lack of funding http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 9 effective implementation of engagement processes (4.2) lack of quality and accessibility of information intensity and number of conflicts too much or too few actors preferred mechanisms used for stakeholder engagement (5) meetings workshops / conferences expert panels web-based communication technologies water associations consultations in regulatory processes surveys / polls river basin organizations others preferred measures and policies for sustainable water resources management (6)(7) control mechanisms (6) control or reduction of water demand by economic instruments control or reduction of pollutants by economic instruments set up of user’s communities as a control mechanism control of water resources by application of satellite remote sensing more intervention of the eu common agricultural policy increase of water control and sanctions by water agencies alternative technical actions (7) efficient conjunctive use of surface water and groundwater use of external water resources by means of transfers use of desalination plants construction of new infrastructures establishment of protected areas greater funding for water resources research others outcome stakeholders' satisfaction outcome: stakeholders’ satisfaction http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 10 table 2. sufficient configurations of factors for stakeholders’ satisfaction. black circles () express the presence of a condition, white circles (⭕) indicate its absence, and blank cells represent ambiguous conditions. in addition, frequency threshold = 1 and consistency threshold = 0.908. configurations (c) factors coverage consistency 1 2 3 4 5 6 7 raw unique c1    ⭕  0.743 0.030 0.948 c2  ⭕   ⭕ ⭕  0.596 0.001 0.893 c3    ⭕  0.680 0.005 0.943 c4  ⭕  ⭕    0.623 0.006 1.000 c5  ⭕   ⭕  ⭕ 0.631 0.003 0.963 c6   ⭕   ⭕ 0.513 0.011 0.959 c7    ⭕ ⭕  0.627 0.001 0.896 c8     ⭕  0.711 0.002 0.914 solution coverage: 0.818 and solution consistency: 0.891 5. conclusion this paper provides insight into stakeholders' conflict resolution by using a fuzzy set/qualitative comparative analysis (fsqca) for determining which combinations of factors are necessary and/or sufficient for leading to stakeholders’ satisfaction throughout the decision-making process of public participation in water resources management. it takes into account a wide range of factors and configurations to obtain the outcome, which allows coming up with the best management practices and policies for a certain watershed with its own particularities. this is because the methodology facilities dialogue http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 11 during the decision-making process between theoretical ideas and empirical evidence and allows the selection and construction of cases and conditions. from all the configurations analyzed results have shown that environmental objectives and socio-economic development of the region are necessary conditions, while for other factors results are imprecise because of stakeholders’ heterogeneity and conflict interests among them. then the outcome do not depend upon single conditions, but result from combinations. eventually this mathematical technique provides a transparent and multidisciplinary framework for informing and optimizing water policy decisions and goes a step further in the implementation of the wfd. http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 12 references berbegal-mirabent, j., & llopis-albert, c. (2015). applications of fuzzy logic for determining the driving forces in collaborative research contracts. journal of business research, doi: 10.1016/j.jbusres.2015.10.123. criado, f., costero, l., cabeza, p. & sánchez-élez, m. (2015). calculator: a hardware design, math and software programming project base learning. multidisciplinary journal for education, social and technological sciences 2(1), 110-127. doi: 10.4995/muse.2015.2275. ec (2000). directive 2000/60/ec of the european parliament and of the council of october 23 2000. establishing a framework for community action in the field of water policy. official journal of the european communities, l 327, 22/12/2000 p. 0001 – 0073. ec (2003). common implementation strategy for the water framework directive (2000/60/ec). guidance document nº 8. public participation in relation to the water framework directive. isbn 92-894-5128-9. llopis-albert, c., palacios-marqués, d., & soto-acosta, p. (2015). decision-making and stakeholders’ constructive participation in environmental projects. journal of business research, 68(7). doi:10.1016/j.jbusres.2015.02.010. http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 13 llopis-albert, c., palacios-marqués, d., & merigó, j. m., (2014). a coupled stochastic inverse-management framework for dealing with nonpoint agriculture pollution under groundwater parameter uncertainty. journal of hydrology, 511, 10–16. doi:10.1016/j.jhydrol.2014.01.021. oecd (2014). stakeholder engagement for effective water governance. organisation for economic co-operation and development (oecd). workshop – iwa world water congress, 24 september 2014, lisbon. oecd (2015). stakeholder engagement for inclusive water governance. oecd (organisation for economic co-operation and development) studies on water, oecd publishing, paris. doi: 10.1787/9789264231122-en. quine, w. v. (1952). the problem of simplifying truth functions. the american mathematical monthly, 59(8), 521–531. doi: 10.2307/2308219 ragin, c. c. (2008). redesigning social inquiry: fuzzy sets and beyond. chicago: university of chicago press. doi: 10.7208/chicago/9780226702797.001.0001 sousa, c. (2016). problem-based learning on cell biology and ecophysiology using integrated laboratory and computational activities. multidisciplinary journal for education, social and technological sciences 3(1), 118-132. doi: 10.4995/muse.2016.3763. http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6679 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 1-14 | 14 srinivasan, v., lambin, e. f., gorelick, s. m., thompson, b. h., & rozelle, s. (2012). the nature and causes of the global water crisis: syndromes from a metaanalysis of coupled human-water studies, water resources research, 48, w10516. doi: 10.1029/2011wr011087. verweij, s., klijn, e. h., edelenbos, j., & van buuren, a. (2013). what makes governance networks work? a fuzzy set qualitative comparative analysis of 14 dutch spatial planning projects. public administration, 91, 1035-1055. doi: 10.1111/padm.12007. http://dx.doi.org/10.4995/muse.2016.6679 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 173 engaging and assessing students through their electronic devices and real time quizzes ferrándiz, e.*, puentes, c., moreno, p.j., flores, e. department of economics, universidad de cádiz *esther.ferrandiz@uca.es; received: 2016-01-23; accepted: 2016-03-13 abstract this paper describes a teaching experience using socrative, a third party electronic tool, for real-time questioning in lectures of econometrics. econometrics is a theoretical-practical subject, but traditionally a large proportion of our students tend to focus on the practical and discard the theory, often skipping classes on theory and avoiding studying its content, probably motivated by its complexity. as a consequence, students’ marks obtained in the theoretical part of the exam are usually low. in this context, we put forward a change in our teaching methodology to include the use of socrative, a freely available app, that allows students to answer teachers’ short, true/false, or multiple choice questions posed during each class using their smartphones (or other electronic devices with internet connection). the objectives of this project are twofold: 1) to engage students and increase attendance at lectures; 2) to improve feedback on the learning process. the results of a survey of a sample of 186 students reveal that socrative has been an effective tool for achieving these objectives. keywords students response system, clicker, quizzes, active learning, student engagement, socrative, gamification http://dx.doi.org/10.4995/muse.2016.6375 mailto:esther.ferrandiz@uca.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 174 1. introduction information technology (it) offers a powerful opportunity for increasing student engagement (roblyer & wiencke 2003). prominent examples of technology include student response systems (srss),1 which allow an instructor to pose questions to the class, allow students to enter their answers in some kind of device (e.g., clicker or smartphone) and instantly summarize and graphically represent results for the instructor (beatty 2004), who can decide whether the responses are anonymous or not. terrion and aceti (2012) have highlighted the importance of student engagement because unengaged students do not listen, process, or attend to the learning process. several papers have analysed the role of srss as a means of engaging students and making them active in the classroom, among other benefits (for a review see aljaloud et al. 2015). although much of the research in srs has traditionally been related to the use of clickers, the development of cloud-based software that enables any device to become an srs overcomes some of the criticism of clickers such as cost, and allows educators to exploit the potential of personal electronic devices, mostly smartphones, for educational purposes. although education is not among the main uses of these devices yet, this use is expected to grow in the coming years as new applications are developed. companies such as apple have already foreseen this possibility, developing itunes u, an application store especially focused on education. the highest rates of smartphone ownership are among the richer economies and the top countries in the rank include south korea (88%), 77% of australia (77%), israel (74%), usa (71%) and spain (71%), with a greater presence among young people (pew research center, 2016). gaming (65%), social networking (54.3%), and instant messaging (48.3%) are among the most common uses minors and younger people have for their smartphones’ advanced features (inteco, 2011). these statistics are in line with yusof et al. (2012), who argue that smartphones are mainly used for entertainment and social networking. therefore, the potential of smartphones for education (m-learning) has yet to be unlocked. 1 srss are also refered to as classroom performance systems, audience response systems, personal response systems, classroom communication systems, electronic response systems, electronic voting systems, polling systems, or clicker systems (aljaloud et al. 2015). http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 175 additionally, the subject of econometrics taught in the business and administration degree program relies on theoretical reasoning and computer practices that students often find difficult to understand. this, jointly with the limited time available in lectures and the complexity of the subject, hinder the learning, comprehension, and motivation of students when they deal with econometrics. motivated by all the above-mentioned reasons, instructors of econometrics at the university of cadiz have proposed an innovation project based on socrative, a cloud-based srs. the main objectives of the project were two: 1) increase attendance and engagement of students in theoretical classes and 2) improve the feedback process to students regarding their learning processes. this paper is structured as follows. section 2 summarizes the advantages and disadvantages of srss. section 3 describes the methodology and the implementation of the study. section 4 describes the results; and finally, in section 5 we draw some conclusions and comment on future work. 2. benefits and drawbacks of using student response systems kay and lesage (2009a) and, more recently, aljaloud et al. (2015) review the literature on the benefits of using srss, which are related not only to engagement, but also to interactivity and academic performance. focusing on the former, previous research has shown that srss enhance student engagement in learning by creating a fun learning environment. further, the researchers found increased attendance and positive student attitudes and an increase in the students’ desire to improve their performance by identifying areas of improvement. because of increased engagement, an srs has the potential to improve academic performance. finally, srss improve the process of instructor feedback since the instructor has more information about the learning processes of their students. however, there are also some drawbacks, such as time required for teachers to formulate good questions and deal with technical problems. other often-cited inconveniences of clickers, such as cost or time to deliver the clickers to students in each class, are overcome thorough the use of cloud-based software such as socrative, which enables any electronic device (such as students’ smartphone or tablets) to be used as a clicker. http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 176 3. methodology this project focused on 385 students enrolled in econometrics classes in the business and administration degree program at the university of cadiz. econometrics is taught in the second semester to third-year students on three different campuses, involving seven teachers. the average number of participants was 160, spread among five classes (i.e., 32 students per class). the project mainly consisted of the regular use of socrative quizzes during lectures. the procedure to develop the activity followed several steps: 1. design and training of teachers. this phase includes the following tasks: a) elaboration of questions in socrative for each of the 10 lessons of the syllabus. each quiz had 3-4 questions and was focused on relevant contents explained in class. we included multiple choice and true/false questions, bearing in mind that the most effective questions to be used in srs are those (kay and lesage 2009b) that allow students to apply knowledge recently acquired; are higher level; focus on process and reasoning as opposed to factual content; identify and help resolve misconceptions; and support a comprehensive review of a specific set of concepts. b) statement of the rules of the game. the participation with socrative is individual unless the teacher says the opposite, and is limited to students who have attended all the classes. the use of electronic devices is only allowed for responding to teachers’ questions. students are responsible of bringing a working electronic device with internet connection. the university provides free working wi-fi for students in their facilities. c) selection of the reward. students who regularly attend class and provide correct answers to at least 75% of the questions would get 0.5 extra points added to their final mark. d) training of teachers. the coordinator of the project developed a training session for teachers and elaborated a visual guide on how to use socrative, covering some of the following aspects: how to create a personal account in socrative.com how to import a quiz previously elaborated by the coordinator and how to start it. regarding the latter, we used the following options:  students pace the quiz, which allows them to navigate through it at their own pace. http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 177  disable immediate feedback. we chose this option because we wanted to keep the correct answer concealed until the entire class had submitted their answers to make cheating less attractive.  random questions and answers. we used these options to make cheating more difficult. how to review the class’ answers, save reports and get bar graphs to give feedback to the students. 2. information session for students and system testing. in the first class of the course, each teacher introduced the subject to his/her class, including the methodology and the evaluation system. regarding the latter, the instructor introduced socrative and presented the rules of the game. subsequently, we made a trial test in class to check that the internet connection worked well in the classroom and to get students familiar with the system. in order to facilitate the link between student names and their marks in socrative, avoiding misspellings, students were asked to log in with their identity card number instead of their names. 3. classes, quizzes and exit tickets. once students and teachers knew how to use socrative, we began the course. in each session, after the teacher explained the theoretical content, he/she gave the students a quiz. by the end of the class, they had to obtain an “exit ticket” that included the following questions: how well did you understand today’s material? what did you learn in today’s class? and, optionally, a teacher’s open question. the latter often polled the student’s desire to explain a point or points again or to develop more deeply some specific materials or concepts that were not clear enough for them. after completing each quiz, the instructor should review and project the aggregated performance of the class for each question, comment on the correct answer, and save the report automatically generated by socrative. the total estimated in-class time devoted to quizzes and exit tickets should be 10-12 minutes. before the next class, lecturers had to review the exit tickets to check students’ needs and their requests in the open question. when all the quizzes had been run, teachers had to merge the marks of each quiz with the students’ names. 4. students’ feedback on the project. by the end of the course, we gathered students’ opinion through a survey to evaluate the contribution of socrative to their learning process. http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 178 4. results in order to get information about students’ satisfaction with the experience, they were asked to answer questions related to the contribution and limitations of socrative. this survey was answered through google forms. we use a likert scale from 1 to 5, where 1 is strongly disagree, 2 somewhat agree, 3 neutral, 4 somewhat agree, and 5 strongly agree. there were 186 respondents. for several questions, a relatively high percentage of answers was concentrated in the central value, 3, but this is not unusual because central tendency is a common bias when using a likert scale. figure 1 shows that 47.31% of students strongly/somewhat agree that socrative enhances comprehension of content and acquisition of competence related to econometrics (20.43% strongly/somewhat disagree). figure 2 shows that 56.99% of students agree that socrative has motivated them to attend to class versus 23.12% who disagree. figure 1. socrative improves comprehension of contents and acquisition of competence related to the subject. figure 2. socrative has motivated me to attend class. it is notable that 73.12% of the students agree that socrative is a motivation for listening to the teacher (figure 3). figure 4 shows that 55.91% of the students agree that socrative has contributed to helping them memorize some basic concepts, versus 20.97% of the students who disagree. according to figure 5, 65.05% of the students agree that socrative 3,23% 17,20% 32,26% 38,71% 8,60% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 9,14% 13,98% 19,89% 32,26% 24,73% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 179 is helpful in testing their level of understanding of the contents explained by the teacher (13.44% disagree). figure 6 shows that 50% of the students agree that socrative has increased their selfconfidence for answering teachers’ questions versus 17.74% who disagree. figure 7 demonstrates that 44.62% of the students agree that socrative increased their selfconfidence in their learning process in contrast with 23.66% of the students, who disagree. further, as represented in figure 8, 56.99% of the students prefer socrative to the traditional system of raising hands (23.66% do not). figure 3. socrative has motivated me to listen to the teacher. figure 4. socrative has helped me to memorize some basic concepts. 4,84% 4,84% 17,20% 45,70% 27,42% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 6,45% 14,52% 23,12% 34,95% 20,97% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 180 figure 5. socrative has contributed to testing my level of understanding. figure 6. socrative has increased my self-confidence in answering teachers’ questions. figure 7. socrative has increased my self-confidence in my learning process. figure 8. socrative is better than traditional questioning systems (e.g., raising hands, direct questions from the instructor). 3,23% 10,22% 21,51% 43,01% 22,04% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 8,06% 9,68% 32,26% 34,95% 15,05% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 7,53% 16,13% 31,72% 34,41% 10,22% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 10,75% 12,90% 19,35% 25,27% 31,72% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 181 figure 9 makes clear the perceived contribution to feedback in comparison to other theoretical subjects, with 67.2% of the students agreeing with this item (only 9.14% disagree). figure 10 shows that 61.29% of the students agree that socrative facilitates interaction while only 15.05% of students disagree with this statement. although we do not present a detailed graph here, 68.81% of the students strongly/somewhat agree that socrative contributes to making classes more pleasant and fun (only 10.75% of the students disagree). finally, table 1 presents the results of questions about the importance of some of the inconveniences students could face with the use of socrative. our results show that the most relevant inconvenience was related to internet connections and problems with the device. our results suggest that using socrative does not hinder class dynamics and does not prevent students from going back to the lesson and listening to the instructor. from the instructors’ viewpoint, the experience is also positively evaluated, and we are motivated by the results obtained in the students’ survey. although it represented an extra effort that required teacher commitment and slight changes in the teaching methodology, the system is easy, fast, and convenient and can be used on a regular base. figure 9. it gives more feedback to the teacher than in other theoretical subjects. figure 10. it facilitates interaction. 4,30% 4,84% 23,66% 35,48% 31,72% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 6,99% 8,06% 23,66% 37,10% 24,19% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 182 however, we also found some drawbacks in the use of socrative. first, time must be devoted to formulate questions that are relevant and appropriate for use in smartphones and other personal electronic devices. second, when we linked data from the reports to students’ names, we found some errors (for example, incomplete identity numbers or students who had logged in with their first name). this may be explained because these students only occasionally attended class or they did not check their login information before submitting. for this reason, it is recommended that the system allow teachers to limit access to a list of users enrolled in the course so spelling errors in login information can be eliminated. this feature will be available in the upcoming socrative pro version, according to the information provided in the system developer website. we also felt that some more gamification and teamwork would be advisable because when using “space races” that allow team collaboration and competition when answering quizzes, students were more motivated than they were by ordinary quizzes. in this regard, other cloud-based response systems like kahoot may be explored. table 1. rate the importance of the following drawbacks (%) 1 strongly disagree 2 somewhat disagree 3 neutral 4 somewhat agree 5 strongly agree problems with the device (i.e., no battery, system errors) 30.90 17.98 25.28 14.04 11.80 problems with the internet connection 17.20 24.19 25.27 16.67 16.67 after using socrative during class it is difficult to go back and and listen to the teacher again ok 34.41 23.66 27.42 8.60 5.91 5. conclusions the experience described in this paper mainly consisted of the regular use in lectures on econometrics of a cloud-based response system, socrative. the project was mainly aimed at engaging students and improving feedback during the lectures. socrative offers lecturers the opportunity to quickly and easily enhance the delivery of their lectures or tutorials in a way that increases interaction with students, leading to a better learning experience. http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 183 socrative is free, easy to use, and our results show that it enhances student engagement and improves the feedback process from learner to teacher and vice versa. for example, 27.42% of the students strongly agree (and 45.7% somewhat agree) that using socrative in lectures has motivated them to listen to the teacher. further, 24.19% of the students strongly agree that using socrative enables interaction (37.10% somewhat agree) and 56.98% agree that it has motivated them to attend lectures. students also positively evaluate the feedback that the system provides them (enabling them to check their level of understanding) and the teacher (providing more feedback than in other subjects). all in all, our results show that the goals of the project have been reached. therefore, student response systems, and specifically socrative, are confirmed as an effective means of engaging students and enhancing the feedback process. although some drawbacks were found such as time required for instructors to develop quizzes and review reports and exit tickets, as well as managing error in the login data, we strongly believe, basing on the survey of students and our own perceptions during lectures, that a student response system is a useful tool for improving our lectures. this is not to say that socrative is the best option available on the market since there are many cloud-based free apps with distinctive features that instructors could also explore, such as kahoot, that add more gamification to the learning process. further research could explore how to increase the effects of response systems on knowledge acquisition and comprehension. http://dx.doi.org/10.4995/muse.2016.6375 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6375 social and technological sciences eissn: 2341-2593 ferrándiz et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 173-184 | 184 references aljaloud, a., gromik, n., billingsley, w. & kwan, p. (2015) research trends in student response systems: a literature review. international journal of learning technology, 10 (4), 313–325. http://dx.doi.org/10.1504/ijlt.2015.074073 beatty, i. (2004). transforming student learning with classroom communication systems. educause research. retrieved june 2016 from: http://cds.cern.ch/record/877215/files/0508129.pdf inteco, instituto nacional de technologías de la comunicación (2011). estudio sobre hábitos seguros en el uso de smartphones por niños y adultos españoles. retrieved may 2016 from: http://www.inteco.es/file/bbzxmkvkx8vg7-0gghlozq kay, r. h., & lesage, a. (2009a). examining the benefits and challenges of using audience response systems: a review of the literature. computers & education, 53(3), 819-827. http://dx.doi.org/10.1016/j.compedu.2009.05.001 kay, r. h., & lesage, a. (2009b). a strategic assessment of audience response systems used in higher education. australasian journal of educational technology, 25(2), 235-249. http://dx.doi.org/10.14742/ajet.1152 pew research center, smith, a. (2016). smartphone ownership and internet usage continues to climb in emerging economies. retrieved may 2016 from: http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usagecontinues-to-climb-in-emerging-economies/ roblyer, m. d., & wiencke, w. r. (2003). design and use of a rubric to assess and encourage interactive qualities in distance courses. the american journal of distance education, 17(2), 77-98. http://dx.doi.org/10.1207/s15389286ajde1702_2 terrion, j. l., & aceti, v. (2012). perceptions of the effects of clicker technology on student learning and engagement: a study of freshmen chemistry students. research in learning technology, 20:16150 http://dx.doi.org/10.3402/rlt.v20i0.16150 yusof, s., goolamally, n., latif, l.a. & fadzil, m. (2012). 12th international conference on information, kuala lumpur (malaysia). http://dx.doi.org/10.4995/muse.2016.6375 http://dx.doi.org/10.1504/ijlt.2015.074073 http://cds.cern.ch/record/877215/files/0508129.pdf http://www.inteco.es/file/bbzxmkvkx8vg7-0gghlozq http://dx.doi.org/10.1016/j.compedu.2009.05.001 http://dx.doi.org/10.14742/ajet.1152 http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/ http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/ http://dx.doi.org/10.1207/s15389286ajde1702_2 http://dx.doi.org/10.3402/rlt.v20i0.16150 microsoft word 8_sousa multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 122 integrating bioethics in sciences’ curricula using values in science and socio-scientific issues c. sousa faculdade de ciências, universidade do porto, portugal. corresponding author: email: up199502480@fc.up.pt received: 2016-06-30; accepted: 2016-11-22 abstract the main objective of the present work is selection of ethical issues that should be addressed with first year undergraduate and k-12 students. since k-12 sciences’ curriculum, in portugal, does not include bioethics content in any discipline explicitly, teachers need to make an effort to include it. some online materials are available to use in high school classes and will be discussed. my proposal combines inquiry learning-teaching methods with the aim of promoting the discussion of bioethics issues in accordance to unesco bioethics core curriculum already adopted by twenty universities throughout the world (darwish 2015). some of the issues that are addressed are: ecology and environment ethics, infectious diseases and vaccination, water for all, intellectual property, genomes and patents, biotechnological advances (genetic modified organisms and synthesis of genomes), future generations, climate changes and natural resources, biomedical advances and human rights, authorship and contributions in scientific publications, and biobanks. in conclusion, this study may constitute an example to facilitate the implementation, by k-12 teachers, of active inquiry strategies, using features of science such as values and socio-scientific issues, and focused on the discussion of concrete ethical issues facing humanity. it also constitutes a proposal of integrating bioethics in undergraduate sciences’ curricula. keywords bioethics, socio-scientific issues, values in science, inquiry, k-12, undergraduate. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ mailto:up199502480@fc.up.pt multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 123 1. introduction the development seen in the last decades in biology has caused individuals to question the consequences, benefits and possible impacts of molecular biology, medicine, genetics and biotechnology at the individual/citizen level and in society. therefore the promotion of scientific literacy is essential and it is known that science education today is the most important factor to promote it. and since some level of understanding of bioethical issues is essential for science literacy, these issues should be included in the k-12 curriculum, as well as at the undergraduate level for all science students. using bioethical issues in the classroom promotes the development of a specific type of questioning, argumentation of different perspectives and, in some cases, proposed solutions (keskin 2013). my proposal constitutes one version of inquiry-based learning using dilemmas, with ill-structured problems and open-ended questions. in a class based on dilemmas scientific argumentation is essential, however including bioethical questions in the classroom will develop other useful skills and prepare students to become citizens who make decisions, such as public policy decisions, for example the ones related with allocating public resources, using scientific data, from different sources, and multiple bioethical views (gutierez 2015). this learning proposal considers that students can understand the complex nature of doing science focusing in some features of science (matthews 2012) such as values in science and socio-scientific issues, and therefore the need of a bioethics reflexion during any scientific activity. since k-12 sciences’ curriculum, in portugal, does not include bioethics content in any discipline explicitly (except one learning objective on 9th grade’s curriculum, *, in table 1), science teachers need to make an effort to include it. recently a portuguese project “education for values and bioethics” addressed some of the known gaps in the middle school curriculum (nunes et al. 2015). http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 124 for undergraduate students, my proposal combines inquiry learning methods with the aim of promoting the discussion of bioethics issues in accordance to unesco bioethics core curriculum already adopted by twenty universities throughout the world (darwish 2015) in a single discipline common to all science minors of a faculty or department of sciences. some of the themes that are addressed in this proposal are: ecology and environment ethics, infectious diseases and vaccination, water for all, intellectual property, genomes and patents, biotechnological advances (genetic modified organisms and synthesis of genomes), future generations, climate changes, exploitation of natural resources, biomedical advances and human rights, authorship and contributions in scientific publications, biobanks and dna databases. in summary, this study may constitute an example to facilitate the implementation, by k 12 teachers, of active inquiry strategies, using features of science such as values in science and socio-scientific issues, and focused on the discussion of concrete ethical issues facing humanity. it also constitutes a proposal of integrating bioethics in undergraduate sciences’ curricula. 2. proposed methodology 2.1. teaching-learning argumentation in k-12 classes, science teachers’ focus – regarding bioethics should be to teach students how to compose a good argument, that includes reasons, facts, evidence to support it and persuasive language, in accordance with other authors, such as in the ideas project (in: https://www.beep.ac.uk/content/284.0.html). i propose the use of inquiry learning-teaching methods to practice argumentation, which will contribute to develop critical thinking skills (chowning et al. 2012) and should also promote the respect for different views of others. college students should know how to compose arguments and i propose the introduction of a curricular discipline of bioethics, for first year undergraduate students enrolled in any faculty or department of sciences, that should focus in promoting the critical thinking of relevant themes and development of decision-making skills. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ http://www.beep.ac.uk/content/284.0.html) multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 125 2.2. learning features of science: socio-scientific issues (ssi) and values in science features of science, that include nature of science topics, constitute an important theme in science education (matthews 2012) that allows students to understand the complexity of science. in this proposal we focused on two features of science socio-scientific issues (ssi) and values in science and also on the use of inquiry-based learning strategy. 2.2.1. socio-scientific issues (ssi) the learning-teaching strategy using socio-scientific issues (ssi) is similar to the strategy of science-technology-society (sts), however ssi includes argumentation and reasoning skills about the benefits and detriments of each issue. therefore, learning bioethics can be done using collaborative work focusing on ssi and promoting students’ claims and justifications (osborne 2010). ssi provide a real-world context for bioethics discussions mainly by including biological advances and the relationship between science and society, which may help k-12 students to engage with the theme, and make decisions on the impact of science, as others described (gutierez, 2015). 2.2.2. values in science some views of science consider that it is based only on facts and is value-free, however recent sociological views of science proposed a science framework including the relation of science-technology-society, the ssi, and science methods, e.g. values in science (allchin 1999). citizens in order to be able to discuss scientific themes, need to learn about the values in science. the use of episodes of history of science to teach these values is suggested, such as the one of pasteur and his “germ theory of disease”. the main values in science, or the epistemic values expressed by science, are (allchin 1999; serageldin 2011): clearness, honesty, collaboratively, acknowledgment of authorship, skepticism, creativity, reliability, testability, accuracy, precision, simplicity, repeatability, novelty, universalism, organized skepticism, interventional experiments and controlled observation. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 126 3. contents: selecting themes and integration in curricula’s themes 3.1. k-12 students all students should learn by the end of k-12 education, in natural sciences and biology disciplines, how to articulate our responsibilities towards nature and future generations when focusing in socio-scientific issues such as biotechnology (o’mathúna 2007) and use of natural resources. biotechnology, mainly by genetic engineering, can ensure the production of saline water-, drought-, herbicidal-resistant plants, as well as producing plants with vitamins-enhanced content (zimmermann and hurrell 2002). while the production of herbicide-resistant plants will benefit agriculture, this will not benefit neither the consumers or the future generations by polluting the soil, and therefore it constitutes a conflict. the production of saline waterand drought-resistant plants will be important for present and future generations living in locations with these conditions, such as in africa. another molecular biology technological advance is related with a novel scientific area named synthetic biology and artificial life: the first synthetic cell was made in 2010 (gibson et al. 2010) and the artificial bacteria with minimal genome (syn 3.0) was obtained recently (hutchison et al. 2016); these two examples are good to promote discussion in the classroom if there should be a limitation to science work, and if so who should decide. another important issue is vaccination: should it be mandatory for all students? and if so, benefits against infectious diseases and potential risks should be learned during k-12 education in natural sciences (at middle school level) and biology (at high school, for example 12th grade). intellectual property, including authorship, is a core issue of ethics and bioethics and should constitute one of the themes of k-12 education since students are frequently asked to perform collaborative work, such as finding digital resources to answer questions or information about a theme proposed by the teacher. k-12 students should know how to cite the work of others as well as defining the authorship of the group’s work and the adequate use of creative commons denominations (in: https://creativecommons.org). http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 127 other issues to be discussed are ecology and environmental ethics, such as natural resources that includes biodiversity and species protection against extinction and geodiversity and non-renewable, in our lifetime, geological resources, such as oil and charcoal. another natural resource is water, that is important for all life forms, and is considered a human right (in: http://waterforall.org). the protection of biodiversity issue also includes the discussion about the risks of introduction of invasive species, genetic modified and synthetic organisms and biobanks, for example of germplasm. the discussion of the respect for future generations rights includes all the alterations made in natural ecosystems and the ones done at a global level, such as climatic changes. the advantages and disadvantages of dna databases can be addressed with a role playing activity dividing students in two groups using either for or against claims of asking all citizens to provide dna for a database; this activity can address another objective of learning to be tolerant to views of others. some biomedical issues should also be addressed with k-12 students as mentioned by other authors “education for health” (nunes et al. 2015), and such as informed consent and biomedical tests. the proposal of ssi to be included in each curriculum’s theme is presented on table 1. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 128 table 1. overview of science topics and the corresponding ssi to be included in k-12 education. discipline (grade) science topic in the curriculum bioethics theme – proposal of ssi to be included in the curriculum natural sustentability on earth protecting geodiversity and geological processes protecting biodiversity and biosphere (7th and 8th grades) climate changes; water as a right future generations scientific and technological development (8th grade) biotechnology and synthetic biology intelectual property and patents sciences (middle individual and public health (9th grade) vaccination school) reproductive system (9th grade) ivf human cloning genetics, its applications and bioethical issues* (9th grade) genetic tests; dna databases consent; privacy; confidentiality transgenic food biology & geology (10th and 11th grade), biology (12th grade) and geology (12th grade) (high school) human interventation on earth’s subsystems (10th grade) future generations and sustainable development water as a right exploitation versus preservation of natural resources hidrographic basins (11th grade) sustainable exploitation of geological resources (11th grade) reproduction (biology 12th grade) ivf human cloning genetic heritage (biology 12th grade) intellectual property and patents food production & sustainability (biology 12th grade) biotechnology and transgenic food immune system (biology 12th grade) vaccination preservation of environment (biology 12th grade) sustainable development water exploration (geology 12th ) water as a right http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 129 on table 1, one can see some repetition of themes comparing middle and high school curricula, however since the 9th grade is the last common year for all students these topics should be mainly approached by the end of middle school. therefore, for the students choosing biology & geology (10th and 11th grades), biology (12th grade) and geology (12th grade) the ssi themes should be explored with more detail and complexity. bioethics principles, such as respect for persons, beneficence/nonmaleficence and justice (chowning et al. 2012; unesco 2011) should be included in 7th grade. in order to learn about values in science i propose the use of the historical episode of louis pasteur and his “germ theory of disease”. 3.2. science undergraduates in a bioethics discipline this proposal includes inquiry-based learning in a bioethics discipline to be created in the first year of each bsc. learning the values in science may help to decide between two competing hypotheses for the best to include in a group report or as self-reasoning; and the use of the historical episode of louis pasteur and his “germ theory of disease” is also recommended as a theme of discussion with increased detail. intellectual property, patents and authorship are themes that should be discussed by undergraduates, allowing learning the benefits and bad consequences of each. controversial issues that should be addressed are associated with patenting of biotechnological inventions, genome and dna tests. the “patentability” of the human genome is not possible since 2013 by a decision of the supreme court of usa that established that dna is not patentable (zaragoza 2015). the discussion of genetic tests (for example, for diseases) and dna databases (for example in the search of missing persons and any crime scene), should include consent, privacy and confidentiality. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 130 the three companies that have more patents related with food production are: monsanto, usa, dupont, usa, and syngenta, switzerland. recently, syngenta obtained the patent of natural pepper, that were obtained by farmers’ breeding activity, without genetic engineering, in the last millions of years. and recently the european patent office revoked monsanto patent on melons (in: http://no-patents-on-seeds.org). it is important to discuss arguments for and against, such as the costs – for the farmer – of seeds obtained by genetic engineering with patent will be higher and in some cases it makes impossible to use it; however, only transgenic plants survive in soil-containing herbicides, such as glyphosate. the protection of biodiversity issue also includes the discussion about the risks of introduction of invasive species and genetic modified and synthetic organisms. the discussion of the respect for future generations rights includes all the alterations made in natural ecosystems and at a global level, such as climatic changes. the use of animals in medical experiments is also important to discuss facing both the benefits to humans and the welfare of the animals. databases obtained by dna profiling are also controversial with benefits in searching for missing persons and as a proof for forensics analyses of any crime scene. another issue that should be addressed is the “right2water” an initiative of european citizens that constitutes the first petition with more than 1.5 million signatures, by citizens of all countries of the european union, that demanded the water and the sanitation as a human right and water as a public good. in the bioethics discipline for all the bsc courses it should also be learned how does an ethics committee work since any of the science professionals may be asked to integrate one. the curriculum should be different between different bsc courses on the deontological issues, since rights and duties are different for each future professional, such as, for example, a biologist and an astronomer, so this module/lesson should be taught by different science professionals. medical bioethics contents should be included in another discipline http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 131 that should constitute an option for senior biology students, that is not in the scope of this paper. learning about the values in science and some examples of misconduct (turrens 2005) are important topics to be addressed so that science professionals are ´able to analyze a research paper and evaluate its relevance. 4. discussion and future perspectives since bioethics can have different meanings for different authors, and is still under discussion within its field (irrazábal 2015), i decided to focus on the contents and on a methodology that can be used in classes to prepare students to make decisions as citizens, as consumers, and as scientists. others have introduced some of these topics as a discipline named “introduction to bioscience ethics” (van roy and pollard, 2002). in portugal bioethics is not required to be included in the curriculum for many biology undergraduate studies. this proposal consists of ill-structured problems about each theme that should motivate students to research about the theme, in small groups, and present, to the class, different arguments and views. an inquiry-based learning strategy should be used, such as problem based learning used by others with success (van roy and pollard 2002). according to other authors integrating socio-scientific issues constitute an approach that enhance the bioethical decision-making of high school students (gutierez 2015). the main objectives of this proposal is that students, at each level, from middle school to undergraduate studies, understand the most important controversies in bioethics and acquire tools to analyze, in the future, bioethical arguments. this strategy may also increase motivation in students and their engagement with science themes, and prepares students with reasoning and justification skills as described by others (chowning et al.2012). bioethics issues are included in science literacy since global issues, such as, for example, climatic changes, vaccination and the use of nuclear energy, should be discussed by all http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 132 citizens, at a regional and a global level, and only upon reaching a bioethical consensus the scientific advances should be either applied or not (van roy and pollard 2002). it is important that future scientists in their activity when facing an ethical decision take in consideration respect for others, minimization of harms and maximization of benefits, fairness and authenticity, and these principles can be learned when facing several bioethical themes in class (loike et al. 2013). others have shown positive results by students of different bsc that were able to engage in all themes (loike et al. 2013; van roy and pollard 2002). bioethics as a discipline in all bsc – in the first undergraduate year in a faculty or department of sciences will prepare students as scientists able to distinguish scientific and bioethical questions, as well as able to address both and to conduct responsible research. this paper constitutes a proposal of what can be established in the future in the faculty of sciences, and may constitute an example for other departments or faculties, as well as for high school teachers to be able to include the themes in the curriculum. 5. acknowledgments and contributes the views expressed in this paper are the author’s own and do not necessarily reflect the present view of faculdade de ciências, universidade do porto. 6. references allchin, d. (1999). values in science: an educational perspective. science & education, 8, 1-12. chowning, j. t., griswold, j. c., kovarik, d. n. and collins, l. j. (2012). fostering critical thinking, reasoning, and argumentation skills through bioethics education. plos one, 7, e36791. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 133 darwish, b. (2015). promoting the future of bioethics. in solinís (ed.), global bioethics: what for? (p107-110). unesco. gibson, d. g., glass, j. i., lartigue, c., noskov, v. n., chuang, r. y., algire, m. a., benders, g. a., montague, m. g., ma, l., moodie, m. m., merryman, c., vashee, s., krishnakumar, r., assad-garcia, n., andrews-pfannkoch, c., denisova, e. a., young, l., qi, z. q., segall-shapiro, t. h., calvey, ch, parmar, p. p., hutchison, c.a. iii, smith, h. o. and venter, j.c. (2010). creation of a bacterial cell controlled by a chemically synthesized genome. science, 329(5987):52-6. gutierez, s. b. (2015). integrating socio-scientific issues to enhance the bioethical decision-making skills of high school students. international education studies, 8, 1, 142-151. hutchison, c. a. iii, chuang, r. y., noskov, v. n., assad-garcia, n., deerinck, t. j., ellisman, m. h., gill, j., kannan, k., karas, b. j., ma, l., pelletier, j. f., qi, z. q., richter, r. a., strychalski, e. a., sun, l., suzuki, y., tsvetanova, b., wise, k. s., smith, h. o., glass, j. i., merryman, c., gibson, d. g. and venter, j. c. (2016). design and synthesis of a minimal bacterial genome. science, 351(6280): aad6253. doi: 10.1126/science.aad6253. irrazábal, g. (2015). on the emergence and consolidation of bioethics as a discipline, as seen from a sociological perspective. história, ciências, saúde, 22, 4. keskin, m. o., samanci, n. k. and yaman, h. (2013). argumentation based bioethics education: sample implementation on genetically modified organisms (gmos) and genetic screening tests. educational research and reviewes, 8 (16), 1383-1391. loike, j. d., rush, b. s., schweber, a. and fischbach, r. l. (2013). lessons learned from undergraduate students in designing a science-based course in bioethics. cbe – life sciences education, 12, 701-710. matthews, m. r. (2012). changing the focus: from nature of science to features of science. in m. s. khine (ed.), advances in nature of science research. dordrecht: springer. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6481 eissn: 2341-2593 sousa (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº1 (2017): 122-134 | 134 nunes, r., duarte, i., santos, c. and rego, g. (2015). education for values and bioethics. springerplus, 4, 45. o’ mathúna, d. p. (2007). bioethics and biotechnology. cytotechnology, 53, 113–119. osborne, j. (2010). arguing to learn in science: the role of collaborative, critical discourse. science, 328, 463-466. serageldin, i. (2011). the values of science. science, 332, 1127. turrens, j. f. (2005). teaching research integrity and bioethics to science undergraduates. cell biology education, 4, 330-334. unesco (2011). bioethics core curriculum, section 2: study materials. 92p. van roy, w. and pollard, i. (2002). teaching and learning about bioscience ethics with undergraduates. education for health, 15, 3, 381-385. zaragoza, (2015). dignity as the foundation for all human rights. in solinís (ed.), global bioethics: what for? (p47-51). unesco. zimmermann, m. b. and hurrell, r. f. (2002). improving iron, zinc and vitamin a nutrition through plant biotechnology. current opinion in biotechnology, 13, 142-145. http://dx.doi.org/10.4995/muse.2017.6481 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 30 feature analysis of the “customer relationship management” systems for higher education institutions de juan-jordán, hugo1*, guijarro-garcía, maría1, hernandez gadea, javier1 marketing department, esic business and marketing school avda. de blasco ibáñez, 55,46002 valencia, spain1 *corresponding author: email:hugo.dejuan@esic.edu; phone: +34 627468823 received: 2018-01-17; accepted: 2018-03-24 abstract universities, business schools and other higher education institutions all over the world are experiencing deep changes in the way they work and interrelate with their ‘clients’, i.e. students and their relatives, alumni and donors, faculty and staff members, because these constituents are demanding more attention through different channels as well as immediate response and service. technology materialized in the form of customer relationship management (crm) systems is the great promise for solving these demands. this article summarizes the features a crm system should possess to make educational institutions thrive in the current digital era, and points out the future trends on this topic. the final objective is neither an analysis of the applications available on the market nor a selection guide, but a recommendation for the end users to utilize a crm system when considering achieving some of the business needs implied in the features available on these crms. keywords crm, education, higher education institution, alumni, student lifecycle management https://doi.org/10.4995/muse.2018.9232 mailto:hugo.dejuan@esic.edu multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 31 1. introduction as (parvatiyar and sheth, 2002) summarize in their article, customer relationship management (crm) is as a comprehensive strategy and the process of gaining, retaining, and partnering with segmented customers to generate superior value for the organization and the customer. it involves the integration of marketing, sales, and customer service functions of the organization to achieve greater efficiency and effectiveness in delivering customer value. nevertheless, when people talk about crm, they are usually referring to a crm system, a software solution that helps with contact management and personalized communications, sales and service management, productivity, and more (salesforce, 2018). so, a customer relationship management (crm) system is a technology for managing all the relationships and interactions of an organization with its customers and potential customers with the purpose of improving business relationships. summarizing, a crm system helps organizations to stay connected to customers, streamline processes, and improve profitability (salesforce, 2018). one differential characteristic of higher education as a service industry that makes crm an ideal strategy is the long-term commitment that students make when they enroll in a university (meyliana, sablan, hidayanto, and budiardjo, 2017). other industries do not have the opportunity to relate to a customer's lifecycle that lasts several years. on the other hand, it is a good practice to consider students as customers since this provides a competitive advantage and enhances an institution’s capacity to attract, retain and serve its students or customers (seeman and o'hara, 2006). when we understand students as customers a crm system helps to achieve student satisfaction (adikaram and khatibi, 2016), loyalty, retention and high service quality (badwan, al shobaki, abu naser, and abu amuna, 2017). higher education institutions are also recognizing the importance of understanding the characteristics and requirements of constituents, especially prospective students, and building stronger relationships with them (lang and pirani, 2014). as another https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 32 example of value, an effective crm programme to improve service quality may induce positive advocacy behaviour from its students (wali and wright, 2016). 2. main processes and typical departments using crm solutions specialized in higher education institutions embracing the opportunities presented by new digital technologies is one of the most urgent challenges companies face today (singh and hess, 2017). universities and other higher education institutions (hereinafter hei) have to deal with the deep changes in the way they work and interact with their ‘customers’, i.e. students and their relatives, alumni and donors, academy and staff members, because these constituents are demanding more attention through different channels and immediate response and service (van vugt and knasys, 2015). according to the study of the american association of collegiate registrars and admissions officers (aacrao, 2015), the hei departments dealing with crm are: • recruiting • admissions/registration • advising • student life • housing • student support services (e.g., tutoring, trio, etc.) • registration • alumni/development • career services/employment https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 33 the typical user cases for a crm in a hei, as detailed by (rigo, pedron, caldeira, and silva de araújo, 2016), are: enterprise relationships, marketing campaigns (personalizing marketing and communications based on student characteristics and actions as well as tracking prospective students’ on-site and online interactions), high-school relationships, student/alumni relationships, internal and external events, university relationships, internal and external communication (building communities), leads management (identifying effective recruiting practices and monitoring student progression from recruitment through enrollment), social networks, crowdsourcing, research centres relationship. moreover, crm systems improve operations and processes (e.g.: admissions staff can automate some tasks and actions such as e-mail or sms communications, freeing their time for a better service such as personal interactions with recruits) as (lang and pirani, 2014) stated. since these are many good examples of possible uses of crm, according to the studies of both the american association of collegiate registrars and admissions officers (aacrao, 2015) and (van vugt and knasys, 2015), crm systems are most likely to be used mainly to support admissions/registration and recruiting, followed by alumni/development, with a poor use of other departments. nevertheless, the crm-related institutional activity is now expanding and diversifying as the management progressively recognizes new user cases and benefits (lang and pirani, 2014). 3. reviewing current crm solutions specialized in higher education institutions when selecting the main crm solutions for higher education institutions with the purpose of analyzing them and finding the best positioned ones, criteria like previous selections of crm systems from other studies (lang and pirani, 2014; prado, 2016; van vugt and knasys, 2015) as well as internet search (google queries on “crm higher education" throws 708.000 results in january 2018) have been considered. with these criteria in mind, the solutions analysed in this study have been: https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 34  radius by hobsons (https://www.hobsons.com/emea/solutions )  ellucian crm (https://www.ellucian.com/crm-for-higher-education/ )  campus nexus crm (https://www.campusmanagement.com/products/radius-bycampus-management/ )  jezebel internet campus solution (https://www.jenzabar.com/blog/2014/01/02/jenzabar-internet-campus-solution/ )  jezebel higher reach crm (https://www.jenzabar.com/higher-educationsolutions/continuing-education-workforce-development/jenzabar-higher-reachcrm/ )  radius by campus management (https://www.campusmanagement.com/products/crm-for-higher-education/ )  salesforce for high education (https://www.salesforce.com/solutions/industries/higher-ed/overview/ )  one world student information system (http://www.oneworldsis.com/ )  azores crm for student recruitment (https://azorus.com/ )  maximiser crm (https://www.maximizer.com/industry/higher-education-crm/ )  edurme educational relationship management engine (https://www.encamina.com/edurme-dynamics365-educacion-crm/ )  leadsquared (https://www.leadsquared.com/higher-education-crm/ ) to perform this study, the websites of the abovementioned solutions have been visited, the product sheets downloaded, the video demos watched and the trials tested where available. https://doi.org/10.4995/muse.2018.9232 https://www.hobsons.com/emea/solutions https://www.ellucian.com/crm-for-higher-education/ https://www.campusmanagement.com/products/radius-by-campus-management/ https://www.campusmanagement.com/products/radius-by-campus-management/ https://www.jenzabar.com/blog/2014/01/02/jenzabar-internet-campus-solution/ https://www.jenzabar.com/higher-education-solutions/continuing-education-workforce-development/jenzabar-higher-reach-crm/ https://www.jenzabar.com/higher-education-solutions/continuing-education-workforce-development/jenzabar-higher-reach-crm/ https://www.jenzabar.com/higher-education-solutions/continuing-education-workforce-development/jenzabar-higher-reach-crm/ https://www.campusmanagement.com/products/crm-for-higher-education/ https://www.salesforce.com/solutions/industries/higher-ed/overview/ http://www.oneworldsis.com/ https://azorus.com/ https://www.maximizer.com/industry/higher-education-crm/ https://www.encamina.com/edurme-dynamics365-educacion-crm/ https://www.leadsquared.com/higher-education-crm/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 35 to summarize the main features, the items have been generalized enough to include the similar features of different systems in just one single item. 4. results the main features of current crm systems for heis are summarized in the next table. the descriptions, in some cases, have been extracted exactly as defined by their producers on the website or in the product sheet of the analyzed crm systems: table 1. results obtained from the analyzed crm features. feature description lead deduplication ensures prospective student’s queries and activities to be tracked under a single record – even if they use multiple channels of communication (email, web-form, phone call, etc.). complete lead profile gets detailed insights into the lead's interests, preferences and hobbies by tracking their activities; has access to the prospective candidate`s social profile. lead distribution instantly distributes student enquiries to the right departments, based on pre-defined triggers. this helps reduce response time and improve student experience. knowing leads inside out tracks prospective students in order to find out their interests and preferences (course of choice, campus location, communication channels etc.). complete website and communication tracking. https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 36 sending leads the right offers the most effective and relevant offers are sent to prospective students at the right time, for instance, scholarship details if they bounce off after viewing the course fee details. reaching out over the right channels these offers are shared with the prospective students across different communication channels email, sms, social media retargeting (facebook etc.), push notifications etc. getting communication insights refers to getting detailed insights into how many prospects are interacting with marketing messages and offers. for instance: email open rates, click rates, best subject lines, best devices etc. setting context to the conversation admissions team has all the information they need to engage in enticing conversations with the prospective students. lead scoring ranks prospect students against a scale that stands for the perceived value each lead represents for the institution. the resulting score is used to determine which leads will engage in order of priority. helps admission officers prioritize their efforts, improves efficiency, and contacts the most relevant prospects first. task assignment sets tasks for the admissions team, when a relevant event occurs. ex: if an old lead who is a high-quality prospective student looks at the courses again, the task of calling them immediately is set. action enablers inbuilt action enablers make it very easy for admissions team to act on the system's recommendations. connected teams a common tool for the institution's marketing and admissions teams that keeps them on the same page, which allows them to coordinate better and more efficiently. https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 37 mobile mobile crm for officers attending education fairs and other college-student interaction events (e.g.: registering enquiries onthe-spot). accesses the system anytime, anywhere, via table or smart phone. complete admissions insights tracks all the marketing, course, and admissions team´s performance. gets complete insights into the performance of teams, courses, campuses etc. integration with other systems and apps integrates with other information systems and apps of the institution such as call centres, email systems, chats, campus on line, student information system, social media apps, zapier/iftt, etc. lead nurturing develops relationships with potential students at every stage of the recruiting and admissions funnel, and through every step of the student's journey. it focuses marketing and communication efforts on listening to the needs of prospects and providing the information and answers they need. social listening monitors digital conversations to understand what students, alumni and other constituents are saying about the brand and institution online. business process automation integrates all the tasks, states, data and insights of every business process in one intelligent and centralized form. preconfigured process management modules functional modules such as:  student accommodation management  disability service management https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 38  assessment service management  employability and placement management  student incident management document management integrated with student info stores, manages and tracks files and digital documents or images related to the student. reporting and analytics real-time reporting and analytics tools that help institutions generate actionable data that can be used to enhance ongoing recruitment marketing activities, report on enrolment and yield, or prove roi of the higher-education crm efforts. alumni engagement and donor management maintains the connection between the institution and alumni to gain donations for the institution, provide a career path for recent graduates, support recruitment efforts, or return to school for additional training and education. automated scheduling manages meetings, events, recruitment travel, and more; completes with build-in reminders. automated data import and export manages lists, creates and sends personalized communications, monitors progress, and updates data without manual uploads. self-service student portal a customizable, web-based app which displays information relevant to the student’s interest with data integration both preapplication and post-application. multi-channel meetings timed, on-demand, web video interviews viewable and trackable from inside the crm system. the interviews include timed written https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 39 responses, collaborative comments and ratings for each response and applicant. student segmentation data segmentation functionality through the use of conditions and expressions within a filtering engine. custom communications delivers personalized content and craft the ultimate 1:1 student experience. 4.1 trends the current digital era is supposing some crucial changes such as an accelerated and deep technological change; an evolution from a physical world to a world of knowledge, a fluid dissemination of information due to globalization; and finally, a more distributed and less hierarchical organizational structure (de juan-jordan, palacios-marques and devece, 2018). the answer to those deep changes is what we call “digital transformation”, able to quickly evolve activities, processes, competencies and business models to take full advantage of those changes and opportunities brought by digital disruption on society, market and its different industries (demirkan, spohrer and welser, 2016), such as education. one of the focuses of the digital transformation is on customer relationship, social media and multichannel management (singh and hess, 2017). so, by mapping customers with students in the world of higher education, social crm can give universities the chance to better connect with students who are accustomed to using social platforms, since they are primarily from the millennial generation and grew up in the era of web 2.0 (meyliana et al, 2017). social crm is an evolution of crm systems that now includes social media as an important component that provides a way to manage the student engagement throughout the student’s academic cycle (meyliana et al, 2017). social crm is clearly one of the trends in crm systems. https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 40 customer satisfaction should also receive a boost as universities will be able to deliver “service personalization” since the social crm platform will be able to better determine student’s needs (wali and wright, 2016). more and more functionalities and apps related to social media and social engagement will be added in the future to crm solutions for higher education institutions. apps as complements to crm, social media integrated in crm and personalization tools are a trend in the crm for the hei world. some of the crm analyzed already include functionalities regarding student success and retention. for example, ellucian crm lets factor student characteristics and behavior over time into course and success scores to accurately identify risk, as long as automated and tailored intervention strategies after identifying at-risk students. although some specific “student success” solutions can be found separately as a product, including these functionalities as part of the crm for heis seems like a tendency. as the digital disruption is transforming education, the presence of technologies such as the internet of things (iot), the internet of everything (ioe), social media applied to the classroom, 3d printing, web 2.0 and web 3.0, chatbots or the artificial intelligence, for instance, are becoming more and more common in higher education (thomas, 2017) and those innovations are also a trend in the crm systems. in the case of einstein, an artificial intelligence module over salesforce crm, it learns from all that data to deliver predictions and recommendations based on the institution`s business processes (henschen, 2017). so, it is more than conceivable that artificial intelligence will be a common component in higher education’s crm systems in the near future. possible uses may include: matching the best academic product for the right student with the optimal argument, predicting problems with the student, identifying the best moments and touchpoints with the constituents, etc. chatbots are also a predictable component of hie’s crm systems, as long as potential students may want to get information about the academic products through different https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 41 channels and different moments or ask questions to the institution’s chatbot in natural language or just report a problem through a conversation. 5. conclusions a crm solution helps institutions to focus on the organization’s relationships with individuals — including customers, service users, colleagues, or suppliers — throughout their lifecycle, including finding new customers, winning their business, and providing support and additional services throughout the relationship (salesforce, 2018). the benefits of implementing crm in a hei include a student‐centric focus, improved customer data and process management, increased student loyalty, retention and satisfaction with the college's programs and services (seeman and o'hara, 2006), but even though the hei's main customer is the student, there are others stakeholders that a crm project must consider (seeman and o'hara, 2006), such as prospective students, alumni, faculty and staff, and current and prospective donors (lang and pirani, 2014). educational institutions leaders, by adopting crm initiatives, try to increase performance, promote better management practices, and improve the institution’s relationship with current and potential students, especially in higher education (badwan et al, 2017) and that may be possible because of the features and functionalities that crm systems enable for them and which have been listed in this article. the wide range of features of crm systems give many ideas of the business needs that hei’s management should implement in their organizations. this study has an ephemeral validity since crm systems are evolving at an accelerated pace and, in a few months, there may be new solutions in the market or new technologies applied to the crm concept that today are even impossible to guess. another limitation of this study is the bandwidth of systems that we can analyze, only focused on the best-known solutions in the market, even though innovation happens not only in the most consolidated https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 42 companies but also in the least known and star-ups. future research could contrast these results with the analysis of new tools and crm systems all over the world. 6. references adikaram, c., and khatibi, a. (2016). the relationship between customer relationship management and customer satisfaction. international journal of arts and commerce, 5(2/8), 6995. http://ijac.org.uk/images/frontimages/gallery/vol._5_no._2/8._69-95.pdf american association of collegiate registrars and admissions officers. (2015). 2014-2015 state of crm use in higher education report. aacrao. http://www.aacrao.org/docs/defaultsource/pdf-files/state-of-crm-use-in-higher-education-report_bw.pdf?sfvrsn=2 badwan, j., al shobaki, m., abu naser, s., and abu amuna, y. (2017). adopting technology for customer relationship management in higher educational institutions. international journal of engineering and information systems (ijeais), 1(1), 20-28. https://hal.archives-ouvertes.fr/hal01500365 de juan jordan, h., palacios-marques, d., and devece, c. (2018). leadership styles and entrepreneurship. en d. r. a. tur porcar, inside the mind of the entrepreneur. contributions to management science (págs. 207-218). cham: springer. doi:10.1007/978-3-319-62455-6_15 demirkan, h., spohrer, j., and welser, j. (2016). digital innovation and strategic transformation. it professional, 18(6), 14 18. doi:10.1109/mitp.2016.115 henschen, d. (2017). inside salesforce einstein artificial intelligence. constellation research, inc. https://www.salesforce.com/content/dam/web/en_us/www/documents/reports/insideeinstein.pdf lang, l., and pirani, j. (2014). maximize institutional relationships with crms. louisville, co: educause center for analysis and research. https://library.educause.edu/~/media/files/library/2014/7/erb1409-pdf.pdf meyliana, sablan, b., hidayanto, a., and budiardjo , e. (2017). the critical success factors (csfs) of social crm implementation in higher education. research and innovation in information systems (icriis). langkawi, malaysia : ieee. doi:10.1109/icriis.2017.8002528 https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9232 social and technological sciences issn: 2341-2593 de juan et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc and tec. sci. vol. 5 nº 1 (2018): 30-43 | 43 parvatiyar, a., and sheth, j. (2002). customer relationship management: emerging practice, process, and discipline. journal of economic and social research, 3(2), 1-34. prado, a. (2016). plataformas tecnológicas para tic360. del crm al lrm. ciudad real. spain: uclm. https://ruidera.uclm.es/xmlui/handle/10578/9032 rigo, g.-e., pedron, c., caldeira, m., and silva de araújo, c. (2016). crm adoption in a higher education institution. jistem journal of information systems and technology management, 13(1), 46-60. doi:10.4301/s1807-17752016000100003 salesforce. (2018). salesforce. retreived on 01/01/2018, from https://www.salesforce.com/crm/what-is-crm/ seeman , e., and o'hara, m. (2006). customer relationship management in higher education: using information systems to improve the student-school relationship. campus-wide information systems, 23(1), 24-34. doi:10.1108/10650740610639714 singh, a., and hess, t. (2017). how chief digital officers promote the digital transformation of their companies. mis quarterly executive, pp1-17. http://misqe.org/ojs2/index.php/misqe/article/viewfile/685/454 thomas , d. (2017). digital disruption: a transformation in graduate management online education. en s. b. khare a., phantom ex machina (págs. 223-233). springer. doi:10.1007/978-3319-44468-0_15 van vugt, t., and knasys, m. (2015). customer relationship management(crm) systems in higher education. the guide to selecting a crm system for higher education institutions. ieandd solutions bv and studyportals bv. http://www.studyportals.com/wpcontent/uploads/2015/12/crm-systems-in-higher-education-main-report.pdf wali, a., and wright, l. (2016). customer relationship management and service quality: influences in higher education. journal of customer behaviour, 15(1), 67-79. doi:10.1362/147539216x14594362873532 https://doi.org/10.4995/muse.2018.9232 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 determinants of entrepreneurial intention among university students: case of albania e. garo*, v. kume, s. basho university of tirana, school of economics, tirana, albania * corresponding author: email: elona_konomi@hks06.harvard.edu; phone: +355692090756 received: 2015-04-03; accepted: 2015-05-08 abstract in albania, during the last decades entrepreneurship has been in focus. it is commonly agreed that developing entrepreneurship in the long term would be translated into sustainable economic development. albanian economy, being an economy in transition greatly and urgently needs to invest into increasing the number of entrepreneurs in the country. entrepreneurial intention is considered to be really important. thus, the purpose of this paper is to observe the determinants of entrepreneurial intention in albania. the target is university students. many programs that foster entrepreneurship such as incubator centers, business plan competitions along with new educational practices have been developed. in addition, even though studies conducting in this field are few, albanian education institutions have shown a willing to study the factors influencing entrepreneurship development. the reason is the belief that developing and supporting entrepreneurial activities among youth will positively influence the economy of the country in terms of economic growth. this study widely represents the albanian students and shows the deteminants of entrepreneurial intention they have. hopefully the findings of this study would be useful to the policy makers and the albanian government to undertake effective policies focused on entrepreneurial activity; targeting economic development of the country. keywords entrepreneurship intention, entrepreneurship determinants, students, albanian universities, albania. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 176 http://polipapers.upv.es/index.php/muse mailto:elona_konomi@hks06.harvard.edu multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 1. introduction entrepreneurship is the engine of any economy, having a positive impact on emerging new and innovative business ventures. these new business opportunities play an important role in job creation and economic development of a country. historically, economists (cantillon, 1755, weber, 1930, cole, 1965) have fully supported the role of entrepreneurship in economic growth and creation of new markets. with the average age of 30 and with 25% of the population in the age group 15-29 years, albania is one of the countries in europe where the population is young. however, as a result of the reduction of mortality and the number of births in recent decades, the albanian population is expected to grow old in a manner visible in the coming years. the albanian labor market faces problems, same ones such as the economies of other countries in transition. high levels of unemployment, together with the participation of cheaper labor and lower levels of employment characterize the labor market of young albanian people. in 2011, the level of employment of the young people (aged 15-29 years) was 42.8% compared with 68.1% for the age group 30-64 years, while unemployment reaches a level of 21.5%, double the value of the unemployment of the adult population.1 over the last five years has been an increasing awareness of the albanian government of the need of entrepreneurship activities. the country has undertaken many reforms to reduce administrative barriers to business through simplification of administrative procedures. the creation of national registration center, a one stop shop is a significant effort. registration at this center are made within one working day and at a minimum cost of 0.81 euros, offered in 32 offices throughout the territory of the republic of albania. regarding the simplification of doing business, with the support of donors, a significant effort is undertaken in public administration activities in four main areas: public 1 "youth employment and migration" report, country brief albania, ilo, mdg achievement fund garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 177 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 procurement, business registration, management tax and licensing. significant efforts are undertaken by the government of albania on the development of information technology to enhance the competitiveness of albanian businesses in regional and global market.2 despite government programs to mitigate this phenomenon, government alone cannot reduce unemployment in albania. there is still a negative perception from authorities, experts and ordinary people related the real contribution of entrepreneurs. as a result of this perception the decision of self employed is made only if people face a difficult situation in the job market where formal and sustainable job opportunities are missing. one of the long-term development priorities of the albanian government is fostering a culture of entrepreneurship, education and training. in particular, the young generation are the beneficiary of learning about entrepreneurship, the essential skills such as creativity, initiative, teamwork, risk taking, and sense of responsibility. world experience shows that governments in several countries have begun to develop policies that provide support for young entrepreneurs willing to support and facilitate them to successfully start the process. learning about entrepreneurship is now recognized as a key competence at the european level and in albania is included in the curriculum of schools in all levels. these processes include not only this, but also a variety of other programs. the aim is to encourage universities to offer specialized courses and programs on entrepreneurship in order to effectively impact the entrepreneurship activities in the country. albania has taken many steps to strengthen the education sector in the last 20 years. entrepreneurial education has been the main focus for both public sector and private education. higher education programs in albania have developed special courses related to entrepreneurial education, to encourage students to think and apply such initiatives in 2 strategy of business and investment, 2014-2020, ministry of economy, trade and entrepreneurship, 2013 garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 178 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 business oriented majors. however, it is unknown whether this knowledge will lead students to self-employment or other factors play a significant role. with the purpose of drafting effective programs, policymakers must recognize the factors influencing young people to become entrepreneurs. this study is undertaken to study the impact of knowledge, skills, previous experience and social norms to the intent of albanian students in the country level. it represents a broad study and the results will serve as a good start to provide input and ideas for policy makers, to influence young people become entrepreneurs and create the right environment in the development of entrepreneurial practices in albania. 1.1 study objectives the aim of the study is to determine the factors that influence the "entrepreneurial intent" among youth in university level. students who did participate in this study are exposed to entrepreneurship knowledge and skills and are towards graduation. so, through them the study determines the perceptions of students regarding entrepreneurship intent. it retests the psychological model of ajzen of planned behavior to identify factors that will influence students become entrepreneurs. as a result, findings of this study will be a valuable asset to policymakers in albania and in the region. 1.2 research questions the objective of this pilot study is to shed light on the following questions: • what do we know regarding the willingness of young entrepreneurs in albania? • what are the achievements of albania in developing entrepreneurial intent of youths? • how can we develop entrepreneurship in albania? • how young people can be influenced to practice this profession? garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 179 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 2. literature review the term entrepreneurship originated from the french word "entrepreneur"-a person who organizes and runs a business or businesses. entrepreneurs are known as self employed. there is a definition for entrepreneurs. researchers in different fields of study perceive and define it differently. an economist defines an entrepreneur as a man who does the combination of resources to make them available. as for the psychologist, entrepreneur is motivated by certain forces, such as the need to achieve or gain something, to experiment and to achieve certain purposes. for a businessman, an entrepreneur can be a risk taker, an aggressive competitors but can also be an ally, a supply source, a customer, or someone who creates wealth for others and finds the best ways to utilize resources, to reduce the losses and to provide for other work (hisrich, peters & shepherd, 2005). some people believe that entrepreneurs are born, not created. however, this myth is broken at the moment when many studies made in the past have reached consensus that can become an entrepreneur is not genetically inherited. everyone has the potential to become entrepreneurs, especially for those who have undertaken educational processes in universities. according to ajzen (1991), the intent is the immediate predecessor of behavior. the researchers stated that the behavior is not done without thinking but it is influenced by reason and information, and stimulated or discouraged by events, rewarding or not. individuals that wish to be self-employed are the one thatperceive it as a proper career (davidsson, 1995), as well as a way to achieve their own personal goals, to realize their ideas and achieve financial benefits (barringer & ireland, 2010: 31 ). entrepreneurial intent is defined as the desire of the individual to perform entrepreneurial behavior, to be involved in entrepreneurial activities, to be self-employed or to create new garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 180 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 businesses (dell, 2008). but this is not enough. it requires inner courage, ambition and desire to stay on your feet (zain, akram & ghani, 2010). birds (1988) proposed that entrepreneurial intent refer to the mental state of individuals that aims to create new enterprises, to develop new business concepts or to create new value in existing firms. it is an important factor associated with the creation of new enterprises and has significant impact on success, survival and growth of entrepreneurship. he suggested that the process will often starts based on the needs, values, desires, habits and personal beliefs entrepreneurs. researchers have shown empirically that entrepreneurial intent is valid predictor for entrepreneurial behavior as entrepreneurial actions fall under the category of the desired behavior. a study of entrepreneurs will provide valuable clues for researchers to understand the entrepreneurial process and to predict entrepreneurial activities in the best way by identifying ancestors of entrepreneurial intent (davidsson, 1995; bird, 1998; krueger et al., 2000; peterman & kennedy , 2003; linan, 2004; kolvereid & isaksen, 2006; krueger, 2007; dell, 2008; mohammad ismail et al., 2009). kolvereid findings and isaksen (2006) found that 297 founders of businesses, the intent to be self employed will define later choosing self-employment. individuals usually do not start a business as a reflex, they do it willingly and not accidentally (krueger et al., 2000; krueger, 2007). according to krueger (2007) the intent will serve as mediators between the entrepreneurial actions and outside factors (the characteristics, demographics, skills, social, cultural and financial support). they suggest that entrepreneurial intention explains the reasons why certain individuals tempted to start their own business before doing the evaluation of opportunities or to decide the type of business that should be included. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 181 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 education entrepreneurs trying to develop in participants, willing to display behaviors, knowledge and desire for entrepreneurial activity (linan, 2004). students are trained and prepared to enter the labor market in the specific area of their study. however if they will work for others or will become entrepreneurs (self employed and job creators) that is determined by their intent. many studies regarding the willingness of students to become entrepreneurs were undertaken. christian (2000) studied the willingness of students to become entrepreneurs and found that 65% of the respondents had it. study of christine (2004) regarding chinese students intent to become entrepreneurs showed that out of 105 survey, 33.66% had the intent and 13.86 had desire. the same kind of studies have been made by gerald (2006) who found that out of 194 respondents, 30.4% had the intent and 22% expressed desire. the study made by autio, keeley, klofsten & ulfstedt (1997) that investigates student entrepreneurial intent, in the context of different cultures, revealed that university environment affects student entrepreneurial confidence positively. educational support through professional education is an effective way to undertake the necessary knowledge regarding entrepreneurship. this is supported by a study made by wang and wong (2004) which stated that dreams of many students lose momentum by the lack of preparation by the academic institutions. the school and the education system also play a critical role in identifying and shaping the characteristics of entrepreneurs (ibrahim & soufani 2002). other studies have shown that entrepreneurial education, especially one that contains technological training, is key to innovative entrepreneurs (gallowawy & brown, 2002, garavan & o'cinneide, 1994). garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 182 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 3. methodology the objective of this research paper is to examine the entrepreneurship intent of young people in albania and students were individuals who became part of the study. this study is part of a large research work with focus on youths at university level which included students from university of tirana, university of elbasan, university of shkoder, university of vlora, university of gjirokaster, university of korca, university "aleksander moisiu", durres, university of new york, tirana and european university. the target group was selected on a broader bases including public and private universities in main cities of the country. entrepreneurial intention questionnaire-eiq (linan et al, 2006) is the instrument of data collection for this study. this questionnaire is evaluated by a series of studies made by linan and chen (2009) and was carried out in different countries and different cultures. regarding the use of this questionnaire in the albanian context there is no evidence. on the use of this questionnaire in this study were requested permission from the copyright holder. eiq questionnaire was created to study the willingness of young entrepreneurs gathering information on entrepreneurial activity, social norms, education and experience, skills, knowledge, objectives of entrepreneurship and the creation of an enterprise. all these factors are measured using a likert scale assessment with 7 points, and the positive response/negative in cases necessary. personal information of students and their contacts (in the case of enabling it to a long term study) were collected through the use of this tool. in total 497 students participated in the study. not all questionnaires are taken into account as a result of missing data in some of the categories. 4. study results the total number of valued observations in the study is 434. during this phase all respondents were studying in university programs, undergraduate or graduate levels, majored in business or related studies. out of all respondents 62.9% (n=273) of participants are females and the average age of participants is 23. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 183 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 the findings of the study show that 68% (n=296) of the students declare having entrepreneurship intention. such a value consists to be high considering that albanian economy is still in transition and people continue to be skeptic about entrepreneurship. overall perception of students about this career opportunity is positive. they feel able and willing to become entrepreneurs. according to the questionnaire the determinants were categorized as below and the results are included also below. 4.1 education and experience according to the study 50.2% (n=218) of students were studying in the bachelor level. majority of students, exactly 56% (n=243) were in the last year of study. among all participants 45.8% (n=199) of them have working experience, whereas among them only 18.2% (n=79) resulted to be self-employed. 4.2 entrepreneurial knowledge the respondents that positively answered the question if they know an entrepreneur were 355 students, which means that 81.7% of participants have been close to an entrepreneur or/and entrepreneurial activities. students exposed to such experiences is believed are influenced by them. another way getting entrepreneurial knowledge is through education on universities, where students get specific and important information related entrepreneurship and entrepreneurial activities. table 1 below shows the level of knowledge for training, financial and technical support in the case of albania. in general, participants show that they are able and have enough skills and knowledge to support undertaking entrepreneurial activities. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 184 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 table 1: entrepreneurial knowledge specific training for young entrepreneurs loans in specially favourable terms technical aid to start the business business centres consulting services in favourable terms mean 4.00 3.85 4.08 4.06 4.20 4.3 professional attraction students in order to evaluate the level of interest in entrepreneurship where asked questions such as the preference they have to start up a firm. they answer in likert scale is 4.41 or in percentage 63% which is considered a moderate preference. out of them 22.8% (n=99) of respondents evaluated starting up a firm as very much preferable. level of attraction towards being an entrepreneur is 5.43 in likert scale or 77.6 %. in addition, 137 participants or 31.5% showed a maximum preference towards becoming an entrepreneur. 4.4 social valuation the social valuation of entrepreneurial activities is positive, creating a healthy environment for entrepreneurship in albania. family is the most supportive of youths to become entrepreneurs valued approximately 83% pro entrepreneurship. according to this study albanian society gives value to the entrepreneurial activities. this point is supported from the values showing the approval of the students' decision to start up a firm from close family, friends, colleagues as described in the table 2. table 2 your close family your friends your colleagues and mates mean 5.79 5.11 4.82 garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 185 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 4.5 entrepreneurial capacity when students feel they have all the ingredients to become entrepreneurs they will undertake the initiative to create a new venture. entrepreneurial capacity is an important ingredient. if students feel being able to undertake such initiative, they will do so. the situation in albania is described in the table below. about all the aspects related entrepreneurial capacity that the respondents were asked, they felt being able as much as the values in likert scale that the table 3 contains. table 3 mean start a firm and keep it working would be easy for me 3.65 i’m prepared to start a viable firm 3.60 i can control the creation process of a new firm 3.94 i know the necessary practical details to start a firm 3.73 i know how to develop an entrepreneurial project 3.92 if i tried to start a firm, i would have a high probability of succeeding 4.28 4.6 entrepreneurial education there are researchers that believe everyone has the potential to become entrepreneur, especially for those who have undertaken educational processes in universities. to test this in the albanian case the questionnaire contained some questions that asked the students' belief if education matter in the issue of entrepreneurial intention. the findings are summarized in the table below. table 4 mean knowledge about the entrepreneurial environment 4.95 greater recognition of the entrepreneur’s figure 4.87 the preference to be an entrepreneur 4.87 the necessary abilities to be an entrepreneur 4.98 the intention to be an entrepreneur 5.22 garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 186 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 the situation above is promising. education does effect the preparation of young entrepreneurs. 4.7 role model analyzing all possible determinants of entrepreneurship intention, role model is a real important one. the family consists to influence young people on a great deal, considering sharing business know how, exposing to opportunities and in understanding benefits of such a career choice. approximately 60% of participants have their fathers being close to entrepreneurship. out of them 26% were self-employed, running their own businesses. in addition 19% of respondents' mothers were self-employed, whereas totally 42% of them were part of the private sector. among respondents' parents approximately 60% of them have university degree. the influence of parents in their children's career decision is very strong. thus we cannot ignore that they are somehow advised from their parents on the study business related major. combined this fact with the highest percentage of parents involved in entrepreneurship experiences is an influential ingredients for the entrepreneurial intention these students developed. 5. conclusion in albania, where this study is focused, it is observed that the entrepreneurial intention among youths exists. according to ajzen's theory of planned behavior the intent is the immediate predecessor of behavior. the intent is influenced by factors and stimulated by the events in life. thus entrepreneurship intention does so too. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 187 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 determinants of entrepreneurship intention consists to be many such as, education; working experience; entrepreneurial capacity, knowledge; social valuation and role models. education is really important and results highly influential. it exposes students to entrepreneurial activities, it teaches them knowledge and skills to become entrepreneur, and provides the whole setting to start a new venture. personal entrepreneurship experience fills students' capacity and makes them able to undertake entrepreneurial activities. in terms of role model parents' experiences involved in entrepreneurship is also influential into having entrepreneurship intention. to complete the list of determinants social norms should be highlighted. social evaluation represents the entrepreneurial environment, a healthy one as it is valuated from this new generation in the case of albania is favorable for having entrepreneurial intention among youth. however, in albania the unemployment level is considerable. for such a situation, the development of new entrepreneurs is the best possible answer specifically on the unemployment problem. this, because entrepreneurship is the best favorable solution for the economic struggle we actually are facing in general. the last paragraph of this study comes with suggestion for youths, readers especially policy makers. in order to develop effective programs, the factors that influence albanian youths that are discussed in this paper must be in focus. the problem students face even though, they follow three or four years business programs is that they do not undertake real steps into developing new ventures. albanian policy makers should undertake concrete actions to support student become entrepreneurs, to transform intent into behavior. also the good efforts undertaken to develop entrepreneurial intention should be continued and improved with future challenges. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 188 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 6. references hisrich robert, peters michael, shepherd dean "entrepreneurship", ninth edition, 2012, pp. 347-382 barringer bruce, ireland duane "entrepreneurship: successfully launching new venture, 3/e, 2010, 255-276. ajzen i. (1991) the theory of planned behavior. organizational behavior and human decision processes, 50, 179-211. http://dx.doi.org/10.1016/0749-5978(91)90020-t ajzen, "perceived behavioral control, self-efficacy, locus of control, and the theory of planned behavior1," journal of applied social psychology, vol. 32, no. 4, pp. 665683, 2002 http://dx.doi.org/10.1111/j.1559-1816.2002.tb00236.x cantillon, r. (1755) essai sur la nature du commerce en general, h. higgs, (1931) macmillan, london. cole, a.h. (1965) an approach to the study of entrepreneurship, in aitken, explorations in enterprise, harvard university press, cambridge, mass, 30-40. http://dx.doi.org/10.4159/harvard.9780674594470.c4 davidsson, p. (1995). culture, structure and regional levels of entrepreneurship. entrepreneurship and regional development, 7(1), 41-62. http://dx.doi.org/10.1080/08985629500000003 dell mcstay (2008), an investigation of undergraduate student self-employment intention and the impact of entrepreneurship education and previous entrepreneurial experience, phd, epublication@bond, faculty of business, technology and sustainable development, pp 138-147. zain zahariah, akram amalina, ghani erlane, entrepeneurship intention among malyasian business students, canadian social science, vol. 6 no. 3, 2010, pp 34-44 bird, b. (1988), "implementing entrepreneurial ideas: the case for intentions", garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 189 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/0749-5978(91)90020-t http://dx.doi.org/10.1111/j.1559-1816.2002.tb00236.x http://dx.doi.org/10.4159/harvard.9780674594470.c4 http://dx.doi.org/10.1080/08985629500000003 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3691 social and technological sciences eissn: 2341-2593 academy of management review, vol. 13 no.3, pp.442-54 kolvereid, l. & isaksen, e. 2006. new business start-up and subsequent entry into self-employment, journal of business venturing, 21(6), 866-885. http://dx.doi.org/10.1016/j.jbusvent.2005.06.008 krueger robert, hicks brian, patrick christopher, carlson scott, iacono william, mcgue matt, etiologic connections among substance dependence, antisocial behavior, and personality: modeling the externalizing spectrum, journa of abnormal pscychology, 2002, vol. iii, no. 3, 411-424. krueger r.f, markon k.e, patrick c.j, benning s.d, kramer m. linking antisocial behavior, substance use, and personality: an integrative quantitative model of the adult externalizing spectrum. j. abnorm. psychol. 2007;116:645–666. http://dx.doi.org/10.1037/0021-843x.116.4.645 li-án, f., educación empresarial y modelo de intenciones. formación para un empresariadode calidad, phd dissertation, dpto. economía aplicada i, universidad de sevilla, sevilla, 2004. li-án, "intention-based models of entrepreneurship education," applied economics, pp. 1-30, 2004. li-án, f. & rodríguez, j.c., "entrepreneurial attitudes of andalusian university students", 44thersa conference, porto (portugal), 25-29 august, 2004. li-án, f., martín, d. & gonzález, r., "characteristics of nascent entrepreneurs in germany",42nd ersa conference, dortmund (germany), 27-31 august, 2002. weber, m. (1930) the protestant work ethic and the spirit of capitalism, allen and unwin, london. garo et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 176-190 | 190 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/j.jbusvent.2005.06.008 http://dx.doi.org/10.1037/0021-843x.116.4.645 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 97 the effectiveness of graphic representation techniques used by industrial designers for the conceptual presentation of new products francisco felip miralles universitat jaume i, departamento de ingeniería de sistemas industriales y diseño universitat jaume i, av. de vicent sos baynat, s/n. 12071 castellón de la plana, ffelip@uji.es, 964 728201 received: 2016-07-06; accepted: 2017-09-13 abstract graphic techniques are very important means for designers. they can improve the conceptual presentation of new products before a client or a company, stimulating the communication of their potential, defending their benefits and making clear their commercial viability. to choose the most suitable graphic languages is essential so that the client can understand the proposal, allowing this way to establish a dialogue with the designer itself in order to improve the solution. today the multiplication of graphic media has made it possible for the designers to have a wide range of ways to conduct and present their ideas, but it may happen that the graphic techniques chosen may not always be the most appropriate. this work aims to demonstrate the importance of graphic expression used as a tool for presenting ideas about new products, and evaluate the effectiveness of several graphic techniques in improving the client's understanding of the product, taking into account the conceptual clarity, the representation of the mode of use, the relationship with the user and the contextualization of the product in its habitat or use environment. keywords graphic media, industrial design, concept, product design, habitat multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 98 1. introduction the emergence and democratization of new screen-equipped devices, the reduction of its size and the increase of the bandwidth in the data transmission have led to the emergence of a global scenario in which communication uses highly visual languages and makes use of increasingly universal graphic resources. through multiple ways, it is possible to access all kinds of information at all times and everywhere, and as a consequence, activities such as leisure, work and interpersonal communication have seen the boundaries of their traditional space diluted to face new ways to develop and new places to be practiced. the emergence of this environment marked by the omnipresence of the various screen formats reminds us of the potential and importance of the image in any communication process. its ability to encode a lot of information and to use referents easily identified by various cultures makes it easily understandable by many communities, overcoming in many cases the barrier that usually imposes verbal language to convey concepts or ideas. many authors have emphasized the importance of the image to communicate ideas, in several fields. graphics have been indispensable tools for social and cultural evolution, and drawing is defended as a key agent in the interaction of knowledge and communication. while some have emphasized the general importance of schemas and diagrams as effective tools for understanding complex concepts, ideas, and relationships (crilly et al., 2006), others go further and argue that graphical representation is not just a means for rapid information uptake, but a vehicle and instrument in the process of acquiring knowledge, so that the development and implementation of graphic language requires a certain maturity (gonzález et al., 2004). in this regard, we consider that although the inclusion of aesthetically attractive elements in a display can favor a better understanding and acceptance of the content, can sometimes reduce the effectiveness of the message if not applied with the correct criterion, and may distract the observer with unnecessary details (brath et al., 2005). multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 99 study drawings, as a medium for graphic thinking, can play a prominent role in any design work process (herbert, 1988). drawings, illustrations and visualizations have a long tradition also in the field of landscape architecture. the designer communicates his or her ideas and concepts through these media to other people involved in the planning process (bishop et al., 2005), favoring the workflow. in the area of product design, it is essential to correctly convey a concept to the client, but also to use the right graphic resources to communicate the ideas to the rest of the work team. nowadays, collaborative work through new technologies has acquired great importance in this discipline (chulvi et al., 2017), making it especially necessary to work with adequate images capable of transmitting ideas in a correct way and thus speed up the teamwork process. the new technologies have favored collaborative work, especially in non-presential design teams (garcía-garcía et al., 2015, mulet et al., 2013), demonstrating that knowing how to generate easily understood images that do not require a verbal explanation for its understanding is an even more justified need, and a challenge that arises in the current information society. nowadays there are many graphic tools for communicating ideas in the field of design (garcia-garcía et al., 2016), but it is worth asking if they are really effective, or if we always choose the most appropriate graphic language. for years, digital tools have coexisted with traditional ones, but making a correct use of them in every part of the work process remains indispensable when it comes to effectively communicating an idea. sometimes we find that when presenting a project to a client or company he or she does not understand the graphic language used by the designer, or that language is not the most appropriate for the proposal that is being presented, and because of this many projects do not prosper. in this context, it is essential for the designers to communicate their ideas effectively through appropriate graphical representations. therefore, it is necessary to study how current designers work graphically the concept of a new product, to see which are the most multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 100 used representation techniques and to reflect on their effectiveness in presenting this concept to a client. 2. main objectives this work aims to demonstrate the importance of graphic expression used as a tool for presenting ideas about new products, and evaluate its effectiveness in the process of communication between the designer and a client or company. for this purpose, we propose to investigate two aspects: a) to determine which graphic representation techniques are the most used in the field of design to conceptually represent a new product. b) evaluate the utility or real effectiveness of these graphic techniques in improving the client's understanding of the product, taking into account the conceptual clarity, the representation of the mode of use, the relationship with the user and the contextualization in its habitat. 3. methodology 3.1 overview two different activities are proposed in the research process. the first is developed with teams of novice designers during several work sessions, and the second is developed in collaboration with professional design studies. the ultimate purpose is to obtain data about the importance of graphic representations during the designer's work process and during the process of communicating an idea. several cases are analyzed, assessing to what extent the graphic representation of the product in its context of use helps to understand it better. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 101 3.2 proposal and working teams the first part of the research compares and evaluates the effectiveness of the graphic media used by several groups of novice designers to present a new product for the habitat that responds to the brief of a fictitious company, and to design the living space where it will be used. it is assessed in each case if the representation technique used by the designers has been effective during the process of communication of the idea to the company and if the graphic representations of the living space where the product will be located have helped to better understand the product, its context of use and its relationship with the user. for this, 10 work teams are formed between 4 and 5 people (t1 to t10) and each will develop the design of a new product. to guarantee homogeneity of the sample, participants who have received similar university training in industrial design, aged between 21 and 37 years old, have been chosen (figure 1). figure 1. sample characteristics (teams of novice designers). multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 102 throughout the activity, each team will be paired with another randomly chosen team, and both will act with a double role: first as company team, elaborating a brief for the other team, and then as a design team, analyzing the brief of the other team (who will assume the role of company) and proposing a design of a product for the habitat, according to their requirements. the teams are paired as follows: t1 with t2; t3 with t4; t5 with t6; t7 with t8 and t9 with t10. 3.3 working sessions once the teams have been set up, the proposal is organized in 4 working sessions: session 1:  each team will begin to work as a company, writing a brief as complete as possible with which to request to other team the creation of a new product, framed in current market trends and related with a previously studied philosophy. the brief should follow the sections: description of the situation, background, target audience, estimated budget, objective of the designed product and main benefit. session 2:  at the beginning of the session, each team (company) will deliver the completed brief to another team, which will act as a designer team from now on.  each team (as a design team) will carefully analyze the brief provided by the other team (company). then they must create a written report to evaluate each section of the brief.  after the analysis, and with the report already written, each team will meet with the other to perform a counter-briefing. first, one team will act as a designer and the other as a company (discussion of the first brief), and then the reverse (discussion multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 103 of the second brief). the objective is to propose changes or improvements in both briefs, in order to better focus the work.  each design team will graphically present a product proposal (concept art, sketches, explanatory drawings of how it works, comments on the functions, etc.) as specified in the brief. the new product is defined to the maximum, with all the drawings, diagrams, general plans, infographics or animations that are necessary. ambiences are created to represent the product in its natural environment of use, in order to improve its understanding by the company team. session 3:  each designer team will deliver to the company team the graphic material that illustrates their product proposal.  each company team will analyze the proposal of the designer team, and will meet with it to discuss it. with the indications of the company, each designer team will continue working on the proposal. session 4:  each team will deliver to the other all the final visual material (concept art, sketches, explanatory drawings, diagrams, infographics, animations, space environments, etc.).  each team, acting as a company, will evaluate the final proposal of the other team according to the following aspects: if the product meets the requirements of the corrected brief (features, uses, benefits, etc.), if the product has taken into account the target audience and is well aimed at them, if the product can meet the established objectives, and if the product complies offering the user the main benefit specified in the brief. after completing the work proposal, the team partners evaluate each other the final material presented, rating on a numerical scale from 1 to 10 several issues related to the graphic multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 104 representation of the product and its use environment, such as the adequacy of the representation technique, if this representation has contributed to improve the conceptual clarity, or if the graphical representations of the living space where the product will be located have helped to better understand its context of use and its relation with the user (figure 2). figure 2. questionnaire to be completed by each company team (evaluating the work of the designer team) finally, each design team is asked to rate on a scale from 1 to 10 the importance of graphic representations in the process of communicating an idea, according to the experience derived from this activity. in the same way, each team marks the representation techniques that has used (figure 3). multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 105 figure 3. questionnaire to be completed by each designer team. 3.4 survey aimed at professional design studios in order to obtain information contrasted with the professional reality, a survey was carried out to ten prestigious design studios based in spain that had a consolidated professional career centered in the field of product design for the habitat, recognized both nationally and internationally. studies surveyed were: aa studio (albertoarza.com), cerámica a mano alzada (ceramicaamanoalzada.com), clausell studio (clausellstudio.com), dsignio (dsignio.com), equipo nómada (equiponomada.es), ignota design (ignotadesign.com), nes (nesestudio.com), joanrojeski studio (joanrojeski.com), vitale (vitale.es) and yonoh (yonoh.es). the survey was conducted electronically. based on their accumulated professional experience, they were asked about the usefulness of the graphic representation to present an idea or concept of a product to a company or client, if representing a product in a habitat or context of use makes the product better understood, and whether to represent a product in its habitat or context of use helps to better understand its relationship with the user. valuations were performed on a numerical scale from 1 to 10, with 1 being the lowest value multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 106 and 10 being the highest. in addition, the studios were asked to indicate which graphic representation techniques they used to present their proposals to their clients, in order to compare them later with the techniques used by the newly graduated designers. 4. results about the teams assuming the role of designers (figure 4), the results obtained show a very high valuation of the support of the graphic resources in the process of presenting and explaining a new product before a fictitious client, in this case, another of the teams that have collaborated in the activity assuming the role of a company. from the same questionnaire, it can be seen that the most used techniques of representation by novice design teams are manual sketching (drawing in pencil, ink and marker), 3d modeling (image) and photomontage (integration of manual drawings and 3d modeling in real backgrounds digitally intervened). figure 4. results of the questionnaire completed by the novice teams in their ‘designer’ role. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 107 figure 5. a selection of final graphic proposals of all novice teams (t1-t10): representation of the product and the living space where it will be used. left to right and top to bottom: t1 and t2; t3 and t4; t5 and t6; t7 and t8; t9 and t10. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 108 figure 6. evaluation of each proposal by the other team: product and environment (habitat). the evaluation of the final graphic proposals of each team by the other (figure 6) has proved to be high in all cases, with an average score higher than 7 points out of 10 in all teams. it is observed that the best rated teams made use of both manual and 3d modeling techniques to represent the product and its environment of use. it should be noted that the team whose proposal obtained the lowest global rating used only manual drawing techniques to represent both the designed product and the natural habitat or context of use. in contrast, proposals that used digital 3d modeling techniques to represent the habitat or product were better rated. while the conceptual clarity was the best rated aspect (9.5 points out of 10), the understanding of the product-user relationship through its representation in the use environment was the worst rated aspect (8.5 points), in many cases because in the multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 109 drawings the user was not explicitly represented using the product in its habitual environment. figure 7. results of the survey to 10 professional studios specialized in habitat design the survey of ten professional design studios (figure 7) reveals that, according to the accumulated experience of their members, the use of graphic representation support in presenting an idea or concept about a product to a company or client is indisputably very multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 110 useful. in the same way, all studios coincide in pointing out that, in the exercise of their profession, the proper representation of a product in its habitat or context of use contributes to a better understanding of the product and helps to understand its relationship with the user. this survey also shows that the most commonly used techniques in the field of product design related to habitat are usually 3d virtual modeling and the photomontage. the techniques of manual and infographic sketching, widely practiced by novice teams, are here moderately significant. 5. conclusions and discussion in light of the present work, and after analyzing the opinion of novice design teams and consolidated studios, it seems that the use of graphic resources is very useful for the designer when presenting and defending a project before a client or company. the immediacy of the image to convey complex ideas and its ease to be understood, often without the need for words, make it a valuable support instrument in any communication process. this study has allowed us to determine that manual sketching, photomontage and 3d virtual modeling are the most commonly used graphic expression techniques in the design field to conceptually present a new product. one possible explanation for the preference of working with these techniques rather than with others (3d animation, for example) may be that they offer the design team a greater immediacy when representing a product concept in early stages, and that such immediacy offers a clearer and direct vision of the main attributes to be highlighted. however, this requires further research to be discussed. in any case, using a more elaborate graphic technique or a simpler one does not always necessarily guarantee that the client to whom the proposal is presented will understand it correctly. other factors such as the complexity of the content to be represented, the details displayed, the quality of the execution or the sequence in which the images are shown are multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 111 possibly involved. it is convenient to consider the simultaneous use of several graphic techniques in the presentation of a design project, since some aspects, such as conceptual ones, maybe can be better understood through more immediate techniques (manual sketches) and others, such as accurate volumetrics or surface qualities, are better understood through more advanced techniques (3d virtual modeling). 6. acknowledgment we wish to thank the collaboration of all novice designers and, specially, the design studios, whose professional experience has made it possible to offer a contrasted view of the importance of graphic expression in the communication of the project to clients or companies. 7. references bishop, i., & lange, e. (2005). presentation style and technology. en i. bishop & e. lange (eds.), visualization in landscape and enviromental planning: technology and applications (pp. 68-77). new york: taylor&francis. brath, r., peters, m., & senior, r. (2005). visualization for communication: the importance of aesthetic sizzle. en e. banissi, m. sarfraz, j. c. sarfraz, b. loften, a. ursyn, r. a. burkhard, a. lee, & g. andrienko (eds.), proceedings ninth international conference on information visualisation (pp. 724-729). los alamitos, ee.uu.: ieee computer society. http://ieeexplore.ieee.org/document/1509153/?reload=true https://doi.org/10.1109/iv.2005.145 crilly, n., blackwell, a. f., & clarkson, p. j. (2006). graphic elicitation: using research http://ieeexplore.ieee.org/document/1509153/?reload=true https://doi.org/10.1109/iv.2005.145 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 112 diagrams as interview stimuli. qualitative research, (6), 341-366. http://journals.sagepub.com/doi/abs/10.1177/1468794106065007 https://doi.org/10.1177/1468794106065007 chulvi, v., mulet, e., felip, f. garcía-garcía, c. (2017) the effect of information and communication technologies on creativity in collaborative design. research in engineering design, 28(1), 7-23. https://doi.org/10.1007/s00163-016-0227-2 garcía-garcía, c., chulvi, v., galán serrano, j., felip, f., royo gonzález, m. (2015) the use of social networks for invigorate the communication during the conceptual design phase in virtual projects. in proceedings of 19th international congress on project management and engineering, granada, spain (pp. 277-290). aeipro garcía-garcía, c., chulvi, v., mulet escrig, e., felip, f. (2016) comparative study of digital sketching tools for conceptual design of new products. in proceedings of 20th international congress on project management and engineering, cartagena, spain (pp. 1028-1040). aeipro. http://www.aeipro.com/files/congresos/2016cartagena/03030.4603.pdf gonzález, j. a., jover, l. (2004). los gráficos en la comunicación y el razonamiento científicos: ¿instrumento u ornamento? medicina clínica, 122(supl 1) 3-10. http://www.elsevier.es/es-revista-medicina-clinica-2-articulo-los-graficos-comunicacionel-razonamiento-13057541 https://doi.org/10.1157/13057541 herbert, d. m. (1988). study drawings in architectural design: their properties as a graphic medium. journal of architectural education, 41(2), 26-38. http://www.tandfonline.com/doi/abs/10.1080/10464883.1988.10758473 https://doi.org/10.1080/10464883.1988.10758473 https://doi.org/10.1177/1468794106065007 https://doi.org/10.1007/s00163-016-0227-2 https://doi.org/10.1157/13057541 https://doi.org/10.1080/10464883.1988.10758473 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7686 social and technological sciences issn: 2341-2593 felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 97-113 | 113 mulet escrig, e.; escamilla, n.; fernández-muyor, h. j.; galán, j.; chulvi, v.; felip, f.; garcía-garcía, c. (2013) analysis and discussion of the methodology for analyzing the interaction level of presential and virtual design groups. in proceedings of 17th international congress on project management and engineering, logroño, spain (pp. 1551-1560). aeipro. http://www.aeipro.com/files/congresos/2013logronio/cidip2013_1551_1560.4072.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 a supervised method for unbiased peer-to-peer evaluation. an experience with engineering students simois, francisco j.* department of signal theory and communications, university of sevilla engineering school, camino de los descubrimientos, s/n, 41092, sevilla (spain) * corresponding author: email: fjsimois@us.es; phone: +34 954 48 81 33 received: 2015-02-14; accepted: 2015-06-25 abstract continuous evaluation is an assessment method which has some appealing advantages but also implies an increase of the teacher’s efforts and it may be unfeasible if the class is large. of course, new technologies may be used to implement automatized evaluations, but it is usually quite difficult to carry them out when a complex task like an engineering problem is to be judged. an interesting alternative is a peer-to-peer evaluation, that is, the students themselves review their works. nevertheless, one drawback is that it is likely that the grades are overrated. although this is a well-known problem, not much effort is usually put into solving it. in this work we propose a novel method to limit this inconvenience, which is that the teacher randomly supervises a fraction of the students tasks. in this paper we present the results of such an experience carried out in a signal processing course within a robotics engineering degree. more precisely, four different sets of problems were solved by the teacher in class. at the same time, they were peer-to-peer reviewed by the students, following the indications given by the professor. later, when the random supervision is performed, a penalty is applied if a major flaw in a student’s evaluation is detected. thanks to this strategy, the scores tended to be more and more accurate according to the teacher’s criteria. finally, the results of a survey anonymously fulfilled by the students to assess this experience are also presented. keywords peer-to-peer evaluation, supervised assessment, engineering course, higher education. simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 65 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 1. introduction the bologna declaration signed on june 19, 1999, proposes, among other aspects, the convergence of the various european higher education systems. this will help to yield a knowledge-based economy capable of growing sustainably and providing more and better jobs and a higher social cohesion (bologna, 1999). the creation of the european higher education area entails a new educational paradigm that enables the development of new educational models and promotes a new skills-based learning to train graduates in solving the problems they will have to address in their jobs (ehea, 2010). these skills are called competencies. it is considered necessary for students to have competencies for regulating individual and group work, for establishing learning goals, planning courses of action, selecting suitable strategies and resources, and reviewing and reorienting tasks in order to meet predetermined objectives (torrano, 2004; pintrich, 2000). moreover, the need to use evaluation for pedagogical ends has been highlighted by numerous authors (see, for example, schunk, 1998; coll, 1999; william, 2000; broadfoot, 2004; mcdonald, 2006). continuous assessment is used for purposes of feedback that can serve to improve the learning of the students and to enhance the teaching. this kind of evaluation allows to identify errors in the process, adjusting and orienting it (delgado, 2005). it has to be performed during the learning process in order to detect the learning gaps. continuous assessment promotes the pace of study established by the professor, which involves the study of learning small units distributed over extended periods of time. this perspective emphasizes “assessment for learning” (birembaum, 2006; nunziati, 1990; allal, 1991), that is, the importance of providing students with information about their own learning process, as well as possible ways for improving it. the process of european convergence has also prompted implementation of teaching methodologies centered on students’ autonomous work (coll, 2007). that is, the simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 66 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 contemporary context wherein higher education operates has determined that students should play a more active role in the teaching-learning process and, likewise, that teachers should play a facilitator role in that process (arias, 2014). in addition, in recent times has emerged a movement called “authentic assessment”, “performance assessment” or “alternative assessment” (schlichting, 2015; biggs, 2006; birembaum, 2006; díaz barriga, 2006), which emphasizes methods that facilitate direct observation of student work. in this line, control of evaluation has been transferred from teacher to student. the agent evaluation are not only the teachers, but students play an important role in their own assessment or that of their peers (mateo, 2005). this requires that the student assumes the learning objectives and criteria to be used for his evaluation. this formative evaluation is compatible with the continuous one since it considers that the learning is better when targets, criteria and standards are known. therefore, both autoevaluation and peer-to-peer (between students) reviews are promoted and encouraged. one well-known issue of peer assessment is its validity (that is, the bias with respect to a professor evaluation) and its reliability (that is, the dispersion of grades with respect to their average). some authors have carried out an extensive review of this matter (falchikov 2000). actually, although some previous works have shown no significant bias in peer assessment (xiao, 2008; marks, 2013), many others have found certain problems in its validity (kommalage, 2011; harris, 2011) or even very important flaws (de grez, 2012). despite of that, previous works seldom propose some ways to reduce the bias and only recommendations about how to manage students’ attitude and responsiveness are provided (lansiquot, 2015; koç, 2011). starting from this perspective, the objective of the present study is to introduce a new peer-to-peer evaluation methodology performed with engineering students. due to the intrinsic difficulty of the proposed tasks, overrated grades are expected. the novelty of the proposed method is that, in order to reduce this bias, the professor supervises a simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 67 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 fraction of the marks and applies penalties to the students that try to cheat. using this approach it is intended that the peer assessment adjusts itself properly during the course, without increasing too much the professor’s effort. 2. methods in the next paragraphs, the proposed methodology will be presented. first, we will describe the current situation of the subject under study. second, we will show the details of the learning experience. 2.1 current situation and problem statement the subject in our study is “procesamiento digital de la señal” (digital signal processing). it is taught in the third course of the electronics, robotics and mechatronics engineering degree at the university of sevilla. the course has an applied nature, having several laboratory sessions. in addition, it is intended that the students try to solve a considerable amount of applied problems to reinforce the theoretical ideas that are taught. these problems are fully graded because they are one important item in the continuous evaluation. there was 49 students participating actively in the course and only one professor. taking into account that the total number of problems to be reviewed is 58, this implies a huge amount of effort and time for the teacher. moreover, the intrinsic nature of the problems is heterogeneous and complex; in addition, the grading criteria is not merely correct/incorrect but they are evaluated as a whole. therefore, an automatic evaluation tool is almost impossible to be performed. for all those reasons, a peer-to-peer assessment was carried out. that is, the students themselves had to grade their classmates. the issue is that it is very likely that the scores simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 68 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 are overrated in such a situation. to overcome this problem, a supervised evaluation is proposed. the details are as follows. 2.1.1 the supervised peer-to-peer evaluation there are four different sets of problems, each of them intended to be solved by the professor in a different session. at the beginning of the session, each student freely chooses a classmate and they swap their problems in order to be graded. the only restriction is that nobody can review the same classmate more than once. when everybody has exchanged his problems, the professor solves them and, at the same time, provides the criteria for grading. these criteria are as detailed as possible in order to the students to award clear and impartial marks. in addition, the teacher answers the questions about grading that the students may pose on the fly. at the end of the class, the students fill a form with three fields (their name, the name of whom they have graded and the marks awarded) and give it to the professor. finally, the teacher chooses randomly 12 students and collects their sets of problems. to be precise, there is one restriction in the randomness: every student will have to be selected at least once and at most three times during the course. later, the professor reviews the 12 students he have chosen and compares his grades to the peer-to-peer ones. if a discrepancy of a 10% or more is found, a penalty of twice the difference is applied to the evaluator. for instance, let us assume that student x grades student y with a 6 and the teacher with a 5 (that is, a difference of 1 point or a 1/6·100 = 16.7%). then, the mark of student x (not student y) is diminished in 2·1 = 2 points. of course, the students were previously warned about this issue. finally, the professor publishes the final grades with all the associated details. the final mark is the teacher’s one, if it exists (since it prevails over the peer-to-peer evaluation), simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 69 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 minus the penalty; or the classmate’s grade, if the professor’s one does not exist, minus the penalty. on the other hand, an anonymous survey was conducted at the end of the course. the purposes were mainly two: assess whether the methodology had helped the students with the subject; and whether the action of grading itself had been puzzling to them. 3. results and discussion on the one hand, we will present the evolution of the peer evaluation compared to the professor’s one along the course. on the second hand, the results of the satisfaction survey will be shown. 3.1. comparison between peer-to-peer and professor’s evaluations as it has been previously explained, two different evaluations were carried out for each set of problems. first of all, all students were graded by a classmate. secondly, some randomly selected students were also reviewed by the professor. both marks would be the same in the ideal case but it is unavoidable that some differences appear. these differences are clearly shown in figure 1. in (a), it is seen that, for the first set of problems, the grades awarded by the classmates were as much as 0.5 points higher than the grades given by the professor. this seems to indicate that, despite the fact that the teacher cautioned the students about the possibility that a sanction was applied, they did not consider it a real warning. probably, many of them thought that the professor was only threatening but he would not apply the penalty; or perhaps they were overconfident that they would not be chosen to be revised. as a consequence, some penalties were imposed. actually, 4 out of 12 students graded by the professor were penalized. to be precise, if a positive difference of a 10% of more between the two evaluations was found, a penalty of twice the difference was applied to the involved student. the details are shown in table 1. simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 70 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 (a) (b) (c) (d) figure 1. grades awarded by both the peer-to-peer and the professor’s evaluation, as well as the penalty applied for (a) first set of problems, (b) second set of problems, (c) third set of problems and (d) fourth set of problems. table 1. details of the grades for the first set of problems. peer-to-peer professor difference difference (%) penalty student #1 7.4 6.8 +0.6 +8.1% 0.0 student #2 7.0 5.9 +1.1 +15.7% 2.2 student #3 9.1 9.3 −0.2 −2.2% 0.0 00 01 02 03 04 05 06 07 6.8 6.3 0.6 problems set #1 00 02 04 06 08 7.1 6.8 0.2 problems set #2 00 01 02 03 04 05 06 07 6.7 6.8 0.0 problems set #3 00 01 02 03 04 05 06 07 6.3 6.4 0.0 problems set #4 simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 71 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 student #4 7.3 6.8 +0.5 +6.8% 0.0 student #5 7.2 6.2 +1.0 +13.9% 2.0 student #6 5.3 4.4 +0.9 +17.0% 1.8 student #7 6.1 5.5 +0.6 +9.8% 0.0 student #8 6.8 7.0 −0.2 −2.9% 0.0 student #9 7.2 6.7 +0.5 +6.9% 0.0 student #10 3.8 3.0 +0.8 +21.1% 1.6 student #11 8.8 8.8 0.0 0.0% 0.0 student #12 5.9 5.6 +0.3 +4.3% 0.0 average 6.8 6.3 +0.5 +8.2% 0.6 of course, it is apparent that, in general, the peer-to-peer grades tends to be overrated, which is not surprising. specifically, there is an average of an extra 8.2% in the marks awarded by the classmates. in figure 1 (b) the situation is quite different. although there is a still a slight tendency to overrating (details are omitted), it is much lower than in (a). indeed, only 1 out of 12 students was penalized. obviously, they realized that the warning was for real and the majority of them did not risked again. finally, in figure 1 (c) and (d) the situation is completely stabilized. all students followed precisely the criteria given by the professor when the problems were solved in class. actually, although some subtle differences between the peer-to-peer and the teacher’s evaluation were present, no penalty was applied and indeed the average grade awarded by the classmates was slightly lower than the professor’s one. in conclusion, it can be stated that the main goal of this experience was fully achieved. that is, the peer evaluation was finally free of bias and this was done even taking into account that the professor did actually grade just a fourth of the students. simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 72 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 3.2. results of the satisfaction survey at the end of the course, an anonymous satisfaction survey was conducted in order to get the students’ opinion on this evaluation methodology. the survey consisted of five questions, which are shown in table 2. each question had to be answered with a number from 1 to 5 (see table 3). the results of the survey are presented in figure 2. it can be seen that the great majority of students believe that the peer-to-peer evaluation has helped them to keep the subject up to date. this is a very important point, since the students usually tend to prepare the subject just at the end, at least in courses where the evaluation consists in just a final exam. table 2. questions in the satisfaction survey. question #1 peer evaluation has helped you to keep the subject up to date? question #2 peer evaluation has helped you to better understand the subject? question #3 do you agree with the grades you received from your classmates? question #4 it has been difficult to you to grade your classmates? question #5 overall, the peer evaluation has been positive to you? table 3. possible answers to the survey. 1 totally disagree 2 disagree 3 partially agree 4 agree 5 totally agree simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 73 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 (a) (b) (c) (d) (e) figure 2. results of the satisfaction survey for (a) first question, (b) second question, (c) third question, (d) fourth question and (e) fifth question. 00% 10% 20% 30% 40% 50% 1 2 3 4 5 4.1% 8.2% 24.5% 42.9% 20.4% question #1 00% 10% 20% 30% 40% 1 2 3 4 5 10.2% 14.3% 32.7% 34.7% 8.2% question #2 00% 10% 20% 30% 40% 50% 1 2 3 4 5 0.0% 0.0% 20.4% 42.9% 36.7% question #3 00% 05% 10% 15% 20% 25% 30% 1 2 3 4 5 6.1% 26.5% 24.5% 20.4% 22.4% question #4 00% 10% 20% 30% 40% 1 2 3 4 5 4.1% 16.3% 28.6% 34.7% 16.3% question #5 simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 74 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 results of second question are also very good, showing that most of the students consider that the peer evaluation has been useful to them to better understand the subject. question number 3 proves that students are not reluctant to be graded by their classmates. actually, none of them disagree on that point. on the contrary, it is usual that some students argue about the grades awarded by the professor. therefore, we can conclude that this is also a good consequence of the methodology. fourth question is the one which has scored worst. up to a 42.8% of the students have found problems in grading the works of their peers. this should be taken into account and some strategies to make this task easier in the future will have to be explored. finally, question number 5 show that most of the students approve the methodology, so we can conclude that it has been a good experience overall. 4. conclusions a peer-to-peer evaluation was carried out with engineering students. four different sets of problems were proposed. the novelty in the methodology is that, to reduce the expected overrate in the classmates’ grades, the professor randomly selected some sets of problems to be reviewed by himself and a penalty was applied if some major flaws in the students’ evaluation were detected. it has been proved that this simple strategy completely cancelled the bias. that is, the peer-to-peer and the professor’s evaluation tended to be the same. a final survey confirmed that this kind of evaluation has helped the students to keep the subject up to date and to better understand it. in addition, all of the students agreed with the grades awarded. the only point to be improved is that many students found difficult to evaluate their classmates’ works but, overall, they were quite satisfied with the experience. simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 75 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 5. references allal, l. (1991). vers une pratique de l’évaluation formative. brussels: de boek. arias macías c.m., arriazu navarro r., casanova arias j.l., fernández arias j., cárdenas rebollo j.m. and rey-stolle m.f. (2014). use of blackboard collaborate platform as a higher education teaching aid. international journal on advances in education research, 1(2), 109-124. biggs, j. (2006). calidad del aprendizaje universitario. [quality of university learning.] madrid: narcea. birembaum, m., breuer, k., cascallar, e., dochy, f., dori, y., ridway, j., wiesemes, r. and nickmans, g. (2006). a learning integrated assessment system. educational re search review, 1(1), 61-67. http://dx.doi.org/10.1016/j.edurev.2006.01.001 bologna (1999). joint declaration of the ministers responsible for higher education convened in bologna on the 19th of june. available at http://www.magnacharta.org/resources/files/bologna_declaration.pdf. broadfoot, p. and black, p. (2004). redefining assessment? the first ten years of “assessment in education”. assessment in education, 11(1), 7-27. http://dx.doi.org/10.1080/0969594042000208976 coll, c. and onrubia, j. (1999). evaluación de los aprendizajes y atención a la diversidad. in c. coll (coord.), psicología de la instrucción. la enseñanza y el aprendizaje en la educación secundaria, (pp. 141-168). barcelona: horsori / ice de la ub. coll c., rochera m.j., mayordomo r.m., naranjo m. (2007). continuous assessment and support for learning: an experience in educational innovation with ict support in higher education. electronic journal of research in educational psychology, 5(3), 783-804. de grez, l., valcke, m. and roozen, i. (2012). how effective are selfand peer assessment of oral presentation skills compared with teachers' assessments? active simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 76 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/j.edurev.2006.01.001 http://www.magna-charta.org/resources/files/bologna_declaration.pdf http://www.magna-charta.org/resources/files/bologna_declaration.pdf http://dx.doi.org/10.1080/0969594042000208976 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 learning in higher education, 13(2), 129-142. http://dx.doi.org/10.1177/1469787412441284 delgado, a. m., borge, r., garcía, j. oliver, r. and salomón, l. (2005). competencias y diseño de la evaluación continua y final en el espacio europeo de educación superior. programa de estudios y análisis (ea2005-0054). madrid: ministerio de educación y ciencia. dirección general de universidades. díaz barriga, f. (2006). la evaluación auténtica centrada en el desempeño: una alternativa para evaluar el aprendizaje y la enseñanza. in f. díaz barriga (coord.). enseñanza situada: vínculo entre la escuela y la vida (pp. 125-163). méxico: mcgrawhill. ehea (2010). european higher education area. available from: http://www.ehea.info/. falchikov, n. and goldfinch, j. student peer assessment in higher education: a metaanalysis comparing peer and teacher marks. review of educational research, 70(3), 287-322. http://dx.doi.org/10.3102/00346543070003287 harris, j. (2011). peer assessment in large undergraduate classes: an evaluation of a procedure for marking laboratory reports and a review of related practices. advances in physiology education, 35(2), 178-187. http://dx.doi.org/10.1152/advan.00115.2010 koç, c. (2011). the views of prospective class teachers about peer assessment in teaching practice. educational sciences: theory & practice, 11(4), 1979-1989. kommalage, m. and gunawardena, s. (2011). advances in physiology education, 35(1), 48-52. http://dx.doi.org/10.1152/advan.00091.2010 lansiquot, r. and rosalia c. (2015). online peer review: encouraging student response and development. journal of interactive learning research, 26(1), 105-123. marks, l. and jackson, m. (2013). student experience of peer assessment on an msc programme. bioscience education, 21(1), 20-28. http://dx.doi.org/10.11120/beej.2013.00015 simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 77 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1177/1469787412441284 http://www.ehea.info/ http://dx.doi.org/10.3102/00346543070003287 http://dx.doi.org/10.1152/advan.00115.2010 http://dx.doi.org/10.1152/advan.00091.2010 http://dx.doi.org/10.11120/beej.2013.00015 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3738 social and technological sciences eissn: 2341-2593 mateo andrés, j. and martínez olmo, francesc (2005). la evaluación alternativa de los aprendizajes. cuadernos de docencia universitaria, nº 3, ice-universidad de barcelona. mcdonald, r. (2006). the use of evaluation to improve practice in learning and teaching. innovations in education and teaching international, 43(1), 3-13. http://dx.doi.org/10.1080/14703290500472087 nunziati, g. (1990). pour construire un dispositif d’évaluation d’aprentissage. cahiers pédagogiques, 280, 47-64. pintrich, p. r. (2000). the role of goal orientation in self-regulated learning. in m. boekaerts, p. r. pintrich, and m. zeidner (eds.). handbook of self-regulation (pp. 451502). san diego, ca: academic press. http://dx.doi.org/10.1016/b978-0121098902/50043-3 schunk, d.m. and zimmerman, b. j. (eds.) (1998). self-regulated learning: from teaching to self-reflective practice. new york: the guilford press. schlichting, k. and fox, k. (2015). an authentic assessment at the graduate level: a reflective capstone experience. teaching education, 26(3), 310-324. http://dx.doi.org/10.1080/10476210.2014.996748 torrano, f. and gonzález, m. c. (2004). self-regulated learning: current and future directions. electronic journal of research in educational psychology, 2(1), 1-34. william, d. (2000). integrating summative and formative functions of assessment. keynote address. first annual conference of the european association for educational assessment. prague, czech republic. xiao, y. and lucking, r. (2008). the impact of two types of peer assessment on students' performance and satisfaction within a wiki environment. the internet and higher education, 11(3-4), 186-193. http://dx.doi.org/10.1016/j.iheduc.2008.06.005 simois (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 65-78 | 78 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1080/14703290500472087 http://dx.doi.org/10.1016/b978-012109890-2/50043-3 http://dx.doi.org/10.1016/b978-012109890-2/50043-3 http://dx.doi.org/10.1080/10476210.2014.996748 http://dx.doi.org/10.1016/j.iheduc.2008.06.005 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 47 gender violence news in british and american press bustinduy, sara university of malaga, fac. letras dpto. filología inglesa campus de teatinos s/n 29071 málaga españa email: sabustinduy@gmail.com; phone: +34 609941324 received: 2014-12-31; accepted: 2015-06-15 abstract this study aims to prove which qualities, from the british and united states actual press, are more adequate for gender violence issues, comparing news on the same stories. it is evident the influence of mass media on public opinion and, therefore, the responsibility that written press has on gender violence messages, avoiding sensationalism. psycholinguistic studies have established the relation language-thought, so language used in journalism is crucial. following the belief that newspapers considered more liberal and independent will lead to a better treatment than traditional ones considered to be more reluctant to change, journals have been selected. furthermore, different cultures can be as objective and respectful but maybe less committed with the issue, as it may arise from the samples. there have been emerging ethic codes giving guidelines to journalists, including discrimination, and more specific on gender sensitive reporting. therefore, the objective to improve public opinion information, stepping away from stereotypes and oversimplifications, is substantial, and will undoubtedly result in a better understanding of equality. keywords gender violence, equality, english language, press. mailto:sabustinduy@gmail.com multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 48 introduction the term gender was first used at the un beijing platform for action, september 1995, where violence against women was mentioned as one of the crucial social mechanisms by which women are forced into a subordinate position by men. it was declared that it was a consequence of the historically unequal power relations between men and women, exacerbated by the absence of educational and other means to address the causes and consequences of violence. on the resolution, institutions, governments and organizations are encouraged to instigate media to examine the impact of gender role stereotypes. this study will contrast the way language is used in messages, from two different newspapers, about the same news. both are considered serious, liberal and free of censorship, from two of the most important countries in the world the uk and the us. the aim is to find out which news processing would serve better for the topic. there are ethic codes for journalists both in the uk and the us where gender violence is considered. the national union of journalists edited the code for gb; inclusive language is used in the text and explicitly refers to gender discrimination declaring it must be avoided. in the code of ethics from the society of professional journalists, from america, inclusive language is not used but all sentences are imperative as a neutral choice. it specifically advises to avoid stereotyping by gender or sexual orientation and to be cautious about identifying victims of sex crimes. one of the main problems related to women issues has been invisibility (alberdi, 2006). there have been different stages concerning the dissemination of written news on periodicals. not so long ago, domestic violence messages were published only in gutter press, specialized in scandals. gender news has been related to sensationalism and stereotypes; these devices were used in order to call for attention and improve selling numbers, which was, even if unintentional, a perverse behaviour. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 49 nowadays most advanced societies are concerned about equality and the structural nature of gender violence. there are many institutions and organizations focused on gender equality aiming to end this scourge. in spite of these efforts, the invisibility of women remains. general press discourse is already divided in a number of different perspectives where it is not easy to find a right spot for gender. education should be revised, van dijk (2011) gives evidence that only in the us can be found some faculties offering ethic diversity studies at university level for journalists, and gender issues should be equally included. studies suggest that rather than looking for a section in gender, every piece of news should be considered under a gender perspective (gallego, 2002). gender education should be taught at all levels, including professionals dealing with language (taillefer, 2011). 1. discrimination, gender and press in this study we will analyze gender violence news trying to discern which processing emerges as more adequate after comparing two english language newspapers, both considered liberal and independent, the guardian from the uk and the new york times from the us that we assume as more formal and diplomatic. studies have found the relevant influence of mass media on public opinion (matas, 2006), and therefore the responsibility they bear on the references most people use in their minds. from a linguistic approach, the power of language influencing thoughts has been one of the most relevant findings, since it is now established that the language we use determines the way we think. by changing the words used in the news to refer to violent acts and people involved in them, society may realize how the situation actually is. mental models (fernández, 2003) are representations anybody can infer about another person or an event. group beliefs and knowledge are the references that need to be changed so models would be more adequate. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 50 literature review includes publications concerning different aspects of this topic. the most important focuses are general education (bengoechea, 2011) and specific journalists education (dijk, 2011) because they are the first steps to unmask the role language plays on building gender references. other focuses are the representation of women on media, the number of women working on news production (gallego, 2002) and the language used (calero, 1999). from the media approach, there are questions on how presence and absence of women in the press affects their visibility (gallego,2002), with the advice ‘not to hide women.’ bach arús (2000) stands for social changes followed by language. the fact that press discourse is never neutral is also pointed by a number of authors (gutierrez, 2009; dijk, 2008; dardis, 2006a) directly linked to the power of manipulating information, so it is imperative to point at the right direction. it is commonly admitted the pressure of androcentric power elites to impose their point of view and therefore their interests (dijk van, 2011; fernández, 2003). discourse works as teun van dijk’s on racism or dardi’s on media marginalization devices, deal with press manipulative strategies. both authors agree on how the discourse of ‘us’ vs. ‘the others’ or the denial of the problem is widely used. these words show how the topic really is (dijk, 2011:5): …it categorizes the out group (the others, them) not only as different, but also as inferior to the in-group (us), so that discrimination can be morally legitimated, or simply taken for granted as ‘normal’ conduct towards them. thus, the symbolic elites of white societies developed racist ideologies that control all perceptions, understanding, beliefs, attitudes, and hence all actions and interactions, geared towards the reproduction of ethnic domination. this shows how difficult is trying to get over discrimination. the ‘us vs. the others’ strategy has been often applied to gender issues where women would be ‘the others.’ the same strategies of discredit apply for all acts of prejudice or bigotry. the invisibility of violence against women is immersed in a more general power control device. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 51 psycholinguistic studies established the relation between mind and language. so that, not only our thoughts are shaped into speech but also our mind is influenced by language. in this way society can be influenced and collective thoughts can be modified by language, proved its capacity to transform our perception of reality and became part of our thoughts following taillefer de haya (2011), who also refers to the fact that language should be an ally to build up a more fair society. it would be good to establish a right frame as where to locate gender news in the newspapers, how to approach the different violent acts, avoiding stereotypes and additional unnecessary information, and how language should be used. so the influence of press should be used to set a more realistic portrayal of the stories. 2. analysis the newspapers reviewed were selected considering their circulation numbers and their national coverage. both journals have in common belonging to developed countries, where press is considered independent and free, as stated by dardis (2006b.) the guardian has been defined as a quality and technical newspaper with a low affective involvement (gonzalez, 2007), and the new york times as a reliable entity for international events reporting specific, real and sure knowledge (dijk, 2008), and as a reference periodical in the us (fondevila, 2012.) this is a qualitative study of the language used for the same piece of news on the same date in two different newspapers. as the newspapers are from two different countries and cultures, the cross-national research must reveal interesting findings based on the socio cultural backgrounds of each country (dardis, 2006b.) using the same event will allow to compare whether a device used by both periodicals may be considered as generic, or cultural differences may arise. several items will be depicted from the most recent research studies. the notion of subject as the main topic or theme within the nominalization group technique, and the rest of the sentence multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 52 as rheme or focus as a secondary form referred to the first one. this may be again divided into main focus and additional focus (gutierrez, 2009). the linguistic topics on fernández (2003) will be arranged to analyze the events where pertinent; and also some of the processing devices used by menendez (2010). we expect more liberal papers to have a more adequate treatment towards gender. both newspapers are considered liberal, serious and independent in their own countries and have an international reputation. the selected examples for this work are soft news, not comments, so no evaluation or opinion is expected, just the facts. the way each editorial reaffirms the “truth” of the story for the intended reader on their discourse (gonzalez, 2007) will show their differences. 3. findings the fact is that there are differences in the way information is dealt in relation to gender. in order to carry out our research, we are going to analyze a news item on sunanda pushkar death. the new york times placed a 58 lines article on the 19th of january 2014, in the asia pacific section, page a10, with the headline ‘after love unravels in media, indian politician’s wife is found dead.’ the guardian placed a 20 lines report on the 20th january 2014, in the international section, page 13, with the headline: ‘indian minister testifies over wife’s sudden death.’ we consider the difference of date a consequence of the time difference between the two countries. the social context is very similar for both periodicals; they are equally serious and address the same class readers with a high socio-cultural level. the words, groups of words and sentences used to refer the story will show the lexical relevance in a technical familiar dualism. findings of affective involvement vs. formal relation of the communicative event and rhetorical mode will indicate the tenor (gonzalez, 2007.) in the new york times piece we find the words: love, unravels, wedding, smooth-talking, reputation, beautiful, middle age, relationship, glare, affair, unnatural, dangerous, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 53 overexposure, pursuing, poisoning, scandal, resigning, influence, wrongdoing, priceless, ridicule, affection, despondent, dazed, divorce, pursued, stupid, intimate, episode, happily, ill, hospitalized, unauthorized, suite, luxury, formerly, infuriated, demise, accused, pursuing, awful and tragic. as for the guardian, we find the words: testifies, accused, adultery, affair, unnatural, injury, death, former, diplomat, investigation, rumors, personal, minister, horrified, reckless, speculation, rampant, pledge, unstinting, indignity, subjected, testimony, magistrate, inquest, comment, scandal, erupted, resurgent, political, rivals, patronage, engulfed, corruption, scandals and account. the references: love, beautiful, smooth-talking, middle age, glare, priceless, ridicule, affection, despondent, dazed, divorce, stupid, intimate, suite, luxury are found only in the new york times article. these words show a closer relation with the reader with some affective involvement, and are not so relevant to refer the information. we find some of these words in the phrases: ‘after love unravels in media’ on the headline; ‘he was a smooth-talking politician’ and ‘she was a beautiful kashmiri from the north’ on the first paragraph. they are strategically located, and set an atmosphere of novel rather than news reporting. other phrases such as: ‘a minister in the indian government’, ‘a high profile member of the governing indian national congress party’ and ‘his resigning as junior minister for foreign affairs’ point at the importance mr. tharoor has. the sentence ‘ms. pushkar, became mr. tharoor’s third wife’ doesn’t add any relevant information. in the sentence ‘he called her “priceless,” leading to some ridicule for his public display of affection’; the adjective ‘ridicule’ shows the journalist’s oppinion. the words used in the guardian to refer the event are more impersonal and technical. there are some phrases in which the importance of the victim’s husband is revealed: ‘an indian government minister’, ‘tharoor, a former un diplomat’, ‘in a letter to india’s home minister’ and ‘tharoor’s congress party, led by rahul gandhi.’ some additional stories are found in the new york times as the way mr. tharoor and ms. pushkar met, the scandal about him misusing his official influence to help her that he denies, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 54 discussing their relationship in social media, and the declarations of ms. tarar, the journalist accused of pursuing mr. tharoor. in the guardian the additional information is on mr. tharoor hoping to end rumours about their personal lives, declaring his horror on speculations on his private life, and on the political relevance of the scandal. we find in the new york times article that sunanda pushkar is the subject on thirteen sentences plus the object on a passive voice one, and mr. tharoor is a subject on fourteen sentences, eleven with the pronoun ‘he’, one with mr. tharoor, one with ‘her husband, shashi tharoor,’ and one omitted; this is: ‘married twice before’ which refers to mr. tharoor. in the guardian there are eight sentences with mr. tharoor as subject, three with ‘he’, three with ‘tharoor’, one with ‘shashi tharoor’ and one with ‘her husband’. ms. pushkar is the subject of one sentence refered with her full name ‘sunanda pushkar.’ 4. final remarks the headlines share the information about the fact that the victim’s husband is a politician, and therefore it may be infered he is an important person belonging to the power elites, and the fact that his wife is dead. in both sentences the topic/theme is mr. tharoor, and the comment/rheme is his wife. the fact that sunanda pushkar accused her husband of adultery previous to her death is revealed in both periodicals as an important aspect of her demise. the way information is placed in the new york times ‘now after public allegations that he was having an affair, the woman is dead’, suggests her death was a consequence of that incident. although both pieces of news refer to the fact that ms.sunanda believed her husband had had an affair, neither of them refers to the psychological mistreatment implicit on adultery, which has been indicated as relevant in her death. the message about the autopsy ‘pushkar’s death was sudden and unnatural, and her body bore injury marks, although doctors said this did not mean the injuries had caused her death.’ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 55 is refered in both newspapers in a very similar way. suicide is a reference left on reader’s minds although not explicitly declared. references to the importance of mr. tharoor are found in both journals as if this fact is what makes his wife’s death appear on the news. there are two remarks on the guardian about mr. tharoor seeking a quick investigation in order to ‘put an end to rumours about their personal lives’ and another on his complaints about the ‘reckless speculation’ on media reports, which suits a serious tone. in spite of the fact that the subject/theme shows to be more balanced on the new york times the processing of the information is more concise on the guardian. the different tone used by the new york times is coherent with the less objective, unnecessary, additional information. 5. references alberdi, i. and matas, n. equipo de investigación género y medios de comunicación. (2006). tratamiento de la violencia de género en la prensa vasca. san sebastian: universidad de deusto. bach arús, m. (2000). el sexo de la noticia. barcelona: icaria. bengoechea, m. (2011). el proceso de nombrar el mundo en femenino y algunos efectos secundarios no buscados. in taillefer de haya, l. (ed.), la igualdad: nuevas perspectivas de género en educación, lingüística y filosofía (pp. 149-181). málaga: cedma calero fernández, m.a. (1999). sexismo lingüístico. análisis y propuestas ante la discriminación sexual en el lenguaje. madrid: narcea dardis, f.e. (2006b) “military accord, media discord. a cross-national comparison of uk vs us press coverage of iraq war protest.” the international multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 56 communication gazette, doi: 10.1177/1748048506068719. dardis, f.e. (2006a) “marginalization devices in u.s. press coverage of iraq war protest: a content analysis.” mass communication & society, 9(2), 117–135. fernández díaz, n. (2003). la violencia sexual y su representación en la prensa española 1989-1993. barcelona: anthropos. fondevila gascón, j.f. (2012) “ el uso de recursos del periodismo digital en la prensa del reino unido, francia, estados unidos y españa”. estudios sobre el mensaje periodístico, 18(1), 73-87. gallego, j. (dir.) (2002). la prensa por dentro. producción informativa y transmisión de estereotipos de género. barcelona: los libros de la frontera. gonzalez rodriguez, m.j. (2007). “on the interpretation of ideology through comment articles: two views in opinion discourse ”. rael-revista elecrónica de lingüística aplicada, vol. mono. 1: 49-68 gutierrez zornoza, m.(et al) (2009). violencia doméstica contra las mujeres en la prensa escrita. cuenca: universidad de castilla-la mancha. menéndez menéndez, m. i. (2010). representación mediática de la violencia de género. análisis de la prensa balear (2004-2008). palma (illes baleasrs): uib ozieblo, b. (ed.) (1993). concepto y metodología en los estudios sobre la mujer. atenea estudios sobre la mujer. málaga: universidad de málaga. taillefer de haya, l.(ed.) (2011) la igualdad: nuevas perspectivas de género en educación, lingüística y filosofía. málaga: cedma vera balanza, m. t.; ballesteros garcía, r. m. (coords.) (2004). mujeres y medios de comunicación. imágenes, mensajes y discursos. málaga: universidad de málaga multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 57 6. appendix news 1. death of sunanda pushkar. new york times, january 19, 2014 page a10, asia pacific. after love unravels in media, indian politician’s wife is found dead by betwa sharma (photograph) sunanda pushkar, left, and shashi tharoor at their wedding reception in 2010. she was found dead in new delhi on friday. creditassociated press new delhi — he was a smooth-talking politician from south india with an international reputation from his time at the united nations. she was a beautiful kashmiri from the north when they met in 2009 in middle age and began a relationship partly played out in the media, and social media, glare. now after public allegations that he was having an affair, the woman is dead, in what a doctor called an “unnatural, sudden death.” her husband, shashi tharoor, briefly checked himself into a hospital with chest pains. and twitter is alive with virtual handwringing over whether the medium allowed a dangerous level of overexposure. the unraveling of the couple’s personal lives began last week, when the wife, sunanda pushkar, was quoted as saying she sent out a series of messages on twitter that she said proved that a pakistani journalist was pursuing her husband. days later, on friday, he reported finding his wife dead in a hotel bed. on saturday, sudhir gupta, one of the doctors who performed an autopsy, told reporters that “there were certain injury marks on the body, but we can’t divulge details at this point.” he ruled out poisoning and said it would take several days to determine the cause of death. (photograph) shashi tharoor, right, a minister in the indian government, sat in an ambulance with the body of his wife, sunanda pushkar, in new delhi, india.creditmoney sharma/european pressphoto agency the death of ms. pushkar, 52, stunned many in new delhi’s journalistic and political circles, many of whom had followed the twists in her life since she met mr. tharoor, a high-profile member of the governing indian national congress party. married twice before, ms. pushkar, became mr. tharoor’s third wife in 2010 in the wake of a scandal that led to his resigning as junior minister for foreign affairs. the scandal centered on accusations that he had used his official influence to help her get an ownership stake in a franchise in india’s cricket league. he denied any financial wrongdoing. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 58 after their marriage, when he was again needled for the scandal, the two of them — both prolific users of social media — turned to twitter and the media to discuss their relationship. he called her “priceless,” leading to some ridicule for his public display of affection. she then told a tv interviewer, “shashi and i show our love with p.d.a.” on wednesday, ms. pushkar, who seemed to have gained access to her husband’s twitter account, began posting messages she said he had received from the pakistani journalist. one said the sender was “in love with you, irrevocably, irreversibly.” the drama between ms. pushkar and mr. tharoor was covered heavily on india’s 24-hour news channels. in interviews, ms. pushkar sounded despondent and sometimes dazed. she told the indian express, a popular newspaper, that her husband had been in a “rip roaring affair” since april, and that she would seek a divorce. “that woman pursued and pursued him,” she was quoted as saying. “men are stupid anyways.” she then added: “for all you know she is a pakistani agent.” the journalist, mehr tarar, denied having an intimate relationship with mr. tharoor, and responded on her own twitter account, “i am not an isi agent,” referring to pakistan’s main spy agency. “or cia. or mossad. or even the dead kgb. may i go now?” ms. pushkar and mr. tharoor then issued a joint statement playing down the episode. “we are happily married and intend to remain that way. sunanda has been ill and hospitalized this week and is seeking to rest,” they said. the statement also vaguely suggested that the messages in question from their accounts had been “unauthorized.” on friday, he returned from a major gathering of the congress party to a suite at a luxury hotel in new delhi, where he found his wife dead, fully clothed and lying on the bed, said abinav kumar, his personal secretary. he said there were no signs of foul play. mr. tharoor, formerly a top united nations official, was one of the first politicians in india to use twitter intensively, and infuriated many hindus in 2009 when he sent out a message describing traveling in coach on a flight as “cattle class out of solidarity with all our holy cows.” hindus revere the cow, and the post almost cost him his political career. on friday night, sunanda puskhar was being discussed in the media and trending on twitter as people posted messages conveying shock and sadness over her death, with many commenting on the role the social media played in her last days. suhel seth, a public relations specialist and a friend of the couple, said on tv that twitter had caused the “decline of many” and that now it had led to the “demise of a very, very fine human being.” ms. tarar, the woman accused of pursuing mr. tharoor, wrote about the announcement of ms. pushkar’s death on twitter: “i just woke up and read this. i’m absolutely shocked. this is too awful for words. so tragic i don’t know what to say. rest in peace, sunanda.” the guardian, jan 20, 2014 page 13, international. indian minister testifies over wife’s sudden death reuters new delhi multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3688 social and technological sciences eissn: 2341-2593 bustinday (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 47-59 | 59 an indian government minister, shashi tharoor, appeared yesterday before a magistrate investigating his wife’s death days after she accused him of adultery. photograph: prakash singh/afp/getty shashi tharoor and his late wife, sunanda pushkar, at the indian parliament in 2012 sunanda pushkar was found dead in a hotel room in delhi on friday after sending tweets that suggested her husband was having an affair with a pakistan-based journalist, mehr tarar. an autopsy found that the 52-year-old pushkar’s death was sudden and unnatural, and her body bore injury marks, although doctors said this did not mean the injuries had caused her death. tharoor, a former un diplomat, called for a quick investigation into his wife’s death, saying he hoped this would put an end to rumours about their personal lives. in a letter to india’s home minister, sushilkumar shinde, tharoor wrote: “i have finally had a chance to catch up with media reports and am horrified to read the reckless speculation rampant there. “i pledge my full and unstinting cooperation. nothing short of truth will end the indignity to which my wife and i are being subjected.” later yesterday, tharoor, 57, gave his testimony to a magistrate, alok sharma, who is conducting the inquest. there was no word on what he had told the magistrate and he made no comment. the scandal has erupted as tharoor’s congress party, led by rahul gandhi, is preparing to fight a tough election against a resurgent main opposition party as well as a new political group that promises clean and open politics. rivals have painted congress as a party of power and patronage, engulfed in corruption scandals and unable to hold its leaders to account for their actions. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 digital signage: an experience of innovation in higher education vanessa roger-monzó*, maría guijarro-garcía, myriam martí-sánchez dpto. de comunicación. esic business & marketing school, blasco ibáñez, 55, 46021 valencia, spain. * corresponding author: email: vanessa.roger@esic.edu; phone: + 34 963614811 abstract this project on teaching innovation was developed in esic business & marketing school valencia and is based on the educational opportunities provided by digital signage. this paper describes the different phases undertaken to implement a flexible teaching method that counted on the participation of 43 third-year undergraduate students of communication and public relations during two consecutive academic years (2013-2015). the objectives of this project are to promote educational innovation through the creation of a work structure led by teachers and to encourage students´ independent learning in the process of constructing knowledge through the development of the most appropriate visual designs for the transmission of information and advertising messages from the school. to achieve this, students have used photoshop, flash and premiere software. the obtained results show that this project helps to encourage assimilation and implementation of key concepts in the field of creativity, graphic design and content management. the project also supports that teachers involved in the project should improve education quality by pooling and making the most of the best practices they have implemented on new methodologies in teaching and the development of their subject matters. keywords digital signage; intra-university management and information planning; teaching innovation; teamwork; independent learning roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 1 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 1. introduction the creation of the european higher education area (ehea) implies various and profound changes in teaching. baena and padilla (2012) consider that the most substantial change might consist in the harmonisation of qualifications across europe in order to be adapted to the current and future needs of students and the labour market. however, an essential aspect of the ehea, which lies at the basis of this innovation project, is to improve teaching quality in universities and boost student skill acquisition (cano, 2008). the evolution of the training needs that society demands has encouraged institutions of higher education to implement an educational strategy meant to ensure students` access to knowledge and skills for lifelong learning. in this context, it is essential to achieve flexibility in teaching methods in order to adapt them to the skills that students must possess. therefore, universities should promote innovative experiences in the teachinglearning processes in order to train students in the skills they will carry out in their professional life, since the current labour market is characterised by being rapidly changing and competitive. this statement is endorsed by the framework document on the integration of the spanish university system in the ehea that emphasises the necessity that european-accredited qualifications should provide access to the labour market. the educational objectives of the official undergraduate training will generally have a vocational orientation, i.e., they must provide university education with a harmonious integration of core generic skills, transversal skills related to the comprehensive personal development and more specific skills for vocational orientation that should ease graduates` integration into the labour market. in this regard, it is imperative to harmonise the design and development process of the official undergraduate training and the officially recognised qualifications in other european countries in each of the scientific, technical and artistic fields. at the same time, there must be a close collaboration between academic coordinators and those in the associations and professional bodies (ministry of education, culture and sports, 2003). canzer (1997) points out that teachers should support students in their acquisition of skills and make sure they will be able to apply them in their professional field. similarly, there is evidence that students who work together and collaborate in the achievement of their goals develop skills and competencies that the traditional method can hardly provide (exley and dennick, 2007). wright et al, (1994) agree with this line and emphasise that active student learning can originate skills like leadership, teamwork and task distribution; on the contrary, a passive learning hinders acquisition. baena (2010) stresses the advantage of developing active learning, which should make students lead their learning process for their satisfactory integration in their workplace. thus, the classical teaching-learning structure based on a lecture given by the teacher and in which students act as passive recipients should be replaced by a strategy to encourage their active participation. the mere transmission of knowledge and procedures must give way to a roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 2 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 scheme of acquisition of new skills, abilities and competencies through self-learning (martin et al, 2014). this new situation can be possible with the introduction of innovative teaching projects that include practical work sessions, debates and seminars where students can have a dynamic personal and team interaction that should lead to the acquisition of skills that, in turn, will help them to lead their own learning (whitehead, 2008). to achieve that, university teachers should alter their relationship with students, so that the vertical teacher-student hierarchy should be replaced by a horizontal structure where the teacher mainly plays the role of an advisor (criado and moreno, 2009). this paper describes the different phases carried out to implement a flexible teaching method that should provide students access to the skills and abilities to work in advertising, marketing and communication. the objective of this project is to promote student autonomous learning of these subjects by developing the most appropriate visual designs for the transmission of informative and advertising messages in the school. 2. description of the educational innovation project digital signage (also called dynamic marketing or dynamic communication) is a system of communication frequently used in the field of digital advertising that allows management, distribution and multimedia content display through various screens. in any case, all terms converge towards a single concept: a communication vehicle that replaces the traditional advertising poster with a "digital sign" that can stream video and reproduce multimedia contents. for this reason, the result of this innovation project is an “audiovisual poster” created and programmed by students. for this purpose, students use graphic design applications, specifically, photoshop, flash and premiere software. digital signage becomes therefore a novel communication channel based on digital content. this instrument provides the basis for the construction of this educational innovation project because it is currently a very popular communication channel for digital content in business. the research on total investment in digital advertising in 2013, prepared by iab spain, includes for the first time the investment in digital signage, which is above €6 million (€6.1 million), which means almost 1% of the total digital investment. digital signage has two major advantages. firstly, the information is not static, since the user can be exposed to different messages that change in a given period of time. secondly, the content grid can be managed automatically from a single source of emission, which means that messages can be programmed or changed in real time from a single computer and displayed in different formats enabled for this purpose. all these factors allow the creation of a communication channel of its own. roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 3 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 as it is adaptable to the needs of each organisation, digital signage offers diverse functionalities such as corporate communication, the transmission of informative messages and live or recorded events. in the present educational project there are three communicative alternatives disseminated through television screens. the main objective of this educational innovation project is to create a flexible teaching method that should encourage the implementation of essential key concepts in terms of creativity, graphic design and content management in order to enable learning professional aspects. on balance, digital signage is considered a tool that adapts to the acquisition of skills required by students. in this case, the new information and communication technologies are closely linked to fields like marketing and advertising. in addition, this experience requires not only the active collaboration of students and the involvement of teachers, but also the cooperation of the whole educational structure. in fact, the novelty of this project consists in the commitment of the university to assume it as a global initiative that involves students, teachers and administrative staff. the different areas of the centre involved the achievement of the educational innovation project were the department of external communication, esic entrepreneurs, the department of international relations, the department of in-company training, inmetur innovation centre, the library and esic language department. regarding the students, the educational innovation in this project is to move certain skills outside the classroom in order to develop them in a professional environment, in this case, in an educational institution. the educational innovation project was conducted in the third year of undergraduate studies in communication and public relations in esic business & marketing school in valencia during the academic years 2013-2014 and 2014-2015. the subject that incorporated the innovation project was art direction and advertising production and counted with the participation of 43 students. the academic departments responsible for the project were two: advertising and it. the implementation of the present educational innovation project covers issues that, according to salinas (2004), should be considered in an experience of this kind: • a support system for teachers that includes their further training in the use of information and communication technologies (ict) in teaching; in this case, teachers and administrative staff in the communication department involved in the project attended a seminar about the functioning of a content management software for digital marketing and advertising. they also had access to ongoing roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 4 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 training in order to take advantage of all the communication options this system offers and which can lead to innovations in the teaching-learning processes. • a support system for students: students require not only training but also technical assistance and promotion policies for the use of ict. in relation to this factor, the students involved in the educational innovation project of digital signage received the necessary guidelines to acquire skills in different fields: selection, information management and planning, graphic and visual design and communication skills, in order to present a digital poster on the new channel. • team policy: the organization of the members of the project is an essential element to consider. in this regard, the presence of a project manager in this project is relevant, as well as that of a content coordinator and a quality supervisor, who worked together throughout the process. thus, although independently listed below, the actions taken by the members of the innovation project were always coordinated, which created synergies that enriched the project. • the role of project manager was to establish a protocol for all the parties involved in the project (teachers, administrative staff and students) with the aim of helping students to get naturally closer to the business side of an educational institution through activities related to the implementation of the digital signage communication channel. • the contents coordinator designed the methodology for learning and active student participation so that students could develop their own channels of communication for the transmission of university-related information. to do so, special attention was given to the teaching and learning about the necessary tools to create such information. in this aspect, students` teamwork is very important, as it is meant to help them to acquire skills that should allow them to address new situations and needs in fields like marketing and communication. the training programme also included an in-person tutorial class structured in short sessions. • finally, the functions of the quality supervisor are transversal because they have at their core an action of control that evaluate initiatives of implementation of innovative methodologies for learning and active participation. the main role of the quality supervisor in the project was to prepare a questionnaire that students had to complete in order to evaluate the extent to which the project was accepted and the objectives were met. • network, hardware and software infrastructure: teaching based on ict needs strategic lines aimed at creating a technology plan that should include specific resources for the development of the innovation project. in this case, the actions taken by the project manager were those of acquisition of licenses for a content management software, as well as of various screens for the transmission of the created messages. furthermore, and prior to the implementation of the dynamic roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 5 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 communication channel, esic had equipped six computer labs with the audiovisual editing and graphics software for the development of digital signage material, as well as other purposes. 3. methodology in summary, the educational innovation project allowed each group of students to work for a week in a collaborative way on the research, selection, editing, design, planning and release of contents related to the academic year, as well as on their presentation as digital posters. specifically, the current project aimed at promoting measures to improve teaching quality through the implementation of active and participatory methodologies meant to facilitate and improve the teaching-learning process. this project considered the following actions: • the development of an innovative methodology for learning and active participation for knowledge construction in order to encourage the development of autonomous learning. • the design of active processes for skill and ability acquisition for professional practice. these processes have therefore been moved from the classroom to the real company. • the study and design of evaluation processes according to this methodological update. • the design and development of specific material and resources for the implementation of these new methodologies. • the development of the tutorial activity. thus, the consequent methodology to carry out this innovative teaching strategy includes the following aspects: • the development of the necessary basic skills to pass the subject of art direction and advertising production: to do this, students had to internalise the theoretical and practical concepts aimed at creating a visual work (or digital poster) so that they could later adapt it to the digital signage channel. in this case, it was essential for students to practice informational, corporate and advertising content development expressed in the most semantically and visually accurate way possible to communicate a message effectively. • use of ict: students had to learn how to operate the content management system to create the communication channel, as well as the graphics and audio-visual editing software for the design of an appealing presentation of the message they wanted to transmit. in addition, they had full access to these tools in the computer rooms. the digital signage content manager a software for programming audio roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 6 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 visual posters was set up on two terminals in the it department in order to have direct control of its use. similarly, the six computer rooms in the university had the necessary software for the design of audio-visual posters. on the other hand, a classroom was booked for the in-person question-answer sessions about information selection or its presentation. • teamwork: students, in groups of three, developed and projected informational, corporate and advertising contents in a digital poster format. cooperative learning means greater motivation for students. however, teachers have to make sure students work in teams and not allow them to work individually on their part of the assignment (polanco, 2005). the structure of the innovation project was designed to specifically address this aspect. thus, the guidelines created for this project underline the active and collaborative participation of each student to achieve the objective to create an adequate message for a visual composition from a communicative point of view and adapted to a dynamic communication channel. • evaluation: the assessment of the practical part considered in the syllabus of the subject involved in the project allocates 30% of the final mark to digital signage content design in groups. the development of highly creative and communicative contents by each team and therefore their possible release on the company`s channel is a sign that the work met the parameters required by the company and, therefore, this aspect is rewarded with a more positive assessment. in any case, whether the designed posters were selected for release or not, every student had access to the software for remote content programming. • revision of results: at the end of the semester, students completed an evaluation questionnaire that assessed the degree of effectiveness of the educational innovation. thus, it was possible to measure students’ perception of the project and the extent to which improvements affected student training. more specifically, the innovation project based on digital signage has the following structure: as mentioned before, students were divided into groups of three. this planning demanded each of the members to develop relevant skills for negotiation, discussion of doubts, productive dialogue and, finally, adequate planning in order to achieve the ultimate objective. each group of dynamic marketing had to complete their work over a week, whose result was two digital signage posters. thus, every wednesday, the team in charge of the practical work chose topics on which they had to obtain relevant references that became the starting point for the messages in the digital poster. consequently, from the data supplied by each of the departments involved in the project students had to select the information likely to become the message for the digital signage channel. they also set a roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 7 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 specific target audience for their messages (undergraduate and graduate students, alumni, etc.). once that the topic and the target of the message were decided, students had to develop the most appropriate communication strategy for each case. therefore, they had to get together, propose, discuss and debate ideas in order to find the most suitable communication design for the intended content, target audience and emission channel. after structuring the message, the group had to design the contents for its presentation in the format of a digital poster. in this case, since the messages was going to be projected on high-definition screens, the images and videos of the digital signage system had to be adapted to 1280x720 pixels. after the digital posters were ready, the group had access to the content management software to learn how to operate it. the content management software allows embedding videos, texts, images, web pages, flash animations, audios, etc., so that users can combine and save them according to their needs. through this specific software for digital signage channels students carried out the remote programming of the messages they had created. their posters were included in the emission grid of the dynamic communication channel. finally, besides the audio-visual posters, each group presented a report on the digital signage practice that included the description of the planning and explanation of the created design. this report was delivered one week after the end of the practical part of the project. 4. obtained results the aim of the educational innovation project is to make students of art direction and advertising production work on real cases and at the same time design specific contents for the digital signage channel of the educational institution. this facilitates collaborative work and encourages a horizontal and egalitarian relationship between participants. the evaluation of the objectives of the innovation project was carried out with a questionnaire based on a five-point likert scale and was completed at the end of the term. table 1 shows the technical sheet of the research, while table 2 reflects the profile of the surveyed students. roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 8 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 table 1. technical sheet of the research universe university students of esic valencia, degree of communication and public relations. subject: art direction and advertising production (3rd year) geographic area valencia, spain. sampling design personal survey sampling size 35 valid surveys field work 2014/12/17 statistical techniques descriptive analysis statistical programme spss versión 21.0 minimum maximum average typ. desv. 1. i was interested in the digital signage project 1 5 3,60 1,090 2. my interest in educational innovation projects has increased after my experience in digital signage 1 5 3,77 0,910 3. i think the structure of the digital signage project is correct 3 5 4,20 0,677 4. the quality of the acquired practices is good 2 5 3,94 0,906 5. a more extended digital signage project could have been carried out 1 5 3,56 1,050 6. i find the digital signage project intellectually stimulating 1 5 3,88 0,880 7. i learned things that i consider valuable 1 5 3,77 1,140 table 2. profile of the surveyed users characteristics % gender men 28,6% women 57,1% age average 22,19 typical deviation 2,370 minimum 20 maximum 29 table 3. descriptive statistical data roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 9 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 8. my interest in the subject has increased as a result of the digital signage project 2 5 3,83 0,822 9. i understood the communication strategies of this project 1 5 3,83 0,954 10. altogether, i improved my collaboration with colleagues 2 5 3,69 0,963 11. i would recommend the digital signage project with similar characteristics to my colleagues 2 5 3,89 0,758 12. overall, i am pleased to have completed this digital signage project 2 5 4,20 0,677 13. i would like more projects of this kind to be carried out 2 5 4,20 0,797 14. i would participate in another digital signage project at the university 1 5 3,89 1,022 in table 3 the following items stand out: 3 "i think the structure of the digital signage project is correct", 12 "overall, i am pleased to have completed this digital signage project", and 13 " i would like more projects of this kind to be carried out ", being the best rated with an average score of 4.20. in contrast, items 1 "i was interested in the digital signage project” and 5 " a longer digital signage project could have been carried out“ are less valued, with an average score of 3.60 and 3.56, respectively. 5. conclusions knowledge is neither final nor fixed but is dynamic and adapted to the context in which it is integrated. that is why it is essential that teaching evolve and adapt to the needs demanded by society at all times. students play an essential part in this project, as they have the opportunity to meet the preferences of their "client" and, thus, they can help improve their corporate image and at the same time get involved in the actual operation of a company, in this case, of an academic nature. for whitehead (2008) teaching should enable students to perform individual work as well as teamwork, which would allow them to become the leading part of their own learning. in this project, the practices to be developed take into account this context. in fact, by using seminars and workshops in order to properly plan and manage information coming from the centre´s different departments, and by mastering tools of graphic and audiovisual creation, students have organized their own college communication channel. with this method of learning, the simple transmission of knowledge has been replaced by a roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 10 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 teaching strategy that involves acquiring new capabilities and skills through self-learning. in this sense, the idea remarked by martin et al (2014) is observed. moreover, the relationship between the teacher and students has acquired a horizontal structure, relegating the traditional vertical hierarchy. as highlighted by criado and moreno (2009), in this project teachers have mainly played an advisory role. to this end, an in-person tutorial class has been developed and organised in short sessions aimed at providing students with the skills and capabilities to enable independent learning for professional practice in advertising and communication. therefore, the objective of this project of teaching innovation is that certain competencies should be transferred outside the classroom, in order to develop them as much as possible in the professional context of an educational institution. the overall assessment of the project is positive on basis of the questionnaires, as the creation of a digital signage channel encourages collaboration among students, university and administrative staff from different areas, because it sets the outcome of the assignments in open environments. thus, students publish their creations on the new communication channel of the university. this fact motivates the collaborative and creative efforts of students under the guidance of the teacher and the indications received from the various departments of the center. similarly, it has been shown that the structure and duration of the project of teaching innovation have been adapted to students´ initial expectations. as reflected in the results, students greatly appreciate such initiatives. however, although these results are above average, it is advisable to develop the necessary actions in a new way to encourage their initial interest in the project and, above all, to increase student teamwork. this is because, as several authors point out, collaboration among students to achieve their goals means further development of skills, capabilities and competencies that the traditional method does not provide (exley and dennick, 2007, wright et al, 1994). it should be noted that item 10 "altogether, i improved my collaboration with colleagues" has a score of 3.69. this is an above average result, but very improvable. it is therefore necessary to determine prior to the development of future editions of this project to what extent students consider they already work together. if they believe their collaboration is adequate, it will be necessary to maintain it during the implementation of the digital signage project. in any case, the need to define the necessary strategies to encourage collaboration with colleagues is imperative. as this experience was well received, the design and management of a digital signage channel by students can help to improve teaching quality because they can provide to a large extent skill acquisition for professional environments. therefore, the project is also roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 11 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 carried out in the official degree of business management and marketing through the subject of multimedia technologies. this innovation is being implemented over the second semester of the academic year 2014-2015. on balance, this strategy can be considered valid to stimulate student acquisition of skills, since it encourages the assimilation and implementation of key concepts related to creativity, graphic design and content management. in addition, the teachers involved in the project can improve education quality by pooling and leveraging the best practices they have implemented in the field of new methodologies for teaching and subject development. 6. references baena, v. and padilla, v. (2012). refuerzo y desarrollo de competencias mediante la elaboración de una campaña real de marketing: la formula uem. revista de docencia universitaria, 10(1), 199-214. baena, v. (2010). innovación docente e identificación de inhibidores del aprendizaje en el área de empresa: una propuesta metodológica. espiral. cuadernos del profesorado, 3(6), 3-14. cano, mª e. (2008). la evaluación por competencias en la educación superior. profesorado. revista de curriculum y formación de profesorado, 12(3), 1-16. canzer, b. (1997). marketing education on the internet: a world wide web based introductory marketing course design for the virtual-u project in distance education at simon fraser university. journal of marketing education, 19(3), 56-65. http://dx.doi.org/10.1177/027347539701900306 criado, r. and moreno, a. (2009). un ejemplo de desarrollo de competencias en el contexto universitario de la tele-enseñanza. relada-revista electrónica de adamadrid, 3(2), 118-126. exley, k. and dennick, r. (2007). enseñanza en pequeños grupos en educación superior: tutorías, seminarios y otros agrupamientos. narcea ediciones. iab spain (2014). inversión publicitaria en medios digitales. resultados de 2013. http://www.iabspain.net/wp-content/uploads/downloads/2014/03/informeinversi%c3%b3n-publicidad-total-a%c3%b1o-2013_reducida.pdf martín, a.; león, c. and garcía, a. (2014). innovación docente para la integración de autoformación y autoevaluación en la plataforma web. pixel-bit: revista de medios y educación, 44, 201-214. http://dx.doi.org/10.12795/pixelbit.2014.i44.14 roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 12 http://dx.doi.org/10.1177/027347539701900306 http://www.iabspain.net/wp-content/uploads/downloads/2014/03/informe-inversi%c3%b3n-publicidad-total-a%c3%b1o-2013_reducida.pdf http://www.iabspain.net/wp-content/uploads/downloads/2014/03/informe-inversi%c3%b3n-publicidad-total-a%c3%b1o-2013_reducida.pdf http://dx.doi.org/10.12795/pixelbit.2014.i44.14 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3769 social and technological sciences issn: 2341-2593 ministerio de educación, cultura y deportes. (2003). la integración del sistema universitario español en el espacio europeo de enseñanza superior. documento marco. madrid: mecd. polanco, a. (2005). la motivación en los estudiantes universitarios, actualidades investigativas en educación, 5(2), 1-13. salinas, j. (2004). innovación docente y uso de las tic en la enseñanza universitaria. revista universidad y sociedad del conocimiento, 1(1), 1-16. wright, l. k.; bitner, m.j. and zeithaml, v. a. (1994). paradigm shifts in business education: using active learning to deliver services marketing content. journal of marketing education, 16 (3), 5-19. http://dx.doi.org/10.1177/027347539401600303 roger-monzó et al. (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 1-13 | 13 http://dx.doi.org/10.1177/027347539401600303 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 diagnosis about integration process of youth foreign catalan marta sabariego puig*, berta palou julián, esther luna gonzález facultat de pedagogía. universidad de barcelona passeig de la vall d’hebrón, 171 2ª planta – edificio llevant 08035 barcelona teléfono: 93 403 52 20 fax: 93 403 50 11 * corresponding author: email: msabariego@ub.edu; received: 2015-2-14; accepted: 2015-4-12 abstract this paper presents the results of a diagnostic study on the real integration process of young migrants in catalonia. the study was carried out using a descriptive survey of 3,830 young catalans from varying cultural backgrounds aged between 14 and 18, with the aim of identifying the key factors influencing the integration of catalan migrant youth. also, in order to analyze these key elements in greater depth as factors easing and/or obstructing integration, four discussion groups were held with the same young people. results reveal achievements and challenges for further study, useful for the design of social and educational policies which may promote the integration process, understood in its structural, social, cognitive-cultural and identitary dimensions. our study confirms the need for a society with pluralistic beliefs, principles and actions, which should be reflected in democratic systems and social and educational policies based on the concept of integration as reciprocity. keywords integration, youth, immigration, citizenship sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 79 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 1. the concept of integration various studies on integration in multicultural areas (aparicio and tornos 2000; favell 2001; borgström et al. 2002; palou 2011a; portes et al. 2011) stress two key aspects of the situation: firstly they verify that the coexistence of people of distinct cultural origins causes problems and difficulties, and secondly they also observe that immigrant groups experience many problems integrating with the host society (pumares, 1996; colectivo ioé, 1992, 1994 and 2002; gonzález, 2002; freixa, 2003; palou, 2011a). in accordance with these findings we have adopted martínez’ (2006) model, which structures the concept of integration into a multi-field framework detailing some of the major factors that explain a successful integration process in terms of its achievement in practice: • the structural field, referring to the individual’s participation in professional activities and social and political institutions on the basis of the principle of equality of opportunities. the variables in this dimension are: the legal situation (citizenship and residence), educational status and position in the labor market, economic resources, and access to housing and welfare systems. • the cognitive-cultural field, which embraces learning cognitive skills and managing the host society’s culture. the variables in this group are: knowledge of the language, political values, cultural values and norms, religious beliefs, and lifestyles. • the social field, referring to the individual’s social relationships inside and out of her/his community, and the availability of avenues of participation. the variables in this field are: the extent and identity of family relationships, contacts with sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 80 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 members of the person’s own community, contacts with local people, friendships, and ties with associations and social bodies. • the field of identity, which includes the subjective dimensions of belonging and personal identification with the community. the variables in this group are: subjective perception of belonging (national and cultural), attitudes of immigrants towards the host society and the latter’s attitudes towards them, the type and degree of identification with the host society, and the orientation of the host society towards intercultural relationships (martínez, 2006: 88-89). from this multi-field model we see that the key condition allowing us to speak of immigrants’ integration is the recognition of their status as citizens, where citizenship is understood to involve ethnic and cultural diversity progressing in this direction, towards a fully intercultural citizenship. this model implies a challenge that, as bartolomé and cabrera (2003) point out, demands open-minded responses taking on board different cultural identities; inclusive responses enabling equality and justice between all citizens; and democratic responses allowing civic participation for all in the construction of the multicultural society we live in. 2. research methodology this study aims to arrive at a diagnosis of the real process of young immigrants’ integration in catalonia, specifically from the ages of 14 to 18. we can break this aim down further into in two more specific objectives: 1. to identify the key elements in the current state of youth migrant integration in catalonia by means of a survey from two viewpoints: that of the youth of foreign origin and that of native young people. sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 81 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 2. to analyze in depth the factors which ease and/or obstruct integration, as well as the patterns of these young people’s integration process in the catalan context, basing ourselves on the key fields of integration (martínez, 2006 and palou, 2011b). to address these objectives a mixed-method study was carried out (creswell, 2003; burke and onwuegbuzie, a. j, 2004), comprising a descriptive survey of the young people combined with discussion groups. in this way complementarity between research methods and techniques is built into our analysis. this strategy enables us to study in greater depth the key factors affecting the current state of migrant youth integration in catalonia, from their own experience and in the first person. the three phases making up the study were: • phase i: selection of and access to informants by the research team. • phase ii: data collection by the team using the questionnaires and discussion groups. • phase iii: analysis by the research team of the data obtained and development of conclusions as a basis for designing initiatives favoring coexistence and social cohesion. 3. data collection techniques our information-gathering techniques conform to the study’s aims and design features. the techniques used were: for the survey, an adapted version of the “social cohesion amongst young people questionnaire” (palou, 2010). a total of 3,830 young people aged sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 82 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 between 14 and 18 took part, from varying cultural backgrounds, intentionally selected from 52 catalan secondary schools, mostly (72%) in compulsory secondary education (up to age 16). four discussion groups with the young people, both migrants and natives, designed to analyze in depth how they themselves experience the process of integration and what they consider to be the key factors facilitating it. 4. results using martínez’ (2006) four fields we present below the results obtained from the diagnostic study. in the structural field, the normative aspects of integration are defined according to the participants’ opinions on who can be considered a citizen. 31.3% of the young people thought that citizens are those who have their “papers in order”, tailed closely by 30.8% whose opinion was that citizens are those who “live and work here”. this concept of citizenship predicated on “legalized” administrative status was accompanied by the stereotyping of the immigrant community as a social category marked even more strongly by the current context of economic and financial crisis. biased representations and negative views, images and attitudes towards immigration, mediated by a logic of excessive differentiation (delgado, 1998), were confirmed, problematizing the “different other” on two levels: 1. the discourse of social panic. when asked what they thought about migrants and what they believed society thinks, almost half the native participants said that they are thieves, criminals, and trouble-makers, that they are bad and violent, that they cause fights, create problems, lack respect, and make coexistence difficult: sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 83 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 “i don’t care if they come … what i don’t like is … that the ones who come, come to steal things …”; “and we don’t realize that … you get all kinds of people all over. …normally it’s true that immigration is associated with vandalism, in general…” (male native students, 11th grade). school students of foreign origin confirm this socially constructed perception of immigration which associates them with poverty, lack of education, and a uniform, stereotyped image (“they’re all the same”): “—this guy’s from colombia, so he’s sure to be one of those low-life types, and this one’s chinese, so he’s sure to be quiet and shy and you can get away with anything with him—so people see it that way …” (female uruguayan student, 11th grade). “people discriminate against you because you’re for example rumanian, and they say that you steal and do bad stuff and that’s the reputation that rumanians and colombians have, that they’re all drug dealers, and that’s the bad image they give you…” (male colombian student, 8th grade). 2. the discourse of competition: immigration is associated with loss of resources (housing, work, access to social services), and this strengthens natives’ rejection of migration. thus most of the native young people claimed that society treats migrants better (it gives them social assistance, aid and grants, they receive more benefits than we do, they have jobs and housing provided, etc.). this criticism of the favored treatment of the immigrant community (positive discrimination) is one of the factors which most obstructs integration, together with a negative reception from the host society which stems from attitudes of aversion and/or social stereotyping: “i knew people too who always held it against me saying i got benefits because i couldn’t buy the school books because they were really expensive. they said, ‘huh! sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 84 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 they help you but my mum had to spend 300€ on those books,’ and that makes me feel bad too, because i didn’t want to be different” (male chilean student, 11th grade). “once it happened to me that they wouldn’t let me into a house. when i didn’t wear a veil they did let me in, and when i wore a veil they didn’t let me in, when i wore it they said, ‘don’t come in!’ it was a spanish friend’s house”. (female maghrebi student, 8th grade) regarding reasons for migration, it is noticeable that the students, both natives and migrant, and although they may evidence a stereotyped view of immigration, agree in recognizing the same factors as expectations and motives behind migration: one of the main reasons for emigrating is to improve your quality of life (economic improvement resulting from wage increases): “here there are loads of things you don’t have in our countries: social services, health, work…” (maghrebi student, open center). “they’re people who emigrate to another country for various reasons. like for example for a better future” (native student, 8th grade). in the social field, the survey and the information yielded by the discussion groups confirmed the following results profile according to the contexts (a) of the school, and (b) of public areas. in the school context, native young people participated more in classroom and school activities, including those involving a certain responsibility, while young people of foreign origin were more active in their neighborhoods. 32% of the young people had been classroom representatives, 31% took part in a range of school meetings and groups, and 24% in organized out-of-school activities and solidarity projects. however, the sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 85 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 migrant young people participated more in all out-of-school activities (x2=10’02, p=0’007, α=0’05), and took part less as year representatives (x2=10’6, p=0’005, α=0’05) and in school meetings (x2=62’4, p=0’000, α=0’05). in general these adolescents did not consider the school as the ideal area for integration and felt more comfortable in out-ofschool activities of a more ludic type, responding more to their needs, thus confirming the results of previous studies (palou, rodríguez and vilà, 2013). amongst the most common mixed activities favoring intercultural coexistence, theatre groups (x2=17’4, p=0’000, α=0’05) and dance groups (x2=121’4, p=0’000, α=0’05) stood out as preferences for the girls, and sports groups (x2=450’4, p=0’000, α=0’05) for the boys. when asked about their assessment of cultural diversity in educational institutions, 38% stated that it awakened interest in other cultures. conversely, 37% also said that it caused more conflict, concurring with previous studies (freixa, 2003; alegre, 2005; palou, 2011b). if this data is analyzed taking into account the origin of students, foreign-born young people are more favorable towards working in multicultural groups. they highlighted the enrichment of the discussions emerging and were less sensitive to the comprehension difficulties and reduction in quality which, on the other hand, the native young people mentioned in their assessments. only when immigration is given a real face and young people think about their migrant classmates do they express a more open-minded, respectful, and inclusive attitude in their personal relationships, along with their personal contacts at school. 43% valued the enrichment of work stemming from the diversity of viewpoints; and 35% also valued positively that discussions and debates were more interesting: “i think it’s good that there are people from other cultures because we can learn about their traditions and the stuff they do in their countries. i don’t know, i like having people from other cultures” (female student, 8th grade). sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 86 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 discussing intercultural coexistence in public space, 53% of participants stated that “i don’t mind if people from different cultures live in my neighborhood as long as everyone minds their own business,” thus favoring acceptance of cultural diversity and the fact of migration within the frame of coexistence, but without the recognition which would imply a step further for coexistence, and which was only considered by 17%. at the same time, a worrying 13% stated openly that “i don’t like people with cultures different from mine living in my neighborhood.” analyzing acceptance and recognition of migration in more depth, it is necessary to look at opinions on coexistence regarding groups of different origin. thus, while 80% of the sample accepted having spanish residents in their neighborhood, only 22% would accept roma neighbors. paradoxically, perception of social rejection in multicultural areas was only expressed by the young migrants. in the discussion groups these young people told stories involving lack of respect and problems encountered participating with natives on an equal basis. there was no evidence that the native young people were aware of this: “well i think it’s different and i go round with asian people, because i had a spanish friend, but she always laughed at me, so since then i didn’t want to have any more spanish friends, because i didn’t know how to speak spanish well, so she laughed at me and i didn’t like it. so i stopped going round with her and since then i’ve always had asian friends” (male bangladeshi student, 6th grade). turning to social relationships, although in general the participants evidenced relatively diverse relationships, their responses show statistically significant differences according to their place of birth. thus while the young catalans and those originating from other provinces of spain were those who associate more with natives (a certain cultural sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 87 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 incestuousness is observed) , the young migrants were those who more frequently have more varied friendships and associated less with natives. this relationship pattern was found in friendships both at school (x2=562’2, p=0’000, α=0’05) and in the neighborhoods. thus one factor which obstructs integration on the part of both young migrants and natives is their lack of affective social networks (lack of social capital) and their exclusive mixing with people of their own communities, stemming from primary socialization processes and ethnocentric and monocultural educational institutions. “well also, when you get here, if people avoid you, it could also be because their parents tell them, ‘don’t hang out with that lot because they’re bad and they do that stuff, don’t go round with them’;” “yes, a lot depends on the family. if parents bring up their kids in one way, you can tell them how things really are as much as you like because they’re not going to understand you, because that’s the way they brought them up” (male chilean student, 11th grade). these are factors germane to the social field of integration, the lack of which fosters neither social relationships nor participation with other members of the community, as this student says: “when i went to school for the first time i got to know other people at school because there was another moroccan kid in my class who helped me at first. i could speak arabic with him” (female maghrebi student, open center). ideally, close relationships and the availability of role models to identify with should play a key part in giving shape to young people’s future directions and ambitions (portes, vickstrom and aparicio, 2011). beyond these contextual factors, the migrant young sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 88 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 people indicated that personal qualities (respect, friendliness, empathy, sociability, extraversion) were features which boosted integration in the current social context: “being yourself. first change yourself and it depends as well. if you want to change the world change yourself first. if you don’t want racism in the world or with the spanish the first person who has to be non-racist is you” (male ecuadorian student, open center). native young people also recognized the need for the reciprocal closeness and empathy which integration involves: the recognition of the “other” starts with interest in and acceptance of their difference, thus confirming the two-way, voluntary and dynamic character of the process: “that’s what it is, also your intentions, if you want to get to know other cultures and people from another country, because it depends on you if you relate more to one person or not. it’s that, and also feeling comfortable being here” (native male student, 8th grade). in the cognitive-cultural field we found that the young people’s linguistic situation did not correspond to the bilingualism found in the catalan context. the majority of participants spoke only spanish with both family and friends (48% speaking spanish at home and 7% speaking both catalan and spanish with friends). the linguistic situation in the schools was somewhat different. the most frequent was the use of both catalan and spanish (47%), followed by 28% who said that they only used catalan as the lingua franca at school, and 21% who spoke only spanish. language and, more specifically, learning the language of the migrant’s host country, is the factor which all participants concur in indicating as the most important for integration into the host society: sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 89 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 “i think that language makes it happen, because it’s not the same if someone comes from latin america, where there are variations on spanish, and we understand each other the same because it’s still spanish, it’s still spanish. so it’s much easier getting across and making friends because they understand you. but if they come from some other country where the language is completely different, it’s going to be much more difficult for them, and it looks like the ones who are adapt best are people from south america” (native male student, 11th grade). their opinion was that lack of cognitive skills and ability to manage the host society’s culture (basically, lack of knowledge of the language) imposed a barrier to integration: “when i came here i didn’t speak catalan or spanish, so it was really hard. and then in first year they taught me catalan but they didn’t teach me any spanish” (male bangladeshi student, 11th grade). thus since catalan is also the linguistic medium in the socioeducational institutions, learning it is important for continuing with studies and attaining the success desired. in this sense both how reception is managed (in terms of the willingness and endeavor of the native population), and the setting up of educational initiatives (in host classroom, in tutorial guidance, in subjects like “education for citizenship”) and socioeducational initiatives (in open centers and social services) by the schools themselves, were considered basic to facilitating contact, relationships and, finally, integration amongst young migrants: “the open center has given us opportunities, and it helps you a lot … they help you find your way. here, apart from doing your homework, they help you find your way. yes, really everyone helps you. you make friends. i met soukaina here. also sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 90 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 the teachers help you a lot” (male migrant student, open center). another important variable in this field is the young people’s understanding and critical judgment of public issues. our diagnosis showed that they tended to be inactive when faced with problems in their surroundings, while their involvement showed a positive relationship to the closeness and meaning of the context in which such problems occurred: thus 53% of participants only took part in school-wide issues if they were asked to, and 28% stated that they always got involved, whereas in the classroom context, closer and more meaningful to them, we observe the opposite tendency: 66% said that they always got involved, compared with 31% who needed to be asked. in these contexts, on the other hand, knowledge and fulfilment of common rules of coexistence were highly stressed for the integration process. this is a key variable which almost half of those surveyed rated as very important and saw as very unfavorable if not fulfilled. as for the field of identity, we found the following on participants’ feelings of belonging: a majority (49%) felt catalan, while 17% felt spanish, 16% felt they belonged to their country of origin, and 10% felt both catalan and spanish. however, the young foreigners maintained two key elements which reaffirmed their ethnic-cultural identity: religion and cuisine. this data can also be read differently according to young foreigners’ length of residence in catalonia. thus 60% living in in the country for less than a year gave more importance to following the rules and behavior of their home countries, contrasting with those living here longer (only 31% of immigrant students resident for between five and ten years had the same opinion (x2=19’6, p=0’003, α=0’05)). sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 91 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 the time factor is a key one favoring integration into the host country. the young immigrants living here for more than five years were those who most often valued the lifestyles of people from both their place of origin and the host country; while almost half the young migrants living here for less than five years valued more highly the lifestyle of their home country (x2=20’1, p=0’002, α=0’05). the more expressive cultural features (cuisine, as against the way of dressing and music of their country) lost weight as models of cultural identity as time went on. lastly, the young people’s sense of civic belonging was mostly linked to the place where they lived. specifically, 71% of those born in catalonia felt that they belonged to the place where they were born, while only 37% of the foreigners felt that they belonged to their place of residence (x2=335’6, p=0’000, α=0’05). 5. conclusions our results in the four fields necessary to integration confirm that a society which is plural in its beliefs, principles and actions should be reflected in its democratic systems and in social and educational policies based on a concept of integration as reciprocity (maalouf 1999; bartolomé 2002; siguán 2003; chaib 2005); with integration understood as a fundamental principle for the management of diversity (delgado 1998; chaib 2005); and a right to difference grounded in the acceptance and recognition of cultural diversity and the building of social cohesion on the basis of equal participation in a common public culture, amongst everyone, with a voluntary acceptance of this responsibility. sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 92 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 6. references aparicio, r. and tornos, a. (2000). immigradoion and integration policy: towards an analysis of spanish integration policy for immigrants and cims. effnatis working paper 32. [en línea]. disponible en: http://www.mmo.gr/pdf/library/spain/paper32_upco.pdf bartolomé, m. (coord.) (2002). identitat y sociedad. un reto a la educación intercultural (131-161). madrid: narcea. bartolomé, m. and cabrera, f. (2003). sociedad multicultural y ciudadanía: hacia una sociedad y ciudadanía multiculturales. revista de educación, número extraordinario ciudadanía y educación, 33-57. burke, r and onwuegbuzie, a. j. (2004). mixed methods research: a research paradigm whose time has come. educational researcher, october 2004, 33, 14-26. http://dx.doi.org/10.3102/0013189x033007014 borgström, m., van boxtel, r., brouwer-vogel, l., casanova, m.a., fleck, e., gundara, j., hegarty, s., illner, j., jones, e., kovats, a., le métais, j., navarro, j., pfaff, u., sánchez, s., solano, c. and taylor, m. (2002). hacia una europa diferente. respuestas educativas a la interculturalidad. consortium of institutions for development and research in education in europe. [en línea]. disponible en: http://www.educacion.gob.es/dctm/ministerio/educacion/ifiie/lineas-investigacioninnovacion/educacion-intercultural/publicaciones-informes/informes/europa-respuestaseducativas-interculturalidad?documentid=0901e72b809c4517 creswell, j.w. (2003). research design: qualitative, quantitative, and mixed approaches. thousand oaks, ca: sage. chaib, m. (2005) ètica per una convivencia. barcelona: la esfera de los libros. delgado, m. (1998). diversitat i integració. barcelona: empúries. favell, a. (2001). integration policy and integration research in europe: a review and critique. in t. a. aleinikoff and d. klusmeyer (eds.). citizenship today: global sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 93 http://polipapers.upv.es/index.php/muse http://www.mmo.gr/pdf/library/spain/paper32_upco.pdf http://dx.doi.org/10.3102/0013189x033007014 http://www.educacion.gob.es/dctm/ministerio/educacion/ifiie/lineas-investigacion-innovacion/educacion-intercultural/publicaciones-informes/informes/europa-respuestas-educativas-interculturalidad?documentid=0901e72b809c4517 http://www.educacion.gob.es/dctm/ministerio/educacion/ifiie/lineas-investigacion-innovacion/educacion-intercultural/publicaciones-informes/informes/europa-respuestas-educativas-interculturalidad?documentid=0901e72b809c4517 http://www.educacion.gob.es/dctm/ministerio/educacion/ifiie/lineas-investigacion-innovacion/educacion-intercultural/publicaciones-informes/informes/europa-respuestas-educativas-interculturalidad?documentid=0901e72b809c4517 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3487 social and technological sciences eissn: 2341-2593 perspectives and practices. (pp. 349-399). washington, dc: brookings institute/carnegie endowment for international peace. martínez, j. (2006). ciudadanía, religión e integración social. en vidal, f. y martínez, j.l. religión e integración social de los inmigrantes: la prueba del ángel (pp. 6485).valencia: ceimigra. palou, b. (2010). cuestionario cohesión social entre jóvenes. en línia disponible: http://diposit.ub.edu/dspace/handle/2445/34192 palou, b. (2011a). la integración de la juventud de origen magrebí en cataluña. tesis doctoral. barcelona: universitat de barcelona. departamemt de mètodes i diagnòstic en educació. [en línea]. disponible a: http://tdx.cat/handle/10803/2363 palou, b. (2011b). análisis de los elementos configurativos de la ciudadanía como condición para la integración de la juventud de origen magrebí en cataluña, revista de investigación educativa, vol. 30, núm. 1, 181-195. palou, b.; rodríguez, m. and vilà, r. (2013). convivencia y participación de jóvenes en sant boi de llobregat (barcelona). revista de educación, 361, 252-278. portes, a.; vickstrom, e. and aparicio, r. (2011). coming of age in spain: the selfidentification, beliefs and self-esteem of the second generation. princeton university. [en línea]. disponible en: http://www.princeton.edu/cmd/working-papers/ilseg-papers. portes, a.; celaya, a.; vickstrom, e. and aparicio, r. (2011). who are we? parental influences on self-identities and self-esteem of second generation youths in spain. princeton university. [en línea]. disponible en: http://www.princeton.edu/cmd/workingpapers/ilseg%20papers/wp1102ewhoarewecorrectedsept2011.pdf siguán, m. (2003). inmigración y adolescencia. los retos de la interculturalidad. barcelona: paidós. sabariego et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 79-94 | 94 http://polipapers.upv.es/index.php/muse http://diposit.ub.edu/dspace/handle/2445/34192 http://tdx.cat/handle/10803/2363 http://www.princeton.edu/cmd/working-papers/ilseg-papers http://www.princeton.edu/cmd/working-papers/ilseg%20papers/wp1102ewhoarewecorrectedsept2011.pdf http://www.princeton.edu/cmd/working-papers/ilseg%20papers/wp1102ewhoarewecorrectedsept2011.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 65 students academic progress analysis at the escola politècnica superior of the universitat de les illes balears: background implications miquel roca*, yolanda gonzález, ramon mas, joan rossello, loren carrasco, francisco forteza, biel cardona, mercè llabrés, carlos guerrero universitat illes balears * corresponding author: miquel.roca@uib.es received: 2016-04-16; accepted: 2016-08-18 abstract in this work, the influence of the background of the university students is analyzed. in particular how the average mark of the students affects their academic progress. an anonymously collected data analysis is performed, among these data are the number of european credit transfer and accumulation system (ects) enrolled, the mark exams, average mark exams, access type, etc. conclusions of each considered degree are presented at the end of the work. keywords background, academic performance, access exams. http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 66 1. introduction one of the main concerns of degree management teams in spanish universities is the analysis of student academic progress, in order to find the main causes when this is not satisfactory, and implement improvement processes wherever possible. successful student progress is of particular interest when it comes to the degrees that have been adapted to the european higher education area. it is supposed that the higher education system has evolved to a system based on competences evaluation, a system focused on student work, a system which considers continuous evaluation processes, that is, a system where the teaching-learning process should imply better results in comparison to the educational systems that was in place before the bologna plan. in the case of engineering degrees, success rates are usually worse than in other degrees. in fact, these degrees are considered as difficult ones in our society, therefore the analysis presented in this paper is highly significant in this kind of degrees. in addition to that, the greater part of engineering schools in spain are involved in the processes of degrees’ accreditation renewal because they have been taught their degrees for six years and an analysis of the results is now mandatory, in order to detect strengths and weaknesses and generate improvement plans wherever needed. the process of renewal accreditation (acredita [1] & acredita + [2]) will end with the drafting of a self-report from the university and their evaluation from a panel of experts appointed by the national evaluation agency aneca or by the regional evaluation agencies, in some cases. due to all the above a special motivation to develop these studies, analysis and reflections as the ones proposed in this work is highlighted. similar studies can be found in the literature [3-6] where colleagues from the euetib analyze these factors at their engineering school. a comparison of results can be found here and differences and similarities between them are commented. http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 67 the paper is structured as follows. in section 2 the main details of how the study is performed and how the data is analyzed are presented. section 3 deals with the presentation of the obtained results and a first comment about their causes. finally in section 4 the main conclusions of the work are pointed out. 2. performed study the main objective of this work is to study the correlation between the average mark of the students and their performance throughout their academic career. at a second phase we have studied the relation between the ects obtained in the first year and the ects obtained along the rest of the degree, as well. marks considered during the study are calculated with the high-school average grades and score on the university specific entrance exam. the study considers all the different degrees taught at the escola politècnica superior of the universitat illes balears (uib): industrial electronic and automation engineering degree (geei), informatics engineering degree (gein), construction degree (geed), agri-food and rural environment engineering degree (geam), telematics engineering degree (gttt) and mathematics degree (gmat). the study takes into account 2625 students from years 2009-10 to 2014-15. the students’ distribution according to the different year of enrollment and degree is presented in table i. http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 68 table i. students distribution (academic years and the different degrees) year 09/10 year 10/11 year 11/12 year 12/13 year 13/14 year 14/15 total geei 126 55 71 65 69 386 gein 175 132 160 158 141 766 geed 86 83 205 249 142 56 821 geam 57 37 54 46 41 235 gttt 41 31 76 50 44 242 gmat 29 26 31 41 17 31 175 the data analyzed have been obtained from the uib students service and from the information technologies center (cti), through spreadsheets files in excel format. the data are anonymous, where every student is indicated by a code, and it is not possible for us to identify the student, the anonymity of the data is guaranteed. the data process has been done using matlab. a first translation from excel to matlab has been made and after the results and graphs have been obtained through matlab programming. as an example, figure 1 shows the evolution of first year students as the ratio between ects overcome and ects registered (normalized ratio) versus the mark, as it has been defined previously. http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 69 figure 1. ects passed/ects enrolled in the first course vs average mark for the case of geei students. each point of the graph represents one student. the x-axis of the graph represents the average mark of the students. the y-axis represents the ratio between the number of ects credits passed in relation to the number of ects credits enrolled in the first year. from these distributions, correlation between data is found and conclusion extracted. 3. results. causes analysis. if we take a look at the case of geei students, we can distinguish three different zones in figure 1. the first one corresponds to the left part of the graph (marks between 5 and 6). in this zone the percentage of ects overcome is very low. this zone includes the students with low marks and a low performance in the first year of study at the engineering school. the second zone covers students with a mark of 6 to 9. in this zone students’ performance varies widely. this group includes the students with medium mark of access exam. in this set, some students are easily adapted to academic demands of their program of study while others aren’t and present some adaptation problems in the transition from high school to 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) geei http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 70 university degrees. finally, the third zone represents students with marks between 9 and 14. in this zone the students’ performance is very high. the observation can be made that there may be a stronger correlation in the case of students of lowest and highest marks and their performance, while in the case of students with medium mark there is no clear correlation: there are some students with high performance, others with medium performance and others with low performance, independently of their mark. figure 2. ects passed/ects enrolled vs average mark for the case of geei students. following the same approach, looking now at the case of student performance during all of their academic career (figure 2), and not only in the first year of study, a very similar result is observed. the pattern is almost identical. finally, figure 3 reflects the success during the degree versus the success in the first year of study. a very high correlation of results can be observed in this graph. students showing a high performance in the first course also have a high performance during all the degree and vice versa, low performance in the first course usually indicates the same trend all throughout the degree. 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( a ll c o u rs e s ) geei http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 71 figure 3. performance all courses vs performance at the first course in the case of geei students. it may seem that the high marks range of access exams is set at high mark (above 9 in the analyzed case), but it must be taken into account that this mark is over 14, where 2 points could be obtained from the specific part of the university entrance exam. in this way, for instance, a mark of 9 could be obtained from scoring 7 in the general part of the exam, as the mark of access is done by the weighted average (60% high school mark and 40% general part of the access exam) plus a value between 0 to 2 depending on the result of the specific part of the exam. in the case of gein degree (figures 4, 5 and 6), it can be observed as the zone for high mark students starts at similar values as in the previous analyzed geei degree case. it can also be deduced that there is also a correlation between these high marks and the students’ performance. on the other hand, there does not seem to be a correlation between low entrance grade and student’s (or this zone is very narrow). there is also a high number of students with medium entrance mark who present extremely low performance throughout their studies. this could be due to a high number of students being very interested in informatics (a very attractive field in our society) who do not know the difference between informatics user and informatics engineer, and consequently they have some difficulties to 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ects passed/enrolled (1st course) e c t s p a s s e d /e n ro ll e d ( a ll c o u rs e s ) geei http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 72 progress in the university degree. these students often have little or no background in mathematics and science aspects. the behavior in the performance throughout the degree is almost the same as when looking at considering first course performance. as in the previous degree a significant correlation between performance throughout the degree and student performance in the first course can be observed, although the correlation is slightly weaker in this case. figure 4. ects passed/ects enrolled in the first course vs average mark for the case of gein students. figure 5. ects passed/ects enrolled vs average mark for the case of gein students. 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) gein 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( a ll c o u rs e s ) gein http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 73 figure 6. performance all courses vs performance at the first course in the case of gein students. figure 7. ects passed/ects enrolled at first course vs average mark for the case of geed students. when looking at geed degree (figures 7, 8 and 9), the patterns observed are very similar to those in geei, although the zone for high marks appears in a range which begins with a slightly upper mark, around 10 over 14. in this case it must be pointed out that there is a significant set of students which present a high student performance during the first course, but whose performance decreases throughout their studies (points x=1 and y<1 in figure 9). this result can be attributed to the fact that some students come from the previous degree technical architecture and in the first course a set of different credits are transferred. if the data from the first years (2009, 2010) is eliminated, this trend becomes less significant. 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ects passed/enrolled (1st course) e c t s p a s s e d /e n ro ll e d ( a ll c o u rs e s ) gein 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) geed http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 74 figure 8. ects passed/ects enrolled vs average mark for the case of geed students. figure 9. performance all courses vs performance at the first course in the case of geed students. when considering geam degree (figure 10), the results reflect almost the same trends as the previously discussed, with the exception of the existence of an important set of students with low and medium marks in the admission exam who later show a high level of performance. it might be due to the vocational character of the students in this degree. 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( a ll c o u rs e s ) geed 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ects passed/enrolled (1st course) e c t s p a s s e d /e n ro ll e d ( a ll c o u rs e s ) geed http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 75 figure 10. ects passed/ects enrolled in the first course vs average mark for the case of geam students. finally, in the cases of gttt (figure 11) and gmat (figure 12) degrees, very similar behaviors are also observed. these degrees have a smaller number of students than all the other degrees in our university. in the case of gttt degree a similar result as in the case of gein degree is observed, that is a set of students with very low performance and with medium mark of access exam. the reason is probably the same. figure 11. ects passed/ects enrolled in the first course vs average mark for the case of gttt students. 4 6 8 10 12 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) geam 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) gttt http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 76 figure 12. ects passed/ects enrolled in the first course vs average mark for the case of gmat students. 4. conclusions there is a high level of difficulty in degrees related to engineering, which implies higher dropout rates and lower student performance than in other fields. for this reason, it is very important to analyze the academic progress of our students in relation to their academic background, average mark, and other significant data. in this work, the different degrees taught at the escola politècnica superior of the universitat de les illes balears have been analyzed. the obtained results show similar trends in all the degrees, although some individual particularities are highlighted. from the results obtained, a correlation between student performance in the degree and mark of access exam is only observed in two cases. first, in the case of students presenting a low mark in the access exam (correlated with low performances) and second, in the case of students that have a high mark in the access exam (correlated with high performances). there is no clear correlation in the zone of medium marks. the statement “the poor performance of engineering students at the eps is because they do not have good average mark” is partially right. the correlation between the average 4 6 8 10 12 0 0.2 0.4 0.6 0.8 1 average mark e c t s p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) gmat http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 77 mark and the performance presented in this paper is significant just in a reduced zone of marks, that is, low marks and very high marks. on the other hand, there is an important correlation between first course and degree performances. the performance in the first course could be used as a predictive parameter in the student progress throughout their studies, and could allow us to propose improvement plans for the students presenting some difficulties. as a future work, we will proceed to evaluate other parameters which could have some influence in the students’ progress such as the students’ background (high school, professional formation …) in order to obtain more information about it. 5. references [1] guía de autoevaluación: renovación de la acreditación de títulos oficiales de grado, máster y doctorado. programa acredita. agencia nacional de evaluación de la calidad y acreditación. aneca 18/06/2014 [2] programa acredita plus. guía de evaluación para la renovación de la acreditación y la obtención del sello eur-ace para títulos oficiales de grado y de máster en ingeniería. agencia nacional de evaluación de la calidad y acreditación. aneca 05/02/2015 [3] j. lópez et alt. “resultados del 2º año de grado en la euetib en función de la titulación y la nota de selectividad de los estudiantes”. xxi congreso universitario de innovación educativa en las enseñanzas técnicas cuieet2013, valencia, 2013 [4] j. lópez et alt. “resultados de la fase inicial de los grados de ingeniería industrial de la euetib en función de la procedencia de los estudiantes y su nota de corte”. xx congreso universitario de innovación educativa en las enseñanzas técnicas cuieet2012, las palmas de gran canaria, 2012 http://dx.doi.org/10.4995/muse.2016.4693 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4693 social and technological sciences eissn: 2341-2593 roca et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 65-78 | 78 [5] j. lópez et alt. “correlación entre la nota de ingreso a los estudios de grado de ámbito industrial y los resultados académicos obtenidos en el primer curso”. xix congreso universitario de innovación educativa en las enseñanzas técnicas cuieet2011, barcelona, 2011 [6] j. lópez et alt. “evolución de los estudiantes de segundo curso de la euetib referenciado ala nota de primer año de carrera y a la de selectividad, análisis de resultados por titulación”. xxii congreso universitario de innovación educativa en las enseñanzas técnicas cuieet2014, almadén (ciudad real), 2014 http://dx.doi.org/10.4995/muse.2016.4693 microsoft word 7_rojas_fernandez_et_al multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 102 http://polipapers.upv.es/index.php/muse/ teaching chemistry with sustainability rojas-fernández, antonia g.1*, aguilar-santelises, leonor1, cruz millán, margarita1, aguilar-santelises, miguel2 and garcía-del valle, araceli1 national autonomous university of mexico1and national polytechnique institute2 * corresponding author: minarojas_fernandezeq@yahoo.com.ar, faculty for higher education (fes) zaragoza, unam. batalla 5 de mayo s/n col. ejército de oriente, iztapalapa, 09230, cd. méxico. phone: +52 5556 23 07 93 received: 2016-02-12; accepted: 2016-06-24 abstract increased awareness on a critical association between the natural environment and human development gave rise multiple projects, aiming to protect the natural environment and to preserve it for future generations. chemists must be acquainted with the principles of green chemistry and the need to practice experimental chemistry with cleaner chemical reactions and sustainability. this is a major concern for all the educators forming new professionals within the chemistry, pharmacology and biology curricula in the faculty for higher studies zaragoza from the national autonomous university of mexico. with this in mind, we start our teachings explaining from the very beginning, how important it is to perform microscale techniques and to follow the principles of green chemistry in the basic science laboratory. furthermore, we have modified, designed and evaluated working procedures related with chemical synthesis, kinetics and calorimetry. by doing this, we managed to greatly reduce the amount of reagents required and residues generated. some laboratory reagents have been substituted with renewable substances. we have also included in our programme a regular treatment of residues generated during everyday laboratory work. our goal is to emphasize the importance of minimizing the environmental impact of chemistry and to prepare environmentally concerned professionals who keep sustainability as main priority and perform chemistry procedures with good laboratory practice routines. keywords education, sustainability, green chemistry, microscale, experimental chemistry. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ mailto:minarojas_fernandezeq@yahoo.com.ar multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 103 http://polipapers.upv.es/index.php/muse/ 1. introduction to recognize that the environment is a fundamental element for human development has led to create programmes specifically designed to improve the utilization of natural resources and to preserve them for future generations (matlin et al. 2015). a multidisciplinary approach has to be taken in order to face many environmental challenges already existing. chemistry, in general, and green chemistry in particular, are key disciplines to achieve these goals with sustainability (fisher, 2012). 1.1 sustainability what does sustainability mean? why do we hear about it more and more often and why should we be concerned about it? this term emerges from the need to counteract the unsustainability that predominates and characterises many technological, industrial and agriculture procedures occurring nowadays in the world (watson, 2012). high utilization and irresponsible consumption of every type of natural resources are unsustainable in long term. equally unsustainable and unacceptable are the effects on human health, wild life and nature, caused by the high generation of pollutants, increased amount of greenhouse chemicals, accelerated and disorganized expansion of human settlements that go along the logarithmic increase of world population, decreased biodiversity, loss of natural resources and socioeconomic destabilization. altogether, the mentioned problems and circumstances are indicating a global emergency, mostly caused by human actions. we are the main actors of the anthropocene, as the nobel prize in chemistry paul crutzen defined the present era of human influence on earth. as dominant species, we are responsible from the critical situation in which earth has been artificially placed. the remediation of problems already existing and the avoidance of new and bigger problems may hopefully also be in our own hands. hans carl von carlowitz used the word “sustainability” for the first time in 1713, in his book “silvicultura oeconómica”, advocating for the conservation, growing and use of wood http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 104 http://polipapers.upv.es/index.php/muse/ in a continuing, stable and sustained manner. he referred to sustainable as not to cut down more trees than those that could be simultaneously planted (schmithü sen, 2013). later on, george perkins marsh published “man and nature”, with a detailed analysis about the environmental impact of human activities. perkins marsh made clear that it was very urgent to find the way to stop and prevent the devastating effects that were already caused by human beings (perkins marsh, 1864). rachel carson, considered as the mother of the ecologist movement, published “silent spring” in 1962, discussing the risks associated with the widespread application of dichlorodiphenyltrichloroethane (ddt) as pesticide, arguing that ddt caused more damage than benefit and that the negative effects on the environment and the human health were still largely unknown (carson, 1962). this work raised awareness on the danger caused by environmental pollution with ddt and contributed to the implementation of regulatory measurements taken by governmental agencies in many countries. the impact of “silent spring” continued with the 1972 report from the ”club of rome”, which gathered a global network of independent and renowned thinkers, set out to push the boundaries of thinking on sustainability (meadows et al., 1972). these publications made evident that the kind of scientific progress that was known until that moment, was not the solution to all problems. on the contrary, science and technology were too often responsible for causing environmental and social problems (schmithü sen, 2013). the world commission on environment and development (wced) from the united nations (un) presented a report entitled “our common future”, which is also known as the bruntdland report. in the first chapter of this report, sustainable development is defined as that which is able to satisfy the needs from the present generation without compromising the capacity from future generations to satisfy their own needs (wced, un, 1987). 1.2 education sustainable development must be as integrated and holistic as science and education (vilches, 2011, 2013). in fact, sustainability is based on education, as it has been discussed in the un conference on human environment, in stockholm, the intergovernmental http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 105 http://polipapers.upv.es/index.php/muse/ conference on environmental education, in tbilisi, the un conference on environment and development (unced) and the un conference on sustainable development that had place during 1972, 1977, 1992 and 2012, respectively. the un proclamation of the decade of education for sustainable development is the biggest and most recent effort to direct education towards sustainability. the un declared 2005-2014 as the decade of education for sustainable development in order to confront the global emergency situation that we are living. according to the un, education alone will not be enough to achieve a more sustainable future. however, sustainability cannot be reached without education. therefore, the un attempt to integrate principles, values and practices of sustainable development in every aspect of education, in order to promote changes of behaviour that are necessary to preserve the future integrity of the environment and economy, as well as social justice for present and future generations (deds, 2012). the time for educators and students to look at environmental problems as spectators has passed. we must ask ourselves what can we do in response to these international calls and act accordingly. how can we, by teaching chemistry at the universities, redirect education towards sustainability? at the present moment, we are working with two proposals that may fulfil this purpose at the faculty for higher studies zaragoza (fes-z) from the national autonomous university of mexico (unam): microscale chemistry and green chemistry. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 106 http://polipapers.upv.es/index.php/muse/ 1.3 experimental chemistry education significant changes are necessary to face the present global emergency. such kind of changes are indispensable within experimental chemistry, one of the most powerful engines to redirect the present socioeconomic crisis towards sustainability (bybee, 1991). accordingly, unesco proclaimed 2011 as the international year of chemistry with the motto: “chemistry-our life, our future”, perfectly stating the strong influence of chemistry on the human society and the opportunities that it may provide to make a transition towards sustainability. thus, how can we contribute with experimental chemistry education towards sustainability? we decided to work with microscale chemistry and green chemistry, since they are powerful tools that have already been effectively used to achieve this goal (nmcc, 2002; cannon and warner, 2011). 1.3.1 microscale chemistry working with microscale means to reduce the amount and utilization of chemical reagents to the minimal levels that are necessary to properly perform the experiments. microscale also represents a group of environmentally safe methods to accomplish chemical processes, avoiding pollution without compromising the quality standards of the results. the amount of chemical reagents needed for our practical sessions in the basic science laboratory is drastically reduced to a rank of 0.5 0.005 g (nmcc, 2015). microscale chemistry provides among others, the following benefits: • less production and utilization of chemicals • less storage space required • less working time in the laboratory • lower glassware costs • lower costs for laboratory work http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 107 http://polipapers.upv.es/index.php/muse/ • improved laboratory techniques • better air quality and minimal exposure to toxic fumes • cleaner and more productive environment • safer laboratory work • improved awareness of risk for people using chemicals • higher awareness of the importance of the environment • promotion of the three r principle (replacement, reduction, refinement) microscale chemistry is very attractive for educators because it addresses general needs for teaching science without special equipment. working with microscale appeared as an alternative to make chemistry teaching less laborious and more accessible for a higher number of students. since these objectives are in line with sustainability, microscale is a valuable tool to facilitate the transition of cumbersome laboratory methods to more sustainable methods, without compromising chemistry education. however, green chemistry is still the best and more complete alternative to reach sustainability. 1.3.2 green chemistry green chemistry is a working philosophy that emerges in response to the need for preserving the environment, with prevention as main objective, instead of remediation (luque, 2012). according to the american chemical society (acs), green and sustainable chemistry is a distinctive way of thinking, based on a set of principles that when used in the design, development and implementation of chemical products and processes, enables scientists to protect and support the economy, the people and the planet. green chemistry consists in finding new ways to create and innovate, as well as to reduce waste and energy consumption and to look for replacement for dangerous substances (anastas and warner, 1998). http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 108 http://polipapers.upv.es/index.php/muse/ the following table contains the 12 principles of green chemistry, conceived by paul anastas and john warner (1998) as a framework for designing or improving materials, in order to make ”greener” products, processes and systems: 1. prevention: it is better to prevent waste than to treat or clean up waste after it has been created. 2. atom economy: synthetic methods should be designed to maximize the incorporation of all materials used in the process into the final product. 3. less hazardous chemical synthesis: wherever practicable, synthetic methods should be designed to use and generate substances that possess little or no toxicity to human health and the environment. 4. designing safer chemicals: chemical products should be designed to affect their desired function while minimizing their toxicity. 5. safer solvents and auxiliaries: the use of auxiliary substances (e.g., solvents, separation agents, etc.) should be made unnecessary whenever possible and innocuous when used. 6. design for energy efficiency: energy requirements of chemical processes should be recognized for their environmental and economic impacts and should be minimized. if possible, synthetic methods should be conducted at ambient temperature and pressure. 7. use of renewable feedstocks: raw materials or feedstocks should be renewable rather than depleting whenever technically and economically practicable. 8. reduce derivatives: unnecessary derivatization should be minimized or avoided, because it requires additional reagents and can generate waste. 9. catalysis: catalytic reagents are superior to stoichiometric reagents. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 109 http://polipapers.upv.es/index.php/muse/ 10. design for degradation: chemical products should be designed so that at the end of their function they breakdown into innocuous degradation products and do not persist in the environment. 11. real-time analysis for pollution prevention: analytical methodologies need to be further developed to allow for real-time, in-process monitoring and control prior to the formation of hazardous substances. 12. inherently safer chemistry for accident prevention: substances and the form of a substance used in a chemical process should be chosen to minimize the potential for chemical accidents, including releases, explosions, and fires. to adhere ourselves to the 12 principles proposed by anastas and warner, we need to have the ability to integrate additional disciplines in order to achieve sustainability. new generations of chemists must be acquainted with green chemistry and sustainability from the very beginning of their professional education in order to be able to prevent pollution and to perform cleaner chemical processes. therefore, we ask ourselves which actions must be taken for teaching experimental chemistry with sustainability. how may we promote the cultural change required among teachers and students during their experimental chemistry work, while keeping full awareness of their roles as academics, professional and citizens? this is an important concern for educators involved in teaching experimental chemistry to students following the chemistry, pharmacology and biology (qfb) curricula in the fes z from the unam. this concern is very much present in the laboratory for basic science (lcb), during the first two semesters of education at professional level and it is the reason why, we have integrated microscale and green chemistry principles in our laboratory work, whereas we keep looking for new designs and evaluate new proposals of experimental work, oriented both to better education and sustainability. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 110 http://polipapers.upv.es/index.php/muse/ 2. methodology the qfb curricula and the lcb i and lcb ii programmes were examined, focusing on design, needs or easiness for modification and evaluation of experimental chemistry proposals for chemical synthesis, kinetics and calorimetry. an exhaustive review of relevant literature was done to find experimental proposals with low cost, easy access and everyday utilization that could be adapted to microscale and green chemistry. then, proposals were studied in relation to subjects and objectives included in the curricula and lcb programme. some proposals were selected, adapted to microscale and green chemistry, evaluated and classified according to the 12 principles of green chemistry by using a likert-type scale (morales-galicia, 2011) to determine how green they were (figure 1). in addition, proposals were also evaluated according to b) reagents toxicity and c) need for treatment of residues, by means of pictograms and ecologic diagrams, respectively. in this work, only a proposal to study chemistry kinetics will be discussed. (10) totally green (9) strongly green (8) very green (7) green (6) lightly green (5) brown to green transition (4) lightly brown (3) brown (2) strongly brown (1) totally brown figure 1. colour code and likerttype scale of analysis to evaluate grades of green http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 111 http://polipapers.upv.es/index.php/muse/ this proposal was developed in the laboratory according to the following procedure: 1. prepare solutions: 1g vitamin c, 0.04 m iodine, 5% cornflour and 3% hydrogen peroxide in 60 ml of water. 2. fill up insulin syringes (1ml). 3. transfer solution to beakers of 10 ml. 4. add h2o2, agitate the mixture and start registering time. 5. stop the time when the iodine-starch complex (deep blue) appears. 6. test by triplicate, using 5 different h2o2 concentrations. 7. perform statistics analysis: obtain average value, standard deviation (s) and relative standard deviation or coefficient variation (% cv). 8. plot and obtain empirical equation. 3. discussion of the proposal and results experimental work designed to study a iodine-vitamin c clock reaction (wright, 2002) by microscale, according to the 12 principles of green chemistry was organized in 4 steps: a, b, c and d, established as follows: measurement instrument chemical source laboratory reagents everyday use products micropipette a c insulin syringe b d here, only results of step d are shown. in this case, we used insulin syringes and everyday use products, such as vitamin c tablets, commercial 3 % hydrogen peroxide, cornflour and tincture of iodine. results are shown in tables 1, 2 y 3, and graph 1. statistics analysis and corresponding equation with an r = 0.9998 indicates that the best function explaining the effect of different hydrogen peroxide concentrations on the speed reaction is potential. the http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 112 http://polipapers.upv.es/index.php/muse/ coefficient variation (cv) obtained in repetitions from every assayed concentration was not higher than 2.20 %, suggesting that the variability was minimal. therefore, systematic experimental error may be considered under control. the proposal was also considered adequate to be performed in lcb to study the speed of a chemical reaction. v h2o2 (ml) 0.20 0.30 0.40 0.60 0.80 # 1 289 s 181 s 124 s 71 s 49 s 2 295 s 181 s 122 s 69 s 49 s 3 292 s 182 s 125 s 70 s 48 s ӯ 292.00 s 181.33 s 123.67 s 70.33 s 48.66 s s 3.00 0.54 1.53 1.53 0.58 % cv 1.03 0.32 1.24 2.17 1.19 r (potential) 0.9995 table 1. endpoints obtained varying h2o2 concentration and function to which data may be adjusted h2o2 time (s) v rx ([h 2 o 2 ] /s) v (ml) molarity (m) 0.2 0.1029 292.00 -4 3.5239 x 10 0.3 0.1544 181.33 -4 8.5149 x 10 0.4 0.2058 123.66 -3 1.6642 x 10 0.6 0.3088 70.33 -3 4.3907 x 10 0.8 0.4117 48.66 -3 8.4607 x 10 http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 113 http://polipapers.upv.es/index.php/muse/ table 2. speed of reaction with different h2o2 concentrations. parameters results potential equation y = bxm y vrx x [h2o2] m 2.3060 b 0.0650 r 0.9998 empirical equation vrx = 0.0650 [h2o2] 2.3060 table 3. minimal square parameters to determine the empirical equation. graph 1. speed of reaction (vrx) in response to different h2o2 concentration. a flux diagram was elaborated, in order to evaluate how green was the experiment (figure 2). safety pictograms are included in this diagram, to facilitate the recognition and handling of dangerous chemicals in every step of the process. afterwards, a likert-type scale and a colours’ code were used to make a numerical qualitative evaluation of the process in relation with the 12 principles of green chemistry (stewart et al., 2016). additional tables were elaborated to show how to obtain the final evaluation (table 4) and to enlist justification for every step of the process (table 5). http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 114 http://polipapers.upv.es/index.php/muse/ figure 2. flux diagram to evaluate an experimental proposal involving the iodine vitamin c clock reaction. a final numerical score of (9) was assigned. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 115 http://polipapers.upv.es/index.php/muse/ the process of making a greener experiment starts with a drastic reduction in the amount of reagents. when the amounts of utilized reagents are smaller than 1 ml, residues, risks, environmental damage and costs are also smaller. costly chemical reagents normally acquired from specialized companies are substituted with low cost, everyday use products. these simple actions take experimental work closer to become green enough for making a transition towards sustainability. an evaluation before and after modifications verifies that these changes actually make the chemical work to become greener. process step principle of green chemistry likert-type scale # of evaluation a) 1 (8) 1 3 (6) 2 4 (10) 3 6 (10) 4 12 (10) 5 b) 12 (10) 6 d) 6 (10) 7 e) 12 (10) 8 g) 6 (10) 9 i) 1 (8) 10 3 (6) 11 12 (10) 12 total points (pt) 108 12 final evaluation = pt / # e 108 / 12 = (9) value of likert-type scale the proposal is strongly green table 4. results of final evaluation procedure of the experimental proposal involving the iodine-vitamin c clock reaction. a final numerical grade of (9) was assigned to this particular experiment. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 116 http://polipapers.upv.es/index.php/muse/ step principle justification a) 1 iodine must be still treated prior to disposal. principle 1 receives a score of 8. 3 amounts of reagents utilized and residues generated are greatly reduced. microscale and every day use products make principle 3 to receive a score of 6 4 replacement of reagents by everyday use products makes principle 4 to receive a score of 10. 6 microscale drastically reduces costs and energy consumption. therefore, principle 6 is evaluated with a score of 10. 12 principle 12 receives also a score of 10, since very small amounts considerably decrease risk for health and the environment. b) 12 as above d) 6 as above e) 12 as above g) 5 using cornflour as indicator makes principle 5 to receive a score of 10. i) 1, 3, 12 as above table 5. justification for making greener an experimental proposal involving the iodine vitamin c clock reaction. an ecologic diagram was also elaborated to show in detail source and handling of residues generated (figure 3). students are also involved, as much as possible, in the treatment of http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.orgl10.4995lmuse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 117 residues, to increase their responsibility, environmental culture and participation in reducing the impact of their work in the environment. figure 3. ecologic treatment of residues generated by the iodine-vitamin c clock reaction. 4. conclusions three experimental proposals were adapted to be performed in the lcb from the qfb curricula at fes-z, unam. here, one of them is discussed as an example from the benefits of incorporating microscale and green chemistry, in search for a transition towards sustainability. http://dx.doi.orgl10.4995lmuse.2017.6462/ http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 118 http://polipapers.upv.es/index.php/muse/ several goals can be reached applying green chemistry principles and using low-cost, everyday use, easily available materials. pollution and dangerous residues generated by work in the laboratory during chemistry education are greatly reduced. increased awareness of the necessity to preserve the environment, the importance of decreasing residues and the responsibility for proper disposal of dangerous chemicals are also key advantages of green chemistry and must be compulsorily incorporated in chemistry education. in the present report, we discuss the evaluation of an experimental proposal in relation with the 12 principles of green chemistry, reagent toxicity and handling of residues. the results of this evaluation indicated a strongly green procedure, with a numerical score of 9, which also indicates that is adequate for teaching experimental chemistry and sustainability. different teaching methods have been evaluated and chosen within the framework of the european higher education area to be implemented into a combustion course delivered at the etsid of the universitat politècnica de valència. such methodology is based upon the interaction between the lecturer and the student, and is being applied for groups with large number of students. among the main results, the high grades obtained by students have to be noted, which are independent of the area that is being evaluated (theoretical lectures, guided work…). in spite of the fact that such an estimator is not definitive to determine the success of the methodology, it can be considered as a first good approximation. however, certain deficiencies have been detected due to the dissemination of information among students when using the evaluation system by means of the upv digital platform, which questions some of the grades obtained by some students. such information dissemination has been quantified numerically. when eliminating such results, grades are still high, showing a high degree of comprehension of the concepts by the students. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 119 http://polipapers.upv.es/index.php/muse/ 5. acknowledgements this project was supported by unam (dgapa papime pe210815). 5. references anastas p.t. and warner j.c. (1998). green chemistry. new york: oxford university press. brixey-williams s. (2015). which countries do the most to protect the environment? available at https://www.weforum.org/agenda/2015/09/which-countries-do-the-most-to protect-the-environment/ accessed november 21, 2016. bybee r.w. (1991). planet earth in crisis: how should science educators respond? am. biol. teach. 53:146–153. http://abt.ucpress.edu/content/53/3/146. doi: 10.2307/4449248 cannon a.s. and warner j.c. (2011). the science of green chemistry and its role in chemicals policy and educational reform. new solut. 21: 499-517. doi: 10.2190/ns.21.3.m. carson r. (1962). silent spring. fawcett publications inc. greenwich, conn. decade of education for sustainable development (deds). shaping the education of tomorrow: 2012 full-length report on the un decade of education for sustainable development. available at http://www.unesco.org/new/es/santiago/education/education for-sustainable-development/decade-of-education-for-sustainable-development-desd/. accessed november 21, 2016. fisher m.a. (2012). chemistry and the challenge of sustainability. j. chem. educ. 89:179 180. http://pubs.acs.org/doi/abs/10.1021/ed2007923. doi: 10.1021/ed2007923. luque r. (2012). green chemistry, chemistry research and applications. nova science publishers. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ http://www.weforum.org/agenda/2015/09/which-countries-do-the-most-tohttp://abt.ucpress.edu/content/53/3/146 http://www.unesco.org/new/es/santiago/education/educationhttp://pubs.acs.org/doi/abs/10.1021/ed2007923 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 120 http://polipapers.upv.es/index.php/muse/ matlin s.a., goverdhan m., hopf h. and krief a. (2015). the role of chemistry in inventing a sustainable future. nature chemistry 7: 941–943. doi:10,1038/nchem.2389. meadows d.h., meadows d.l., randers j. and behrens w.w. (1972). the limits of growth. a report for the club of rome’s project on the predicament of mankind. a potomac associates book. available at http://www.sustainablelifestyles.ac.uk/news/tj-club-of-rome accessed november 21, 2016. morales-galicia m.l., martínez j.o., reyes-sánchez l.b., hernández o.m., arroyo razo g.a., valdivia a.o. (2011). ¿qué tan verde es un experimento? educ. quim. 22: 240-248. national microscale chemistry centre (nmcc). (2002). available at http://www.microscale.org/who.asp. accessed november 21, 2016. perkins marsh g. (1864). man and nature: or physical geography as modified by human action. kessinger publishing co., whitefish, montana, usa. schmithu· sen f. (2013). three hundred years of sustainable forestry. unasylva 240; 64: 2013/1. http://www.fao.org/docrep/018/i3364e/i3364e.pdf. accessed november 21, 2016. singh m.m., szafran z. and pike r.m. (1999). microscale chemistry and green chemistry: complementary pedagogies. j. chem. educ. 76: 1684-1684. http://pubs.acs.org/doi/abs/10.1021/ed076p1684. doi: 10.1021/ed076p1684. stewart a.f., williams a.l., lofgreen j.e., edgar l.j.g., hoch l.b. and dicks a.p. (2016). chemistry writing instruction and training: implementing a comprehensive approach to improving student communication skills. j. chem. educ. 93: 86–92. doi: 10.1021/acs.jchemed.5b00373. vilches a., gil-pérez d. (2011). papel de la química y su enseñanza en la construcción de un futuro sostenible. educ. quim. 22: 103–116. vilches a., gil-pérez d. (2013). ciencia de la sostenibilidad: un nuevo campo de http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ http://www.sustainablelifestyles.ac.uk/news/tj-club-of-rome http://www.microscale.org/who.asp http://www.microscale.org/who.asp http://www.fao.org/docrep/018/i3364e/i3364e.pdf http://pubs.acs.org/doi/abs/10.1021/ed076p1684 http://pubs.acs.org/doi/abs/10.1021/ed076p1684 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6462 eissn: 2341-2593 rojas-fernández et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 102-121 | 121 http://polipapers.upv.es/index.php/muse/ conocimientos al que la química y la educación química están contribuyendo. educ. quim. 24: 199-206. watson w.j.w. (2012). how do the fine chemical pharmaceutical, and related industries approach green chemistry and sustainability? green chem. 14: 251-259. http://pubs.rsc.org/en/content/articlepdf/2012/gc/c1gc15904f. doi: 10.1039/c1gc15904f. world commission on environment and development (wced). (1987). our common future. available at http://www.un-documents.net/our-common-future.pdf. accessed november 21, 2016. wright s.w. (2002). tick tock, a vitamin c clock. j. chem. educ. 79: 40a. http://pubs.acs.org/doi/pdf/10.1021/ed079p40. doi: 10.1021/ed079p40. http://dx.doi.org/10.4995/muse.2017.6462 http://polipapers.upv.es/index.php/muse/ http://pubs.rsc.org/en/content/articlepdf/2012/gc/c1gc15904f http://pubs.rsc.org/en/content/articlepdf/2012/gc/c1gc15904f http://www.un-documents.net/our-common-future.pdf http://pubs.acs.org/doi/pdf/10.1021/ed079p40 http://pubs.acs.org/doi/pdf/10.1021/ed079p40 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 mentoring in the acquisition of professional skills through practices in companies by degree students t.m. monllau jaques, n. rodríguez-ávila*, miralles de imperial, j departamento de economía y empresa, universitat pompeu fabra e-mail: teresa.monllau@upf.edu departamento de sociología y análisis de las organizaciones, universitat de barcelona e-mail: nrodriguez@ub.edu * corresponding author: email: nrodriguez@ub.edu; phone: + 34 934021801 received: 2013-12-31; accepted: 2014-08-11 abstract this work has had as objective to analyze the skills acquired through internships in business companies by students of the faculty of economics and business at the university pompeu fabra. the internship is a basic item in order to obtain a hard connection between the university and socialeconomic world where university and enterprises develop their activity. in this study we want to know about two aspects. the first one, we want to know the profit that is obtained from the student as a consequence of internship and mentoring. also, we want to study about the importance of mentoring as a principal element that establishes the relationship between the student and the company. moreover, it has sought to analyze if certain factors such as the size of the company where the practices has been performed, the study rank level that was achieved or the fact of being a man or a woman, were among the determining factors at the time of acquiring the skills. the results presented here indicate that the size of the company that have been making the practices and the gender of the student are related to the acquisition of certain skills. there was not a statistically significant relationship related to the rank level have by the students in the practice. in the future we are going to study if the labor market integration is easier if the student has performed work placement. keywords skills; employability; internship; meatoring. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 124 mailto:teresa.monllau@upf.edu mailto:nrodriguez@ub.edu multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 1. introduction the economic, social and professional environment in which we live is changing. to survive professionally in this type of environment is not only necessary to have a baggage of important knowledge, but it is also necessary to have the capacity to adapt to these changes. that is why in the existing literature speaks of employability to define the technical and human qualities that increase the likelihood that the candidate has the job that you choose. we define 'employability' (van de heijden et al. 2006), (de vos et al. 2011) as the use of skills that increase the likelihood of: 1. achieve the objectives for which it has been hired. 2. get a new job. 3. create new jobs. this is why the professional development of a person depends not only the intellectual baggage and knowledge available, but also of the competencies and skills that characterize the person (kinicki 2008), (van de heijden et al. 2006). the literature suggest that when employers evaluate a candidate not only considered the specific knowledge the future candidate may have, but also the skills we describe as cross skills.there are three types of skills that are used to assess the employability of the future professional (kuijpers 2006), (kuijpers et al. 2006): 1. those related to the capacity for reflection, which are characterized by a reflective behavior. 2. those related to the educational career, which are identified by active behavior. 3. those related to the networking, which is manifested through an interactive behavior. the implementation of practices in enterprise on the part of the university students has clear advantages for the students since: 1. it allows to acquire relevant skills to move into the labour market. in this sense, some authors (matthews et al. 1999) concluded that students who have done work experience monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 125 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 during their training have more ability to develop critical thinking, problem solving etc, than those who have not been exposed with the world of work. 2. increases the chance of finding a job upon completion of the studies (coco 2000). in this study, we analyze whether there are factors that are inherent to the student in practice, and the company in which have been carried out the practices which determine the acquisition of a certain type of competency. 2. context: the state of affairs the analysis of the factors that facilitate the acquisition of a particular type of competence is of interest to both employers and prospective employees. employers will know through realized studies, which are the inherent characteristics of the people who increase the acquisition and development of certain types of powers which are the inherent characteristics of the people who increase the acquisition and development of certain types of competencies. meanwhile, prospective employees may, on the one hand, to concentrate the work search on that type of companies and of work in which the personal characteristics adapt themselves better to the characteristics of the vacant job. moreover, they can evaluate the type of skills they have to develop and improve if they want to access a particular job or develop professionally. we have used different methods to identify the major work published on practices in companies. first, we searched through digital metasearch engines using keywords like: mentoring, internship, skills and competencies. second, we analyze the search for articles published in the last four years of specialized journals such as: journal of vocational behavior, career development, international, and journal of organizational behavior. these magazines were taken as reference since they had already used as reference in earlier studies that analyzed the state of the art (allen et al. 2008). from the results we conclude that there are basically three areas of interest to researchers: monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 126 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 • the study of the figure of the tutor and for its role as a companion to the student during the carrying out of the practices. the large number of articles published on the figure of the tutor and the variety of approaches that develop (allen et al. 2008), (fletcher, 2012) (russell et al. 2010), (chun et al. 2012), (eby et al. 2008) requires a specific analysis that we will do in future work. • the role of the student training in a company improves the relation university-company. it is also a basic element to increase the job satisfaction and rise the integration to the labor market. new pedagogic approaches gives importance to developing skills and ways of learning (gonzalez et al. 2003; ibernom et al. 2005). the benefit to the students conducting business practices was studied in order to improve the relationship between academia and industry (narayan et al. 2010; matterhews et al 1999; coco 2000). the educational paradigm of bologna has an active role of the student involved in learning and training, this produced a shift of the professional profile towards more concrete functions (benito et al., 2005). among the competences highlights the transversal ones are considered as instrumentals, interpersonal and systemic. this way are identified the competitions that must be acquired by today’s students and the professionals of the future. the entrepreneurs, on their behalf, value as key competencies the initiative and creativity (monllau 2010 a, b). • the role of the practice in the acquisition of the so-called professional competencies. in this sense, some authors conclude that many companies manage their human resources in terms of the powers they may have (de la poza 2008; escalona 2005). according to brockmann's the skills influence facilitates the adaptation to a constantly changing environment (brockmann et al., 2009). for meijers it is important the relationship between the acquisition and the development of new skills in conjunction with the worker commitment to the company and his work (meijers et al. 2013). on the other hand, there is a relationship between skills development, employability and career success (de vos et al. 2011; meijers et al 2011; akkermans et al. 2013.). there is also a significant weight monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 127 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 between general and specific skills with the salary level and the own employability (rotundo et al. 2004, rotwell et al. 2008). company practices are the maximum exponent of the university-industry collaboration. is for this reason that there are studies that analyze what factors may make this more efficient collaboration. a key factor is communication, and that positively affects trust and satisfaction (frasquet et al. 2012). likewise, studies that analyze the methodologies to be developed at the university for collaboration between universities and companies work better. these studies conclude that for new problems that arise in business is necessary creativity (generation of ideas). the techniques for generation videos help manage the company, allowing the identification of problems and their solutions. likewise, binding of university knowledge with business knowledge improves idea generation, participation and student learning (montoya rios et al. 2011). other analyzes in the field of pre university studies conclude that business practices are positive as facilitate the student seeking a better professional future (gamboa et al. 2013). finally, there are studies, especially in the field of psychology, which examines how the practices in companies facilitate the personnel selection. most of these studies used quantitative methodologies to analyze the information obtained through the completion of questionnaires. when talking about skills related to business practices, reference is made to professional skills. these skills are related to the capacity of reflection, training and networking. in this regard, the studies analyze which factors enhance the acquisition of these skills and if there is a relationship between skills and professional success. thus, we analyze whether the environment in which learning and personal variables such as gender, age and culture develops, facilitate the acquisition of such skills. the conclusions are that in high school studies the methodologies developed reflection, but do not contribute to vocational training or networking. reflection occurs more in secondary education than in the pre professional education (kuijpers, et al. 2006). in addition, the studies reviewed conclude that the development of professional skills promotes success (de vos, et al. 2011), monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 128 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 professional commitment (meijers et al. 2013) and the proper well-being of the worker (akkermans et al. 2013). 3. description of methodology our objective is the analysis of competencies related to the employability that have acquired the students of the faculty of economics and business at the university pompeu fabra practices that have been made in companies. specifically, the questions that guide our analysis are: 1. does he have followed a particular study; is a determining factor for the acquisition of a certain type of skills? 2. does the size of the company in which the practices are carried out is a determining factor for the acquisition and development of a particular type of competence? 3. is gender a determining factor for the acquisition and development of a particular type of competence? to perform the study, we defined two types of variables: independent and dependent. we defined the independent variables based on the size of the company that have been made the practice, the attending grade and gender of the intern. in relation to its size we defined companies as small and medium when they have less than 250 employees, and we have considered large companies with more than 250 employees. from our point of view, the fact of studying a degree can make the acquisition of certain competencies since the academic focus of the degrees is different. the degree of economy is characterized by a higher proportion of subjects requiring numerical skills and a strong mathematical formation, while in the degree of management and in ade there is a greater number of subjects of a practical nature and high enterprise content. this diversity can make a difference in the starting point of the acquisition of skills. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 129 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 finally, we have considered the gender of the trainee as it is a factor considered in other studies (kuijpers et al. 2013). we have taken as dependent variables the related skills of the student employability. in the definition of the competencies we have taken into account the specific and generic competences raised by the bologna paradigm and, in particular, the tuning project, which are related to the level of employability. specifically, a question was included in the questionnaire that was used to collect data on up to what extent had acquired a set of competencies during practice. subsequently, each of these competencies was classified into three groups: those that promote training, reflection, and interaction or networking. the skills of those asked were classified according to the following groups: c1. technical capacity (formative). c2. learning ability, (formative). c3. labour administration , ( reflection). c4. oral communication skills, (interaction). c5. sense of responsibility, (reflection). c6. adaptability (interaction). c7. creativity and initiative, (reflection). c8. personal involvement, (interaction). c9. motivation (education). c10. responsiveness of the reviews (interaction). c11. punctuality, (not been analyzed). c12. relationship to the work environment (interaction). c13. ability to work in team (interaction). to carry out the research a questionnaire was provided to all students who have undertaken curricular practices at the faculty of economics and business studies during the course 2012-13. table 1 shows the distribution of the sample. in this case we have not monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 130 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 considered students of international business administration (ibe). the response rate of questionnaires was 28.7% (until july 2013). table 1 . distribution of the sample characteristics big business pime total n % n % n % ade 20 55.6% 16 44.4% 36 100.0% men 5 13.9% 2 5.6% 7 19.4% women 15 41.7% 14 38.9% 29 80.6% economy 11 61.1% 7 38.9% 18 100.0% men 5 27.8% 5 27.8% 10 55.6% women 6 33.3% 2 11.1% 8 44.4% management 10 41.7% 14 58.3% 24 100.0% men 4 16.7% 6 25.0% 10 41.7% women 6 25.0% 8 33.3% 14 58.3% total 41 52.6% 37 47.4% 78 100% source : authors. the methodology used was quantitative. quantitative research is useful when trying to see how often an event occurs, or if you want to get a general idea without losing too much information (kirk et al. 1990). this type of analysis is appropriate when you want to obtain a first description of the population and the heterogeneity that characterizes it. to analyze whether there is a significant relationship from the statistically analyzed between skills and independent variables we used the analysis of variance. the software used was excel 2011. moreover, a qualitative analysis was realized on the importance that the students give to the practices and the importance of the role of the tutor in the company as well as the academic tutor. in this case, we used a quantitative methodology that allows us to obtain a first approximation to the roles played by mentors in conducting the practices. to review the tutors performance we performed in-depth interviews to all students who have completed the curriculum practices. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 131 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 the working hypotheses were: 1. during the practices, students give more importance to the generals and specific competencies. 2. the work in large companies requires and facilitates the acquisition of skills that facilitate interaction with other people. 3. the structure of economics studies facilitates the acquisition and development of skills that encourage reflective behavior. 4. the structure of the studies that are directly related to the business environment (ade and business management) facilitate the acquisition and development of training skills. 5. the women have more developed than the men, the competitions related to communication. 6. the both mentoring business and academia are considered as a factor guaranty success and utility practices. 4. discussion of the proposal and results the results obtained in the degree of importance that has been given to the competencies are shown in table 2. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 132 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 table 2. rating competencies that have been developed during practice (in relative terms) 1 2 3 4 no value total c1 28.2% 53.8% 16.7% 1.3% 0.0% 100.0% c2 74.4% 20.5% 1.3% 3.8% 0.0% 100.0% c3 71.8% 19.2% 5.1% 2.6% 1.3% 100.0% c4 47.4% 34.6% 15.4% 2.6% 0.0% 100.0% c5 74.4% 21.8% 2.6% 1.3% 0.0% 100.0% c6 41.0% 48.7% 9.0% 1.3% 0.0% 100.0% c7 26.9% 42.3% 26.9% 2.6% 1.3% 100.0% c8 39.7% 53.8% 5.1% 1.3% 0.0% 100.0% c9 56.4% 32.1% 9.0% 1.3% 1.3% 100.0% c10 39.7% 43.6% 14.1% 2.6% 0.0% 100.0% c11 50.0% 29.5% 20.5% 0.0% 0.0% 100.0% c12 57.7% 30.8% 9.0% 0.0% 2.6% 100.0% c13 56.4% 26.9% 11.5% 3.8% 1.3% 100.0% source: prepared from the results obtained we see that the most valued skills have been the learning ability, the ability to manage work, sense of responsibility, motivation to work, be on time , the relationship with the working environment and the ability to team work. we wish to note that, during the interviews that are made to students to assess academically the practice carried out, many of them have emphasized the importance of managing the work and the learning of relations in the working environment. these results are consistent with performed studies showing that a relationship between wage levels and the acquisition of a cognitive factor (a general cognitive factor) has more to do with general competencies than with specific skills (rotundo et al. 2004). our first hypothesis which states that the acquisition of general skills has an important weight in work development and therefore in the future employability of workers is confirmed. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 133 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 the least valued competencies were technical ability, creativity and initiative. the fact that technical capacity is one of the least valued skills is consistent with the fact that for a large academic assessment of students stated that in their opinion the companies " hired " interns in the last years of the degree, not for having a specific knowledge, but because throughout the studies they had learned to act in a more analytical manner. the fact that the initiative and creativity is not valued in contrast to the study done in 2010 (monllau et al. 2010a) in which one of the results was that one of the skills most valued by employers was the initiative and entrepreneurship. in our opinion, this difference is caused by the fact that the objective characteristics and dynamics of the practice may condition the objectivity of this response. typically, students are hired to do odd jobs where little initiative is required. the statistical contrast done may be noted that, when you relate the competencies analyzed with the size of the company, there are statistically significant differences in competencies c3: labor administration and c6: ease of adaptation. specifically, the fact of making practices in a large firm makes easier to learn to manage the job. in addition, working in large companies requires a greater ability to adapt to situations and environment. having completed a degree does not make a significant difference with any of the competencies under study. ability to work in team: when we relate skills and a statistically significant gender difference in the case of the c13 competition is observed. the women considered, to a greater degree than men, that they have acquired and needed the ability to work in a team. however, it should be noted that the sample used for the analysis are many more women than men. statistical test results are shown in tables 3, 4 and 5. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 134 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 table 3. relationship between firm size and labor administration: mean and variance. groups account sum average variance big business 41 62 1.51219512 0.75609756 pime 37 44 1.18918919 0.21321321 anova source of variation sum of squares degrees of freedom mean square f probabiliy critical value for f between groups 2.02913983 1 2.02913983 4.0668 0.0472 3.9667 within groups 37.9195781 76 0.49894182 total 39.9487179 77 source: prepared table 4. relationship between the size and ease adaptation work groups account sum average variance big business 41 77 1.87804878 0.5597561 pime 37 56 1.51351351 0.31231231 anova source of variation sum of squares degrees of freedom mean square f probabiliy critical value for f between groups 2.58446157 1 2.58446157 5.839985 0.018065 3.96675 within groups 33.6334871 76 0.44254588 total 36.2179487 77 table 5. relationship between the gender of the student in practice and teamwork groups account sum average variance big business 27 36 1.33333333 0.38461538 pime 51 89 1.74509804 0.87372549 anova source of variation sum of squares degrees of freedom mean square f probabiliy critical value for f between groups 2.99321267 1 2.99321267 4.2372871 8 0.0429729 9 3.9667597 8 within groups 53.6862745 76 0.70639835 total 56.6794872 77 source: prepared 1. from the results obtained we can conclude that students who have done the practices in large companies develop a greater ability to manage work and require monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 135 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 greater adaptability. in addition, more women than men develop the ability to work in teams. therefore, we can conclude that working in big companies requires some different skills than those required in smaller companies; the larger companies require organizational skills rather than interaction with other staff. on the other hand, we have not obtained any statistically significant relationship between the completed study and the type of competition that is of importance for the development of practice, we could demonstrate that test 3 and 4 hypotheses. we were observed that there are differences by gender and the women value teamwork. it is this sense our 5 hypothesis is confirmed. all the students who were asked about the role of mentor consider that the mentor in the business is a cornerstone. it seems that there is a relationship between having a mentor in the company who is dedicated to the intern and the type of work the company does. students who have a mentor committed to practices in the company claimed that they obtained higher level of knowledge. on the other hand, the importance given by students to the academic mentor is not as much about knowledge as is as key element that helps them solving administrative problems. furthermore, in cases where there have been problems between the intern and the company tutor, the students perceived the academic tutor as a mediator in the conflict. 5. conclusions the study of the competencies is important because they are a cornerstone of the learning process and integration into the labor market. for this reason, it is important to know what is the perception and importance given by teachers and employers to competencies directly related to the professional future employability. in the literature we can observe some aspects of competence have been analyzed from the academic and vocational perspective. in this sense, it has tried to identify which monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 136 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 competencies should be further encouraged for graduate studies. from a business perspective has studied the relationship between skills and professional success. moreover, it is important to examine whether certain characteristics of the future worker increases the probability that acquire and develop certain skills. this work is part of a larger project which the authors are pursuing the study of competence. initially, were analyzed the competencies, according to teachers and employers, that students must have after graduation (monllau et al. 2010a). subsequently, we have analyzed the differences in skills acquisition among students of degrees in economics and business administration and management and acquired by students of political science (monllau et al. 2010 b). also there has been analyzed the vision that the students have with regard to the competitions acquired during its studies (monllau et al. 2010a). in this paper, we have focused on studies that refer to the acquisition of skills and employability (de vos et al. 2011), skills and wage level (rotundo et al. 2004) or competencies, employability and demographic variables (rothwell et al . 2007). our findings are in this line. we can say that at the time of entering into the labor market the development of general or cross-skills plays an important role as the acquisition of the specific skills. certain differences between skills development and demographic aspects are detected. thus, the results indicate that women values different types of skills than men. on the other hand, the size of the company can also play an important role in the development of skills related to the work organization. our study has some limitations, since the skills were analyzed from the trainee perspective. the type of work assigned to the trainees is limited in certain skills such as the capacity of decision-making or specific cognitive abilities. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 137 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 acknowledgements the data of this work belong to the project: "the acquisition of skills in college grads: vision and opinion of students, teachers and entrepreneurs (viopes)" funded by the university of barcelona in the call for research projects in social sciences and human 2012-2013. also, we would like to thank the project: "practice in company with indicator and the relationship between the world, socioeconomic and university" funded by the university pompeu fabra in the call for the projects support to the quality innovation (2013-2014 school year). we thank alba miralpeix vinas, ba in economics and business administration from the university pompeu fabra, for help in tabulating the data. 5. references akkermans, j. schaufelia , w.b. brenninkmeijera , b r.w.b. blonk , (2013). the role of career competencies in the job demands -resources model. journal of vocational behavior, 83, 356 -366. doi: 10.1016/j.jvb.2013.06.011 allen , t.d. eby , l.t. o'brien , k.e. lentz , e. (2008). the state of mentoring research: a qualitative review of current research methods and future research implications. journal of vocational behavior, 73, 343-357. doi: 10.1016/j.jvb.2007.08.004 benito, a., cruz, a. (2005). nuevas claves para la docencia universitaria. en el espacio europeo de educación superior. madrid: narcea, s.a. de ediciones. brockmann, m.; clarke, l.; winch, c.(2009). competence and competency in the eqf and in european vet systems. journal of european industrial training, 33(8/9), 787-799. doi: 10.1108/03090590910993634 chun, j. i. sosik , jj, yun , ny, (2012). a longitudinal study of mentor and protégé outcomes in a formal mentoring relationship", journal of organizational behavior, 33,1071-1094. doi: 10.1002/job.1781 coco, m. (2000). internship: a try before you buy arrangement. advanced management journal, 65(2), 41-43. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 138 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 de la poza, j. (2008). la selección de recursos humanos basadas en las competencias. propuesta de un modelo para la selección de recursos humanos. aedipe 4(marzo), 1819. de vos a, de hauw, s. van der heijden , b.i.j.m. (2011). competency development and career succes: the mediating role of employability. journal of vocational behavior, 79, 438-447. doi: 10.1016/j.jvb.2011.05.010 eby, lt, allen, td, evans , sc, ng , t. dubois , dl (2008). does mentoring matter? a multidisciplinary metaanalysis comparing mentored and non mentored individuals. journal of vocational behavior, 72(2), 254-267. doi: 10.1016/j.jvb.2007.04.005 escalona, a.(2009). actividades para la enseñanza y el apredizaje de competencias genéricas en el marco del espacio europeo de educación superior. zaragoza: prensas universitarias. fletcher, e., (2012). business education university supervisors perspectives of mentor teachers' competencies. journal of career and technical education, 27(1)56-71. frasquet, m. calderon, h. cervera, a. (2012). university -industry collaboration from a relationship marketing perspective: an analysis empirial in a spanish university. high education, 64, 85-98. doi: 10.1007/s10734-011-9482-3 gamboa, v. paixao, m. p. neves de jesus, s. (2013). predicts internship quality career exploration of high school students, journal of vocational behavior, 83, 78-87. doi: 10.1016/j.jvb.2013.02.009 gonzalez, j., i wagenaar, r., ed. (2003). tuning educational structures in europe. informe final. proyecto piloto – fase 1. bilbao: universidad de deusto. imbernon, f., medina, j.l. (2005). metodologia participativa a l’aula universitària. la participació de l’alumnat. barcelona: universitat de barcelona. kinicki m. fugate, a.j. (2008). a dispositional approach to employability: development of a measure and test of implications for employee reactions to organizational change, journal of occupational & organizational psychology, 81(3), 503-527. doi: 10.1348/096317907x241579 kirk, j miller, m.l. (1990). reliability and validity in quantitative research, qualitative research methods series, 1, newbury park, london, nova delhi , a sage university paper. monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 139 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 kuijpers m.a.c.t. and j. (2006). career competencies for the modern career, journal of career development, 32, 303-319. doi: 10.1177/0894845305283006 kuijpers, m. schyns, b. y scheerens, j. (2006). “career competencies for career success”, the career development quartely, 55,168-179. doi: 10.1002/j.21610045.2006.tb00011.x matthews, c., y zimmerman, b.b., (1999). integrating service learning and technical communication: benefits and challenges, technical communication quartely. 8, 383404. doi: 10.1080/10572259909364676 meijers, f. kuijpers, m. gundy, c. (2013), the relationship between career competencies, career identiy, motivation and quality of choice, international journal for educational and vocational guidance, 13 (1), 47-66. doi: 10.1007/s10775-012-9237-4 monllau jaques, m.t. rodríguez ávila, n. (2010 b), “the new docent frame: the comparative competencies studies between business and social policy degree”, enhancing strategies for global quality learning in higher education, international consortium for educational development (iced), https://congresos.ultramarexpressevents.com/congress/en/iced-2010/inicio/. monllau jaques, m.t. rodríguez ávila, n. (2010a), “el nou paradigma docent i la problemàtica de la implantació a les escoles d’empresarials d’acord amb el pla bolonya. els estudis a les empreses cobreixen les expectatives dels empresaris?”, papers: revista de sociología, 95(1), 95-213. montoya rios, f.j., aguilar zambrano, j.j. (2011) “the linkage of university-industry within business internships: a conceptual model using idea generation techniques”, journal of technology management & innovation, 11,196-208. nrayanan, v.k., olk, p.m., fukami, c.v. (2010). determinants of internship effectiveness: an exploratori model”, academy of management learning & education, 9(1), 61-80. doi: 10.5465/amle.2010.48661191 rothwell, a., herbert, i., y rothwell, f., (2008). self-preceived employability: construction and initial validation of a scale for university students, journal of vocational behavior, 73. 1-12. doi: 10.1016/j.jvb.2007.12.001 rotundo, m y sackett, p., (2004). specific versus general skills and abilities: a job level examination of relationships with wage, journal of occupational and organizational psychology, 77, 127-148. doi: 10.1348/096317904774202108 monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 140 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2306 social and technological sciences eissn: 2341-2593 van der heijde, cm. van der heijden, b.i.j.m. (2006), a competence-based and multidimensional operationalization and measurement of employability”, human resource management, 45(3), 449-476. doi: 10.1002/hrm.20119 monllau jaques et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 124-141 | 141 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 80 theoretical biomimetics: a biological design-driven concept for creative problem-solving as applied to the optimal sequencing of active learning techniques in educational theory la shun l. carroll university at buffalo (usa) lcarroll@buffalo.edu received: 2016-06-1; accepted: 2017-09-12 abstract the article introduces the author’s concept of applying principles of biological design to drive creative problem-solving. it provides a brief background of the field of biomimetics, which serves as a context for the reader to appreciate how it began with an established field upon which he constructed and adapted his concept to apply to human-made intangibles. a discussion of theoretical biomimetics specifies the differences between it and its predecessor. the final section provides the opportunity to see theoretical biomimetics in detail applied to education theory to address a problem related to student learning in higher education and how best to establish optimized sequences to implement evidence-based active-learning techniques to fill a void in the literature demonstrating from nature what has worked. the originality lies in the author taking a multidisciplinary approach to synergize a sequence of existing active-learning techniques and apply them to a new area in a new way. keywords biomimetics, creative problem-solving, educational theory, active learning, teaching methodologies, pedagogy 1 introduction what do aircraft, cpr, the camera, eyeglasses, and swimming all have in common? they all exist as results of the successful comprehension and application of data obtained multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 81 from the observation of principles that guide biological processes or designs to create, develop, or advance human-made tangibles. if one wants to fly, then studying birds is the best option. likewise, if the goal is to sustain life then understanding how life is sustained should provide guidance on which essential principles to incorporate. therefore, it requires no stretch of the imagination to understand the potential benefits of studying and applying principles that govern biological processes and designs in nature to address problems of all sorts. problems do come in many forms, such as naturally occurring events. while it may not be the case that the conditions are the cause of the natural event occurring; the natural event occurs because of the conditions and may provide clues as to how best to deal with it in the future. responsibility is either directly attributed (i.e., the cause of), or indirectly attributed (i.e., because of). although it is unfathomable to either encounter or create a problem for which there is no solution, without insight into the workings of the most creative of problem-solvers known to man, it may prove too daunting a task to solve on our own. our reality consists of a constant juggling act where we become aware of a problem only to reveal another problem in the process of solving the first. there is apparently no way to address every problem without creating “new” ones. nevertheless, there may be hope for finding a new application that borrows from an existing field known as biomimetics, which the author has envisioned and he believes has the potential to be a discipline in its own right. 2 biomimetics: the established field architects have studied plants and incorporated these principles into the structural design of towering edifices (knippers and speck, 2012). likewise, engineers can apply principles learned from data obtained observing wing span and migration in nature to design wings for aircraft with specific purposes in mind, such as long flights, fast flight, and energy minimization (rasmussen et al., 1995). the most efficient and enduring things made by multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 82 man are modeled with nature in mind. it is no surprise why biomimesis (bar-cohen, 2006), or mimicry of natural processes and designs to create structures and materials (tangibles), has been recognized and widely adopted as both an effective and efficient approach to human-made things (bond et al., 1995). with this in mind, the author wondered with the completely logical application to vehicles, buildings, aircraft, and other human-made tangibles, “why couldn’t biomimetic principles be applied to the creation of human-made intangibles with as much effectiveness?” from this, natural and biological phenomena serving as the basis for conceptual problem-solving was born. although the author cannot be credited with the concept of using biological design or processing approaches to creating human-made tangibles also known as biomimesis, the author is contributing the idea of using biomimetic principles as an approach to creating, developing or advancing human-made intangibles such as theories. researchers have studied biological processes and design in order to understand them and explain, which we know as science. many have certainly taken science a step further by exploring natural processes and designs and applied principles to create human-made things or tangibles, which is known as biomimetics. the author has taken biomimetics one step further by implementing the principles of biological processes and designs to create, develop, or advance human-made intangibles. the author has not come across anything in the literature. whether it will come to be known as a branch of biomimetics, or be to biomimetics what biomimetics is to science is yet to be known. nonetheless, the author describes and defines his contribution as the concept of the study and application of principles guiding biological processes and designs in creating, developing, or advancing human-made intangibles and calls it conceptual or theoretical biomimetics. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 83 3 theoretical biomimetics: the new field nature does not operate in terms of right, or wrong, with respect to processes or design; man interprets them as such relative to subjective experience. we have studied things that are good or work well in nature and applied the underlying principles to the development of human-made physical processes and designs successfully with good reason. nevertheless, as much can be learned from how biological processes and designs that are deemed inadequate or ineffective operate. as an example, for scientists to study the underlying biological principles of the “pathological” processes of metastasis in malignancies and apply them analogously to create, develop, or advance a human-made cascading system responsible for interception, encoding, encryption, and dissemination of information would be considered biomimetics since the resultant application would be tangible. the point that the author is making is that when biological processes and design work well using them as inspiration is a good idea. however, it is when these biological processes and designs are violated then studied, as in the case of malignancies, that we have the best opportunity to capitalize on the potential for the greatest breakthroughs. mother nature does not often have design flaws, and when she does, they are not many. to study and learn about how they affect her and how she handles them reciprocally the author feels would provide so many answers to problems that we are yet to encounter and is why “conceptual” problems should be explored with such abstract intellectual framework. it is with this approach in mind that the author employed theoretical biomimetics to conceive of an abstract yet creative approach to establishing a sequence for implementing evidence-based active learning (bonwell and eison, 1991) techniques to synergize student learning and achievement in educational settings. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 84 4 application of theoretical biomimetics to education for the first demonstration of how theoretical biomimetics may be used in creative problem-solving, the author attempted to address the issue with student learning in higher education after reading a disturbing fact. research conducted on more than 2,300 undergraduates found that 45 percent of students show no significant improvement in the key measures of critical thinking, complex reasoning and writing by the end of their sophomore years(arum and roksa, 2011). something definitely in need of revisiting, according to this statistic, is how the process of educating students occurs in the classroom. so the author contemplated on the issue as an intellectual exercise to put theoretical biomimetics to the test. the approach that the author took was to conceptualize how to establish a sequence for implementing well-accepted, evidence-based active learning techniques to synergize the educational process. the goal was to devise a “recipe” for instructors to use in the classroom, which could provide guidance based on desired outcomes, materials taught, or time constraints. nevertheless, there will be some who oppose implementing such a thing, and the author has already expected naysayers to anticipatorily criticize this approach as being too mechanistic and rigid to be of any practical use to educators. the reader will be able to see that this could not be any more inaccurate a statement if they understand the what natural process inspired the idea. 4.1 selecting a natural process for comparison many studies advocate active learning techniques or strategies individually. nevertheless, though there are lists that indicate the types of things educators are recommended to incorporate, the author found nothing close to an established protocol or flowchart detailing a sequence of steps in which to implement the techniques. therefore, the author set out to multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 85 integrate some of the best techniques to include from the literature and figure out how an optimal sequence might be modeled. the optimal sequence was born of the idea that, like ingredients for a recipe that are added in a specific order to yield optimal results, there must be an optimal order to introduce active learning techniques. more specifically, there must be an optimal order or sequence for different types of educational outcomes, various kinds of subjects, and different time constraints at the least. that notwithstanding, the key was to determine which natural biological process or design could be superimposed on the learning process successfully. after much consideration to natural processes whose design could provide the scaffolding necessary to implement an optimal sequence, the author determined he would model the instructional and learning process after his personal experience with a type of cardiac arrhythmia known as supraventricular tachycardia (svt). 4.2 biological process, design, and solution although there exist varieties, svt may be explained as a reentrant loop in which nerve impulses occasionally in the heart are conducted through accessory pathways at different rates allows the faster one to travel back up on the path used by the slower one instead of continuing in the original direction (badhwar, 2010). the author has had this condition his entire life. those unmistakable rapid heartbeats were more annoying and frightening than anything. what made having an episode or svt event so frustrating, yet intriguing, was how it occurred. the author was unable to pinpoint what triggered them due to seemingly unpredictable occurrences. he was to avoid anything that increased heart rate because the faster the rate, the increased likelihood and episode would occur. the author had to wear a holter monitor continuously for a period to be able to record an event before an official diagnosis was confirmed. eventually, while wearing the monitor 24hrs daily, the author had an episode multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 86 that was recorded for the author’s cardiologist who confirmed the diagnosis of supraventricular tachycardia. when this occurs, the nerve impulse becomes trapped in this “reentrant loop” resulting in tachycardia and the associated symptoms that are experienced (colucci et al., 2010). trying to “break” the episodes when the author first had them, the author noticed that it seemed as though the longer the author had an episode, the easier it was to break it. there was something about spending more time in the reentrant loop that the author believed made escaping or “breaking” it easier. the author had success utilizing the valsalva maneuver as needed with a daily regimen of beta-blockers. a valsalva maneuver is one method to break the svt episode that involves trying to forcibly expel air while bearing down against a closed glottis to slow the heart rate and break the reentrant loop (pandya and lang, 2015). beta-blockers are a useful class of drug routinely used to prevent the onset of the reentry phenomenon that occurs with svt by keeping the heart rate low by depressing nodal automaticity and inhibiting function (kowey et al., 1987). 4.3 abstraction and application: sequence is everything after the author had finished writing sections of this paper that he had not started, he began to format the completed sections he had not yet written. next, the preceding sections that followed were what motivated him to come up with the idea to start writing about theoretical biomimetics in the first place. if the reader finds something not quite right or is having difficulty comprehending the previous two sentences, then author’s point has been made: sequence is everything. moreover, if those same two sentences made sense to the reader, the author’s point has also been achieved. he or she was presented with a group of individual words serving as information inefficiently juxtaposed that the mind rearranged in the most appropriate sequence to result in the information being intelligible and useful. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 87 whether it is dna, coding for amino acids, speech, and conversation, or logic and mathematics, sequences determine economy, effectiveness, and efficiency. even with culinary arts like cooking and baking, creative arts like dance and drawing, construction, architecture, or simply daily activities like getting dressed and ready to work, there is not a single aspect of existence that has not been affected, inspired, or optimized by sequencing. the author believes that as important as individual active learning evidence-based techniques may be, without established sequences in which to implement them they have nothing more to offer. unfortunately, their full potential to collectively become greater than the sum of individual parts will never be realized. theoretical biomimetics aims to prevent that from occurring and is a prime example of how concept abstraction and application works. theoretical biomimetics led to the abstraction of the biological process, design, and therapeutics involved in svt to act as a framework for conceptualizing how one might best sequence evidence-based active learning techniques to optimize student learning, achievement, and educational outcomes. analogous to individual ingredients each of which has individual properties when combined in a variety of sequences that yields specific results, the author believes that the particular arrangement or sequence of steps implemented in an optimized instructional method will lead to different outcomes. experiments will be needed to validate which sequences are best for particular disciplines, educational goals, student learning types, and time requirements. it must be stressed that the author is not advocating mechanical rigidity as some may contend. active learning does require some flexibility, and the author has conceptually accounted for that in the process of establishing a sequence. one must remember that all that is minimally necessary for a sequence are two techniques where one must precede the other. in this fashion, the overall optimal sequence of techniques will be determined by combining the results from a series of individual sequence experiments performed on pairs of techniques at a time. conducting the experiments this way not only will allow an overall multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 88 optimal sequence, but technique-pair results individually can be organized such that algorithms may be formulated to suggest sequence choices for which techniques work best when immediately following others. ultimately, these flow diagrams for use by both educators and students alike will facilitate instruction and study while simultaneously providing the sort of flexibility required to deal with ever changing classroom dynamics. 4.4 selected active learning techniques more than a quarter century ago a review was done, which concluded that traditional lecture methods were not as effective as interactive discussion approaches in significantly improving retention, interest, and participation among students (mckeachie, 1987). for a while now there has been a shift in favor of incorporating more dynamic or active techniques categorized as active learning. active learning techniques are activities designed to get students involved in doing things and thinking about what they do (bonwell and eison, 1991). the author believes that characteristics of an optimal sequence of active-learning techniques should include that which minimizes the likelihood of students becoming stuck in a metaphoric “reentrant loop.” additional features that should be built into the optimal sequence is that, in the unfortunate event that students do become stuck having to “reenter the loop,” the optimal sequence should allow correction of weaknesses before students progress. incorporating features like employing demonstration (milner-bolotin, 2007), think, pair, share (mills, 1995), and classroom assessment techniques (angelo and cross, 1993) will ensure that students will have the opportunity to comprehend. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 89 table 1 example of instruction sequence using active learning strategies active learning and evidence-based techniques step in sequence classroom assessment 1st teacher expectation 2nd-3rd explicit teaching 4th demonstration 6th small portions of info 6th interactive 7th group collaboration 8-9th summative assessment 10th 4.5 theoretical biomimetically suggested sequence in which to implement techniques the steps involved in the optimal sequence that the author suggests are as follows: (1) the teacher administers a diagnostic assessment to determine individual and class baseline levels with respect to the material to be covered in the course, (2) the teacher must assume students are capable; initially assuming the opposite in the absence of proof is selfdefeating and self-fulfilling, (3) based on the assumption they are capable, he or she establishes high expectations for the students, (4) the teacher must be sure that expectations are clearly communicated (teaching) and incorporate students teaching into course goals, (5) the high expectations imply confidence in students who then feel obligated to meet these expectations and fully commit to the educational experience, (6) having obtained student commitment, the educator begins instruction with ice-breaker activity that ties into the topic followed with an optimized sequence pair of demonstration (milner multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 90 bolotin, 2007) and deconstruction (rosenshine, 2012), or any other appropriate pairing of active learning techniques determine using an algorithm for the allotted time, material or educational outcome desired, (7) students are encouraged to ask questions to solidify understanding of the material immediately afterward making the process interactive or dynamic. the instructor will reciprocate with purposeful questions appropriate to ability level, (8) students may then have an opportunity to demonstrate they understand, which gives students a chance to allow for teacher feedback, modification, and refinement before misconceptions are set in. this step displays the ability students have to teach one another (paul) and collaborate with each other about what they have learned (johnson et al., 1991) [steps 6 thru 8 are the reentrant loop that gets repeated and students only move to the next step after full comprehension, and students’ total commitment is positively reinforced by comprehension resulting in a continued effort to thoroughly master material], (9) the educator tests for students’ mastery, which they may readily exhibit by applying what they have newly mastered to address novel problems. mastery is defined by knowing what to do, how to do it, and why it is done the author coined “the interrogative trinity,” and (10) the demonstration-deconstruction of material to allow students to comprehend and engage themselves in doing serves a dual purpose: to display material knowledge for assessment while building confidence, and to reinforce understanding of the material, which also builds confidence. all components of an optimal sequence will be substantial. nevertheless, the analogous reentrant loop design overlaying steps six to eight are the heart of the sequence and continues indefinitely until the material is mastered by the student. each cycle of reentry reinforces learning the material such that subsequent cycles become less likely while permitting progression in the most efficient manner possible. this design will ultimately lead to the successful application of newly mastered material to address new problems by ensuring identified weaknesses are corrected before progressing. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 91 5 conclusion the suggested sequence of techniques composed of sequence pairings serves only as an example and has not been validated experimentally yet. once data is obtained, it may be found that group collaboration yields the best results when following demonstrations as a sequence pair, in which case the sequence above can be further optimized by rearranging the techniques to reflect this. the sequencing process is dynamic and as different variables are discovered needs to be flexible enough to accommodate such newly discovered information. the author’s concept is structured, yet flexible enough to meet such demands. the author’s response to criticism about his “recipe” of sequence pair optimization being too rigid and mechanical to be practical is best understood with an analogy to culinary training. the critics should know that “recipes” are to be considered to be structured, sequenced steps that provide roadmaps to guide aspiring chefs. these established protocols have been tested and sequenced based on the individual ingredients and what result is desired. for instance, when preparing a dish that consists of a liquid ingredient and a finely ground starch ingredient, there is an enormous difference between adding the liquid to the starch and adding the starch to the fluid. one sequence with the pair of ingredients results in a clumpy consistency while the other leads to a smooth consistency, which is why knowing that one wants to achieve a certain result requires that a specific order be followed and is exactly what a recipe provides. important to realize, however, is that a recipe containing five ingredients consists of units composed of four sequenced ingredient pairings. eventually, through understanding why the ingredient pairs in a recipe are sequenced a certain way the novice chef will transition from the apparent rigidity of an exact recipe to the next level of improvisational cooking. if the reader has ever had the opportunity to witness an improvised meal made by a chef, then he or she undoubtedly was in awe at what was accomplished. to the unfamiliar, such improvisation may seem random and unstructured. however, the spontaneous dish was created with microstructures. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 92 the multitude of culinary complexities one observes in end dishes derive from the chef’s ability to employ an algorithmic approach subconsciously. this tree is comprised of sequencing paired ingredients such that given the availability of n number of ingredients and a result in mind, they have at least n! many ways to order them individually. that is, in the case of using eight ingredients to keep it simple, the chef has a total of eight factorial, or 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1 = 40,320 possible eight-ingredient sequences. the 40,320 total permutations composed of sequenced pairings means that there are n(n-1) = 56 distinct sequenced pairings available to combine to yield the global order. it is the ordering of sequence pairs by the chef that produce the relatively endless possibilities when one accounts for the actual n ingredients available in any well-stocked chef’s environment. as instructors who are tasked with putting structure in students, utilizing a recipe to provide a structured approach is the most efficient method convey information. with our hypothetical chef analogous to instructors who have at their disposal n number of actual evidence-based active learning techniques, given n techniques and an outcome in mind, they have n! many ways in which to sequence the techniques overall using n(n-1) sequence pairs to achieve their goal. the example used eight ingredients intentionally to draw a comparison to an existing binary system in common use. the same way zeros and ones are employed in groups of eight to encode an alphanumeric character, each node of any eight ingredient sequence will be yes-no, or zero-one, such that active learning sequences may be conceived of as strings of a combination of decisions incorporating anywhere from no techniques (i.e., all no’s) to all eight (i.e., all yes’s). so how might this work for active learning techniques at an instructor’s disposal? let us assume that data have been obtained experimentally for sequence pairs that rank them according to effectiveness using a zero to five score; zero is less efficient, and five is most effective. certain techniques are obviously more appropriate to begin sequences, such as classroom assessment techniques (cats), which can be used to obtain a baseline, are anonymous, ungraded, and for planning lessons. so, we select cat, which is technique multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 93 one. although there are seven techniques to choose to follow technique one that we selected, evidence on pairings to which we refer shows that when technique four or seven follows the results are greatest. we then select number four and have our first sequence pairing, which we have chosen to provide the best results when used in this order. our next technique must follow number four. so we consider the remaining sequence pairings with techniques that follow four and select the pair that provides best results according to evidence. in this case, for example, the best results are obtained when technique six follows four. therefore, we have techniques sequenced one-four-six obtained from the sequence pairings of one-four and four-six. utilizing a binary system of visualization, a row of eight nodes in numeric order left to right would indicate sequence pair one-four by yes under one and yes under four. to represent four-six, a yes would be in the node underneath four and six as well. when combined, nodes for one, four and six would result. to represent the inverted pair fourone, for instance, the nodes under one and four would be no and all others yes. in this manner, a pair and its inversion could be represented correctly. the reason that the author mentions how this may be done is to show how easily data obtained from experiments could be used to devise an algorithm for active learning decision-making programs to assist educators in choosing the most appropriate sequences for their desired purpose. the utilization of an algorithmic approach to incorporating active learning and evidencebased techniques will help instructors select optimal sequences of any length based on desired outcomes, time constraints, and material covered using available evidence-based data from individual pairings. contrary to what critics may believe, although particular “recipes,” or sequences, result in unique outcomes that can only be achieved with one order, there is no rigidity at all since sequence pairings are the units of which exact recipes are comprised and may be rearranged in nearly any order imaginable. in fact, sequencing is the opposite of rigidity; it provides flexibility as well as structure instead of guesswork and its associated unpredictability. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 94 the field of theoretical biomimetics as envisioned by the author allows individuals to become intellectual chefs of creative problem-solving. the examples from which to draw inspiration abound and their longevity are a testament to their brilliance. despite conceptually being in its infancy, there is so much potential for theoretical biomimetics to blossom into a full-fledged field with the capabilities to create and develop both humanmade intangibles directly, as well as human-made tangibles (i.e., biomimetics) indirectly. in this fashion, theoretical biomimetics not only possesses a “meta-“ aspect that will result in theory about theorizing and concepts on how to conceptualize, but the author conceived of it after what inspired its development yet it will be the inspiration for the creation of that which the idea that inspired its own creation was supposed to do thereby surpassing it! it is hoped that the author’s ideas and article have inspired the reader. if there is one thing that the reader should take from this paper, it should be to understand the importance of sequence in that coming after does not preclude the possibility to take one further than that which has come before. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 95 6 references angelo ta and cross kp. (1993) classroom assessment techniques: a handbook for college teachers, san francisco: jossey-bass publishers. arum r and roksa j. (2011) academically adrift: limited learning on college campuses, chicago: university of chicago press. badhwar n. (2010) introduction to supraventricular tachycardia. cardiac electrophysiology clinics. 179-181. https://doi.org/10.1016/j.ccep.2010.01.001 bar-cohen y. (2006) biomimetics: biologically inspired technologies, boca raton, fl: crc/taylor & francis. bond gm, richman rh and mcnaughton wp. (1995) mimicry of natural material designs and processes. journal of materials engineering and performance 4: 334-345. https://doi.org/10.1007/bf02649071 bonwell cc and eison ja. (1991) active learning: creating excitement in the classroom, washington, dc: school of education and human development, george washington university. colucci ra, silver mj and shubrook j. (2010) common types of supraventricular tachycardia: diagnosis and management. american family physician 82: 942-952. johnson dw, johnson rt and smith ka. (1991) cooperative learning: increasing college faculty instructional productivity, washington, dc: school of education and human development, george washington university. knippers j and speck t. (2012) design and construction principles in nature and architecture. bioinspir biomim 7: 015002. https://doi.org/10.1088/1748-3182/7/1/015002 https://doi.org/10.1016/j.ccep.2010.01.001 https://doi.org/10.1007/bf02649071 https://doi.org/10.1088/1748-3182/7/1/015002 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7078 social and technological sciences issn: 2341-2593 la shun l. carroll (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 80-96 | 96 kowey pr, friehling td and marinchak ra. (1987) electrophysiology of beta blockers in supraventricular arrhythmias. the american journal of cardiology 60: 32-38. https://doi.org/10.1016/0002-9149(87)90706-5 mckeachie wj, pintrich, p. r., lin, y. g., & smith, d. a. (1987) teaching and learning in the college classroom: a review of the literature. ann arbor: national center for research to improve postsecondary teaching and learning. mills b, lyman, f. t., & davidson, n. (1995). (1995) interactive learning in the higher education classroom (pp. 204-225). washington, dc: national education association. education: 204-225. milner-bolotin mk, andrzej; rieger, georg. (2007) can students learn from lecture demonstrations?: the role and place of interactive lecture experiments in large introductory science courses. journal of college science teaching: p45-49. pandya a and lang e. (2015) valsalva maneuver for termination of supraventricular tachycardia. annals of emergency medicine. 27-29. https://doi.org/10.1016/j.annemergmed.2013.07.012 paul am. the protégé effect. time. rasmussen bh, norberg a and sandman po. (1995) stories about becoming a hospice nurse. reasons, expectations, hopes and concerns. cancer nurs 18: 344-354. https://doi.org/10.1097/00002820-199510000-00003 rosenshine b. (2012) principles of instruction: research-based strategies that all teachers should know. american educator 36: 12. teaching asfeb. explicit teaching: a lesson structure that delivers results. the australian society for evidence-based teaching. https://doi.org/10.1016/0002-9149(87)90706-5 https://doi.org/10.1016/j.annemergmed.2013.07.012 https://doi.org/10.1097/00002820-199510000-00003 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 152 development of generic and strategic skills through a wiki plat-form in industrial engineering programs l. celorrio-barragué*,1, a. lopes-ramalho2, m.r. calvete-gaspar2 1dpto. de ingeniería mecánica. universidad de la rioja, san josé de calasanz 31, 26004 logroño, spain. 2polytechnic institute of castelo branco, av. do empresário, 6000 castelo branco, portugal * corresponding author: email: luis.celorrio@unirioja.es; phone: + 34 941299542 received: 2015-7-10; accepted: 2016-1-3 abstract this paper describes the collaborative learning task carried out in the framework of a strength of materials course in an undergraduate degree program in industrial engineering using a wiki platform as a support tool. stages of the teaching/learning process are described. these are: organization of the wiki to host the work of student groups, creation of working groups and assignation of projects, delivery of guidance material and model, editing the project and publishing pages on the wiki, assessments performed using rubrics, and the results obtained. the wiki platform provides interesting statistics for evaluating cooperation between students and time variation of students’ effort. the final quality of the projects is very high because a midterm evaluation in the form of a “peer review rubric” is carried out. then, students improve their projects. the paper concludes by gathering the opinions made by students; the benefits provided by this teaching experience are then analyzed. keywords wikis, cooperative learning, transversal competences, group working, rubrics. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 153 1. introduction this article outlines the development of generic and strategic skills by students in all three degree programs currently studying industrial engineering at the universidad de la rioja in the strength of materials course. the latest guidelines drawn from the european higher education area stipulate that, when designing, planning and evaluating courses, the primary focus should be on skills. one must consider not only the skills specific to each course or degree, but also the generic and strategic skills that every university hopes to instill in its students. the most sought-after generic skills in the current labor market are interpersonal skills. collectively, this skill set is popularly known as “teamwork” and consists of the ability to work on an interdisciplinary team and to communicate in other languages. these skills can be sharpened by having students work in groups to collaboratively complete one shared or several different projects. the current development of information and communications technologies (icts) has enabled the creation of very powerful group-work tools, not just in the field of education but also in the professional sphere. social networks and messaging apps for smartphones facilitate effective communication between group members. it is vital that our students learn to apply these ict tools to their learning practices. the project that each group must complete consists of describing an emblematic or singular feat of engineering or architecture. these are works that have roused the interest of the general public and are internationally recognized for their touristic or historic importance, for their large size, or because they are emblems of a certain population, were converted into internationally renowned monuments, apply the most innovative construction techniques available or, in some cases, because they were poorly designed, famously collapsed or required extensive repairs. these works have received special attention by the mainstream media and audiovisual content producers. we could cite numerous very wellknown tv shows that help a general audience understand the inner workings of the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 154 construction process, such as megastructures, megaconstructions and extreme engineering. these shows are constantly in reruns and they doubtless influenced more than one student’s decision to pursue a degree in engineering. taking these reports as our point of departure, we proposed that second-year students pursuing bachelor’s degrees in industrial engineering and enrolled in the course strength of materials complete a group project on singular works of engineering and architecture. we asked them to describe these works in greater depth than the televised report, highlighting the chief points of interest for industrial engineers, such as the building type, the new materials used, innovations in the field of construction technology, installations, maintenance and monitoring. to complete the project, students used the best ict tool available for group learning: wikis. this technology is described in the following section. 2. wikis a wiki is a server-based computer application that can be accessed from any computer connected to the internet. from the user’s point of view, a wiki is just another webpage, but with the advantage that allows for information sharing. wikis, along with blogs, social networks and a significant number of applications make up what is known as web 2.0. the distinctive feature of web 2.0 is that it is no longer a single user who shares information, but rather the entire community that makes contributions. working on a wiki is very simple: the user simply registers, logs on and gets to work. all registered users can edit or modify content and create new pages (leuf and cunningham, 2001). there are typically one or two administrators who ensure that the wiki is functioning properly and grant permission to other users. it is most convenient if this is the professor of the subject in which the project is assigned. however, granting administrative permissions to a handful of students (representatives from each group, for example) can ease the professor’s task load and give students the opportunity to explore the wiki to its full potential. furthermore, as multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 155 leaders on a higher hierarchical level than the other members of the group, these student administrators feel more responsible. they are more motivated to perform their duties, and this motivation carries over to the rest of the group. wikis are among the best tools for collaborative learning and teamwork. they make it possible for students to stay in contact, share and contrast their ideas and opinions on a wide range of subjects, complete a group project, and make consensus-driven decisions when solving problems (moral and villalustre, 2007, 2008). wikis also have numerous advantages for professors. they facilitate visibility in the educational process and direct communication between the professor and the students, giving professors the chance to guide projects towards fixed goals. the teacher can evaluate the participation and individual contributions each student in the group makes to the project, since on the wiki there is a record of contributions and their authors. this helps professors continually assess the progress of the project, giving grades or feedback every week, two weeks, etc. (adell, 2007). wikis are the perfect tool for peer and group assessment when the project is complete. for example, a group can evaluate another group’s work according to a specific peerassessment rubric. in actual practice, students use other ict tools to support the material of the other work they do, too, and to communicate between group members and share information. it is normal for members of the group to create a group on messaging apps such as whatsapp, or on a social network such as facebook, to help them communicate better and set up group meetings. another key element is a shared folder on a file storage service on the cloud, such as dropbox or google drive. these folders hold project files and drafts of text that will make up part of the final draft. this makes students’ work more comfortable, since they are not constantly under the watchful eyes of the professor and/or of classmates in other groups. still, this way of working has a drawback for professors—they cannot track the progress of the assignment without imposing mid-project evaluations of the wiki. e multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 156 mail is essential, but it has largely been dethroned by other, more direct means of communication, such as instant messaging. on some occasions the wiki’s internal messaging function is used. figure 1 shows the homepage of the wiki described in this article. figure 1: singular works wiki 3. goals students complete their projects within the framework of the course strength of materials as part of the degrees in industrial engineering (mechanical engineering, electrical engineering, and industrial electronics and automation engineering) at the university of la rioja. strength of materials forms part of the mandatory training module in the industrial branch. it is taught in the second quarter of the second years for all three bachelor’s degrees in industrial engineering. the teaching team is made up of one professor who is also in charge of the course. strength of materials carries 6 ects credits, the equivalent of 60 class hours and 90 hours of student study and work. the group wiki project requires two hours of explanation and classroom work, with 10 of the 90 hours assigned to the project outside of the classroom. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 157 because the project is done in groups of five or six, each group should end up dedicating a total of 50 or 60 hours to their project. the 60 class hours are distributed as follows: 40 large group hours dedicated to theory and problem solving, and 20 hours in smaller groups, of which 16 are set aside for conventional classroom problem solving and questions; the remaining four are for practice time with the educational software mdsolids (philpot, 2014) and interactive mecmovies pages (philpot, 2011). the criteria fixed in the teaching guide for evaluation in the course are: 60% for the final written test, 20% for the group project described in this article and 20% for individual internship projects. the goals of the wiki-based group project are:  to improve students’ general skills, especially: o the ability to gather and process information. in order to acquire this skill, it is essential that students be familiar with advanced searching techniques, such as searching within interest domains, use of suitable search terms and how to search in various languages. o interpersonal skills, which consist of two abilities: the ability to work on an interdisciplinary team and the ability to communicate in foreign languages. o digital literacy. although engineering students are already familiar with icts, it is essential that they learn how to use them to their maximum potential.  to acquire specific skills, including: o describing practical applications in the field of structural engineering and construction and the theory covered in the course strength of materials. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 158 o describing the chief advances that have been made in the fields of structural engineering and construction technology. o creating a glossary of terms used in structural engineering and construction.  to maximize the university’s strategic skills: this collaborative working experience has much in common with project based learning methodology (pbl), which students will develop if they go on to do postgraduate studies at the universidad de la rioja.  to get teaching experience using collaborative learning methodologies through the use of icts, especially wikis.  to share the completed projects among the students enrolled in the course and, eventually, with a general audience. 4. stages in collaborative learning process in this section we outline the stages in our proposed collaborative learning process. after describing the goals and methodologies to be employed, choosing the wiki platform to be used is essential. our chosen platform is wikispaces. although the free educational version based on proprietary code does not offer the same range of possibilities as open-source wikis, it has a simple interface and does not require any programming background or the maintenance of a server for storing the wiki. a free version of wikispaces is available for educational purposes, with a maximum storage capacity of 50 gigabytes and unlimited users. it boasts greater functionality than the wiki tool available in the university’s official virtual learning environment (vle): it allows administrators to leave the wiki open for reading, unaffected by the annual changes made on the course pages without the professor’s consent. wikispaces has been the tool of choice for a wide range of teaching applications multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 159 and experiences. it is very easy to use. students can also access manuals (palomo et al, 2015), video tutorials on youtube, and real wikis online. 4.1 group structure groups of five or six will be made in the first two weeks of the quarter. through their representatives, every group will keep the professor informed on the group’s components and suggest several possible works on which they may base their projects. finally, since there are often overlaps in group preferences, the professor will ultimately assign the group’s work from among their proposals. this improves group motivation, since their work was not imposed on them, but the product of a consensus. a total of 33 groups are formed. on the wiki, a wikispaces project is created for each group under the name of the work on which they are to focus. these projects are set up in such a way that only the members assigned to each group can work on the project; they do not have access to other groups’ projects. the shared glossary will require creating a specific project that all members of the wiki can access. by the third week of the quarter all of the groups should already be formed and ready to start working. 4.2 guidance material and model providing students with guidance materials is essential for producing the high quality of work expected. a bibliographic paper, or a paper based on gathering and processing information, means more than simply “cutting and pasting.” rather, students must check several sources, extract the information of interest to the project and compile it in a coherent way, including graphics, charts, photos, references, conclusions, etc. students are given a set of rules that they must adhere to while working on their project and the criteria by which they will be evaluated. they are enumerated in these documents: multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 160  a list of the groups, their members and each group’s representative.  a guide to the aspects that must be addressed by the project, with insistence that, as future engineers, they must focus on the work’s most technical elements and eschew promotional and tourist information. and of course, the professor must point out that the groups can add new aspects or remove those that do not conform to the specific work that they are discussing.  project rules: dates for the midterm revision, final due date, rules for coordinating as a group and with the professor, rules on how to edit pages, as well as on the glossary and formal aspects of the project.  rubric or evaluation matrix for the midterm evaluation by the groups.  rubric or evaluation matrix for the final evaluation of the project by the professor. both the guide and the rubrics will give the work direction, given that they enumerate the objective elements to be evaluated. for example, to receive the highest possible score on the glossary aspect, the following requirements must be met: “at least 10 terms have been added to the glossary. the definitions and/or descriptions provided are in accordance with the use of these same terms in the project. the terms in the glossary itself have been included in the shared glossary. should some term already be entered into the glossary, the two entries should be merged by the groups.” two classroom hours should be dedicated to explaining the content of these documents and answering any questions students may have on the work assigned, or on how to use wikispaces. special emphasis should be placed on these two points, which will determine the success of the group project: advanced information searching: the project aims to teach students to reach a high skill level with the usual online search engines (e.g. google), how to use article databases (dialnet, google scholar), paper and presentation repositories (slideshare), videos (youtube, vimeo), etc. they should be heartily encouraged to search in english, since this multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 161 will return much more substantial and interesting information for the project, given that the works to be analyzed have attained international recognition. students should also be made familiar with the importance of blogs, social networks, message boards and, of course, how to participate in these media by asking questions and giving answers. using wikispaces: even though wikispaces is very easy to use, brief tutorials should be given on how to insert images, videos and presentations through editing add-ons called widgets. professors should also request the inclusion of a table of contents and links to pages both external to and within the wiki to make navigation easier. 4.3 editing the project and publishing pages on the wiki over the course of the following eight weeks, students will work in groups, and questions that arise will be addressed in class. this will settle formatting questions and help identify some of wikispaces’ editing abilities and limitations. for example, students cannot change the names of pages or delete them since they only have “member”-level permissions. to solve this problem, one can temporarily give the representatives of each group “administrator” permissions so that they can make these changes. 4.4 rubric-guided group evaluation in the twelfth week of the quarter, students conduct a midterm evaluation among their own groups using a rubric prepared specifically for that purpose. the evaluation will be conducted anonymously: each group knows the group that it is going to evaluate, but does not know the group by which it will be evaluated. afterwards, each group will send the professor a completed rubric in which the ratings given in the different fields will be justified—that is to say, merely assigning numeric evaluations will not be enough. the professor will evaluate the quality of the rubrics, bearing in mind the reasoning given for every aspect and the rubric’s objectivity. one important aspect, in terms of improving the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 162 projects, will be the recommendations provided by the evaluators. in general, these recommendations were well-considered and useful. the rubric is sent to the interested group through its representative. these comments are transferred to the group undergoing evaluation through the professor in order to maintain the evaluating group’s anonymity. of the 33 rubrics requested, the groups completed 32. the evaluations were fairly objective, although some of the reasoning did not achieve the degree of detail desired. they were a good reference for the project improvement phase. the negative aspects highlighted in the rubrics in the midterm revision period consisted of the lack of certain important aspects in the project, such as: a glossary, conclusions and bibliographic references. this last aspect is very important, since it helps separate those who have truly done the work, culling and analyzing information from diverse sources, from those who have simply “copy and pasted”. one measure that encourages more objective evaluation is requesting at least three rubrics for each project, completed by three different groups. in this way, the group evaluated has more information to improve the work. at the start of the thirteenth week of the quarter, the editing feature on the wiki is re-enabled so that groups can make improvements on their projects based on their peers’ recommendations. they have a 10-day period to make these changes. 4.5 final evaluation by the professor in the fifteenth and sixteenth weeks of the semester, the professor will conduct a final evaluation of the projects using the professor rubric which, naturally enough, is very similar to the rubric used by the groups. it includes additional fields for which professors must incorporate the progress of student activity by adding new information to wikispaces. thus, the evaluation is based on:  participation by all members of the group in the publishing of pages multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 163  continual work, throughout the work period and not just at the end of the quarter  revision of the pages  true teamwork as seen through shared editing of the pages  consideration of the recommendations made during the group evaluation the project grade will be the same for all members of the group, except in exceptional cases, and will be determined as follows: 25% grade received on the group-evaluation rubric 25% quality of the evaluation that the group made of another group’s work 50% professor’s rubric evaluation 5. results first, we will analyze the general data from the wiki. later we will analyze the momentto-moment evolution of editions, editors, visits and visitors produced on the wiki. the total number of wiki members was 183. the number of projects created for the completion of the group project was 33. at the time of the midterm revision, the number of edited pages included in the shared glossary was 652. the number of terms and definitions in the glossary was 218. together, the wiki pages were accessed (that is, viewed) 20,300 times. it is interesting to analyze the moment-to-moment evolution of some of the wiki’s parameters. wikispaces provides a few statistics that can be graphically represented with a spread-sheet. figure 2 shows the moment-to-moment evolution of the number of editors and the number of wiki pages edited in the period prior to the midterm evaluation. it is clear in the chart that wiki activity increases as the evaluation date approaches. the night prior to the midterm due date sees the highest number of students editing pages, 64, and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 164 the highest number of pages edited, 948. it is important to bear in mind that many groups prepared drafts of the pages and spent these last days solidifying their final versions. similarly, figure 3 shows the number of visits the wiki received, as well as the number of pages visited. this number saw notable growth in the days prior to the midterm due date. in both graphics we can see a period with no activity in the first days of april, caused by the holy week break. wikispaces also offers statistics on the number of messages exchanged. however, very few messages were sent, since students communicated directly or coordinated using instant messaging apps. figure 2: trajectory of page edits figure 3: progress of page visits progress of editors and edited pages progress of page visits per day and visitors editors edit pages visitors page visits multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 165 figure 4: histogram, edits per member besides the graphics above on general wiki activity’s document progress time, it is also possible to determine activity on a project-by-project basis, and even the activity of each member in a given group. it is possible to determine what time pages were edited, and by whom. it is important for the professor to consult this information when it comes time to evaluate the projects. wikispaces also allows the administrator to generate statistics for individual members; there are the statistics that were used to determine the number of editions, shown in figure 4. in this histogram, the classes correspond to the number of edits made to pages. we can see that the average student has made between 5 and 20 pages edits, while only 3 have made more than 100. this data corresponds to the state of the wiki during the midterm evaluation period. in the peer evaluation process, only two projects received failing grades. the rest of the projects received good grades, with an average of 7.1. this could give rise to the low activity in the wiki improvement phase. this is reflected in the following data. the final number of wiki pages included in the glossary was 736, reflecting a 94-page increase in the midterm revision. nine projects/groups made no changes. histogram, page edits per member frequency multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 166 as for the glossary, the final number of terms was 241, from which we can conclude that 23 terms were added in the improvement phase. (these 23 terms, however, were added by only three groups.) in the final evaluation, according to the professor’s rubric, the grades were, in general, better than those given in the peer-evaluation stage. the discrepancies between the professor’s evaluation and the peer evaluation were more drastic in the groups that received unjustified evaluations. 6. conclusions this paper describes a collaborative learning task completed by students in the second year of the degree programs in industrial engineering; it makes use of a wiki platform as a fundamental part of the learning process. this task consists, essentially, of gathering and processing information, including peer revision carried out among student groups. the quality of the projects completed is higher than that of tradition projects that only have a final due date. the causes behind this quality increase are, at their core, the introduction of a tool well suited for the collaborative project, such as the wiki and the implementation of revisions throughout the quarter. the wiki is also a key tool for making these revisions. managing such a large number of students, nearly 180, is practically impossible without the use of some sort of digital assistance, such as the wiki, which offers the functionalities described above. with this setup, members of the group relate to one another better: not only must they agree on the content, but also the form—that is, on the layout of the pages on the wiki. group members take on greater responsibility both individually and in the way they participate in the group. now their work can be scrutinized by others in real time. greater responsibility entails greater respect and greater group connection. in this way, students acquire abilities related to group work and cooperation. we can conclude that the projects completed multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 167 received better grades as consequence of students’ greater academic performance. this fact is motivation enough for students, given that they can improve their grade by nearly two points on the written test when the projects are well done, which in turn improves academic performance. we gathered students’ opinions on the methodology used. the general opinion is that the wikispaces interface is simple and preferable to other ict tools. the group evaluation was accepted in practically all cases, even among those groups that received low grades. it is important to highlight that now, with the implementation of the bologna process, students must not only complete “written tests,” but also projects, internship reports, portfolios, oral presentations, etc. they are subjected to an excessive workload that may lead them to leave their tasks to the last minute if they do not organize their time well. coordination between teachers in the same course is critical to avoiding this excessive workload. a cross-disciplinary seminar in which students learn time-management and productivity-boosting techniques is advisable. the group learning activity carried out is not just of importance for students’ professional future; it is also transferable to their postgraduate studies. those who go on to pursue the university master’s in industrial engineering at the universidad de la rioja will have to apply the group-work and cooperation skills acquired through the assignments such as the one described in this article. the university master’s is structured around the project based learning methodology, which proposes the solution of an integrated project to first-year master’s students. the integrated project is common to several courses and requires sound coordination between professors and students. a wiki can facilitate this coordination. this experience with wikis acquired during the undergraduate years will yield long-term benefits that will become apparent during students’ master’s studies or in their academic and professional future. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4486 social and technological sciences eissn: 2341-2593 celorrio-barragué et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 152-168 | 168 7. acknowledgements this project was funded by the vice-rectorate of faculty, planning and teaching innovation at the university of la rioja through the academic management of training and teaching innovation, within the framework of the announcement of teaching innovation project for the 2013-2014 academic year. and, of course, we want to thank the students in the course for their participation and their good work. 5. references adell, j. (2007). “wikis en educación.” in j. cabero & j. barroso (eds.), p. 323-333. editorial octaedro, granada. leuf, bo; ward cunningham (2001): the wiki way: quick collaboration on the web. boston: addison wesley. sánchez-rodriguez j. and ruiz-palmero j. manual de wikispaces. universidad de málaga. http://tecnologiaedu.uma.es/index.php/materiales/27-curso-de-elaboracion-de-wiki-conwikispaces moral-pérez m. e. and villalustre-martínez, l. (2007). las wikis: construcción compartida del conocimiento y desarrollo de competencias. iv jornadas internacionales de innovación universitaria, villaviciosa de odón (madrid), july 12-13. http://abacus.universidadeuropea.es/handle/11268/3352?show=full moral-pérez m. e. and villalustre-martínez l. (2008). las wikis, facilitadoras de aprendizaje colaborativo y el desarrollo de competencias. comunicación y pedagogía: nuevas tecnologías y recursos didácticos, issn 1136-7733, no. 226, 2008 , p. 13-17. timothy a. philpot. (2014) mdsolids, www.mdsolids.com timothy a. philpot (2011). mecmovies. http://web.mst.edu/~mecmovie/ http://tecnologiaedu.uma.es/index.php/materiales/27-curso-de-elaboracion-de-wiki-con-wikispaces http://tecnologiaedu.uma.es/index.php/materiales/27-curso-de-elaboracion-de-wiki-con-wikispaces http://abacus.universidadeuropea.es/handle/11268/3352?show=full http://www.mdsolids.com/ http://web.mst.edu/~mecmovie/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 history and nature of science enriched problem-based learning on the origins of biodiversity and of continents and oceans cristina sousa faculdade de ciências, universidade do porto. rua campo alegre, s/n, 4169-007 porto, portugal; email: up199502480@fc.up.pt. received: 2013-12-31; accepted: 2014-08-11 abstract the episode of the history of science (hos) on the theory of continental drift proposed by alfred wegener has been considered an excellent example for teaching students aspects of nature of science (nos) and the relation of science with social and technological contexts. we implemented a nos and hos-enriched problem-based learning environment at the middle (year 7 of the portuguese national curriculum) and secondary level (year 10) for teaching the origins of biodiversity and of continents and oceans (mobilism). the main objective of this study was to provide implementation practices to address the lack of guidelines of how to use problem-based learning (pbl) in classrooms for 12 to 16 year old students. this constitutes the first practical example of implementation using wegener’s episode of hos for teaching geological mobilism integrated with evolution. therefore, in this study, we provide specific suggestions for supporting teachers’ classroom efforts in implementing pbl, such as the appropriate scaffolding. the ill-problem presented to students was based on the phylogeny of extant and extinct ratite birds, described by charles darwin and the present geographical distribution. the evaluation of the students was focused on the chain of reasoning employed, and we performed a comparison analysis of the problem’s solution presented by the students of both classes regarding the explanation of the phylogeny of ratites based on geological mobilism. we observed an overall improvement (25-77%) of the percentages of students pre and post-instruction adequate answers; therefore our pbl strategy was efficient. keywords problem-based learning; history of science; nature of science; scaffolding; geological mobilism; biodiversity. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 142 mailto:up199502480@fc.up.pt multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 1. introduction problem-based learning (pbl) is an instructional and curricular student-centered approach that requires that students apply knowledge and skills to propose a viable solution to a given problem (savery 2006). while pbl can come in a variety of forms, it is generally characterized by using problems as triggers of learning and is usually associated to educational aids that support learning, including: the collaboration of the students in small groups, the metacognitive guidance of a tutor and the self-directed learning activities of the students (sousa 2007). according to ertmer and simons (2006), pbl has a long history of successful use at college level, mainly in medical and preprofessional students, but has yet to be widely adopted by k–12 teachers, due to several challenges and constraints, such as, in case of portuguese schools, time available and mandatory number of students in the classroom. the understanding of scientific epistemology is beneficial and encourages public engagement since knowledge about how science is done is useful because citizens need to make systematic, rational decisions about scientific projects, either about their ethical implications or about the public funding to be allocated for scientific projects, for example by voting in referendum. so, scientifically literate students should exit formal schooling (at the end of year 12 of the portuguese national curriculum) with knowledge about science core ideas and aspects of nature of science (nos); this expression nature of science typically refers to the epistemology of science and to the science as a way of knowing (abd-el-khalick and lederman 2000). during part of an academic year we implemented a history of science (hos) and nos enriched problem-based learning environment at the middle (year 7 of the portuguese national curriculum) and secondary level (year 10) for teaching the origins of biodiversity and of continents and oceans (geological mobilism). at the level of middle school the discipline in the portuguese curricula that includes biology and geology sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 143 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 contents is named natural sciences, and according to the curriculum guidelines it is divided into four organizational themes: “earth in space”, “earth in transformation”, “sustainability in the earth” and “living better on earth”. the study presented in this article focuses on the theme “earth in transformation”. within the curricular organization, corresponding to the level of middle school, there are two main guiding texts, such as the essential competences (deb 2001) and the curriculum guidelines (deb 2002), that defines the competences to be developed across the various disciplines and defines the specific competences of each discipline, respectively. according to ferreira and morais (2013) the absence of guidance, in these texts, has led teachers to merely teaching science topics and the relation between science and technology. therefore, in this study, we provide specific suggestions for pbl implementation and supporting teachers’ classroom efforts in scaffolding student learning including aspects of nos. the ill-problem that we presented to students, without a single correct answer, was based on episodes of hos, including charles darwin’s work, mainly evolution mechanisms that explain the biodiversity, in his book the origin of species: by means of natural selection, and phylogeny of some ratite birds, as well as alfred wegener’s continental drift theory, in his book die entstehung der kontinente und ozeane (the origin of continents and oceans), since both episodes have been considered excellent independent examples for teaching students central aspects of nos (jacoby 2012; lederman et al. 2002). khishfe and lederman (2007) have considered that the aspects of nos might be taught successfully if integrated in real-life controversial science issues, therefore we decided to choose the historical controversies, due to the socio-scientific context, around the acceptance of the idea of geological mobilism and of evolution, which both illustrate several aspects of nos. furthermore, this study constitutes the first practical example of implementation using this episode of hos for teaching geological mobilism integrated with biodiversity and evolution. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 144 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 we used inquiry activities also known as hands-on and minds-on activities and promoted the discussion of aspects of nos by the students that is a partial match of the discussions occurred throughout history, raising their dissatisfaction with some of their own misconceptions on aspects of nos. this is the first step for conceptual change and it motivates students since it constitutes a challenge for further learning. several studies showed that middle school students in optional-scaffold conditions performed significantly better than students in a no-scaffold condition (kirschner et al. 2006; schmidt et al. 2011), so, in this study, we used hard scaffolds for the seventh graders, and in both classes, the author (in the role of pbl’s tutor) used challenging and motivation questions soft scaffolds to generate cognitive conflicts and to guide students to establish linkages and relationships between concepts. pbl implementation in classrooms, of 12 to 14 year old students, is still relatively uncommon because of many challenges and barriers (liu et al. 2012), however, we propose a modified pbl strategy that includes adequate scaffolding for each grade level in order to improve learning outcomes. we developed hard scaffolds, for seventh graders, based on expected learner difficulties associated with the task, such as: an inquiry activity based on a puzzle activity adapted from usgs (2008) that includes wegener´s arguments, and an exploration worksheet for google earth (tectonics), including questions that the students should go through, and that constitute intermediate results, of the problem-solving learning process on the type of plate limits associated with each geological phenomenon. some metacognitive tools were also used for both grade levels and can be considered learning scaffolds, such as: concept maps and heuristic v diagram. recent issues related with these episodes of hos were also included either in the problem or as a learning outcome, such as, the phylogeny of extant and extinct ratite birds and their present geographical distribution (lieberman 2005) and the tectonics theory that integrates the continental drift theory. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 145 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 in summary, the purpose of this educational action research was to propose an effective teaching-learning strategy that improves understanding of concepts, and their relationships, as well as students’ views of nature of science, using episodes of history of science. we tested whether there was an improvement on students’ views of nature of science and selected scientific topics by a problem-based learning that highlights two relevant episodes of the history of science. 2. methodology this study is the result of a one-academic year educational action research project (during a 7 months period) and constitutes an exploratory study containing a qualitative analysis of results. 2.1. participants study participants were students from the same public school, at a portuguese public school (oporto city, portugal), and either of the seventh grade class (middle school, with age from 11 to 14 years, and 12 years as median age) or the tenth grade class (high school, with age from 14 to 16 years, and 15 years as median age) that willing to participate in the study had the corresponding informed consent signed by the parent/person responsible for education. the author participated in the instruction by teaching the corresponding curricular units to all the students of both classes. 2.2. educational interventions our teaching strategy was mainly student-centered, and we used a pbl methodology that included an ill-structured problem, as well as appropriate learning scaffold activities for each level. and for the tenth grade students, we also included lectures, by scientists sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 146 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 (principal investigators) with relevant expertise regarding each of the focus themes (geodynamics, paleontology and biogeography), that included questioning and discussion (60min total, each). all the themes are included in curricular units of geology (continental drift theory and its role on in the development of tectonics theory) and biology (biodiversity and evolutionary history of birds’ species). the aspects of nos were distributed throughout the units and taught as embedded within the curricular content by promoting discussion on these aspects, in small groups and with all students in class. we summarize the characterization of our learning environment for 7th grade level and for 10th grade level, that may constitute future guidelines for other teachers (table 1). sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 147 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 table 1. overview of pbl’s learning environment for each science topic of 7th and 10th grade level. discipline science topic pbl’s learning environment natural sciences (7th grade level) continental drift theory plate tectonic theory biodiversity and species’ geographic distribution nos integrated-pbl in the lesson about hos: alternative explanations to continental drift theory (before 1912 and 1912-1960), wegener’s evidences and inferences, social context and oppositors to the theory, technological advances and theory acceptance. problem-solving presented in an exploration worksheet for google earth (tectonics) on the type of plate limits associated with each geological phenomenon. nos integrated-pbl in the lesson about the historic episode of darwin including explanation of geographic distribution of ostrich, rheas and their common ancestral (in gondwana). biology and geology (10th grade level) continental drift theory plate tectonic theory biodiversity and phylogeny nos integrated-pbl on the historic episode of wegener: alternative explanations to continental drift theory (before 1912 and 1912-1960), wegener’s evidences and inferences, social context and those contrary to the theory, pbl lesson on technological advances and integration of continental drift theory in plate tectonic theory and subsequent lectures by experts, problem-solving presented in an exploration worksheet for google earth (tectonics) on the type of plate limits associated with each geological phenomenon and predictable future geological events for each geographic area. nos integrated-pbl in the lesson about the historic episode about darwin including description of geographic distribution and phylogeny of extinct and extant ratites species and common ancestral of ratites (ostrich, rheas and moa); and the role of geological mobilism on these. we started with a small problem a pbl subunit that included the main question “of all the arguments presented that support his theory, which one do you think is the most convincing argument? why?”, since the students were unfamiliar with pbl. in the seventh grader level we provided, as handouts, the description of the arguments that one considered relevant for their age/level: morphological, paleontological, paleoclimatic and lithological arguments. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 148 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 the handouts containing the description of the arguments constituted a hard scaffold that facilitated the research of relevant information by each student. for the tenth graders we provided, to each small-group of students, a handout containing additional, and more complex, arguments (geophysical and stratigraphical), and they were asked to propose an answer to the same problem (“of all the arguments presented that support his theory, which one do you think is the most convincing argument? why?”). on this first learning phase, we also proposed to the seventh grade students a concept map to be completed by each small-group, since the students were unfamiliar with this tool, based on the concepts about the continental drift theory, by wegener in 1912, that proposes the existence of horizontal movement of continents responsible for the supercontinent pangea fragmentation from 300ma ago, and in his theory supported by scientific arguments. then, we proposed to the seventh graders, an inquiry activity, based on a puzzle activity adapted from usgs (2008), that includes four types of wegener´s arguments and requires the use of all arguments to be successfully completed (sousa 2013). subsequently each small-group explained to the author/tutor their proposed solution for the activity and to the question proposed before. this activity allowed students to understand the importance of considering all the evidences obtained by different sciences. it was also provided an hard scaffold to all students an exploration worksheet for google earth (tectonics) that includes questions that the students should go through, and that constitute intermediate results of the problem-solving learning process on the type of plate limits associated with each geological phenomenon. biodiversity and non-flying birds (ratites) species’ geographic distribution themes were introduced integrated with the theme geological mobilism and in a history of science context, with the visualization and tutor-promoted discussion of selected parts of the film “darwin’s lost voyage”, produced in 2008 by national geographic, containing parts of darwin’s book. nos integrated-pbl in the lesson about the historic episode of darwin included a simple phylogenetic tree regarding the ratites group: the focus of seventh sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 149 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 graders was oestrich and rheas, while the tenth graders studied the phylogeny of seven extinct and extant ratites species. students were asked to characterize the geographic distribution of their common ancestral and explain the geographic distribution of the other species based on their knowledges on geological mobilism. 2.3 instruments and data collection the questionnaire for seventh grade students consisted of open-ended and multiple choices items, and the questionnaire for tenth grade students consisted of open-ended items. these questionnaires were composed of several types of questions: questions about single concepts, questions about the relationships between concepts and questions that assess alternative conceptions and prerequisite knowledge. the items were developed and, next, a panel of experts examined these items to establish their validity (one university professor and a science educator), and then the items were modified according to the panel's comments and suggestions for improvement. data was collected using pre and post-test students’ responses to the questionnaire (prior and by the end of the unit, respectively). 3. results the most common proposed answer to the small problem “of all the arguments presented that support his theory, which one do you think is the most convincing argument? why?” was different in each class, that is, the most frequent answer by tenth graders was paleontologic arguments (4 out of the total 6 groups), while morphological arguments was the most frequent answer by seventh graders (4 out of the total 7 groups). sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 150 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 the overall improvement of the seventh grade student’s views of nos and knowledge of the science topics was evaluated by their responses to each aspect, corresponding to an item of the questionnaire (fig. 1). figure 1. comparison of pre and post-test results of middle-school students: pre-instruction and postinstruction adequate answers of seventh grade students for each question (as percentage). note: n=16; adequate answers consists of either complete answer (comp.) or correct answer (cor.). the overall interval of gain in the percentages of adequate answers (including complete and correct answers) is 25% to 58%. all the groups were able to explain and justify their answer to the other students in the classroom, and by the end of this small pbl unit all the students understood the importance of all the types of arguments for wegener’s theory, as well as several consensus aspects of nos, such as: empirical, creative and inferential. we observed an sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 151 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 increase of percentage of adequate answers to items regarding aspects of nos between 55% and 58% (fig. 1). the final product asked to the students, by the end of the instruction, was a power point presentation and a written mini-essay, elaborated in small-groups, on “what is the role of geological phenomena in the evolution and geographic distribution of species through earth’s history?”, that was successful for all the groups evaluated with good or very good. we observed an overall improvement of the responses by the participants (tenth graders) for each aspect, corresponding to an item of the questionnaire (fig. 2). figure 2. comparison of pre and post-test results of high-school students: pre-instruction and postinstruction adequate answers of tenth grade students for each question (as percentage). note: n=20; adequate answers consists of either complete answer (comp.) or correct answer (cor.). sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 152 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 at the end of the unit, 77% of the high-school students (fig. 2) demonstrated an informed understanding of the importance of observations and inferences in science, since wegener’s observations of the contours of the south-american and african continents contributed to the inference of the existence of a supercontinent, 300ma years ago, he named pangea, that also shows the importance of creativity. the students’ views on the subjective and dynamic nature of scientific knowledge at the end of the study were improved, since we observed an increase of 35% of high-school students (fig. 2) with adequate answer/view, corresponding to an adequate understanding of the role of the technological and social contexts in the acceptance of the theories at any time point. the understanding of central concepts on the new theory that explains the origins of continents and oceans plate tectonic theory was assessed by several items in the questionnaire, such as holmes’s contribute (corresponding to 35% increase of correct answers) and the type of lithospheric plate´s limits found at regions of origin of a new ocean (corresponding to 30% increase of correct answers). the scaffolding used was adequate to each grade, for example we organized talks (minilectures) by experts for the tenth grade, and this activity was one of the activities that the students considered that have a higher contribute on their own learning. 4. discussion and conclusions in this article, we describe some specific guidelines for implementation of pbl in k-12 contexts integrating nos and hos aspects into biology and geology classes. we showed an improvement on students’ views of nature of science and selected scientific topics by a problem-based learning that highlights two relevant episodes of the history of science. in accordance, other authors have described the efficacy of inquiry activities, on sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 153 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 different themes, with aspects of nos embedded within the content (khishfe and lederman 2007). our pbl strategy was guided by hard scaffolds, that were provided as handouts, to the students, during various stages of the learning process that helped students perform a variety of inquiry-related tasks, such as more efficient search for relevant information. we also used soft scaffolding such as some questioning during the learning process by the teacher. the strategy described here was effective as shown in the results obtained in post-tests, however, future research should shed further light on the role of scaffolding to support student performance in pbl. results showed that the context of celebrating the centenary of the first public presentation by wegener of his theory may constitute an excellent learning environment as previously described (san román, 2012). we consider that in order to be successful in the implementation of such a novel strategy it is important to start with small problems, mini-pbl subunits, to initiate and engage students, unfamiliar with pbl, as according to other studies (ertmer and simons 2006). our pbl strategy allowed students to experience multiple opportunities to establish the links between epistemological ideas and biological and geological concepts due to the integration of different views of aspects of nos within the content from the curriculum. therefore, the expected efficacy was observed as an increase of percentage of adequate answers to items regarding aspects of nos between 55-58% (seventh graders) and 3577% (tenth graders). some consensus aspects of nos (abd-el-khalick 2012) were the focus of this pbl strategy, such as: scientific theories as systems of explanation which includes results obtained in several fields of investigation; and the characterization of scientific knowledge as empirical, tentative, creative and inferential. students were able to overcame some of their misconceptions, about oceanic expansion and maintenance of earth’s dimensions, since we observed an increase of 56% (seventh graders) and 50% sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 154 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 (tenth graders) in the percentage of correct answers to the corresponding item (figures 1 and 2). by the end of the unit the students had a good idea of what is known about the relation of the common ancestral of some species of ratites, mainly ostrich, rheas (rhea pennata and rhea americana) and moa and geological mobilism as it is hypothesized that it lived throughtout gondwana 100 to 120 millions years ago (100-200ma) according to lieberman (2005); however the phylogeny of their family of species is still quite controversal (haddrath and baker 2012). our pbl strategy allowed the improvement of attitudes, as described by other authors (ferreira and trudel 2012), such as collaboration and may, also, have contributed to the development of friendships among some students that we observed. some limitations of this study are recognized, that are mainly due to the number of participants and the specific context within the research project was conducted, however our results are consistent with other studies and allowed the author to contribute with suggestions to the new national curriculum guidelines, some of which were included in the final document that will be mandatory for the forthcoming academic years. we will pursue the research with other grades so we can generalize the results to wider contexts. we recognize the necessity for the development of national and international collaborative structures and actions to support and encourage k-12 teachers in using a pbl approach successfully, since there is a lack of high quality instructional materials and courses for teachers, and an insufficient link between hos and nos and science topics in national curricula (ferreira and morais 2013) and in textbooks (höttecke and silva 2011). hence, we hope this article will constitute a background that enables other teachers to take their first steps into pbl. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 155 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 5. acknowledgments the author is grateful to the faculty administration for encouraging this study and for providing funds. cristina sousa, is the author, and conceived, designed and performed the study, analyzed the data and wrote the paper. the author is grateful for the suggestions made to the questionnaire by dr. clara vasconcelos, phd (cgup & fcup) and prof. lucinda motta (escola secundária aurélia de sousa). the author would like to thank to professor dr. fernando noronha, phd (fcup & cgup), professor dr. helena couto, phd (fcup & cgup) and professor dr. joão honrado, phd (fcup & cibio), for their contribution with lectures in the context of the celebration of the centenary of wegener’s talk within the conference cycle “centenary of novel ideas on the origin of continents and oceans, of alfred wegener”, organized by the author. this study was made possible by support from the staff at the escola secundária aurélia de sousa (porto, portugal). the importance of the participants in this project cannot be overemphasized, i would like to extend a very big thank you to all the students who took part in the process, since their time and insights were critical, invaluable, and essential to this project. 6. references abd-el-khalick, f. and lederman, n. g. (2000). the influence of history of science courses on students' views of nature of science. journal of research in science teaching 37(10),1057-1095. abd-el-khalick, f. (2012). examining the sources for our understandings about science: enduring conflations and critical issues in research on nature of science in science education, international journal of science education, 34(3), 353-374. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 156 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 deb departamento de educação básica. (2001). currículo nacional do ensino básico competências essenciais [national curriculum for compulsory education: essential competences]. lisbon: ministry of education. deb departamento de educação básica. (2002). orientações curriculares para o 3º ciclo do ensino básico [curriculum guidelines for middle school]. lisbon: ministry of education. ertmer, p. a. and simons, k. d. (2006). jumping the pbl implementation hurdle: supporting the efforts of k–12 teachers. interdisciplinary journal of problem-based learning, 1(1), 40-54. ferreira, m. m. and trudel, a. r. (2012). the impact of problem based learning (pbl) on student attitudes toward science, problem-solving skills, and sense of community in the classroom. journal of classroom interaction, 47(1), 23-30 ferreira, s. and morais, a. m. (2013). the nature of science in science curricula: methods and concepts of analysis. international journal of science education, 35(16), 2670–2691. jacoby, w. r. (2012). alfred wegener 100 years of mobilism. geoscientist, 22(9), 1217. haddrath, o. and baker, a. j. (2012). multiple nuclear genes and retroposons support vicariance and dispersal of the palaeognaths, and an early cretaceous origin of modern birds. proc. r. soc. b 279, 4617–4625. höttecke, d. and silva, c. c. (2011). why implementing history and philosophy in school science education is a challenge: an analysis of obstacles. sci & educ, 20, 293–316. khishfe, r. and lederman, n. g.. (2007). relationship between instructional context and understandings of nature of science. international journal of science education, 29(8), 939-961. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 157 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 kirschner, p. a., sweller, j. and clarck, r. e. (2006). why minimal guidance instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experiential and inquiry-based teaching. educational psychologist, 41(2), 75-86. lederman, n. g., abd-el-khalick, f., bell, r. l., schwartz, rene´e s. (2002). views of nature of science questionnaire: toward valid and meaningful assessment of learners’ conceptions of nature of science. journal of research in science teaching, 39(6), 497– 521. lieberman, b. s. (2005). geobiology and paleobiogeography: tracking the coevolution of the earth and its biota. palaeogeography, palaeoclimatology, palaeoecology, 219 (1-2), 23-33. liu, m. , wivagg, j. , geurtz, r. , lee, s. , and chang, h. m. (2012). examining how middle school science teachers implement a multimedia-enriched problem-based learning environment. interdisciplinary journal of problem-based learning, 6(2), 46-84. mamlok-naaman, r., ben-zvi, r., hofstein, a., menis, j., and erduran, s. (2005). learning science through a historical approach: does it affect the attitudes of nonscience-oriented students towards science?. international journal of science & math education, 3(3), 485-507. san román, l. s. (2012). aniversarios en ciencias: algunas orientaciones para su uso didáctico [anniversaries in sciences: some orientations for teaching use]. enseñanza de las ciencias de la tierra, 20.1, 96-104. savery, j. r. (2006). overview of problem-based learning: definitions and distinctions. the interdisciplinary journal of problem-based learning, 1(1), 9-20. schmidt, h. g., rotgans, j. i. and yew, e. h. j. (2011). the process of problem-based learning: what works and why. medical education, 45, 792–806. sousa, c. (2007). abordagem por resolução de problemas em aulas práticas de disciplinas na área da biologia: pbl e resolução de problemas. [using problem solving in sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 158 http://www.sciencedirect.com/science/article/pii/s0031018204005887 http://www.sciencedirect.com/science/article/pii/s0031018204005887 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2249 social and technological sciences eissn: 2341-2593 practical classes of disciplines in the area of biology: pbl and problem solving]. in: i. cardoso, e. martins, z. paiva (eds.). actas do colóquio da investigação à prática: interacções e debates, e-book (isbn: 978-972-789-253-2). ddte e cidtff da universidade de aveiro, aveiro. 244 253. sousa, c. (2013). pensar a pangeia como wegener. kit educacional mobilismo geológico, volume 1 atividades práticas para o 7º ano de escolaridade. [think pangea as wegener. educational kit gelogical mobilism, volume 1 practical activities for senventh grade level]. isbn: 978-989-97682-2-2. (e-book). casa das ciências. http://imagem.casadasciencias.org/online/39116006/39116006.php. usgs (2008). wegener´s puzzling continental drift evidence in this dynamic planet: a teaching companion. resource document. http://volcanoes.usgs.gov/about/edu/dynamicplanet. sousa (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 142-159 | 159 http://www.casadasciencias.org/ http://volcanoes.usgs.gov/about/edu/dynamicplanet pbl’s learning environment science topic discipline multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 26 sketching as a communication tool and a vehicle for exploring new ideas in higher education of industrial design. implementation and analysis of new methodologies josé luis navarro francisco felip miralles universitat jaume i, departamento de ingeniería de sistemas industriales y diseño, email: jnavarro@uji.es universitat jaume i, av. de vicent sos baynat, s/n. 12071 castellón de la plana, email: ffelip@uji.es received: 2017-03-30; accepted: 2017-08-11 abstract freehand drawing is a basic tool to explore shapes and visualize ideas during the first phase of projective process in industrial design. sketches, expressive and quickly executed, allow designers to highlight the most interesting aspects of the product to be designed and help to define its shape and general features. due to the immediacy that characterizes the manual drafting industrial designers in training learn to become able to translate any ideas graphically and thus able to communicate with other designers, initiating a dialogue on the product to be designed. in the bachelor's degree in industrial design and product development engineering at the jaume i university of castellón (spain), the subject 'artistic expression ii' maintains and strengthens the knowledge gained in other subjects about the different graphic techniques, making drawing a means of expression for represent, synthesize and define the shape of objects. however, it has been found that the subject procedures were not adequate to achieve the learning results proposed, so it was necessary to apply a new approach. this paper describes the implementation of a new methodological strategy on this subject, adapting it to the european higher education area, with the aim that students in multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 27 industrial design acquire the four core competences of this matter. we describe the activities designed to achieve each of the competences and the implemented methodology for each of these activities. finally, the results are analyzed and the overall positive impact of the measures taken is discussed. keywords product design, drawing, sketching, creative process 1. introduction 1.1 the role of drawing in design drawing as a projective tool for shaping new ideas is indispensable in the academic training of industrial designer. hand drawing favors the development of creativity in the field of design, while allowing communicating the first ideas about new concepts. to practice the basic components of hand drawing, as the line, composition, color, perspective and chiaroscuro, is necessary for teaching disciplines such as architecture and the design of spaces (travis 2014), but can also be relevant for industrial design, since they share many of their creative approaches. some studies have also defended the hand drawing and model making as activities closely related to the design process, to help familiarize the user with the geometry of objects, that is, to understand the space around us and forms contained therein (carnevale 2015). lasseau (2001) highlights the need to develop graphical manual skills to practice graphic thinking and enhance this way the acquisition of perceptual skills, emphasizing its importance for architects and designers as a mechanism of representation, conceptualization and abstraction for communication between the creator and his audience. design drawings, as a means to graphic thinking, therefore have a prominent role in any process of design work (herbert 1988). drawing requires learning certain skills, but is merely a rational part of the design process, which is more complex and requires a creative mind. it is possible to draw multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 28 without designing, but not vice versa. that is why teaching drawing in the field of industrial design is so necessary. 1.2 'artistic expression ii' subject: description and detection of shortcomings this subject is taught from september to december in the second year of the bachelor's degree in industrial design and product development engineering, maintaining and strengthening the knowledge acquired in the different graphic techniques introduced in 'artistic expression i', by making drawing a means of expression to represent, synthesize and define the shape of objects. the skills to be acquired by students in this subject are important because they will use them throughout the degree in many other subjects focused on conceptualizing new products, such as 'model making workshop, 'conceptual design', 'product presentation' or 'product design prototyping workshop', where it is necessary to know how to represent graphically the ideas and present them in public, in order to receive feedback and improve them before building physical models or prototypes. in this subject students work on issues such as the composition and shape analysis (shape psychology and compositional principles), light, volume and formal expressiveness (point, line, stroke, spot color, chiaroscuro and techniques of artistic expression and presentation), color (theory, perception, semantics and symbolism of the colors and texture of the industrial product), descriptive illustration (the human figure and its relation to objects, proportion, stylization and synthesis) and architectural space (intuitive representation of perspective). the methodology used has always been eminently practical, but the approach conducted so far has not been entirely satisfactory, since it has been found that students fail to acquire the necessary skills and do not apply them correctly in later courses. among the shortcomings that have been detected in students from other years after completing this subject, we found a lack of confidence to represent new ideas. they find it very difficult to draw what they can’t see, and they don’t know how to represent it. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 29 normally students often prefer verbal explanations to drawing when presenting their new products and ideas. in addition, when drawing these new products they always use twodimensional representations (front or side views), although in reality these products would have three dimensions and they will be understood more quickly if they were represented in perspective. the explanation of these shortcomings is found in the methodology carried out so far on the subject, too focused on copying and tracing photographs of real objects to interpret them graphically later, instead of drawing physical objects after manipulating them or watching them directly. to learn how to graphically represent and interpret the geometry of the objects around us is necessary to observe our environment carefully and become familiar with spatial depth that defines it, which has direct influence on our perception and understanding of space. thus, once the industrial designer learns how to represent on paper a volume that sees in real space, it may be easier for him or her to represent other volumes not seen. additionally, to improve manual sketching skills it is also required daily, continuous and intensive practice during the entire course, using hand-drawing in each design project developed by students (dong et al. 2013). 2. methodology 2.1 analyzing the students in order to learn more about the students of this subject an anonymous and voluntary survey was conducted early in the course. the intention was to find a number of factors that could influence the development of learning during the course, and thus better align the contents and schedule of activities of the subject. this survey asked the students: 1) what was their previous training in high school: technical, artistic or humanities. 2) the perception of the importance of the subject 'artistic expression ii' for their training in the grade (evaluating it on a 10 point grading scale). multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 30 3) what was their skill level in subjects related to artistic or graphic expression before starting the bachelor’s degree (evaluating it on a 10 point grading scale). 4) what was their skill level in subjects related to artistic expression before starting the subject 'artistic expression ii' (evaluating it on a 10 point grading scale). 5) rate from 0 to 10 points the interest or love for drawing and the use of techniques of artistic expression in their daily lives. 6) to point out with which of this five branches related to design feels more affinity, from highest to lowest technical nature: a) engineering and industrial technology, b) engineering and product design, c) product design and graphic design, d) design and visual arts, and e) fine arts. the survey was answered by 80 students (80% of students enrolled). the first question allowed us to see that 70 students came from technical high school, 6 came from art school and 4 came from humanities high school. this data corroborated the affinity of the previous formation of most students with the technical character of the degree in industrial design that they are studying. regarding the second question, the average perception of the importance of the subject proved to be 8.56 points out of 10, detecting 4 points as the lowest value (one student) and 10 points as highest value (31 students). in the third and fourth questions the students reported having an average skill of 4.73 points out of 10 in artistic expression matters before the bachelor’s degree, and 6.04 points right at the beginning of this subject, which indicates that the perception of their skills improved during the first year of the degree. the fifth question allowed us to find out that their average interest in drawing outside academic environment was 7.73 points out of 10, indicating a willingness of students towards the use of tools of artistic expression in their daily lives. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 31 the answers to the last question indicated a dominant trend towards engineering and product design, followed by product design and graphic design, engineering and industrial technology, design and the visual arts and finally fine arts (see figure 1). figure 1. affinity of students in 'artistic expression ii’ to 5 design related branches: a) engineering and industrial technology, b) engineering and product design, c) product design and graphic design, d) design and visual arts, and e) fine arts. overall, the initial survey allowed to prove that although most students had a solid technical formation prior to university and a natural tendency to the most technical fields of design, they considered very important artistic training for the exercise of the designer profession. although they claimed to have poor skills in artistic expression, a predisposition to keep improving is observed; therefore it is possible to achieve an improvement in their skills after the course by adapting exercises to their initial abilities. 2.2 preparing exercises to achieve the skills of the subject multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 32 the skills to achieve through this subject, previously defined in the curriculum of bachelor’s degree, were as follows:  ce23 creativity and innovation in the field of design  ce24 being able to communicate effectively orally and in writing, written reports and all the written and graphic documentation necessary to communicate ideas and projects.  ce5 to handle techniques and tools for artistic expression and representation.  cg5 motivation for quality to ensure that students could finish the subject acquiring the abilities necessary to develop the planned skills, a series of exercises were designed (see table 1) and were distributed in 12 categories:  category 1: analysis and interpretation of geometric volumes (part 1). through 7 conceptual experimentation exercises (1a-1g) students attempt to reactivate the knowledge acquired in the previous course. such practices also serve to disinhibit the capacity to respond to new artistic expression problems that arise.  category 2: analysis and interpretation of geometric volumes (part 2). through various exercises students practice tactile perception of volume or shape that is imaged with their hands (exercises 2a and 2b) practicing both geometric and organic shapes. another exercise (2c) uses an object of cylindrical shapes, and the student has to imagine and draw a nonexistent membrane to give a sense of continuity between different volumes, as if it were a slightly stretched skin over a structure. through these topological surfaces students discover that things are not just as they look, but they can be the internal structure of an organic form. exercises 2d and 2e are interrelated, and are to capture the essential shape and structure of a moving object, to achieve a mental synthesis of the structure and basic volume of the object (scale and proportion), but eliminating curves and multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 33 converting the object into a complex polyhedron. in the next phase (exercise 2e) students had to draw the same figure but with the material texture of the real object: the glass as glass, liquid as liquid and metal as metal, since it was a simple transparent bottle of perfume.  category 3: analysis of everyday objects: seesaw toy. through 3 exercises (3a-3c) students had to try to understand the product observed from different points of view by representing and interpreting it with different graphic techniques.  category 4: analysis of everyday objects: taps. through 4 exercises (4a-4d) students had to become familiar with the formal identity of a very common household industrial product, learning its mode of operation from handling, direct observation of its volume and the representation of its parts.  category 5: analysis of everyday objects: electrical appliances (part 1). through 3 exercises (5a-5c) students had to familiarize themselves with the formal identity of various small electrical appliances, representing them from various points of view and using various graphic techniques.  category 6: analysis of everyday objects: electrical appliances (part 2). deepening understanding of domestic appliances by means of 3 exercises (6a-6c), practicing exploded views to become familiar with the formal and functional meaning of the components of an industrial product, while chromatic interpretation is used as a resource to represent the volume of a product without using perspective.  category 7: analysis of everyday objects: electrical appliances (part 3). by means of 4 exercises of handling and direct observation (7a-7d) is intended for students to understand that the volume of products is directly related to its ergonomic features, and learn to propose functional improvements and represent them by sketching. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 34  category 8: analysis of everyday objects: vehicle. 3 exercises (8a-8c) to practice the observation of large industrial products and its representation with different techniques from different points of view.  category 9: street / domestic furniture and architecture. through 3 exercises (9a9c) is intended for students to associate the products for habitat with their spatial context, representing both as a single whole. the concept of 'graphic animation' is introduced: to enhance the visual expressiveness of the drawings by textures or tracings of the students’ own unique style, avoiding representing the surfaces with excessive blurring or smoothing.  category 10: design a mascot for an event or for a company. through this exercise (10a) is intended that students learn to work together in an agile way sharing their ideas through sketches and working collaboratively.  category 11: resources for show ideas: presentations and digital compositions. exercise 11a familiarizes students with the use of digital photo retouching tools, learning to integrate several photos into a single one changing various parameters such as color, brightness, contrast and saturation. exercise 11b serves the students to practice their oral communication skills to objectively defend in public their own personal portfolio.  category 12: design project: design of a piece of furniture or lighting. through exercise 12a students practice the graphic composition of various elements in a bidimensional space, learning how to distribute them with the intention of creating a visual narrative discourse. table 1. distribution of exercises be performed, the competences to achieve and the methodology implemented for this purpose in each case. competences activities or exercises to achieve the implemented methodology multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 35 c e 2 3 c e 2 4 c e 5 c g 5 competences x 1a. interpretation of a mental image from a descriptive narration with geometric volumes. linear perspective drawing with 2b pencil on paper. din a4. x 1b. interpretation of exercise 1a changing the organization of the construction elements and keeping the plane of symmetry. linear perspective drawing with 2b pencil on paper. din a4. x 1c. formal reproduction through tactile experiences (several elements). linear perspective drawing with 2b pencil on paper. din a4. x 1d. interpreting cylindrical volumes of the proposed model using prismatic shapes linear perspective drawing with 2b pencil on paper. din a4. x 1e. conceptual interpretation of exercise 1d from a different point of view, without observing the analyzed model. linear perspective drawing with ballpen on paper. din a4. x x 1f. chiaroscuro drawing of a model, perspective view. 2b pencil on paper. din a4. x x 1g. pointillist chiaroscuro interpretation from exercise 1f. ballpen or fine tip marker on paper. blue or black color. din a4. x 2a. shape recognition by tactile experiences. linear perspective drawing with 2b pencil on paper. din a4. x 2b. formal reproduction through tactile experiences (several elements). linear perspective drawing with 2b pencil or ballpen, on paper. din a4. x 2c. interpret by membranes or topological surfaces the cylindrical volumes of the model analyzed in the exercise 1f. chiaroscuro drawing in perspective. free technique. din a4. x 2d. geometric synthesis drawing in perspective view of a moving object. ballpen on paper. din a4. x x 2e. free color interpretation of the 2d exercise, emphasizing the materiality of the object. crayons or markers on paper. din a4. x 3a. linear perspective drawing of the proposed model. 2b pencil on paper (2 different points of view). din a4. x x 3b. interpretation of a new point of view of the model by chiaroscuro of linear tracings fine tip marker pen, pen nib and ink, stylographs or other permanent means of black, brown or blue color, on paper. din a4. x x 3c. interpretation by spot colors of one of the points of view studied in exercise 3a or 3b. marker pens, colored inks or tempera paint on paper. din a4. x x 4a. sketching a tap considering its volume and not its materiality. linear perspective drawing with 2b pencil on paper (2 different points of view). din a4. x x 4b. monochromatic interpretation by line and spot of a new point of view of the tap or another marker pen, ink or watercolor pencil on paper. din a4. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 36 different model. x x x 4c. linear perspective drawing showing the operation of a tap by representing the hand in attitude of use. indication of elements and / or parts of the object by reference lines or arrows with explanatory text. hb pencil on paper, reinforcing lines using black marker (fine and medium thickness). din a4. x x 4d. linear perspective drawings of 2 different taps belonging to the same formal family (sink, toilet, bidet, kitchen). ballpen on paper. din a4. x x 5a. linear perspective drawing of one of the proposed models (hair dryer, drill, electric saw). blue ballpen on paper (2 points of view). din a4. x x 5b. chiaroscuro drawing in both color and perspective, showing the selected object in the exercise 5a from another point of view. crayons on paper. din a4. x x 5c. linear perspective drawing of a typical small household electrical appliance (blender, coffee maker, juicer, etc.). free technique on paper. din a4. x x x 6a. linear perspective drawing of exploded view of one of the proposed models (hair dryer, drill, electric saw). 2b pencil on paper. din a4. x x 6b. sketch of one of the models proposed in the exercise 6a, showing by a single orthogonal projection (elevation view or profile view) as a basic representation of a product. hb pencil on paper. din a4. x x 6c. chromatic interpretation of sketch 6b in order to generate volumetric sensation free technique on paper. din a4. x x x 7a. ergonomic study. linear perspective drawing of one of the proposed models (hair dryer, drill, electric saw) representing the hand in gripping or clamping position (two points of view). 2b pencil on paper. din a4. in one of the drawings the student must reinforce the object’s contour line using a black marker pen, while the hand drawing will remain in pencil. x x x 7b. redesign of one of the models proposed in practice 7a through chromatic chiaroscuro drawing in perspective. free technique on paper. din a4. x x 7c. line drawing of a domestic power tool, strengthening contour and main areas using thicker lines. fine, medium and/or thick tip marker pen. din a4 or din a3. x x 7d. color or monochrome interpretation of exercise 7c, in order to represent threedimensional appearance by spot colors. marker pens, inks, watercolors or watercolor pencils on paper. din a4 or din a3. x x 8a. line perspective drawing of a vehicle (car or motorcycle) blue ballpen on paper. din a4. x x 8b. chiaroscuro perspective drawing of the object (another point of view, different from exercise 8a). 2b pencil on paper. din a4. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 37 x x x 8c. drawing of complete redesign of a vehicle which may be based on exercise 8a or 8b, in chromatic chiaroscuro. free expression technique. din a4 or din a3. x x 9a. perspective drawing of a furniture element ballpen and/or fine or medium tip marker (one point of view). din a4. x x 9b. linear representation of the same furniture element (another point of view, different from exercise 9a), considering the architectural environment in which it is located. pencil on paper. din a4 or din a3. x x 9c. free interpretation of exercise 9b by graphic animation (color or monochrome). free technique. din a3. x x x x 10. design a mascot for an event or for a company (teamwork, three to four people). set of linear and expressive drawings representing the shape and nature of the character teamwork of three to four people. pencils, ballpens or fine tip markers. din a3. x x x 11a. integration of images in different pictures. digital image retouching. recommended software: adobe photoshop, gimp or similar. x x x 11b. individual presentation of personal portfolio (in 3 minutes). the presentation should show only a selection of the exercises done in the subject (the most successful). microsoft power point or open office impress. x x x x 12. furniture design project according to five different product alternatives. representation of the design through a visual panel that includes several previous concept sketches, the orthogonal projections with basic dimensions, elemental product specifications and a color rendering of the final design. din a3. 2.3 use of rubrics one of the weaknesses identified in previous years referred to the subjectivity with which the students addressed the exercises and their inability to see their own mistakes. in order to direct the student to the acquisition of objective criteria for assessing their own work impartially, some self-assessment rubrics were created to be applied to each group of exercises. each rubric specifies the various aspects to be evaluated and the criteria to evaluate each of them. the purpose of these rubrics was to provide students with the objective criteria of an industrial designer, with which detect effectively the own multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 38 shortcomings and learn to solve them. all exercises, in turn, were evaluated by teachers following the same criteria: fitting, proportion, presentation, formal expression and technique. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 39 figure 2. example of rubric used in the subject to assess the first group of exercises multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 40 3. results 3.1 observable results following the implementation of the exercises the implemented methodology has helped nearly all students to acquire the expected skills. the results of each group of exercises are described below:  the exercises in category 1 (analysis and interpretation of geometric volumes 1) got an average mark of 5.3 out of 10 points. it is observed that after a long period of drawing inactivity since they finished the subject 'artistic expression i' (4 months), analytical skills and agility to draw are depleted. the results, although most of them are not remarkable, do maintain a similar level, so that 28% of students do not reach the minimum mark of 5 points, but the lowest mark reach 4 points.  in category 2 (analysis and interpretation of geometric volumes 2) the same average mark of category 1 is obtained. 27% of students do not reach the minimum mark of 5 points, but the lowest mark reach 4 points. it is noted that students have trouble understanding the conceptual experimentation and expression proposals. students do not fully understand the need to interpret the visible forms instead of copying them as if they made a picture, or copy them directly from the photos. it is observed that most of them have fear of making mistakes, so they draw very slowly and carefully.  in category 3 (analysis of everyday objects: toy seesaw) an average mark of 6.0 points is obtained. 91% of students exceed 5 points, an improvement which is a slight recovery of the skills acquired in the previous semester with the subject 'artistic expression i'. we can observe greater confidence in the development of exercises compared to initiation exercises 1 and 2.  in category 4 (analysis of everyday objects: taps) an average mark of 6.1 points is obtained. 98% of students exceed 5 points, indicating the acquisition of a greater multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 41 ease when using the proposed techniques. there are still some issues that they need to learn in order to express themselves more freely, because they are accustomed to draw only what they see.  exercises in category 5 (analysis of everyday objects: electrical appliances 1) an average mark of 6.6 points is obtained. 98% of students exceed 5 points, although the average quality of the work is only acceptable. students have difficulties to simplify and reduce the amount of visual information of the objects observed.  for category 6 (analysis of everyday objects: electrical appliances 2) an average mark of 6.7 points is obtained. 98% of students exceed 5 points. it is noted a great improvement in the agility to draw and conceptualize the visible forms, showing more adequate strokes.  the exercises in category 7 (analysis of everyday objects: electrical appliances 3) got an average mark of 6.8 points. 100% of students exceed 5 points. although the proposals in categories 5, 6 and 7 are varied and with different approaches, students seem tired of working with similar models and prefer to draw other objects.  in exercises in category 8 (analysis of everyday objects: vehicle) an average mark of 6.7 is obtained. 99% of students exceed 5 points. it is noted that students are not accustomed to draw large objects and at first they are not able to interpret the proportions from the point of view of the observer, drawing more conceptually and with great disproportions.  for category 9 (street / domestic furnishings and architecture) an average mark of 6.5 is obtained. 96% of students exceed 5 points. it is observed that changing the format is more uncomfortable and results are more irregular and of lower quality, in general. most students are not prepared to represent the interior space intuitively, being more used to drawing objects. however, the incursion on an multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 42 issue of this nature has allowed them to exercise in the synthesis of the elements represented, with a consequent improvement in their way of expressing space.  in the category 10 (design a mascot for an event or for a company) an average mark of 7.6 points is obtained. 100% of students exceed 5 points. the assessment of the work is done jointly (students evaluate each other) and we observe that they are quite critical in their assessments.  the exercises in category 11 (resources for show ideas: presentations and digital compositions) allowed to note that very few students handle well the image processing programs. except for a few students, most of them are motivated and strives to achieve a good result. it is also noted that most of the students feel uncomfortable in public oral presentation of their portfolio, probably because it is their first oral presentation in the bachelor’s degree.  finally, the last category (design project: design of a piece of furniture or lighting) got an average mark of 6.5 points out of 10, 98% exceeding 5 points. it is detected that most students have completed the exercise without consulting enough teachers to meet formal, technical and representation issues, probably due to excessive confidence in the abilities acquired and accumulation deliverables of other subjects, which it makes the students accelerate the completion of this exercise and neglect important aspects. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 43 figure 3. graphic sketches of a student of the subject (laura zas): conceptual representation of various industrial products. 3.2 reflection and self-evaluation after the course, students were asked anonymously to rate on a scale from 0 to 10 points the level of improvement they considered had reached during the subject in the analysis and expression of everyday objects by drawing, over the previous year. 80 students of 100 enrolled answered: 49 reported having improved their skills and 31 said they did not consider to have improved (see figure 4). figure 4. students’ perception on the improvement of their skills at the end of the course multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 44 in the survey the students were also asked to objectively assess the ability of expression they thought they had reached at the end of the course, assessing their overall skill, the visual quality of their work and the adequacy of strokes. the average mark of the students on their own skills was 7.21 points out of 10, which was very close to the actual average grade achieved in the course: 7.0 out of 10. this fact corroborated that they had acquired the ability to be critical of their own work and evaluate it according to objective criteria, learned by using rubrics during the course. 3.3 performance and overall results the subject was successfully passed in the first round by 93 of the 100 students enrolled. the average mark was 7.0 points out of 10, the lowest mark was 5.1 points and the highest 9.5 points (see figure 5). the attendance was very high throughout the course, which reveals a very high interest by students in this area, a finding consistent with the answers to the second question of the survey conducted at the beginning of the course. figure 5. normal distribution of marks. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 45 4. conclusions the methodology implemented in this subject has shown positive results. direct observation of many real industrial products, to become familiar with them through their manipulation and tactile experience have proven effective for students to understand their particular formal geometry, the first step in learning to represent them without copying, during the design process of similar products. using different drawing tools and techniques for quick hand representation has allowed the students to learn express themselves through various graphic languages, helping them learn to choose the one that best suits each time. the exercises implemented and their sequence during the course have allowed almost all students achieve the desired competencies satisfactorily. it is worth mentioning that students who did not pass the subject was because they did not come to class or did not perform any of the exercises. this confirms that the methodology implemented allows achieving the necessary skills to all students of the degree in design regularly attending classes, regardless of their previous education. the first observations carried out on the subject 'model making workshop', which is taught following 'artistic expression ii', verify that the students already draw with more ease than students from the last year, representing their ideas for new products quicker and with greater skill. however, closer observation is desirable to see if these skills are maintained or improved over time, and quantify the long-term impact that this methodology can have on other related subjects of the degree, in order to identify potential areas for improvement. 5. acknowledgment the methodologies conducted in this subject are part of the educational improvement project with reference 3049/15, which has received funding from the educational support multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6349 social and technological sciences issn: 2341-2593 navarro & felip (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 26-46 | 46 unit (u.s.e.) of the jaume i university, through the annual program 'aids for educational innovation projects at the jaume i university in the 2015/16 course'. 6. references carnevale, v. (2015) "to design" or "to draw": two different verbs, two different abilities, one result. the international journal of designed objects, 8(1), 17-25. https://doi.org/10.18848/2325-1379/cgp/v08i01/38695 dong, h., selami cifter, a. & fan, z. (2013) methods for improving undergraduate students' sketching skills. international journal of mechanical engineering education, 41(nº4), 329-336. https://doi.org/10.7227/ijmee.41.4.7 herbert, d.m. (1988). study drawings in architectural design: their properties as a graphic medium. journal of architectural education, 41(2), 26-38. https://doi.org/10.1080/10464883.1988.10758473 laseau, p. (2001). graphic thinking for architects and designers. new york: john wiley & sons. travis, s. (2014). sketching as a tool for seeing: a method for teaching drawing to architecture and interior design students. the international journal of design education, 7(3), 63-74. https://doi.org/10.18848/2325-128x/cgp/v07i03/38452 https://doi.org/10.18848/2325-1379/cgp/v08i01/38695 https://doi.org/10.7227/ijmee.41.4.7 https://doi.org/10.1080/10464883.1988.10758473 https://doi.org/10.18848/2325-128x/cgp/v07i03/38452 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 beliefs and tensions of kindergarten pre service students: a three case longitudinal study m. pérez-peitx*, m. fons-esteve dpt. de didàctica de la llengua i la literatura. universitat de barcelona, pg de la vall d’hebron 171, 08035, barcelona, spain. * corresponding author: email: mireia.perez@ub.edu; phone: + 34 934035077 received: 2013-12-29; accepted: 2014-04-15 abstract this present work is part of a four-year-project related to the pre service teachers’ beliefs’ about initial literacy from the university of barcelona. in particular, this paper presents the evolution of beliefs and tensions from the first to the second year of training from three pre service kindergarten students. the final aim of the project is to make proposals in order to improve teacher education. so far, it can be confirmed that beliefs are resistant to change but not immovable. some core beliefs have been identified and they have incorporated new information acquired during the second year of training, specifically, from the practice. furthermore, tensions and the awareness (or not) of them have become a very important issue since they show conflictive areas to work with students in order to make them conscious of their thoughts. in conclusion, beliefs and tensions appear to be the quid of the question in teacher education. keywords beliefs; tensions; pre service students; initial literacy; teacher training pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 63 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 1. introduction initial literacy is one of the most important learning to be achieved in a literate society. thus teachers’ education regarding this complex learning is extremely relevant because an important part of the knowledge takes place during school time. therefore, researchers should address their effort and interest to have a better understanding of what happens in those years. this was the starting point of the present paper, which is part of a larger longitudinal study. in particular, the questions guiding this paper are related with how do teacher thoughts’ evolve (if they do) during teacher education. lortie (1975), back in the seventies, made an accurate explanation of the teachers’ socialization, trying to identify different non-traditional contexts were teachers learn how to deal with the teaching profession. this would explain why some teacher practices are perpetuated even though there are new approaches and new ways of teaching taught in pre service education and continuing education. basically, the apprenticeship of observation makes emphasis on the extended period of time all the students are exposed to as a consequence of the formal schooling. thereby, all the pre service teachers arrive to teacher education programs with a large experience as observers that may filter the new information acquired in the degree. besides, in a long-term view, this would be one of the reasons that would explain the perpetuation of the educational system in general. chong, wong & queck (2005) pointed out that the main reason to explain this perpetuation of the educational system relies on the lack of belief exploration in teacher training programs. however, it cannot be forgotten one of the most important characteristics of the beliefs system, the resistance to change noted by pajares (1992) and more recently by peacock (2001). there can be found many studies working with teachers’ beliefs. studies with either pre service teachers and/or with in service teacher beliefs, as well as investigations that conclude that change in beliefs systems have occurred (or not). for the topic on the current paper, those most interesting studies are those who put the emphasis on wether pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 64 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 there has been change or not in the beliefs systems during teacher education programs. basically, those studies can be divided into two groups, those that confirm that after the course no changes are stated such as kagan (1992) or peacock (2001) and those that confirm belief change (nettle 1998; matheoudakis 2007). in relation to the same faculty teacher training from the university of barcelona, cambra (2008) conducted a research were concluded that the degree didn’t had enough impact on students’ beliefs. nonetheless, all the authors agree that teacher education programs should take students beliefs as the starting point (graden 1996; cambra 2000; borg 2003). in order to measure the impact of the course the model used is based on woods (1996) proposal of beliefs, assumptions and knowledge (bak). the model used in this paper is one of the several revisions made of the system. in particular, cambra’s (2000) proposal has ben used because it offers a wider vision of the second term, assumptions, including the french approach of representations (jodelet 1994). therefore, the system used is compounded by beliefs, representations and knowledge (brk). despite the use of this compounded system – with the presence of three different elements there will be no attempt to differentiate between what is belief, what is a representation or what is knowledge because they are intrinsically bounded and it is almost impossible to draw the borders between them. the brk system is used as a synonym of beliefs, but the preference to talk in terms of a compounded system is because it visualizes the complexity of the teachers’ thoughts. another extremely relevant issue when talking about pre service teachers and their bkr systems is tensions. traditionally, this term has been used to talk about the differences between beliefs and actions even though other authors suggested concepts such as incongruences (graden 1996) or mismatches (lee 2008). phipps & borg (2009) alert that these differences have been perceived as something negative and they suggest the use of the term tension, a more positive perspective. this term is based in freeman (1993: 488) study where he defined tension as “divergences among different forces or elements in the pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 65 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 teacher’s understanding of the school context, the subject matter or the students”. borg (2011) used this broader positive understanding of the term to talk about further differences than just thoughts and practice or beliefs and action. specifically, he used this term to talk about the tension between two perceptions; the perception of belief change and the real belief change experienced by one of the participants of his investigation. along these lines, pérez-peitx (2013) identified tensions between the actions and the beliefs but also in the same discourse. this new tensions showed areas where the participants kept different points of view depending on the moment of the interview. this broader vision of tension is the one being used in this study, because it makes emphasis on the potential contribution of tensions, like johnson defended (1994), and shows areas where teacher education programs can improve. therefore, the questions guiding this paper are: do teachers’ beliefs change between the first and the second year of teacher education in relation to learning to read and write? and also, are there any tensions in the students’ interviews? if the answer is yes, which are they? 2. data collection and methodology as it has been said, this paper is part of a four-year qualitative longitudinal study. the original study also works with pre service primary teachers beliefs and with other data than interviews, but for this paper some samples have been selected. for the designing of the project borg (2003) and richards (2009) demand for longitudinal qualitative studies was taken into account study, because it makes emphasis on the potential contribution of tensions, like johnson defended (1994), and shows areas where teacher education programs can improve. data is collected through interviews as suggested by barcelos (2003) as one of the best ways of obtaining qualitative data, concretely, with an oriented talk (palou & fons 2013). this type of talk allows the interviewer to raise new issues for discussion as they appear pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 66 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 as an interesting topic in the conversation and also gives space to the interviewees to make their contributions. all the interviews were collected and transcribed by one of the researchers. the length of the interviews depends on each subject but the first interviews lasted around 13 minutes while the second ones were about 30 minutes -with an exception of 75 minutes-. participants begun the degree on september 2011 and they are supposed to finish it on june 2015. the interviews were collected at the end of the first course (academic year 2011-2012) and at the end of the second (academic year 20122013). at the end of the second course (april and may of 2012), they coursed a practice subject, which consists of attending two schools (one month in each one) study, because it makes emphasis on the potential contribution of tensions, like johnson defended (1994), and shows areas where teacher education programs can improve. data is analyzed with an ethnographic-interpretative approach, always insisting in the holistic perspective that allows the researchers to infer beliefs and make emerge the tensions from the discourse. in order to achieve this goal, data is analyzed taking kerbratorechioni (2005) model of interactionist conversational analysis as the base. the original model includes three dimensions to analyze, the interlocutive, the thematic and the declarative. because of the nature of the data, an interview, the interlocutive dimension is not as relevant as the other two because it focuses on whom is talking or with whom are people interacting. whereas the other two dimensions (thematic and declarative) center the attention on what is being said and how is it being said. 3. participants all the subjects are studying the kindergarten teacher degree (grau de mestre en educació infantil) in the university of barcelona (barcelona, spain). • carolina is a 27 year-old student with a terrible initial literacy experience. her first interview is impregnated with these bad memories that she uses to explain her point of view. in her second interview, this experience is less obvious because pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 67 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 she makes continuous explicit reference to the “practice subject” she recently attended, and she uses this new available experience to justify her point of view. • sara is a 23 year-old student with the capacity to think in scenarios different from those experienced in her practice subject. she expects to change her point of view about initial literacy during the teacher-training program. • diana is a 24 year-old student. she believes in what they say to her, without reflecting if it is true or not. as a consequence of this feature she has propensity to have tensions in those issues she has not reflected. 4. results for each of the three students two tables will be presented. the first one reflects the evolution of some of the inferred beliefs (there are many for every case) and the second one refers to the tensions identified during interview analysis. more tensions are expected in the second data collection because the interviews were more extensive and the topics where more deep but also because the students are in the second year if the degree and they have more knowledge related to initial literacy. 1.1. caroline figure 1: comparison of beliefs inferred from the analysis of carolina’s interviews. beliefs inferred from the first interview beliefs inferred from the second interview 1. there is too much pressure on the school regarding the learning to read and write 1 1. there is too much pressure on the school regarding the learning to read and write the society puts pressure on the school, the school puts the pressure on teachers and those to the children the origin of the pressure is the fear of the adults the problem is between the rhythm the children needs and the one imposed 1 the formulation of beliefs may be bizarre but they are almost literally translated either from catalan or spanish in order to maintain the same meaning pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 68 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 2. in order to learn to read and write, time and space must be given 2. in order to learn to read and write, time and space must be given teachers must trust children when the children are not pressured they have more capacity part of the learning is leaving the children do all the process by their own 3. orthographic errors are not important in p4. 3. in p42, is more important the message than how it is written 4. for a good learning the child must show interest for initial literacy 4. for a good learning the child must show interest for initial literacy, and he will because we live in a literacy context 5. a multicultural and multilingual classroom means more work for the teacher 5. the teacher has the key of the success of a multicultural and multilingual classroom. it can be confirmed the existence of change in carolina’s beliefs, even though the core remains almost the same. two types of change can be differentiated. on one hand, there are beliefs that have incorporated new information to the previous existent core belief such as beliefs number 1, 2 and 4. in the first example the new information acquired is about the origin of the social pressure, what agents are implied and finally, carolina has identified a tension in the educational system, which would explain the inception of this social pressure. in the second example, beliefs are about the conditions of the learning (when she talks about teachers trust) or the consequence when she formulates the belief in a different way (how does this learning happen when there is no pressure). finally, she adds new information about the usefulness of letting time and space when she talks about the importance the children being the main actors of the process. on the other hand, there are beliefs that have been reformulated, for example number 3 or 5, maintaining part of the core belief, but with significance alterations. in the example number 3 the change of perspective is quite illustrative. the belief has moved from a specific situation (orthographic errors) to a wider conception of the learning process where the inferred belief is that the message is more important than the form. at last, example number 5 shows again a change of perspective from the specific anecdote (more work for the teacher) to the identification of the key of the success. 2 p4 means “parvulari” (kindergarten) “4 years” pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 69 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 figure 2: tensions identified in carolina’s interviews tensions from the first interview tensions from the second interview a. learning to read and write is a complex simultaneous process vs learning to read and write has a well established hierarchy process b. there is a tension in the educational system between children learning rhythms vs social pressure to learn to read and write c. the children have to make the whole process vs the teacher makes parts of the process d. the pedagogical advantages of group work vs use group work to have all the students under control in her first interview, cristina expressed a tension associated with the nature of the learning (a), whether it is more free or there is a particular order to be followed. in her second interview, inferred beliefs indicate that she is more supportive of the idea that learning to read and write is a complex and simultaneous process. so, even though it doesn’t appear in the figure 2 – because it is no longer a tension in her second interview-, the tension has evolved and has been solve, at least temporarily. also, other tensions appeared. in particular, carolina identified a tension in the educational system (b). although this tension is outside her belief system because is information related to a specific situation or reality-. also, during her interview some tensions between her beliefs and her practice have arose. in particular, those tensions are between her belief system, as said, and the account of her practice subject done in the interview. basically, they are narrated actions from the practice subject not matching the correspondent beliefs (c, d). 1.2. sara figure 3: comparison of beliefs inferred from the analysis of sara’s interviews. beliefs inferred from the first interview beliefs inferred from the second interview 1. knowing to read is more important than knowing to write 1. knowing to read is more important than knowing to write 2. the sooner the children know t read and write the better 2. the sooner the children know to read and write the better 3. teacher’s role is the most important thing in a multicultural classroom 3. multicultural classrooms are more difficult (compared with a non multicultural) because the teacher doesn’t pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 70 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 know all the languages 4. multiculturalism is a rich situation for the children 4. the languages that children bring to multilingual classrooms will have an impact on their future learning (possibility or not of making transferences) 5. 5. the system is the same as when she as a scholar in sara’s case, other type of belief change can be identified. on one side, two beliefs (number 1 and 2) have emerged the same way on both interviews, which indicates that they haven’t been altered at the moment of the second interview. on the other side, there is one belief (number 5) that has been inferred in the second interview but not in the first one. the other two beliefs (number 3 and 4) are reformulations referred to multiculturalism. they are specifically interesting because her practice experience was in a multicultural classroom. again, there has been a shift of perspective but just the opposite way than in carolina’s analysis. sara’s beliefs have shift from a general perspective of the situation -the importance of the teacher’s role in the classtowards a more detailed dimension of the reality –the added difficulty of not understanding all the languages that children bring to the classroom. so, it can be said that she places the she places the challenge of multicultural classrooms in the communicative competence. figure 4: tensions identified in sara’s interviews tensions from the first interview tensions from the second interview a. about a specficic activity: first she defines it in terms of boring and impossible activity and then she says she liked it and that it was a useful activity. b. what i expected to find in the classrooms vs what she really found in the classrooms during the first interview, sara didn’t express any idea that indicates any tension in her belief system. however, in her second interview, sara expresses two opposite opinions about a specific activity (a) done in the class when she was attending the practice subject and shows signs of being aware of the situation. in particular, she literally says “well, it pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 71 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 may seem not but” which includes this double opinion of the activity. more relevant is the tension between what she expected to find in this practice experience and what she really found (b). she is shocked because the system has not changed in more than fifteen years and she lives this lack of change as something negative. 1.3. diana figure 5: comparison of beliefs inferred from the analysis of diana’s interviews. beliefs inferred from the first interview beliefs inferred from the second interview 1. you have always to cheer up the children 1. you have always to cheer up the children 2. initial literacy and calculus are the most important things to be taught 2. 3.multilinguism is a good thing because knowing more languages means major future mobility. 3. 4. 4. reading and writing well in p5 is a good thing 5. 5. when you work oral skills you are also learning to read and write diana maintains exactly the same belief (number 1) in the first and second interview, which indicates lack of change. basically, in figure number 5 it can be see how some of the beliefs didn’t emerge in both in the interviews. is the case of belief number 2 and 3, that emerged in the first but not in the second interview, and the case of beliefs number 4 and 5, which appear in the second but not in the first interview. figure 6: tensions identified in diana’s interviews. tensions from the first interview tensions from the second interview a. about correction: you have to correct children’s’ writings vs you don’t have to correct them b. how can you measure a good level of english: decoding vs understanding c. learning to read and write: the sooner the better vs there is a perfect moment to initiate this learning d. what is project work pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 72 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 in the first interview no tensions emerged. however, in the second interview several tensions appeared, basically related to conceptual issues such as –i.e. what is (or not) project work (d), what defines a good level in literacy (b)but also there were ideas been expressed in almost contradictory ways such as example (a) about whether you have to correct or not children’s errors in a composition or if it is true that the sooner you learn to read and write the better (c). furthermore diana is not aware of any of her tensions. therefore, they indicate a gap in the belief system that teacher training should fill offering a wide variety of paradigms, theories, models, etc to assist reflection in order to solve those tensions. 5. discussion this study presents several limitations that have to be taken into account. first of all, it is part of a larger study not concluded, so it will be interesting to see if the final conclusions go in the same direction as the ones exposed here. for sure, it will complete these results, since they will offer a wider vision of the learning process in teacher education programs. second, this is a case study with three subjects. this means that there is no attempt to generalize these conclusions to all the students in all pre service educational programs. finally, and not least, the way data was obtained has unquestionably an impact on the deepness of the inferred beliefs. this qualitative study is very different than quantitative studies, even though the topic may be the same. with this data it cannot be assured that beliefs are immovable. it can be demonstrated that some of the beliefs remain perfectly stable through the year of teacher education, which would confirm the stability noted by pajares (1992). furthermore, both three subjects kept, at least, one of the beliefs stable between the interviews. but, at the same time, some beliefs have changed. the interesting part is to see how these beliefs have changed, whether some new information has been acquired to the previous existent structure or if it has been reformulated in order to express the idea in a entire different way. nevertheless, the amount of change can be so small, that it can be unperceived with some data pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 73 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 collection instruments. in the present paper, in-depth interviews have been made and the exhaustive discursive analysis has proved to state changes. this may be one of the most importance difference between the current study and the one reported by peacock (2001), were he used a questioner called beliefs about language learning inventory (balli) to obtain data. the impact of the one-year degree is low, but to agree with cambra (2008) the investigation should be terminated, because there is still one last interview in the final course of the degree. clearly, what this data shows is the existence of previous shaped beliefs, acquired before teacher education, what confirms lortie’s (1975) theory about the impact of the formal schooling configuring teachers’ beliefs. another relevant issue is the appearance (or not) of the same topic beliefs, which complicates the beliefs tracking. it is especially characteristic in diana’s case, where most of the beliefs that appeared in the first interview didn’t appear in the second, and the same way around. this be caused by the dynamic condition of beliefs (dufva 2003) and also because it is impossible to infer the complete belief system of each student from one interview. beliefs are also being constructed when interacting and not all the issues may arise in one interview, which means that some of the beliefs may keep submerged. for these reasons teacher-training programs should take beliefs systems as the starting point (graden 1996; cambra 2000; borg 2003; chong, wong & queck 2005). it would make a difference to work individually with students in order to see what beliefs are they bringing in to the classroom. if that is not possible, at least a belief exploration should be done at the beginning and at the end of every course to see the impact of the program. also, tensions appear to be the key factor because they show conflictive areas where students are no comfortable yet, because they lack either reflection or more theory to incorporate to their thoughts. in this sense, self-reflection or auto exploration should be encouraged because tensions, as well as beliefs, will accompany them for life and it can be useful to make our students aware of their thoughts. thereby, looking at discursive pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 74 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 tensions as positive elements (johnson 1994) can make the difference both for teachers and students, and certainly for the researchers. 6. conclusions the methods used in this investigation to obtain data and to analyze it have proved to be appropriate for achieving the objectives described in the introduction. the way beliefs have been inferred shows an interesting approach to study this non-visible part of learning-to-be-a-teacher process that takes place in students’ minds. moreover, it has demonstrated to be able to show the evolution of the beliefs as well as tensions, which was the main objective, but also it has brought deepness and an intensity that grants a better understanding of the complicated learning process. furthermore, it has been confirmed the existence of change and stability in students’ beliefs at the same time. this fact changes the narrow vision of what happens during teacher education, whether only change or the absence of change can be the deduced. it shifts the understanding of the process towards a much wider, complex and pluralistic perception were some of the beliefs change while some don’t. these conclusions stimulate the researchers to keep investigating teachers’ beliefs, and specially trying to identify the reasons of shift. 7. references barcelos, a.m.f. (2003). researching beliefs about sla: a critical review. in a. m. barcelos & kalaja, p. (eds.), new approaches to research on beliefs about sla (pp. 7-33) dordrecht: kluwer. borg, s. (2003). teacher cognition in language teaching. a review of research on what teachers think, know, believe, and do. language teaching 36, 81-109. borg, s. (2011). the impact of in-service teacher education on language teachers’ beliefs, system, 39: 370-380. pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 75 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 cambra, m. (2000). el pensament del professor: formació per a la pràctica reflexive. in a. camps & ríos, i. & cambra, m. (eds.), recerca i formació en didàctica de la llengua (pp. 161-172). barcelona: graó. cambra, m. (2008). enseigner l’oral à l’école primaire. les représentations des futurs enseignants catalans en stage dans des classes de français langue étrangère. in c. cortier & bouchard, r. (eds.), quel oral enseigner, cinquante ans après le français fondamental? le français dans le monde. recherches et applications (pp. 158-170). paris: cle international chong, s. & wong, i. y-f & quek, c.l. (2005). pre-service teacher's beliefs, attitudes, and expectations: a review of the literature. centre for research in pedagogy and practice international conference on education, singapore. retrieved 28 august, 2012. http://conference.nie.edu.sg/paper/covert/ab00613.pdf dufva, h. (2003). beliefs in dialogue: bakhtinian view. in a. m. barcelos & kalaja, p. (eds.), new approaches to research on beliefs about sla (pp. 136-146) ) dordrecht: kluwer. freeman, d. (1993). renaming experience/reconstructiong practice: developing new understandings of teaching, teaching and teacher education, 9 (576): 485-497. graden, e.c. (1996). how language teachers’ beliefs about reading instructuin are mediated by their beliefs about students, foreign language annals, 29 (3): 387-395. jodelet, d. (ed) (1994): les représentations sociales. paris: puf. johnson, s.e. (1994). the emerging beliefs and instructional practices of preservice english as a second language teachers. teaching & teacher education, 10 (4), 439-452. kagan, d. m. (1992). professional growth among preservice teacher education. review of educational research 62, 129-169. kerbratorecchioni, c. (2005). le discours en interaction. paris: armand colin. lee, i. (2008). ten mismatches between teachers’ beliefs and written feedback practice, elt journal, 63 (1): 13-22. lortie, d. (1975). schoolteacher: a sociologycal study. chicago: university of chicago press. pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 76 http://conference.nie.edu.sg/paper/covert/ab00613.pdf multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.2235 social and technological sciences issn: 2341-2593 mattheoudakis, m. (2007). tracking changes in pre-service efl teachers beliefs in greece: a longitudinal study, teaching and teacher education, 23: 1272-1288. nettle, e. b. (1998). stability and change in the beliefs of student teachers during practice teaching, teaching and teacher education, 14(2): 193-204. pajares, m. f. (1992). teachers’ beliefs and educational research: cleaning up a messy construct, review of educational research, 62 (3) : 307-332. palou, j.; fons, m. (2013). investigar en un escenario social denominado aula, cultura & educación 25 (4). doi: 10.1174/113564013808906816 peacock, m. (2001). pre-service esl teachers’ beliefs about second language learning: a longitudinal study. system 29, 177-195. pérez-peitx, m. (2013). els perfils de pràctiques docents i el sistema de creences: un estudi de cas de dues mestres de parvulari. bellaterra journal of teaching & learning language & literature, 6 (1), 56-71. phipps, s.; s. borg (2009). exploring tensions between teachers’ grammar teaching beliefs and practices. system, 37 (3): 380-390. richards, k. (2009). trends in qualitative research in language teaching since 2000, language teaching 42/2, 147-180. woods, d. (1996). teacher cognition in language teaching: beliefs, decision-making and classroom practice. cambridge: cambridge university press. pérez-peitx and fons-esteve (2015) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 2 nº 1 (2015): 63-77 | 77 beliefs and tensions of kindergarten pre service students: a three case longitudinal study figure 2: tensions identified in carolina’s interviews figure 4: tensions identified in sara’s interviews figure 6: tensions identified in diana’s interviews. 5. discussion 6. conclusions 7. references jodelet, d. (ed) (1994): les représentations sociales. paris: puf. kerbratorecchioni, c. (2005). le discours en interaction. paris: armand colin. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 84 inquiry learning for gender equity using history of science in life and earth sciences’ learning environments c. sousa faculdade de ciências, universidade do porto, portugal e-mail: up199502480@fc.up.pt received: 2015-03-02; accepted: 2016-01-03 abstract the main objective of the present work is the selection and integration of objectives and methods of education for gender equity within the life and earth sciences’ learning environments in the current portuguese frameworks of middle and high school. my proposal combines inquiry learning-teaching methods with the aim of promoting gender equity, mainly focusing in relevant 20th century women-scientists with a huge contribute to the history of science. the hands-on and minds-on activities proposed for high school students of life and earth sciences constitute a learning environment enriched in features of science by focusing on the work of two scientists: lynn margulis (1938-2011) and her endosymbiosis theory of the origin of life on earth and inge leehman (1888-1993) responsible for a breakthrough regarding the internal structure of earth, by characterizing a discontinuity within the nucleus, contributing to the current geophysical model. for middle school students the learning environment includes inge leehman and mary tharp (1920-2006) and her first world map of the ocean floor. my strategy includes features of science, such as: theory-laden nature of scientific knowledge, models, values and socio-scientific issues, technology contributes to science and feminism. in conclusion, i consider that this study may constitute an example to facilitate the implementation, by other teachers, of active inquiry strategies focused on features of science within a framework of social responsibility of science, as well as the basis for future research. keywords features of science, gender equity, inquiry. mailto:up199502480@fc.up.pt multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 85 1. introduction one of the fundamental goals of science education is the development of scientifically literate citizens (conant, 1947) who can understand the nature of scientific knowledge, and this should be achieved by the end of high school (k-12 education). according to pisa (2015), science literacy is an understanding of the major facts, concepts and explanatory theories that form the basis of scientific knowledge, including content procedural and epistemic knowledge and also the ability to explain phenomena scientifically and interpret data and evidence scientifically. this learning proposal considers that students can understand the nature of explanations within the larger context of the construction of scientific explanations of the natural world, such as scientific models and theories, through the use of historical case studies corresponding to the 18th and 19th centuries for biology and 19th century for geology (conant, 1947) as well as contemporary case studies (focusing in relevant 20th century female scientists) and of the enterprise itself which “assumes a human face” (ngss, 2013). in this study i suggest the use of an inclusive inquiry learning strategy focused on the role of female scientists in the scientific enterprise, for implementation within life and earth sciences classes, in middle and high school portuguese curricula. in order to promote an inclusive learning environment it is necessary to understand students’ interests and backgrounds, as well as to include significant contributions of women and of people from diverse cultures and ethnicities to be able to engage them, for example from areas not evidenced by nobel lauretes, such as al gore, neil degrasee, sagan, and e. o. wilson (biodiversity and science communication) and margulis (evolution), leehman (seismology), tharp (ocean mapping). doing science occurs in complex settings of epistemic and social practices so our science learning environments should be designed around these features such as the embeddedness of science. since gender equality has been considered both as a human right and a development goal (unesco, 2014) this proposal is based on an inclusive science learning environment that promotes gender equity and its main focus is one of the features of science feminism (matthews, 2012). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 86 historical episodes may be read, presented in movies, may involve role-play, simulated debates or problem-solving scenarios and may focus on ideas, individuals, or controversies. this paper summarizes the guidelines to use 3 relevant case studies of life and earth sciences 3 women scientists and their corresponding outstanding ideas from history of science and were designed to be focused on “how we know”. since a consistent characteristic of scientific knowledge across different disciplines is that scientific knowledge itself is open to revision in light of new evidence (tentativeness) either due to new technology or novel interpretations these features are referred in each case study. 2. teaching-learning strategy proposal this teaching-learning strategy is based in a novel philosophy of science that goes beyond the epistemic role and introduces the social context in which science is conducted (kourany, 2012). in our current social context one can find social injustices experienced by women, occurring in their workplaces and homes, that may interfere with career choices to be made by middle and high school students. therefore, we propose an inclusive and inquiry-based learning environment, promoting the use of gender-neutral language, collaborative work (in small groups of mixed gender with up to 4 students), scientific argumentation, and focused on “how we know”. in life and earth sciences curricula one can find several male-scientists, that are always studied in portuguese and world classrooms, such as: (a) earth internal structure beno gutenberg, andrija mohorovičić, (b) evolution charles darwin, alfred r. wallace, (c) continental drift and plate tectonics alfred wegener, arthur holmes, john t. wilson, william jason morgan, (d) sea floor (spreading and characterization) robert s. dietz, harry hess, frederick vine, drummond h. matthews, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 87 (e) classification of life on earth carl woese and robert whittaker. therefore, we propose a learning environment focused on contemporary female-scientists that contributed to the above themes and constitute relevant episodes of history of science of the 20th century, in order to promote gender equity in the classroom. in table 1, one can find the summary of the science topic, within the curricula, and the corresponding proposed episode of history of science (hos). table 1. overview of science topics and the corresponding episodes of hos. discipline (grade) science topic episode of hos for inclusive inquiry learning natural sciences (middle school) understanding the internal structure of earth (7th grade) inge leehman and geophysical model understanding the principles of earth’s structure and dynamics (7th grade) mary tharp and mapping the ocean floor technological advances; using google earth. understanding earth as a system able to generate life (8th grade) lynn margulis and endosymbiotic theory earth as a system subdivided in subsystems biology & geology (high school) earth subsystems and their interaction (10th grade) earth as a system subdivided in subsystems (margulis’ s gaia hypothesis) lynn margulis and endosymbiotic theory margulis’s life classification system biological evolution (11 th grade) classification systems (11th grade) face of the earth: continents and ocean floor (10th grade) mary tharp and mapping the ocean floor new technological advances and correlation with plate tectonics; using google earth. earth internal structure: geophysical model (10th grade) inge leehman and geophysical model new technologies: seismic tomography multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 88 3. case-studies: relevant episodes of the history of science the historical case studies presented have in common the focus on a feature of science feminism (matthews, 2012). however, each case study highlights, not all but a certain set of features of science, such as: empirical basis, scientific theories and models, creativity, tentativeness, idealization, values and socio-scientific issues, technology and worldviews. some guidelines for teachers are presented for each case study focused on a female scientist, in each of the following sections. 3.1. inge leehman (1888-1993) inge lehmann was born in 1888 and received her degree in mathematics in 1920. in 1928, she became chief of the seismological department of the geodetical institute of denmark, a position she held until retirement in 1953. leehman also contributed to the foundation of scientific societies that are important in both promoting studies and public understanding, so it is suggested to discuss with students the role of scientific and professional societies using this episode. inge cofounded the danish geophysical society in 1936 and served as its chair in 1941 and 1944, and in 1950, she was elected the first president of the european seismological federation (hjortenberg, 2009). inge leehman was the seismologist responsible for a breakthrough regarding the internal structure of earth, by characterizing a discontinuity within the nucleus, describing an inner core, in 1936, contributing to the current geophysical model. an interesting question teachers should ask high school students is: analyze leehman’s illustration of how seismic rays propagate inside the earth’s inner layers (lehmann, 1936) and hypothesize about earth internal structure. students should suggest something like leehman’s own interpretations, such as that when a p-wave (the first phase to arrive) hits the core-mantle boundary, passes through the core and is detected on the far side of the earth or is deflected and received in the p-wave shadow zone suggesting the existence of solid inner core. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 89 leehman worked in the reliability of european seismological stations so they could be used in accurate geological inferences and subsequent independent investigations by birch in 1940 and bullen in 1946 establish the lehmann’s hypothesis of a solid inner core (lehmann, 1987), and in 1970 high-angle reflections of seismic p waves were observed unequivocally on seismograms upon underground nuclear explosions (engdahl et al. 1970). in small groups, students should read selected articles about inge lehmann and focus on: ‣ discussion of lehmann’s life and work, mainly identifying aspects of her personality and attitudes towards her work that contributed to her success (bolt, 1997): inge lehmann experienced her first earthquake when she was growing up in østerbro, denmark, at age 15 or 16, and she was educated at a progressive high school where girls and boys were treated equally. throughout her long career in seismology, inge maintained a keen physical intuition and in firm support for international collaboration. she had to fight social stereo-types and prejudice: “no difference between the intellect of boys and girls was recognized, a fact that brought some disappointment later in life when i had to recognize that this was not the general attitude.” at college, inge experienced “severe restrictions inflicted on the conduct of young girls” and returned home from exhaustion, but then returned and graduated in 1920. she was described as “probably not always very diplomatic”, since she said: “you should know how many incompetent men i had to compete with – in vain.” the award of bowie medal by the american geophysical union, in 1971, to lehmann was her highest distinction, and at this ceremony it one said that: “the lehmann discontinuity was discovered through exacting scrutiny of seismic records by a master of a black art for which no amount of computerization is likely to be a complete substitute...“. in 1996, the american geophysical union established the inge lehmann medal to acknowledge “outstanding contributions toward the understanding of the structure, composition and/or dynamics of the earth’s mantle and core.” multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 90 today’s advances by seismic tomography allows the creation of colourfull images of earth’s interior using three-dimensional seismic data, improved modeling techniques and powerful supercomputers (clabby, 2015), that are responsible for increased resolution of seismic waves speed and consequently a higher accuracy of inferences made about earth’s internal layers. teachers should ask students to search for information about the project earth scope (www.earthscope.org) that has been installing 20 new seismometers every month, in usa, since 2013 and promote the discussion of the information available, such as: ‣ discuss, in small group, of the tectonic significance of one seismic tomography image of their choice; then share the findings and explanations with the class. 3.2. mary tharp (1920-2006) marie tharp was born in 1920 in ypsilanti, michigan, and graduated from the university of ohio in 1943 with a bachelor’s degree in english and music after graduating. then she enrolled in michigan’s geology program, a rare opportunity, normally only open to males, due to the ii world war; in 1944 marie graduated with a master’s degree and started her first job at the standlind oil & gas company in tulsa, oklahoma (tharp, 2006). however female geologists were not allowed to conduct field work, and only office work was available, coordinating maps and data from male colleagues (tharp, 2006). so while working, she also attended the university of tulsa and in 1948 she received her bachelor’s degree in mathematics. then she started to work as a research assistant at lamont geological laboratory in columbia university, coordinated by dr. maurice ewing, and collaborating with bruce heezen (1924-1977). ewing did not allowed females to work at sea, so mary tharp was responsible for the analysis of thousands of depth measurements obtained by echo sounders in navy ships, drafting and plotting ocean floor profiles. in 1952 she discovered what came to be called the mid-atlantic rift valley, but her claims were dismissed by bruce heezen: “it cannot http://www.earthscope.org/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 91 be. it looks too much like continental drift.” and later her collaborator said in an interview that “i discounted it as girl talk and didn’t believe it for a year.” (tharp, 2006). due to mary tharp perseverance, bruce heezen "eventually gave up the idea of an expanding earth for a form of continental drift...” and in 1957, bruce heezen presented their model on the mid-ocean rift system, at a conference, at princeton, and used a globe (made by heezen and tharp) that showed how the rift system extended all around the world. after the talk, the princeton geologist harry hess who later developed the theory of seafloor spreading stood up and said: “young man, you have shaken the foundations of geology!” (tharp, 2006). teachers should promote discussion, in small groups, about the social context (e. g. using the above paragraph) and also the scientific context (using text of earth institute, 2006) of tharp’s discovery, using texts such as: “i had a blank canvas to fill with extraordinary possibilities, a fascinating jigsaw puzzle to piece together: mapping the world’s vast hidden seafloor. it was a oncein-a-lifetime and a once-in-the-history-of-the-world opportunity for anyone, but especially for a woman in the 1940s. the nature of the times, the state of the science, and events large and small, logical and illogical, combined to make it all happen. (...)establishing the rift valley and the mid-ocean ridge that went all the way around the world for 40 000 miles that was something important! you could only do that once. you can’t find anything bigger than that, at least on this planet.” some questions that the students should focus on are: ‣ in what way the existence of a mid-atlantic rift supports the continental drift theory? ‣ at some point tharp’s claims were dismissed by her co-workers, and she latter explained her thoughts: "i figured i'd show them a picture of where the rift valley was and where it pulled apart. there's truth to the old cliché that a picture is worth a thousand words and that seeing is believing!". discuss about tharp’s characteristics that allowed her to succeed as a woman in a field dominated for decades by men. (teachers should empathize her imagination and creativity, curiosity, perseverance, intelligence and intuition). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 92 mary tharp and bruce heezen collaborated for 30 years producing the first maps of the ocean floor, in summary they are co-authors of published in 1959 north atlantic physiographic diagram published, in 1961 south atlantic physiographic diagram, published in 1964 indian ocean physiographic diagram, in 1967 indian ocean panorama in national geographic magazine and in 1977 world ocean floor panorama published by the office of naval research (earth institute, 2006) bruce heezen, died in 1977, but marie tharp continued to work at the university of columbia till 1983, and then she opened a map making business in new york. in 1998, she was honored at the 100th anniversary of the library of congress' geography and map division and in 1999, she received the women pioneer in oceanography award, awarded by the woods hole oceanographic institution (earth institute, 2006). in 2001, tharp received the lamont-doherty heritage award and in 2004, the marie tharp visiting fellowship program a financial aid given to promising women researchers was created by lamont. tharp worked with pens, ink and rulers, drawing underwater details described by thousands of sonar readings taken by several researchers, and her world ocean floor panorama has become a modern scientific and popular icon, described as: "a map produced as a supreme act of rigorous creativity”, “one of the most remarkable achievements in modern cartography”. state of the art includes satellite altimetry, by missions cryosat-2 and jason-1, that allowed high resolution imaging of tectonic structures of the ocean basins (e. g. unknown areas covered by sediments) and represented in novel marine gravity anomaly maps (sandwell et al., 2014). teachers should guide high school students through these technological advances by suggesting them to read “seafloor secrets revealed” in science (hwang & chang, 2014). for middle school students, i recommend to introduce the theme by watching some movies, such as: “deep sea challenge”, by james cameron and national geographic, including the facebook and website (http://www.deepseachallenge.com/) and focusing http://www.deepseachallenge.com/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 93 on the mariana trench and “cosmos: a space-time odyssey”, episode 9 of 2014, in which neil degrasse tyson talks about mary tharp, and includes some animations about her work. then, students can be motivated and connected to the real-world using a virtual globe or geobrowser such as google earth, that includes satellite and aerial imagery and ocean bathymetry images encouraging them to investigate different areas of the sea floor. it is also interesting to connect tharp’s work with hesse’s theory and vine’s work on the ocean floor using paleomagnetism, by using the layer, for google earth, “age-of-earth crust.kmz”, that allows the overlay of the age of ocean isochrones. some hands-on and minds-on activities have been proposed to be performed in small groups of 3 students per computer (sousa, 2013), including questions to be used to guide students in their investigative activity using google earth, such as: ‣ explore the overlay “age-of-earthcrust.kmz” and correlate the maximum age of each ocean with the tectonic event responsible for their origin. ‣ tharp, heezen and collaborators were responsible for producing the first topographical maps of the ocean floor and of the first map of oceanic earthquake locations, both created at the same scale. argue about the importance of their contributes to the plate tectonics theory. suggestion: using the overlay “usgs magnitude 2.5+ earthquakes” correlate the occurence of underwater earthquakes with the ocean bathymetry. ‣ explore the overlay “global_grav.kmz” corresponding to the suggested literature on global gravity anomaly (hwang & chang, 2014). note: suggestion for high school students, only. 3.3. lynn margulis (1938-2011) lynn margulis (born lynn petra alexander, former lynn sagan) obtained a bachelor degree at age 19 and her master's degree in genetics and zoology at age 22 from the university of chicago. she received her phd from the university of california, berkeley, at age 27. then, in 1966, she joined the faculty of boston university. in 1988 she became distinguished professor of botany, and in 1997, distinguished professor of geosciences multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 94 at the university of massachusetts at amherst. she is honorary doctorate from 15 universities. with middle school students, teachers can explore some of lynn margulis sentences and discuss the differences in the attributes of men and women and gender-based roles over time in the past and current cultural contexts, and empathizing the importance of equity between genders in the expectations and roles and sharing of responsibilities by men and women, such as: margulis said about her marriage with carl sagan in 1957, and divorce in 1965 and her second marriage in 1967 with thomas n. margulis and posterior divorce: "i quit my job as a wife twice!". and also said that "it’s not humanly possible to be a good wife, a good mother and a first-class scientist. no one can do it something has to go.” margulis’s first paper on endosymbiosis was finally published in 1967 after being rejected twenty times in the journal of theoretical biology, she suggested that mitochondria and plastids evolved from bacteria hundreds of millions of years ago, after bacterial cells started to live symbiotically with one another, then named endosymbiosis theory. her theory is considered one of the “greatest achievements of twentieth-century evolutionary biology” (richard dawkins, 1995 in brockman, 1995). teachers should emphasize the role of criticism and controversies in the revision of theories and acceptance of new theories, such as the endosymbiosis theory, by selecting some texts for discussion in the classroom, such as the one bellow (w. daniel hillis, 1995 in brockman, 1995): “most of the science that gets done gets done within a rigid set of rules, where you know exactly who your peers are, and things get evaluated according to a very strict set of standards. that works, when you're not trying to change the structure. it works in what stephen gould calls incremental science. but when you try to change the structure, that system doesn't work very well. when you try to do something that doesn't fit into a discipline or a standard theory, you usually make some enemies. (...) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 95 she had a way of looking at symbiosis which didn't fit into the popular theories and structure. in the minds of many people, she went around the powers that be and took her theories directly to the public, which annoyed them all. it particularly annoyed them because she turned out to be right. if it's a sin to take your theories to the public, then it is a double sin to take your theories to the public and be right.” ( teachers should ask high school students to discuss the role of the technological advances in the resolution of the scientific controversies about margulis’s theory, by guiding them to answer the following question: ‣ what evidence could support the hypothesis that mitochondria really did evolve from free-living bacteria? students should be able to refer to dna analysis such as in phylogenetic studies that was not available at the time of margulis’s proposal, since dna sequencing only started at the end 1970s. in fact in 1982, other scientists showed that the plastid dna was much more similar to the dna of free-living photosynthetic bacteria than it was to the nuclear dna of the host cell, and one year later they showed the similarities of mitochondrial dna and free-living aerobic bacteria dna (gray, 1983). students should also discuss in small groups, and then in class, the opposing views about evolution, using for guidance adequate sentences, such as: ‣ “for two centuries, male biologists had emphasized the role of struggle, competition, and war in evolution, and lynn margulis was offering a very different view.” (dunn, 2013). high school students should be able to refer that lynn margulis showed that genetic mutations are not the only source of variation and that competition is not the only strategy that drives evolution. her view the novel view is that through symbiosis organisms can cooperate to form a new entity and that with time the relationship can become definitive, named endosymbiosis, in which distinct species become one and that is responsible for the origin of the first eukaryotic organisms. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 96 margulis contribution to the classification system should also be referred and high school students may be asked to compare different systems such as: copeland’s, whittaker’s, woese’s and margulis’s (that includes 2 domains and 5 kingdmons). gaia hypothesis, according to margulis, defines that all earth systems are connected, and the biosphere overlaps with the atmosphere, the upper part of the geosphere, and almost all of the hydrosphere (margulis & sagan, 2002). while james lovelock, a chemist, proposed “gaia” as a kind of living organism, however according to margulis: gaia is "an emergent property of interaction among organisms" but “not a organism” (margulis,1998). 4. discussion and future perspectives an inclusive learning environment is important in order to avoid students be led away from science-related careers due to misperceptions that science is difficult, uncreative and “men’s world”. the use of historical case studies in science education has long been shown to be efficient in both motivating students and promoting significate learning (conant, 1947). however, the necessity of an update of these case studies and the construction of novel ones has been described (allchin, 2011). this paper presents a structured inquiry learning strategy that was designed to promote critical thinking, scientific argumentation, problem-solving and communication, in middle and high school classrooms. the construction of these contextualized historical case studies was made upon the study of several materials in order to create historically, philosophically and sociologically well informed case studies that include biographical and social context factors, as well as the gender equity. the aim was to create significant case studies, and not anecdotes and stories, to facilitate the development of classes focused on history and philosophy of science, that may be used by other teachers and constitute the focus of my current research. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 97 acknowledgments special thanks to the ones that inspired me. the views expressed in this paper are those of the author. the author would like to thank universidade do porto, portugal, for financial support. references allchin, d. (2011). the minnesota case study collection: new historical inquiry case studies for nature of science education. sci & educ. doi 10.1007/s11191-011-9368-x. bolt, b. a. (1997). inge lehmann. biographical memoirs of fellows of the royal society. 43, 285-301. http://www.physics.ucla.edu/%7ecwp/articles/bolt.html. brockman, j. (1995). the third culture: beyond the scientific revolution. new york: touchstone. clabby, c. (2015). seismic visions of middle earth. american scientist, 103 (2), 102. doi: 10.1511/2015.113.102. conant, j. (1947). on understanding science: a historical approach. cambridge, ma: harvard university press. dunn, r. (2013). remembering lynn margulis. http://www.robrdunn.com/2013/04/remembering-lynn-margulis/ earth institute (2006). remembered: marie tharp, pioneering mapmaker of the ocean floor in the earth institute columbia university news archive. http://www.earth. columbia.edu/news/2006/story08-24-06.php.html. engdahl, e.r., flinn, e. a. & romney, c. f. (1970). seismic waves reflected from the earth's inner core. nature, 228, 852 853. gray, m. w. (1983). the bacterial ancestry of plastids and mitochondria. bioscience, 33, 693-699. http://www.physics.ucla.edu/%257ecwp/articles/bolt.html http://dx.doi.org/10.1511/2015.113.102 http://www.robrdunn.com/2013/04/remembering-lynn-margulis/ http://www.earth.columbia.edu/news/2006/story08-24-06.php.html http://www.earth.columbia.edu/news/2006/story08-24-06.php.html multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.3762 social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 98 hjortenberg, e. (2009). inge lehmann’s work materials and seismological epistolary archive. annals of geophysics, 52 (6), 679-698. holmegaard, h. t., madsen, l. m. & ulriksen, l. (2014). to choose or not to choose science: constructions of desirable identities among young people considering a stem higher education programme. international journal of science education, 36 (2), 186215. hwang, c. & chang, e. t. y. (2014). seafloor secrets revealed. science, 346 (6205), 3233. kourany, j. a. (2012). philosophy of science after feminism. oxford: oxford university press. lehmann, i. (1936). p’. union geodesique at geophysique internationale, serie a. travaux scientifiques, 14, 87. lehmann, i. (1987). seismology, in the days of old. eos, 69 (3), 33-35. margulis, l. (1998). symbiotic planet: a new look at evolution. london: weidenfeld & nicolson. margulis, l. & sagan, d. (2002). acquiring genomes: a theory of the origins of species. new york: basic books. masnick, a. m., valenti, s. s., cox, b. d. & osman, c. j. (2010). a multidimensional scaling analysis of students’ attitudes about science careers. international journal of science education, 32 (5), 653-667. matthews, m. r. (2012). changing the focus: from nature of science to features of science. in m. s. khine (ed.), advances in nature of science research. dordrecht: springer. nrc, national research council (2013). next generation science standards. washington, dc: national academies press. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.xxxx social and technological sciences eissn: 2341-2593 sousa (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 1 (2016): 84-99 | 99 oecd, organisation for economic co-operation and development (2013). pisa 2015 draft science framework. paris: oecd. htp://www.oecd.org/pisa/pisaproducts/ draft%20pisa%202015%20science%20framework%livepage.apple. com20.pdf. richardson, e. (2014). inge lehmann: discoverer of earth’s inner core. in earth 520: plate tectonics and people: foundations of solid earth science. https://www.eeducation.psu.edu/earth520/content/l2_p23.html. sandwell, d. t., müller, r. d., smith, w. h. f., garcia, e. & francis, r.(2014). new global marine gravity model from cryosat-2 and jason-1 reveals buried tectonic structure. science, 346 (6205), 65-67. sousa, c. (2013). pensar a pangeia como wegener. kit educacional mobilismo geológico, volume 1 atividades práticas para o 7º ano de escolaridade. [think pangea as wegener. educational kit gelogical mobilism, volume 1 practical activities for senventh grade level]. isbn: 978‐ 989‐ 97682‐ 2‐ 2. (e-book). casa das ciências. http://imagem.casadasciencias.org/online/39116006/39116006.php. sousa, c. (2015). educação para a igualdade de género: reflexões e ambientes de aprendizagem com recurso a história da ciência. [education for gender equality: reflection and learning environments using the history of science.]. u. porto (under development). tharp, m. (2006). marie tharp bio in mary sears woman pioneer in oceanography award, woods hole oceanographic institution (whoi). http://www.whoi.edu/sbl/litesite.do?litesiteid=9092&articleid=13407. unesco, united nations educational, scientific and cultural organization (2014). gender equality, heritage and creativity. paris: unesco. zeidler, d. l., sadler, t. d., simmons, m. l., & howes, e. v. (2005). beyond sts: a research-based framework for socioscientific issues education. science education, 89, 357–377. http://www.oecd.org/pisa/pisaproducts/%20draft%2520pisa%25202015%2520science%2520framework%2520.pdf http://www.oecd.org/pisa/pisaproducts/%20draft%2520pisa%25202015%2520science%2520framework%2520.pdf https://www.e-education.psu.edu/earth520/content/l2_p23.html https://www.e-education.psu.edu/earth520/content/l2_p23.html http://www.casadasciencias.org/ http://www.whoi.edu/sbl/litesite.do?litesiteid=9092&articleid=13407 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 17 using structured positive and negative reinforcement to change student behavior in educational settings in order to achieve student academic success j. kelly*, b. pohl. department of urban educationdisability studies. university of houston – downtown, department of urban education. one main street, houston, texas, 77002, usa. *corresponding author: email: kellyj@uhd.edu; phone: 713-221-8952 received: 2017-03-03; accepted: 2018-01-11 abstract typically, classroom management approaches for dealing with disruptions and misbehavior from students involve the use of various forms of punishment: removal from the classroom, fines, in-school and out-ofschool suspensions, or expulsions (garret, 2015). however, traditional classroom management methods have yielded very little positive results. some would even argue that classroom behaviors are escalating out of control. using research based approaches; this article’s goal is to help teachers discover student-centered approaches that will positively improve discipline inside the classroom. keywords reinforcement, behavior, punishment, special education https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 18 1. introduction traditionally, approaches for dealing with student disturbances and disobedience are based on various forms of punishment: removal from the classroom, fines, in-school and out-of school suspensions, or expulsions (garret, 2015). for the most part, schools continue to base their discipline policies on a strict adherence to obedience (goodman, 2006) and zerotolerance policies (maag, 2012; skiba & peterson, 2003). these practices are stated in most every code of conduct handbook in public schools throughout the united states. some of these approaches may give educators the false sense that schools and classrooms have become safer when disruptive students are removed. surprisingly, research shows that punishment and severe reprimands have minimal effect on helping students perform more socially acceptable behaviors because removal from the classroom does not come with appropriate behavior instruction – just ejection (carter & pool, 2012; skiba & peterson, 2003). in recent years, policy makers at the local, state, and federal level have expressed concerns about the effectiveness of zero-tolerance disciplinary approaches, encouraging schools to adopt more constructive approaches that would create more healthy learning environments (skiba, 2014). despite the evidence to the contrary, educators and schools continue to find zero-tolerance policies and punishments as acceptable approaches to deter misbehavior. for the most part, according to skiba (2014), schools and districts do not get a chance to consider best practices and alternatives to their zero-tolerance policy. teachers and administrators working in schools with a high implementation of zero-tolerance policies see themselves navigating high-stress environments where alternatives to harsh disciplinary practices are rarely available and discussed (hernandes-melis, fenning, & lawrence, 2016; robinett, 2012). in addition, the successful support to these disciplinary alternatives might not exist, as it does require a major investment in resources and training by the districts and schools (skiba & losen, 2016). https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 19 in this article, we discuss positive-research based practices that help educators efficiently change disruptive student behaviors in educational settings in order to achieve student academic success for all their students. we begin by examining the historical implementation of zero-tolerance policy in the united states. we continue by discussing the foundational understanding of positive and negative classroom reinforces. finally, we discuss the implementation of alternatives that have successfully worked in the classroom, which have created healthier learning environment. 2. the “get-tough” legacy historically in the united states, schools have been expected to be learning environments where students acquire the virtues of becoming productive citizens (pohl, 2013; purpel & mclaurin, 2004); however, the implementation of harsh discipline techniques have not only been accepted, but it has become an integral part of schools, achieving a primordial place in the curricular agenda (parsons, 2015). in the united states, zero-tolerance policies and rigid school environments became popular as the industrialization of the country demanded a labor-ready workforce available for the factories (pohl, 2013). the demand for workers promoted a factory-like environment in the schools, encouraging educational settings with rigid schedules, standardized curriculums, and regimented physical lay-outs (goodman, 2006; purpel & mclaurin, 2004). in the early second half of the 20th century in the united states, zero-tolerance policies became popular in schools and criminal justice systems (skiba & losen, 2016) as the public demanded schools to be training places for the workforce (pohl, 2013)—policies which affected a disproportionate number of minorities such as latinos and african americans as time passed (thompson, 2016). the increase use of these harsh discipline methods were due to several factors; however, the primary factors that prompted schools to pursue harsher discipline policies were the get-tough federal drug and crime policies of the late 1970s and early 1980s (shah & mcneil, 2013). the implementation of severe discipline polices were deemed necessary to https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 20 send the message that certain behaviors were not going to be tolerated. in the 1980s, as a way to calm the fears of school violence, calls were made to severely punish guns, drugs, and weapons violations with suspensions, expulsions, and even jail time (thompson, 2016); moreover, the use of zero-tolerance policies also promoted the increase of technology and security personnel. in the 1990s, president bill clinton signed into law the gun-free school act, which required any school that received federal funds to expel for a year any student who was caught with the possession of a firearm; additionally, with the passing of time, calls were made to implement the same severe reprimands to much lesser violations and misbehaviors (skiba, 2014). as a result, schools and districts throughout the nation began to severely punish much lesser offenses such as truancy, dress code violations, classroom disruptions, and inappropriate use of language with more severe consequences such as expulsion, suspensions, fines, and even criminal charges (robinett, 2012; shah & mcneil, 2013). since the implementation of these zero-tolerance policies, the number of students affected by these practices have been disproportionate (thompson, 2016). these disciplinary measures have affected more african american, native americans, and latinos than any other sector of the population (hernandes-melis et al., 2016), increasing the possibility of these groups of students to become permanent members of the judicial prison system, struggle in school, and never graduate. another important factors is that these severe measures do not appear to discriminate between socio-economics status, as minorities are more likely to be suspended and expelled in equal percentage from affluent schools (skiba & losen, 2016). in addition, recent data shows other minority groups, such as the disabled, black and latina females, and lbgt students, are also the target of more severe punishment (skiba, arredondo, & williams, 2014). despite the social acceptance of more severe punitive measures in american schools, data and research shows that these measures have not provided the desired results. since the 1980s, schools and districts have increased their investment in safety resources, such as https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 21 cameras and security personnel, including the creation of police forces by many school districts. however, studies have shown that these measures have provided very little relief in the decrease of violent and disruptive behavior (skiba & losen, 2016). an increasing number of studies are also starting to show the negative effects of severe punitive measures in schools, failing to corroborate the belief that harsher punishments result in better school safety (shah & mcneil, 2013). these results indicate that schools with higher suspensions and expulsion rates tends to have lower academic performance (skiba et al., 2014). additionally, students with a history of suspensions are more likely to be repeat offenders, developing a higher risk for anti-social behavior. finally, the use of severe reprimands increase the likelihood of a student becoming part of the juvenile justice system. according to skiba and losen (2016), this has led to organizations, such as the american psychological association, to highlight the ineffectiveness of severe reprimands in decreasing misbehaviors and improving school safety. 3. the positive strategies in recent years, however, a number of positive interventions have been implemented that have been found to be effective in improving behavior and school safety. these measures are characterized by the common use of three components: 1) the promotion of positive relationships within the schools; 2) the use of approaches that encourage students to become more conscious of their social behavior; and 3) the institutional use of positive intervention practices and student-friendly disciplinary codes. an increasing number of research is showing that a better teacher-student relationship promote better academic results and reduce misbehavior, especially for minorities. the practice of promoting better relationships are based in some fundamental principles: 1) the creation of schools as centers of community; 2) a personal relationship between teachers and students; and 3) the implementation of rigorous academic expectations. the result is https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 22 that schools that encourage such practices tend to severely decrease their misbehavior cases, while noticeably improving academic performance (robinett, 2012). such is the case of denver public schools, which saw their suspensions and expulsions decrease by 47 percent after the adoptions of programs that encouraged teachers to seek better relationship with their students (skiba & losen, 2016). schools that have positive discipline outcomes tend to develop learning environments that incite students to become more aware of their social behavior. these kind of programs that promote social-emotional learning, as defined by skiba and losen (2016), vary widely across the nation; however, these programs tend to share some common characteristics: 1) teach students behavioral management skills; 2) establish the appreciation for diverse views; 3) promote the creation of student-centered learning and social goals; and 4) encourage the students to handle interpersonal situations. in recent years, the creation of student-lead support groups is becoming more common, which have helped teachers and administrators in dealing with the early warning of potential misbehavior, which has led to reduction in the implementation of severe reprimands by schools that have successfully implemented these measures (shah & mcneil, 2013). the institutional use of more positive intervention practices and student-friendly disciplinary codes can have a major impact in the reduction of discipline misbehaviors (robinett, 2012; thompson, 2016). studies have shown that changes in the structure of discipline codes that implement more positive interventions have resulted in the reduction of expulsions and suspensions in school, yielding positive academic results. however, as skiba and losen (2016) argue, these practices will require schools and districts to become aware of their social disparities, including identifying the social ‘enablers’ and ‘barriers’ that students encounters in the learning environment. for example, research has shown that schools that implement early response teams that identify potential scenarios of misbehavior tend to reduce violent and disruptive incidents by at least 25 percent among minority groups (skiba & losen, 2016). https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 23 finally, the implementation of more student-friendly codes of conduct can also positively impact a school. recent calls have been made to limit expulsions and suspensions for the most severe of disciplinary cases. in recent years, according to shah and mcneil (2013), schools district across the nation are revising their codes of conducts to revise the consequences for minor incidents such as tardiness, inappropriate language, and classroom disruptions (robinett, 2012), focusing on more positive alternatives to suspensions and expulsions. however, we must be aware that such changes must be accompanied with better training, more resources, and effective administrative support. 4. practical understanding of positive reinforcement skinner (1953) introduced the theory of operant conditioning—a system of learning that occurs through the association of rewards and punishments through the use of reinforcement. with operant conditioning, good behavior is associated with positive or negative reinforcements, and bad behavior is associated with positive or negative punishment. for example, if kate does all her homework and behaves well during a particular week, the teacher may reward kate with extra playing time and the removal of a low grade. in this instance, the addition of extra playing time is a positive reinforcement, while the removal of a low grade is an example of a negative reinforcement. however, if kate misbehaves and does not do her homework, the teacher might punish her by taking away her cell phone and making kate stay an extra hour after school. the removal of the cell phone is an example of negative punishment, while staying an extra hour after school is an example of positive punishment. despite its obvious benefits, most teachers and administrators continue to have reservations about using reinforcements as a tool for discipline and classroom management (maag, 2003). it is speculated that teachers continue to underutilize behavior-managing tools to positively and productively confront possible challenging behaviors from students because they lack the confidence to implement them. https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 24 this has shown to be problematic for educational settings because students’ behaviors become especially challenging when traditional approaches do not yield the expected results. with the new inclusion paradigm and different cultural values, the old punishment models do not seem to work with about 5% of the students (maag, 2003). the negative results to teaching and learning are significant, especially when the actions of a few students can disrupt the learning of the majority in a given class. the successful learning environment often indicates a successful classroom management plan (clement, 2010). disruptive environments are not conducive to good learning experiences, which affects the overall academic achievement. 5. implementation since the implementation of skinner’s (1953) work, techniques based on positive reinforcement have been well developed and implemented, yielding positive results (ditullio, 2014). these techniques produce better academic success; moreover, they are adaptable and easy to implement in any behavior situation or discipline challenge. several of them are presented here: 5.1 rewarding good behavior. this is one of the easiest and most effective ways for dealing with challenging behaviors (maag, 1999). i am perplexed at why teachers ignore students behaving well; this method is easy to use and provides great teachable moments for the whole class. i use this technique frequently. any time student exhibits an appropriate behavior, i always take time out to praise the student and use it as a teachable moment. one of the kids who had a bad reputation called me “coach kelly” one day. i took the time to praise him in front of the class for addressing me appropriately, using it as a teaching moment to discuss manners. all the kids began to follow suit. they even started correcting each other when someone regressed back to simply “mister.” this takes less time than punishing bad behavior, because the students feel good about themselves and we move on. punishment takes away class time and can invite other bad behavior. https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 25 5.2 small unit thinking. when dealing with many challenging behaviors, or one major one, it is better to deal with them one at a time or in small units in the case of a big one (roberts, 2002). example: i had a self-contained classroom and was dealing with many behavior problems, including tardiness every period. the other teachers were complaining, so i was forced to deal with the tardiness issue immediately. it was here where i learned to think small. after stopping their tardiness using a token economy, i learned it was easier and more effective to deal with their other behaviors one at a time. a benefit to this success is the positive effect it had on the students’ attitudes. they seemed to gain a sense of confidence that they could effectively handle their other behavior problems. thinking small also works in presenting academic work; present material in small amounts and let the students achieve small successes. they will gain so much confidence that future failures will not set them back. they will remain on task longer and cause fewer disruptions. 5.3 promoting a group management plan. this is the most important ingredient in developing a positive classroom management system. there are two dynamics at work: it allows students to have ownership, and it provides for group (peer) influence (rhode et al, 1995). the control is in ownership. the more the students have a choice on what they are going to do the better they behave and perform. ownership works equally well with both behavior and performance. example: i got the class together and we made our class rules a class project. we discussed issues like the type of teacher they wanted, how they wanted to be treated by each other and me, and what the consequences would be. we put it into a contract that was signed by all and displayed prominently in the classroom for everyone to see and for a reminder when necessary. we still had issues from time to time, but they were always handled in class. in addition to control, the power of peer pressure can be very positive and is really effective in a group setting. it helps build community. 5.4 promoting trust. try to look for ways to establish trust. plan them if needed. do what you say you are going to do (roberts, 2002). your students must recognize the honor in your actions. it is very powerful because it changes the entire class into a family. when my https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 26 students were in the general classroom they liked to get in trouble so they could go home. once they were assigned to me, i told them that they were never going to the office. we would handle all problems in the classroom. what was said and done in our room would stay inside the room. when my students understood that i meant what i said they responded superbly by improving their attendance and doing much more class work. they felt safe. 5.5 promote choices and consequences the teacher and the students must establish together both positive and negative consequences for an action or inaction (roberts, 2002). it is very important that the students know they have a choice and understand what consequences will be implemented when the student makes a choice. “you do this and you get to do this". "you stop this behavior and you will be able to do this”. i used outside playtime as a positive consequence for completing their lessons. the negative consequence was less or no outside activity. i cannot emphasize enough how well this procedure worked in changing their behavior on a permanent basis. 6. conclusion these techniques are a common sense approach, and they are easy to apply; however, teachers are more accustomed to the use of punishment. it is easier for them to apply it, while administrators continue to advocate for it use. in the end, it will require teachers and administrators to change their own behaviors if they wish to positively change the students’ behavior. however, this change can prove to be difficult. four things must happen in order for this transformation to occur: first, a proper training and implementation system must be developed that helps the teachers, novice and veterans, to learn and apply new behavior modification techniques such as the five suggestions described in this paper; second, administrators must support this new system, encouraging the teachers with incentives, money, and professional development opportunities; third, teachers and administrators must buy into the new paradigm of today’s classroom and the reality of a more diverse https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 27 learning environment; and fourth, education colleges and teacher preparation programs will have to be more in-tune with the current realities and challenges of today’s public schools. research indicates that positive behavior modification techniques are more effective than punishment. structured positive and negative reinforcement foster learning by reducing classroom disruptions and increasing student attention. if these four suggestions are put in place, then, we can witness the successful use of scientifically proven procedures and methods for proactively and positively managing student behavior. students will be empowered to acquire the knowledge and skills that they will need to achieve academic success. https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 28 references carter, d., & pool, j. (2012). appropriate social behavior: teaching expectations to young children. early childhood education journal, 40(5), 315–321. clement, m. c. (2010). preparing teacher for classroom management: the teacher educator’s role. delta kappa gamma bulletin, 77(1), 41–44. ditullio, g. (2014). classroom culture promotes academic resiliency. phi delta kappan, 96(2), 37–40. garret, t. (2015). misconceptions and goals of classroom management. education digest, 80(5), 45–49. goodman, j. f. (2006). school discipline in moral disarray. journal of moral education, 35(2), 213–230. hernandes-melis, c., fenning, p., & lawrence, e. (2016). effects of an alternative to suspension intervention in a therapeutic high school. preventing school failure, 60(3), 252–258. maag, j. w. (2003). behavior management: from theoretical implications to practical applications. independence, ky: cengage learning. maag, j. w. (2012). school-wide discipline and the intransigency of exclusion. children & youth services review, 34(10), 2094–2100. parsons, b. (2015). the way of the rod: the functions of beating in late medieval pedagogy. modern philology, 113(1), 1–26. pohl, b. (2013). the moral debate on special education. new york, ny: peter lang. purpel, d. e., & mclaurin, w. m. (2004). reflections on the moral and spiritual crisis in education. new york, ny: peter lang. robinett, d. (2012). alternatives to student suspension. leadership, 42(1), 32–36. shah, n., & mcneil, m. (2013). discipline policies squeezed as views on what works. education week, 32(16), 4–11. https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.6370 social and technological sciences issn: 2341-2593 kelly and pohl (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 17-29 | 29 skiba, r. (2014). the failure of zero tolerance. reclaiming children & youth, 22(4), 27– 33. skiba, r., arredondo, m., & williams, n. (2014). more than a metaphor: the contribution of exclusionary discipline to a school-to-prison pipeline. equity & excellence in education, 47(4), 546–564. skiba, r., & losen, d. (2016). from reaction to prevention: turning the page on school discipline. american educator, 39(4), 4–11,. skiba, r., & peterson, r. (2003). teaching the social curriculum: school discipline as instruction. preventing school failure, 47(2), 66. skinner, b. f. (1953). science and human behavior. new york: macmillan. thompson, j. (2016). eliminating zero tolerance policies in schools: miami-dade county public schools approach. brigham young university education & law journal, (2), 325– 349. https://doi.org/10.4995/muse.2018.6370 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 the interdisciplinary approach in textbooks: a study on energy issues c. martín*, t. prieto, a. r. muñoz departamento de didáctica de la matemática, de las ciencias sociales y de las ciencias experimentales. facultad de ciencias de la educación de la universidad de málaga. campus de teatinos s/n, 29071. málaga, spain. * corresponding author: email: cmartin@uma.es; phone: + 34 952132449 received: 2013-12-15; accepted: 2014-08-04 abstract in a globalized and constantly changing world, education must be continuously updated. this requirement plays a major role in scientific and technological education, since technology may undergo significant changes in short periods of time. in such context, our students need educational opportunities that allow them to link scientific and technological aspects to many other subjects and situations which are closer to their personal interests and responsible actions. how to help our young people to discuss issues that need the consideration and integration of a variety of disciplines? how to encourage them to participate in a responsible way? in this study we focused on teaching and learning the nature of energy, and other related concepts, in science from an interdisciplinary point of view. we analyzed contents of secondary education textbooks and curricular guidelines, taking into account their importance as classroom resources, and discussed the level to which controversial issues were displayed, considering a multidisciplinary approach. results highlight a lack of consideration of environmental and social impacts derived from energy production and consumption, and also a lack of attention to interdisciplinary aspects. attention given to renewable energies does not fit the present role of these kinds of energies in society, focusing more on nonrenewable energies. generally texts emphasize traditional energies without regard to the future possibilities. keywords interdisciplinary contents, energy issues, secondary education, textbooks. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 90 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 1. introduction at present, education, whatever the matter from which it addresses its duties, has to take responsibility, and also the challenge, to include comprehensive and updated approaches. education has to face up to this problem now more than ever because its mission is to enable each of us, to develop our talents to the full and to gain consciousness of our creative potential, including responsibility for our own lives and achievement of our personal aims (delors, 1996). that is, education has to offer each person the tools to continue learning throughout life, in the context of a globalized world in constant change. considering the features of the new society and the extent and speed with which the changes are taking place, this line of education must be continuously updated (hodson, 2003; lemke, 2006). these requirements play a major role in scientific and technological education. if our goal is training citizens with skills for decision-making and participation, we must offer our students work contexts where scientific and technological components could be related to many other subjects and situations. those contexts, sometimes, have to do with aspects at the frontier of science, comprising a variety of disciplines (colucci-gray, camino, barbiero and gray, 2006; nordine, krajcik and fortus, 2011). additionally, science education should become more humanized and closer to personal interests in order to provide meaningful educational scenarios. zembylas (2007) emphasized the importance of emotions, which have to be fostered by science’s teachers. edwards, gil, vilches, and praia (2004) call for attention on global situation of the world, and hodson (2003) gaves great importance to some abilities as comparing, selecting or discussing topics, because they affect the health of democracy. he refers to "politicize" the science curriculum, as the analysis of some of the new challenges concerning humanity would train students for responsible actions. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 91 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 holbrook and rannikmäe (2007; 2009), ask themselves about the nature of science education for enhancing scientific literacy and about the meaning of scientific literacy. how to join the interdisciplinary content and values this changing society demand? how to help our young people to discuss issues that need to consider a variety of disciplines? and how promote their social participation? searching for answers to these questions, the teaching approach based on socio-scientific issues, which gives prominence to ethical and moral reasoning, becomes relevant (zeidler, sadler, simmons, and howes, 2005). another relevant approach is what roth and lee (2004) called "citizen science", which incorporates elements of science that are common in both social and personal spheres. similarly, lemke (2001) recognized that the view of science as a single valid path to knowledge, disconnected from the social, political, cultural beliefs and values of society, is as unrealistic as useless in teaching. holbrook and rannikmäe (2007; 2009) proposed less emphasis on specific scientific content knowledge and more emphasis on teaching based on interdisciplinary contexts and socio-scientific decision making. thus, priority is given to opening science teaching to interdisciplinary and social considerations, in order to promote the necessary understanding of science in its social context (roth and lee, 2004). 2. the study in this study we had into account those consideaations in the analysis of proposals of the present spanish curriculum of secondary education (educacion secundaria obligatoria, eso) on energy-related issues, and how these topics are treated in a sample of textbooks for students. the energy is a fundamental and unifying concept in science. given its importance, and how close it is in everyday life, it is one of those concepts students have direct contact with, even before facing teacher explanations (nordine, krajcik & fortus, 2011). martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 92 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 its teaching and learning processes are complex, as it is reflected in the large number of studies that have been addressed over it (solomon, 1983; bliss and ogborn, 1985; kruger, 1990; trumper, 1993 and 1998, liu and mckeough, 2005; domenech gil-perez, gras-marti, guisasola, martinez-torregrosa, salinas and trumper, 2007, lee and liu, 2010, among others). for these authors teaching and learning the energy concept involves the analysis of other essential aspects, related to environmental, political and economic problems. a rigorous treatment of the concept is needed but, indeed, superficial and limited approaches dealing only with some individual aspects are predominant. the problem of production and consumption of energy in our days, related to the need to promote sustainable development for humans and environment involves us and affects everyone in the planet (martin and prieto, 2011). we can find in it a large volume of content and a great deal of controversy. moreover, current energy consumption is one of the most widely used measures of progress and welfare of society, and its steady increase is striking and unquestionable. there are many consequences associated to this problem, as the exploitation of natural resources. we must take decisions about what to investigate or how innovate in science and technology fields, in relation to energy which encompasses finding a resource consumption model that is sustainable for humans and for environment and, in turn, to meet the production demands of the economy in today's society (pedrosa, 2008; nordine, krajcik and fortus, 2011). this problematic area is closely linked to some of the most current environmental problems, especially, those about climate change. controlling greenhouse gases emissions requires reducing demands of energy from fossil fuels in favor of alternative energy sources. it involves and affects us all as inhabitants of the planet. in its analysis and its possible solutions, many factors are protagonist. some can be highlight, as: • social factors associated with concern for the survival of our living’s standards which are linked to depletion of some resources, and the high availability of other martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 93 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 natural resources, that in the case of suitable technological advances, could be exploited at reasonable costs. • environmental factors, linked to the impact that the use of different energy sources, such as fossil fuels or radioactive components, have on the environment. • economic and political factors, linked to the high cost that citizens have to pay for the enjoyment of them, associated, in most cases, to the political interests of certain countries. it is therefore a case where controversy is a key element, as well as the personal and social domain (martin, prieto and jimenez, 2013). this raises the question that, in the current situation of "planetary emergency" (gil and vilches, 2006), the approach to energy issue should respond to different perspectives (ethical, political and cultural), and not only scientists issues (jimenez & sampedro, 2006). pedrosa (2008) and thomas, jennings & lloyd (2008) highlighted the importance of relating school learning of energy to everyday actions, because this will promote understanding about the environmental problems that are involved to obtain it, as well as its consumption. furthermore, with this approach we encourage our students to take interest in science. the education system has a key role in educating about the importance of the energy problem, and it is necessary to cooperate in taking the necessary measures. a relevant aspect to search is the grade of presence that the highliter factors have in the way in which the science curriculum and textbooks address the energy problem. 3. the energy issue in the science curriculum of compulsory secondary education in spain the spanish curriculum of secondary education (mec, 2007 and caa, 2007), determines what students should learn in relation to the energy issue. this curriculum, focuses on skill acquisition, states that the goal of compulsory education is that students martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 94 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 must acquire the necessary tools to understand the world around them, in order to be able to intervene actively and critically in society. according to oecd (2002), the concept of competence is the ability to respond to complex demands and perform different tasks properly, which involves a combination of practical skills, knowledge, motivation, ethical values, attitudes, emotions, and other components and social behaviors. our analysis was focused our on the different sections of the curriculum in which energy issues have presence: competences, skills, aims, content, assessment criteria and teaching methods. our first aim was to determine what skills, related to energy issues, are included in the curriculum proposal to students acquire and develop. we focused on the "competence in knowledge and interaction with the physical world". a royal decree-law (mec, 2007) refers to it as the ability to interact with the physical world, both in its natural aspects and those generated by human action, to enable understanding events, making predictions of consequences and to improve and preserve conditions of life, of other people and other living beings. central elements in this competence are those aspects more related to energy issues, as the responsible use of natural resources, environmental care, rational and responsible consumption and protection of individual and collective health as key elements of the quality of life. in addition, it highlights that the interaction with the physical environment demands an awareness development of the influence and changes that might be produced. addressing energy issues in the classroom can help to achieve certain specific aims which are important for scientific literacy, as, for instance: • to aply basic concepts of natural sciences to explain natural phenomena, as well as to analyze and assess the impact of techno-scientific developments and applications on society. • to adopt knowledge-based critical attitudes to analyze, individually or in groups, scientific and technological issues. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 95 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 • to understand and assess the interactions between science and technology on society and the environment, with particular attention to the problems humanity is facing today. the royal decree-law 1631/2006 (mec, 2007) emphasizes that students should understand problems associated to production, transport and use of energy, as well as the importance of energy saving. the sequence of the above mentiones contents throughout the courses is: second year: it is included the analysis and assessment of different energy sources, renewable and nonrenewable, and also about the problems associated with production, transportation and use of energy, as well as awareness of the importance of energy saving. third year: it is included the study of the consequences that human waste of energy has in the environment, considering the relationships between people and environment. geological changes caused by the external energy. the energy we get from de sun. fourth year: it is included the analysis of different energies in our lives, as well as advantages and disadvantages of different sources of energy. an analysis of the contribution of science to a sustainable future is also proposed, as well as problems and global challenges that humanity faces today: global pollution, climate change, resource depletion, etc. if we do the same in the order of august 10, 2007 (caa, 2007), which develops the secondary education curriculum in andalusia, we observe that it emphasizes the responsible use of natural resources and the current energetic crisis and its possible solutions. we can see how questions are raised in order to describe contents: martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 96 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 natural resources and their responsible use among the relevant issues, directly related with the topic at hand, we find air pollution and other socio-environmental problems. these topics are introduced with the following questions: what are the main causes of air pollution? how does it affect us? is it harmful the greenhouse effect? how could decrease air pollution? what proposals are made worldwide to achieve it? is there any connection between our use of a resource (for instance, fossil fuels or forests) and some problems in our environment (such as global warming and desertification)? the possible solutions to the energy’s crisis related to the energy problem: which of your daily activities need energy? from where do we get that energy? how much energy is available, how much does it cost? how is that energy distributed? what is the energy problem? is there really such a problem? which are the worldwide proposed measures to reduce it? what do scientists and media mean when they refer to global warming? what might happen in the world if an increase on the average temperature would occur? martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 97 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 what countries would be affected mainly by this increase and in which way? what might happen in andalusia? what measures are proposed to address this problem globally? which of these measures are the most appropriate? related to energy savings: what elements could we implement at home to make a better use of solar energy? how could we save energy along its transport? which could be the consequences worldwide of saving energy on issues such as air pollution, noise, etc.? could we help to save energy by changing our habits in terms of everyday consumer goods, modes of transport we use, etc..? how? regarding energy sources: what alternative sources could be used to replace fossil fuels? which are the advantages and disadvantages of these alternative energies? which are the environmental impacts of renewable energy sources? as we analyze the content of our national curriculum (mec, 2007) and the curriculum of the community of andalusia (caa, 2007), we can see that both laws give importance, as content to be treated, to the energy conservation, analysis and evaluation of using different energy sources and their environmental consequences. in both cases there are recommendations to deal with the energy problems that society is facing today, and the possible solutions that must be developed to alleviate the situation. in relation to teaching methods, the andalusian curriculum (caa, 2007, blocks 4 and 5), recommends to develop the content around solving energy problems. explanations of martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 98 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 teachers must originate reflection on how we use natural resources more connected to us, and get on with the analysis of the consequences arising from such use, both individually and locally. it expresses that they should end with a reasoned assessment of the solutions that may be applied in relation to that issue. suggestions are made in order to start with those content that is more related to the world which is directly perceived (activities and everyday situations), and move to the study of more complex phenomena as students are progressing. concerning assessment, both laws (mec, 2007; caa, 2007), recommend to evaluate the capabilities that were developed to recognize problems related to the energy crisis, to analyze and evaluate information from different sources, and to evaluate proposals for energy savings that society is posing. they also propose the assessment of knowledge and level of awareness of the fact of abuse being made on various natural sources, as well as creativity and appropriateness of the proposals made in relation to the energy problem and the responsible use of natural resources. in general, this analysis shows how the curriculum and, therefore, the laws that are governing education in spain, whether at the national or regional level, are well aligned with the teaching approach based on socio-scientific issues. in particular, these considerations demonstrate the importance given to the treatment of energy issues through an approach that highlight the most controversial aspects that concur therein, as are the pros and cons associated to the use of energy, or the social implications that derive from them (pedrosa, 2008; fernandez, 2010). 4. research questions since a majority of teachers usually use textbooks as major resource, it is important to know how curriculum recommendations are reflected in them, and especially whether or not the contents related to the most controversial issues are dealt from a multidisciplinary approach. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 99 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 the research has been guided by the following questions: in the declarative content of the topics that are dedicated to energy and its different aspects: • what presence do have the interdisciplinary aspects related to the issue of energy in the world today? • what presence does have the impact that energy uses have in the environment and the society? 5. data analysis and results a sample of 36 textbooks from nine publishers with great presence in andalusian schools was selected. the texts, 4 per publisher, correspond to the subject of natural sciences for 1st and 2nd eso, and physics and chemistry, of 3rd and 4th year, addressed to pupil from 13 to 16 years old. data analysis began with a selection of chapters and sections in which energy, sustainability and renewable energy had a presence. usually, these concepts appear in the context of issues devoted to the study of energy. the analysis was conducted by looking for, in the different parts in the declarative content of these chapters, the interdisciplinary aspects on the energy issue that are emphasized by the curriculum. the search focused on content related to the kind of use made of energy from different sources and the social consequences of it. proceeding in this way, we configured, in a flexible manner, a set of categories within them (bliss, monk and ogborn, 1983). uses associated with different energy sources descriptions and explanations about the use of energy from different sources in two different contexts, the industrial and domestic/commercial, were collected and organized. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 100 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 in the industrial context, we have considered explanations about the use of energy that has been produced from large facilities, referring to the type of font, and how the process has been performed. in this way, eight categories were defined, which allude to the different uses depending on the facilities of origin (wind farms, hydroelectric plants, solar plants, geothermal plants, tidal energy plants, biomass plants, nuclear and thermal power plants). in the domestic/commercial context, we collected explanations about the uses of energy from different sources, in everyday activities. as in the previous case, a subdivision was performed, but now, we considered the origin of the energy source (sun, coal, natural gas, oil and radioactive materials). from our data, we detected a high interest on the uses given to energies from large industries in comparison to renewable energy sources. it highlights a tendency to emphasise energy production of wind farms, hydroelectric and solar plants. in contrast, plants that use energy sources from fossil fuels and radioactive materials were less considered. attention that textbooks pay to the energy problem in the industrial context is even smaller. as in the previous case, less attention is paid to aspects related to renewable energies. as a positive aspect, we find out that three publishers collect at least three of the categories considered, and they do it in courses where the subject is compulsory for every student (1st and 2nd year of eso). social consequences they refer to contents that are related to the consequences that are generated by the consumption of energy from the different sources. we considered three kinds: environmental, economic and political. for all of them, we distinguished between nonrenewable and renewable energy sources. data showed that implications which are more directly related to the impact on the natural environment have a greater presence than those associated with the social aspect, both economic and political. having a look in detail, we saw that in relation to the martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 101 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 environmental consequences of non-renewable energy sources, all publishers included, in some of their courses, acid rain, greenhouse effect and production of waste radioactive nuclear plants, as adverse effects. only three of them allude to contamination produced by released products that can be generated in its production and use. regarding renewable energy sources, all texts emphasized absence of polluting effects as advantages, and most of them alluded to disruption in ecosystems and visual impact as disadvantages. with regard to their presence in each of the courses of the eso, the aspects that have been addressed in any case are included in more than two courses. it is of concern to see how other aspects are poorly treated and, when they are, in most of the cases, appear only in the texts of the second cycle. most publishers refer to a number of aspects between 4 and 6. they all present the trend to emphasize the advantages of nonrenewable energy and the drawbacks of renewable, from the economic point of view. the political consequences arising from the use of one or another energy source are reflected to a lesser extent than the economic and environmental consequences. the political aspect that received greater attention (it appears in 5 of the 9 book publishers) is the dependency that exists between those nations possessing energy sources and those without them. in the development of this aspect, emphasis is on the difficulties that may arise in a country where the power supply is dependent from another country, and therefore, depending on its economic and political relationship. only one of the publishers includes the 4 aspects, in a differentiated way, and even 2 of them on 2 years of eso. we could say that this publisher shows a different trend from the remaining 8 in terms of attention given to the political implications. it is also notable for being one of those that address a wider range of economic implications. with regard to the other publishers, three of them did not consider this type of consequences and three did so only on one aspect and in a single course. it is also martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 102 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 interesting to note how, in the total sample of textbooks, only four publishers included at least one political aspect in their texts (1st and 2nd). 6. final thoughts our results show a lack of attention to interdisciplinary aspects, which may provide us the learning opportunities we are looking for. we also find that the implications of economic and political aspects of the problem appear in a lesser extent than environmental implications. moreover, our results also show that attention and space given to nonrenewable energy in some key aspects, such as production aspects and implications, are greater than that devoted to renewable energy. in this respect, we see that the texts emphasize traditional possibilities and not future possibilities, what leads us to conclude that the focus on the problem that curriculum presents is not reflected in the content of the texts we analyzed. edwards, gil, vilches & praia (2004), in the same line of reasoning, argued that textbooks should make greater use of the context of real problems that currently humanity faces. we consider that, around the current energy problems, it is more likely we can promote awareness about the importance of energy in our lifestyle by promoting awareness of how far we depend on it. our findings are also in accordance with those of garcia-carmona (2008), who argued that concepts, laws, rules and techniques appear as first order, compared to technology and society aspects, which appear as 2nd order. we also highlighted a lack of consideration to environmental and, specially, social impacts, and their influence on political and administrative decisions on the uses and applications of electronics. the most controversial aspects of the problem, as well as its interdisciplinary nature, cannot be overlooked in a rigorous treatment of it, as they also contribute to the development of attitudes and values. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 103 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 considering our results as starting point, we agree with the proposal made by coluccigray, camino, barbiero & gray (2006), who emphasize the relevance of spending time and effort in the classroom on working issues such as: a) the concern for sustainability and savings in energy consumption, b) the impact of energy in our daily lives, c) our responses: kind of problems and kind of solutions, d) energy consuming activities, problems arising, etc. and we also agree with lemke (2006) when he says that one of the contributions that science education can make to students and society in the xxi century is a science education with a focus on concerns and social issues of students, in order of getting arouse their emotional engagement with science learning. we strongly believe that this goal is a clue in order to achieve these new educational aims. some contributions to teaching practice. considering the importance that have the issues and situations that lead students to understand the links between consumption of certain energy products and the necessary resources to get the energy, as well as the advantages and disadvantages of the modes of production and consumption of energy from different sources, we think that they represent useful instruments in the development of the personal responsibility, and so they would receive more protagonism in the science activities included in teaching and learning of energy. the same argument is valid for the different consequences in the environment and society. the interdisciplinary nature of the analysis of these problems has potential in promoting a meaningful understanding, through the use of different variables that play a role on them. our data show that many of them are frequently poorly developed or simply not included. we consider it would be advisable to treat the energy aspect in more courses, due to the importance of the problem and because it can help to understand many of the real problems that exist in society. the way that many of the texts analyze these contents may hamper educational developmentin students, if previously we do not promote the martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 104 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 motivation to deal with the problem and, hence, to commit their participation in the search for solutions (pedrosa, 2008). in our opinion, treatment of some real issues about energy in the science classroom should be considered together with an analysis of the society itself, and this includes economic and political aspects, which are privileged agents in decisions making. limitations regarding the research this piece of research has been centered on the interdisciplinary nature of the energy problem and the way it is reflected in the science curriculum of spanish compulsory secondary education and in a sample of texts books. we have focused the study on the declarative content of the different chapters in search of the interdisciplinary aspects on the energy issue that are emphasized by the curriculum. other aspects, for instance the analisys of concept and processes involved in the activities that books propose, as wel as the kind of activities themselves, has not been included. we are also aware that, in order to promote interdisciplinarity in the treatment of topics like this one of energy, we have to bring with us the science teachers’ complicity. we need they open their mind in order to accept that, today, they cannot rely on them and consider they are doing enough, and recognize that what teachers need is not always included in textbooks. we are going to keep this research in the way of trying to promote and to stimulate teacher’s willingness to apply these approaches and to open their perspectives from pure science to the interdisciplinary interactions, as well as their ethical, political and moral components which are involved in many of the problems that we, as humans, have to face in the twenty-first century. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 105 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 7. references bliss, j., monk, m. & ogborn, j. (1983). qualitative data analysis for educational research. london: croom-helm. bliss, j. & ogborn, j. (1985). children’s choices of uses of energy. european journal of science education, 7(2), 195-203. doi: 10.1080/0140528850070210 caa (2007). orden de agosto de 2007, por la que se desarrolla el currículo correspondiente a la educación secundaria obligatoria en andalucía. (boja 5 de enero de 2007). colucci-gray, l., camino, e., barbiero g., & gray, d. (2006). from scientific literacy to sustainability literacy: an ecological framework for education. science education, 90, 227-252. doi: 10.1002/sce.20109 delors, j. (1996). la educación encierra un tesoro. méxico: ediciones unesco. domenech, j. l., gil-perez, d., gras-martí, a., guisasola, j., martıneztorregrosa, j., salinas, j. & trumper, r. (2007). teaching of energy issues: a debate proposal for a global reorientation. science and education, 16, 43-64. doi: 10.1007/s11191-005-5036-3 edwards, m., gil, d., vilches, a. & praia, j. (2004). la atención a la situación del mundo en la educación científica. enseñanza de las ciencias, 22 (1), 47-64. fernandez, m. a. (2010). o ensino das enerxias renovabeis factor clave na educación enerxética. boletín das ciencias, 69, 103-115. garcía-carmona, a. (2008). relaciones cts en la educación científica básica. i. un análisis desde los textos escolares en la enseñanza electrónica. enseñanza de las ciencias, 26(3), 375–388. gil, d. & vilches, a. (2006). algunos obstáculos e incomprensiones en torno a la sostenibilidad. revista eureka sobre enseñanza y divulgación de la ciencia, 3(3), 507516. martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 106 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 hodson, d. (2003). time for action: science education for an aternative future. international journal of science education, 25(6), 645-670. doi: 10.1080/09500690305021 holbrook, j. & rannikmäe, m. (2007). nature of science education for enhancing scientific literacy. international journal of science education, 29, 1347 1362. doi: 10.1080/09500690601007549 holbrook, j. & rannikmäe, m. (2009). the meaning of scientific literacy. international journal of environmental and science education, 4 (3), 275-288. jiménez, j. d. & sampedro, c. (2006). ¿son las energías alternativas la solución del futuro? alambique, 49, 71-80. kruger, c. (1990). some primary teachers’ ideas about energy. physics education, 25, 8691. doi: 10.1088/0031-9120/25/2/002 lee, h. s. & liu, o. l. (2010). assessing learning progression of energy concepts across middle school grades: the knowledge integration perspective. science education, 94, 665-688. doi: 10.1002/sce.20382 lemke, j. l. (2001). articulating communities: sociocultural perspectives on science education. journal of research in science teaching, 38(3), 296-316. doi: 10.1002/10982736(200103)38:3<296::aid-tea1007>3.0.co;2-r lemke, j. l. (2006). investigar para el futuro de la educación científica: nuevas formas de aprender, nuevas formas de vivir. enseñanza de las ciencias, 24(1), 5–12. liu, x. & mckeough, a. (2005). developmental growth in students’ concept of energy: analysis of selected items from the timss database. journal of research in science teaching, 42(5), 493 517. doi: 10.1002/tea.20060 martín, c. & prieto, t. (2011). el potencial educativo del problema energético en la sociedad actual. en j. j. maquilón, a. b. mirete, a. escarbajal & a. m. giménez (eds.), martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 107 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 cambios educativos y formativos para el desarrollo humano y sostenible (pp. 29-38). murcia: edit.um. martín, c., prieto, t. & jiménez, m.a. (2013). el problema de la producción y el consumo de energía: ¿como es tratado en los libros de texto de educación secundaria? enseñanza de las ciencias, 31(2), 153-171. mec. (2007). real decreto 1631/2006 de 29 de diciembre por el que se establecen las enseñanzas mínimas correspondientes a la educación secundaria obligatoria (boe 5 de enero de 2007). nordine, j., krajcik, j. & fortus, d. (2011). transforming energy instruction in middle schools to support integrated understanding and future learning. science education, 95, 670-699. doi: 10.1002/sce.20423 oecd (2002). definition and selection of competences (deseco): theoretical and conceptual foundations. retrieved december 21, 2010 in http://www.deseco.admin.ch/bfs/deseco/en/index/02.html. pedrosa, m. a. (2008). metas de desenvolvimiento do milenio e competenciasenergía e recursos energéticos em educaçao científica para todos. comunicación presentada en el xxi congreso de enciga, o carballiño, orense. roth, w. m. y lee, s. (2004). science education as/for participation in the community. science education, 88, 263–291. doi: 10.1002/sce.10113 solomon, j. (1983). messy, contradictory, and obstinately persistent: a study of children’s out-of-school ideas about energy. school science review, 65(231), 225 -233. thomas, c., jennings, p. & lloyd, b. (2008). issues in renewable energy education. australian journal of environmental education, 24, 67-73. trumper, r. (1993). children’s energy concepts: a cross-age study. international journal of science education, 5(2), 139-148. doi: 10.1080/0950069930150203 martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 108 http://www.deseco.admin.ch/bfs/deseco/en/index/02.html multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2264 social and technological sciences eissn: 2341-2593 trumper, r. (1998). a longitudinal study of physics students’ conceptions of energy in pre-service training for high school teachers. journal of science education and technology, 7(4), 311-317. doi: 10.1023/a:1021867108330 zeidler, d. l., sadler, t. d., simmons, m. l. & howes, e. v. (2005). a research based framework for socio-scientific issues education. science education, 89(3), 357–377. doi: 10.1002/sce.20048 zembylas, m. (2007). emotional ecology: the intersection of emotional knowledge and pedagogical content knowledge in teaching. teaching and teacher education, 23, 355– 367. doi: 10.1016/j.tate.2006.12.002 martín et al. (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 90-109 | 109 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 30 mentoring program for students newly enrolled in an engineering degree juan pedro peña martín*, eva gonzález-parada, carmen garcía-berdonés and eduardo javier pérez rodríguez dpto. de tecnología electrónica. universidad de málaga, e.t.s.i. telecomunicación, boulevar louis pasteur nº 35 málaga, spain. * corresponding author: email: jppena@uma.es; phone: + 34 952132732 received: 2016-06-01; accepted: 2016-08-15 abstract this work presents a mentoring program for first year engineering students in the telecommunications engineering college (etsit) at the university of malaga (uma). actors involved in the program are professors from staff, veterans mentoring students and, of course, freshmen. all of them has been organized trough the moodle based virtual learning environment platform of the uma. the program has gone through several phases over three years. this paper shows the main objectives of this mentoring program, the initial design to get them where professors played mentor role, and successive changes made to try to improve the results, including the assumption of the mentor role by senior students (peer mentoring). the tools used for program evaluation are shown too. despite the low participation, it has been a framework for the development of various educational and socializing activities (for mentors and mentees) focused on developing generic competences. furthermore, it has been a research tool to get a better understanding of problems affecting students newly enrolled. keywords advising program, mentoring program, european higher education area, engineering degree, peer mentoring http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 31 1. introduction the european higher education area (ehea) involves the adoption of new methods of teaching and learning, as well as a new approach to mentoring (pallisera, 2010). the proposed mentoring program offers a help and a guidance in specific academic issues, as well as a framework to support to students from joining the university. therefore, it is a framework for the teaching-learning process. these general guidelines set by the european commission have been translated into spanish law in different ways. firstly, rd-1393(2007), as amended by rd-861 (2010), included the "student orientation", asking to the bachelor's degree "support and guidance systems for students". secondly, rd-1791 (2010) provides in chapter v that the general principles of the mentoring systems integrate, in a coordinated manner, the actions of information, guidance and training support to students, developed by teachers and specialized staff. this approach distinguishes between general mentoring and subject mentoring. the first one, referred to by experts as integral mentoring, considers not only instructive aspects but also social, personal and emotional aspects. a mentoring program is defined as the design and development of support mechanisms, information, guidance and training to students, in response to the needs identified in the academic, administrative and social-professional fields. a possible approach to implement this integral mentoring is to use peer mentoring. several spanish universities have been performing mentoring projects for more than a decade. in spain, the origin of those activities is the simus project (valverde, 2001), sponsored by the university of seville. other universities (polytechnic university of madrid, university of burgos, university of las palmas, etc.) subsequently joined this initiative. engineering degrees has often been scenarios of this kind of programs, as in our case. engineering degrees in public universities have high dropout and failure in the first http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 32 year (mecd, 2014), so it seems reasonable to focus the mentoring towards students newly enrolled in an engineering degree, both freshmen and students of international exchange. moreover, and following ehea (enqa, 2015), the verified memories of the bachelor's degree collect a methodology for managing and improving the teaching-learning process, which is included under the so-called quality assurance system (qas). within the qas implemented by the telecommunications engineering college (etsit) of the malaga university (uma), the key process pc05 "guidance to students" establishes the way in which the college revised, updated and improved procedures relating to host-driven actions, mentoring actions and support actions for training and orientation of their students. for all these reasons, our college started in the year 2012 a mentoring program (henceforth called pat from the spanish expression ‘plan de acción tutorial’), under the qas-pc05. the first pat (version 0, v0) has continued the following two courses (v1 and v2) up to date, although it has been annually reviewed in a continuous improvement process. this paper will present the design of the pat v0 and following changes in v1 and v2. an orientation program must be adapted to the characteristics and context of the university and college where it is to be implanted. however, you can set a general structure and working methods from which the action will be organized. therefore, first step to carry out our pat was to analyze diverse experiences of different universities in order to get the pat´s structure and methodology. among others, we took as reference some similar active programs as buddy-program (ugr, 2015), program-orient (uclm, 2015), patmentor (ubu, 2015), and gou (uma, 2015). the last one, an experience conducted by colleagues from university of malaga, has been a particularly important support to our mentoring program. the organization of this paper is as follows. after the introduction, the following section presents the general lines and objectives of the pat, which constitute its essence beyond the necessary adjustments. the development and results of the first year and the pat http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 33 redesign, implementation and results of the pat v1 and v2, will be the target of the next two sections. finally, conclusions and future lines of this work will be presented. 2. outline of the pat in etsit as stated in the previous section, we considered developing actions to improve guidance of students, primarily of freshmen. so, the general lines of intervention (goals) will focus on: a) facilitate the transition of new university students from undergraduate education, including orientation towards more efficient working methods. b) detecting and coping with the most relevant typical problems that arise newly enrolled students. c) facilitate the integration of students in the institution, that is, make an orientation, in addition to academic, administrative and social. figure 1. overall organizational structure of the pat. relating to the design, it was initially left open to develop the guidance through teaching staff (classic style) or peer mentors, students of higher levels supervised by a teacher, whose http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 34 objective is the guidance and advice of a group of freshmen (mentees) to achieve the above objectives. the second case involves an additional objective related to training of mentors in order to enhance their social skills (relationships and leadership), that is, mentors become both collaborators and beneficiaries. the overall organizational structure that we assume is presented in figure 1, where main actors involved in the pat are shown. the functions of each one are briefly shown in table 1. table 1. functions of the main actors in the program management team head of the pat qac -quality assurance commission it approves and evaluates all improvement actions, such as the pat, and develops the qas pat coordinator he is responsible for the organization, coordination and development mentor (volunteers) they develop the actions specified in the pat on a group of students mentees mentees they receive the pat’s activities. they are first-year students who volunteer to participate secretary and library staff (pas) they collaborate to develop guidance activities in their respective areas virtual campus it is a virtual learning environment where you can neatly manage the interaction between different users, and the exchange of documentation and materials involving the pat; it was created as a course in moodle, with different sections and use privileges for each of the actors, with private communication forums groups, with wikis for compilation of experiences and with questionnaires for evaluation purposes http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 35 figure 2 shows a partial view of how is organized the moodle based virtual learning environment (virtual campus, vc) in the pat v21. on the other hand, the general working methodology is presented in figure 3. as it is shown, a guide to advice and to collect information (hegae) was performed, and a tool to evaluate the pat itself (heep) was envisaged. figure 2. virtual campus pat v02. hegae, is a mentor’ guide. the guide summarizes all the possible key aspects for successful integration of junior student, so, it would help mentors, while they are talking 1 we must highlight the difficulty of the various roles of the participants: a) faculty coordinator with full access as managers; b) collaborating lecturers without edit permission; c) mentors, with restricted areas and resources for intercom use; d) pat students; constituted by young students enrolled in the pat and therefore members of a steering group led by a mentor, who also had an own communication space, e) students in general which included, in addition to the above groups, all novice students. most of the documentary resources generated in the activities had just placed at the disposal of this large group. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 36 with the pat students, to get information about whether these aspects were really key for students (it was also a socializing tool to facilitate the conversation and to identify topics of common interest). in addition, it was intended to gather information that could identify potential problems on their integration into this college, so, that could be the seed of improvement actions within the qac. the table 2 summarizes the issues included in the hegae and related pat objectives. note that it has two different blocks, one for each semester, because novel students evolve rapidly during their first months in the college. regarding the assessment tool for the pat (heep), its main objective is to gather information about the level of satisfaction of the participants in program, mentor and mentee. figure 3. pat general working methodology. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 37 the design of both, the hegae and the heep, has been changing in new versions of the pat. henceforth, we will show only the results of the issues highlighted in italic in table 2, which are directly related to the work presented here. table 2. hegae design and goals. issues related to … goal first semester input profile (type of degree, option of choice) to be treated by the qac expectations (motivation, prospects for success, coping strategies planned) advising / to be treated by the qac level of knowledge of the degree itself advising / check need for advice level of knowledge of resources (rules of residence and registration, library and virtual campus) advising / check need for advice lack of study forecast for intrinsic reasons (work, family problems ...) advising second semester problems with the pre-university training and experience with the zero course to be treated by the qac degree of difficulty for each subject and experience with continuous assessment to be treated by the qac coping strategies (time spent studying, methodology, attendance, tutorials) advising / check need for advice results (reflection on the results and their relation to strategy) and overall satisfaction with the course advising / to be treated by the qac 3. design, implementation and results of the pat v0 (year 2012-13) 3.1. final design of the tools in the pat v0, we marked a further objective: improve the communication between teachers and student, which traditionally came being reported as unsatisfactory. therefore, the choice of student-mentors was postponed and the pat v0 stage responds to figure 1 with the role of mentor played by teachers. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 38 regarding the hegae, we chose to materialize it in two ways. first, mentees were asked to complete a vc questionnaire. after, an interview, made for mentors to mentees, allows completing the information gathered by the questionnaire. for this interview, it was generated a guide (semi-structured interview) with the philosophy explained in the previous section. finally, interview reports was collected (without identifying the students2) through virtual campus in order to be analized by the pat coordinators and the qac. on the other hand, the heep was materialized in a vc questionnaire, asking information on user satisfaction, on compliance with the additional goal of the pat v0 and on the possible use of peer mentoring in future editions. 3.2. results and analysis for redesign table 3 shows the level of participation of students and mentors during the first year. the initial turnout was about 20% of 250 new students in total. table 3. participation data. initial participation student participation in hegae participation in heep students mentors survey 1 interview 1 survey 2 interview 2 students mentors 53 23 52 39 34 19 8 23 table 3 is only for mented students. participation data for other students (new students but non-participant in the program) were: 44 (first semester) and 22 (second semester) in the hegae, and 3 in the heep. the provided data for both groups scarcely showed significant 2 from the outset, it was requires to the tutors a confidentiality agreement and transmit the information without identifying t he source to give confidence to the students. so were transmitted to them too. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 39 differences, so we decided not distinguish between them in later editions. as it is shown, the participation of students was declining as the year progressed3. table 4 shows the relevant global data from the first survey, while table 5 gives specific information about students enrolled in the pat. table 6 shows the data collected by heep. table 4. global data collected with hegae. level of knowledge of the degree itself 33% have a clear idea of the degree to which they have enrolled, the rest reported substantial doubt part-time regulations it is known by 16% minimal credit requirements regulations it is known by 28% regulations about maximal permanence it is known by 22% registration fee it is known by 28% table 5. data collected with hegae about students in the pat. level of knowledge of resources (library and virtual campus) 64% of students considered to have sufficient knowledge and 4% depth of both resources. teacher comments reflect that the library is the most unknown among both resource. coping strategies (time spent studying, study methodology, attendance, tutorial services) there are large spread among the interviewees. the main problems reported are lack of use of mentor services and the lack of organization in the study. it seems to have enough hours of study but not well distributed throughout the semester or dedicated especially to matters they love. conclusion: the need for better information of the regulations that affect them is appreciated. also it was decided, for subsequent editions, introducing some changes as 3 basically, mented students did not respond to tutors e-mails, although it was occasionally reported the imposibility to close an appointment because of scheduling problems between both parts. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 40 incorporating secretary and library staff or providing to students some study skills workshops. table 6 data collected with heep. compliance with the target studentteacher approach 62% of teachers believes that it has reached a high or very high level. 62% of the students understand a high level, 0% very high. advisability of introducing mentor students 85% of teachers understand it would be worthwhile 100% of students understand that it would be worthwhile overall satisfaction 3,8/5 teachers appreciation 3,5/5 students appreciation pat utility for students 100% rating of teachers (very useful). not asked to students broadcasting 100% of students not participating in the pat says he has not heard of it conclusion: we decided to evolve into mentors students, despite losing in terms of teacherstudent relationship, on the other hand, it has not been fully satisfactory (62%). 4. redesign, implementation and results of the pat v1 and v2 (years 2013-14 and 2014-15) the main novelty of these two versions is that the scenario to develop the pat responds to figure 1 but with the mentor's role played by veteran student volunteers instead of professors. the participation of these students had incentives: a) ects, because orientation activities that are covered by the uma (up to 2 per year); b) specific training for their role of guidance and mentoring, that is, develop of generic skills which are projectable to their professional future (requires external monitors, such as psychologists and educators) http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 41 c) certainly, and in view of their attitudes and results, the main incentive was simply their willingness to help peers who are going through what they are gone before. therefore, a dual training scheme was planned: on the one hand, for mentors, enhancing the skills expected of them; and on the other hand, for mentees, helping them to acquire some generic skills that can facilitate adaptation to overcome the university, but without forgetting aid for socialization, aspect often neglected in engineering colleges4. in addition, the participation of two members of secretary and library staff, has enabled the development and provision of two students guides in faq format ("frequently asked questions") of such services. finally, the tools hegae and heep were gradually simplified, transferring part of questionnaires towards the below indicated semi-structured interviews. 4.1 redesign of the tools regarding the hegae, and aware that students currently have excess prospective surveys, each year we have tried to simplify and reduce them. in the pat v2, we just keep the semistructured interviews (one per semester), and we have released quite content of mentor´s guide. we think this simplified guide continues to fulfill its dual mission of informing freshmen of little known interesting issues and gathering information on their specific difficulties. the delivery of the information collected by mentors through virtual campus, remains as a control that the mentor was properly performed his duties. as there is a relatively low percentage of students involved in the pat and also many of them leave during the year, we developed a specific questionnaire to find out the reasons (heep), addressed to all novice students. it asked about the students expectations regarding the pat. if they have been part of the program, it asked too about relations with their 4 there have been seminars and workshops on coping with exams, time management and study skills for different subjects, and there are several plans for future editions. in the pat v0 only fit a small workshop by time limitation. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 42 mentors. to improve participation in this survey -the first year was held on virtual campus and was very low-, the second year it was decided to spend class time students. we desisted from repeating it in the pat v2 because student’s petitions seem clear from the information gathered by heep in the pat v1. table 7. participation data; v1 and v2. initial participation participation mentors / mentees in hegae participation in heep mentees mentors survey 1 interview 1 survey 2 interview 2 mentees mentors p a t v 1 92 16 -/78 7/11 with second interview 5/7 159 with second report p a t v 2 112 24 with first interview 8/18 with second interview 4/7 not performed with second report 4.2 results table 7 shows the level of participation of students and mentors along the two years. the initial participation of new students is about 25% in both editions. regarding the data collected with hegae on the pat students, they are shown in table 8. table 8. data on pat students (with hegae). level of knowledge of resources (library and virtual campus) librarylevel of knowledge reported v1 4,0/5 - v2 4,2/5 virtual campus level of knowledge reported v1 4,0/5 - v2 4,3/5 coping strategies no data was collected during the interviews http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 43 conclusion: the data collection by the mentors is poorer than the one made by teachers in v0. it is proposed to associate faculty to the mentor-mentee groups in following editions to get better guidance or to let teachers themselves collect information on certain issues. hegae data collected on all students are very similar to those of v0. the data contained in the mentors reports reflect high degree of satisfaction with the received training (all of them mentioned it) and the desire to continue as pat mentor. also they made recommendations for improvement for future editions. the findings are summarized in starting the activities earlier5 and carrying out some joint recreational activity to introduce mentor-mentees team. last one action will be proposed to the v3. a summary of the data collected on all students (heep), is shown in table 9. the most relevant data were selected. table 9. data on all first year students (heep). survey completed on schedule class time in mid-course. student expectations regarding the pat only 65% read all e-mails they receive from the virtual campus. 50% of those who signed up for the pat declare that they have not followed the activity later and another 25% say they have not had occasion to contact. 40% left because they thought he was going to lose time to other things. relationships with mentors most students gave more value to more advice on specific curricular subjects, followed distantly by global information titling and moral support. to a lesser extent, they ask a teacher mentor directly as well as transversal and specific workshops. all of them declare their intention to continue the pat. perspective of those who do not participate 56% claims guidance from someone who had overcome the same difficulties in the subjects they (contradictory). 38% demand workshops on how to study different subjects. 31% demand an external control of working time 5 in the v1 we could not meet the team of mentors to 2 months after the start of the course, which was partially remedied in the v2 to select that team at the end of the previous year. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 44 conclusions: there was a communication problem with the email (in later editions, phone number was additionally asked). the high percentage of students not enrolled, who ask for something very similar to what the pat is, revealed that the communication problem is really deep. the technical problems of curricular subjects outweigh any other concerns and students fail to perceive the benefit of transferable skills, which clearly constitutes a future line of work. 4.3 planning the pat v2 figure 4 shows the schedule of activities in the pat v2. each activity involves a significant workload in what, for brevity, we cannot enter. figure 4. summary and approximate annual schedule tasks. 4.4 process outputs a process of size and complexity described here has multiple outputs / results that are summarized in table 10. http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 45 table 10. main process outputs. process outputs (documents and materials) topics of interest for new students, as documentation of the workshops and seminars organized for them or other guidance documents that can provide important information (e.g. guides using the services of secretary and library). training subjects for mentors: allow the formation on the functions of guidance, mentoring and evaluation, as well as action strategies with peers. materials for the evaluation and monitoring of the activity (interview guides, questionnaires, etc.) the generated virtual learning environment is in itself a valuable technology resource, which mostly represents work done for the future. 5. conclusions and future lines mentoring program that we have conducted over the last three years was conceived as a process of continuous improvement, which has been keeping its achievements and modifying the parameters that have been problematic. in table 11 are shown schematically conclusions as detected, by one side benefits and on the other problems and challenges for the future. we must highlight our excellent experience with veteran student as mentors but also, clearly, we have failed to convey the message of the group of new students. in addition, we say that we agree fully with (zabalza, 2013) about "the importance of mentoring, guidance and support to students are called to be the new university of the xxi century". therefore, it has already started the pat v3 with new actions aimed at first courses students. table 11. conclusions. benefits problems and challenges  we have obtained high actual knowledge of the thought and feeling of new students.  the decline of the participation requires retention activities (under study). http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 46  activities, despite poor attendance, have been highly regarded, so we expect better future.  in the group of mentors abandonment was much less (50%), high motivation, success of the training course and curriculum improvement in their social skills.  dynamic designed and can be expected to have lower labor cost on these courses.  also for improving retention and because it is demanded, we must plan collective activities at very early stages of the year.  the priority of approving outweighs everything else, but we do not manage to associate these activities with an indirect benefit on their exams success (under study).  program coordination has cost too many hours, which cannot be maintained over time. the challenge is to automate the process where possible.  increase training offered by the project (note that the workshops / seminars are opened to all new students). 6. acknowledgement to all those who have been mentors of the pat, teachers and students, and the contributor staff, for their dedication and enthusiasm. this project was partially funded by the university of malaga through the educational innovation project (pie13-046). http://dx.doi.org/10.4995/muse.2016.4609 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 47 7. references enqa (european association for quality assurance in higher education) (2015) standards and guidelines for quality assurance in the european higher education area. http://www.enqa.eu/index.php/home/esg/. accessed: 24 january 2016. mecd (ministerio de educación, cultura y deporte) (2014). datos básicos del sistema universitario español. curso 2013/14. http://www.mecd.gob.es/dms/mecd/educacionmecd/areas-educacion/universidades/estadisticas-informes/datoscifras/datos_cifras_13_14.pdf. accessed: 24 january 2016. pallisera, m., fullana, j., planas, a., y del valle, a. (2010). la adaptación al espacio europeo de educación superior en españa los cambios/retos que implica la enseñanza basada en competencias y orientaciones para responder a ellos, revista iberoamericana de educación 52(4). real decreto 1393/2007, de 29 de octubre, por el que se establece la ordenación de las enseñanzas universitarias oficiales. boe núm. 260, de 30/10/2007. real decreto 1791/2010, de 30 de diciembre, por el que se aprueba el estatuto del estudiante universitario. boe núm. 318, de 31/12/2010. real decreto 861/2010, de 2 de julio, por el que se modifica el real decreto 1393/2007, de 29 de octubre, por el que se establece la ordenación de las enseñanzas universitarias oficiales. boe núm. 161, de 3/7/2010. ubu (university of burgos) pat-mentor (orientación y tutoría de apoyo) (2015) http://wwww.ubu.es/servicio-de-informacion-y-extension-universitaria/servicios-unidad-deinformacion/orientacion-y-tutoria-de-apoyo. accessed: 24 january 2016. uclm (universidad de castilla la mancha) (2015) programa orient@esii. http://www.esiiab.uclm.es/news.php?codnoti=334. accessed: 24 january 2016 http://dx.doi.org/10.4995/muse.2016.4609 http://www.enqa.eu/index.php/home/esg/ http://www.mecd.gob.es/dms/mecd/educacion-mecd/areas-educacion/universidades/estadisticas-informes/datos-cifras/datos_cifras_13_14.pdf http://www.mecd.gob.es/dms/mecd/educacion-mecd/areas-educacion/universidades/estadisticas-informes/datos-cifras/datos_cifras_13_14.pdf http://www.mecd.gob.es/dms/mecd/educacion-mecd/areas-educacion/universidades/estadisticas-informes/datos-cifras/datos_cifras_13_14.pdf http://wwww.ubu.es/servicio-de-informacion-y-extension-universitaria/servicios-unidad-de-informacion/orientacion-y-tutoria-de-apoyo http://wwww.ubu.es/servicio-de-informacion-y-extension-universitaria/servicios-unidad-de-informacion/orientacion-y-tutoria-de-apoyo http://www.esiiab.uclm.es/news.php?codnoti=334 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4609 social and technological sciences eissn: 2341-2593 peña et al. (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 30-48 | 48 uma (university of málaga) grupo de orientación universitaria (gou) (2015). http://www.webgou.uma.es/. accessed: 24 january 2016 ugr (university of granada) (2015) buddy programme. http://internacional.ugr.es/pages/perfiles/estudiantes/programa-mentor-de-la-ugr. accessed: 24 january 2016 valverde, a., garcía, e. y romero, s. (2001). desarrollo de un sistema de alumnos mentores en la universidad de sevilla. aportación al simposio de investigación sobre educación universitaria, x congreso nacional de modelos de investigación educativa. revista de investigación educativa 19(2), 626-627. zabalza, m. (2013). ser docente es más que ser enseñante. revista de docencia universitaria 11(2), 11-13. http://dx.doi.org/10.4995/muse.2016.4609 http://internacional.ugr.es/pages/perfiles/estudiantes/programa-mentor-de-la-ugr microsoft word 3_garcia_garcia_et_al multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 37 http://polipapers.upv.es/index.php/muse/ social live streaming tools for the development of virtual workshops c. garcía-garcía*, j. galán, r. izquierdo dpto. de ingeniería de sistemas industriales y diseño. universitat jaume i de castellón. av. de vicent sos baynat s/n, 12071 castellón de la plana, spain. *corresponding author: email: cgarciag@uji.es: phone: +34 964 72 8190 received: 2016-01-06; accepted: 2016-09-18 abstract the increase of social media on the internet has brought an unprecedented revolution which has changed the existing social communication systems to date. currently, a high percentage of population in the developed world has a smartphone with internet connection that allows being permanently connected. this enables new ways of approaching some types of tasks that have traditionally required of simultaneity in space and time, such as the development of creative proposals by a large number of people. technological advances have allowed, in a relatively short period of time, the size reduction of a computer to today's mobile devices. among the multitude of specifications offered by next-generation devices, the continuous evolution of the imaging capture systems is a highlight. the existence of both front-facing and rear cameras, both capable of taking pictures or record video has become widespread in the majority of mobile devices. this paper proposes the use of social live streaming tools in mobile devices in order to facilitate the development of creative workshops, using the virtual territory as a co-creation area with the aim of promoting one-to-many communications, so that a lecturer can perform a mass communication, in real time and delocalized, without losing the possibility of interacting with the audience. these tools also allow the possibility for each member of a creative team to swap between different roles (viewer at some times or lecturer at others), thus stimulating the creative process through social participation. keywords social media; virtual workshop; creativity; live streaming; co-creation; social learning. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ mailto:cgarciag@uji.es multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 38 http://polipapers.upv.es/index.php/muse/ 1. introduction the arrival of web 2.0, about a decade ago (o’reily, 2006) made users change from passive agents into active elements of the system, so being called prosumers (adell, 2012; ritzer, 2013). this fact has led to a virulent evolution of a great number of internet-based social communication platforms, commonly known as social media (couldry, 2012; ponce, 2012). during the last five years, smartphones have also joined this revolution. this has been feasible thanks to the increase of processing capacities of this type of devices and the appearing of several applications or apps (mostly free) which have increased their utility range, as well as with the democratization of mobile data rates. thus, a huge number of users are permanently connected to the internet and, consequently, to their social, personal or professional circles (telefonica, 2016). this permanent connection between users encourages the development of collaborative creation processes among a great number of users, by using social media as social communication platforms (cenich and santos, 2006; chulvi et al., 2016). this trend has also been used in educational frameworks by means of massive open online course (mooc) (cormier, 2008; downes, 2013; siemens, 2013), trying to expand the range of a local educational action towards an open and massive space. the incorporation of image capturing devices in smartphones has also contributed to a change in the paradigm of social communication. currently, almost every smartphone has one or two high definition cameras, capable of capturing both high quality steady images and video sequences, either of the own user (selfies, video conferences, video calls… by using the frontal camera) or third parties’ (by using the rear camera). thus, audiovisual media have been incorporated as native and usual elements of social communications. in this sense, some platforms allowing the broadcasting of live videos throughout the internet have been recently developed for smartphones. they use an app to record the http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 39 http://polipapers.upv.es/index.php/muse/ videos and a general-interest social network to notify the potential audience of their broadcast. this feature is known as social live streaming video. 2. aims of this work here, it is proposed the use of social live streaming tools in mobile devices (rugg and burroughs, 2016; stewart and littau, 2016) as a way of organizing and developing creative workshops, trying to make the most of the virtual territory as a co-creation space (prahalad and ramaswamy, 2004; sanders and simons, 2009). the aim in doing so is adapting the usual organizational structure of a creative workshop to the new social and educational paradigm. so, the analysis of the new possibilities that live streaming offers as a one-to-many communication tool, taking into account features such as the possibility of avoiding interaction limitations with the event’s audience or of considering rotating roles, thus enabling every spectator, or group of spectators, to develop the role of lecturer at a certain moment. in this way, massive real time and delocalised communications among a high number of creative professionals could be favoured, thus enhancing creative processes and social participations in a work group (hart, 2011). the methodology proposed in this work, which has been devised focusing in the development of massive workshops, could even have an application not only in creative fields but also in educational ones, thus enabling a displacement from the traditional academic approaches towards informal learning. in this sense, baser et al. (2013) define informal learning as a continuous learning process that happens throughout life. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 40 http://polipapers.upv.es/index.php/muse/ 3. methodological proposal prior to define the methodological steps of the proposal, a preliminary analysis of the various available technological tools that are capable of enabling the development of creative workshops would be required. 3.1. preliminary analysis of live streaming tools two types of tools, close to the concept of life streaming tools, which could be useful for developing creative workshops, may be differentiated: video conferencing tools and video streaming tools. 3.1.1 video conferencing tools these tools allow bidirectional communication between a limited number of users, in a way that all of them can see and listen simultaneously the other ones. in order to establish a communication, there has to exist a previous relationship between the users. examples of this type of tools are the following: • google hangouts (https://hangouts.google.com/). it allows the simultaneous connection between ten (smartphone app) or fifteen users (web platform). the broadcast quality is very good and this system pioneered free group video conferences. • skype (https://www.skype.com/). it allows the possibility of establishing video conferences between twenty-five users. this is one of the most extended communication tools, actually under property of microsoft. (https://www.microsoft.com/). 3.1.2 video streaming tools video streaming tools allow unidirectional one-to-many communication. they differ in the way the broadcast is accessed and the emitter’s feedback possibilities. the following ones are examples of this type of tools: http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ http://www.skype.com/) http://www.microsoft.com/) multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 41 http://polipapers.upv.es/index.php/muse/ • google hangouts on air (https://hangouts.google.com/). apart from the video conference features, the google hangouts tool allows streaming broadcasts by means of the service “hangouts on air”. this tool has the peculiarity of requiring receiving an invitation from the emitter to be able to access the broadcast. usually this is done by means of the timeline of the social network google+ (https://plus.google.com/). once the broadcast ends, it remains recorded in the youtube (https://www.youtube.com/) channel of the emitter, so it can be subsequently viewed by the rest of receivers. • facebook live (https://live.fb.com/). it allows live broadcasting from the own facebook app for smartphones, with a maximum time length of ninety minutes. once the broadcast ends, it remains published in the emitter’s facebook profile. • ustream (http://www.ustream.tv/). this application, under property of the multinational ibm (https://www.ibm.com/), is one of the first video streaming platforms acting from a smartphone app. there is a basic trial version, as well as an advanced one (ustream pro) with different price plans according to the offered features and the amount of viewer hours. • livestream (https://livestream.com/). formerly known as mogulus, it allows the creation of a television channel on its own for live broadcasting. it has a smartphone app which allows broadcasting from anywhere. • bambuser (http://bambuser.com/). this application allows video streaming both from smartphones and from computers. • twitcasting live (http://es.twitcasting.tv/). it allows live video broadcasting from a smartphone, while using facebook or twitter (https://twitter.com/) to notify the broadcast airing to followers. it is possible to receive the broadcast via twitcasting viewer. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ http://www.youtube.com/) http://www.ustream.tv/) http://www.ibm.com/) http://bambuser.com/) http://es.twitcasting.tv/) multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 42 http://polipapers.upv.es/index.php/muse/ • periscope (https://www.periscope.tv/). this popular tool allows video streaming via a smartphone app. at the beginning of the broadcast, it is published at the emitter’s twitter profile, thus enabling followers to join the live transmission. once the broadcast ends, it remains registered in the twitter timeline, as well as in the emitter’s periscope channel. this application also allows spectators’ interaction by means of text messages that appear on the screen. this way interaction is enhanced not only towards the emitter but also among all the broadcast receivers. • meerkat (http://meerkatapp.co/). this tool, also very popular, allows live video broadcasting from a smartphone and notifies the broadcast airing to followers via facebook or twitter, as well. this is currently periscope’s direct competitor, in terms of audience and popularity. this tool also allows spectators’ interaction via text messages shown on the screen, in a similar way periscope does. 3.2 proposal of a live streaming tool after analysing the current main streaming tools, periscope is considered as the most suitable tool for developing creative workshops (siekkinen, masala and kämäräinen, 2016). therefore our methodological proposal will focus on this application taking into account the features that would help achieving the various objectives that are intended with de development of this type of activity: massive real time and delocalised communication and interaction among participants, and dynamic roles definition, in order to obtain a final co-creative result. on a first step of this choice, video conferencing tools have been ruled out as they are limited to a concrete number of users, which also need a previous relationship between themselves in order to start the interaction process. regarding the video streaming tools, periscope, apart from having a high number of active users, is spreading fast, so it is easily accessible to prospective users. in this way, it can easily increase the potential audience of any planned event. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ http://www.periscope.tv/) http://meerkatapp.co/) multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 43 http://polipapers.upv.es/index.php/muse/ moreover, its linkage with twitter allows followers in that social network participating in the workshop as lecturers to receive a notification in their timeline at the beginning of the broadcast, or to access later to the recorded contents. twitter also allows identifying each publication with one or many labels, created by users, known as hashtags, which are identifiable by starting with the # symbol. thus, all the publications concerning the same creative event could be easily identified by using a specific label/hashtag (for instance, #edureworkshop16). another interesting point about the use of these technologies consists of the existence of tools that ease the tracking of an event, or of diverse topics of interests, based on hashtags (#hashtag) or, even, on specific users (@user). in this sense, tweetdeck (https://tweetdeck.twitter.com/) is a twitter client that allows tracking of various contents and concepts by means of a very intuitive system of thematic columns. therefore, this application can easily make the most of the ple (personal learning environment) (norman, 2008) of every creative participating in the workshop, allowing some, or many, of their followers to be added to the massive event, thus yielding to a pln (personal learning network) (adell and castañeda, 2010). these concepts are intimately linked to the connectivism theory, which states learning can reside outside us and focuses on connecting specialized information groups and in considering rather more important the establishment of good connections than our current knowledge state (siemens, 2004). from that point of view, the extent of the event can grow exponentially, by mainly including spectators who are interested in the topics or background of the event, plus some spontaneous lecturers who might decide to participate in a more active way in it, either individually or collectively. finally, another good point of the use of periscope is that this application allows accessing the active broadcasts from a map. in this way, it is easy locating the place where all http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 44 http://polipapers.upv.es/index.php/muse/ broadcasts regarding the virtual workshop are taking place, thus enabling a visual sample on its extent. 3.3 workshop development considerations for the development of the workshop itself, different aspects should be taken into account: • event organization. an organizing committee has to be created to sort out the topics of the event and publicize it throughout social media, taking advantage of their close and potential contacts. • event promotion. prior to announcing the event, a positioning strategy could be developed. to do so, it can be used the same means as the ones that are planned to be used as a spreading platform of the event: the social media (garcía-garcía, felip and royo, 2016; pérez ortega, 2006). it could be interesting the creation of specific web spaces, such as a blog, or a cms-system based web. this space can even act as a neuralgic point in which the study cases could be presented, place news feed according to the hashtags of the event or link to the resulting broadcasts related to the workshop. • users’ sign in. all users interested in participating in the event should have an active twitter account in order to follow it. it could also be interesting to have the tweetdeck application for google chrome web browser installed on a computer, so it could ease the tracking of the whole event. • apps installation and smartphones configuration. only the users, or groups of users, willing to participate with any kind of communication or lecture should have a smartphone capable of admitting the installation of the periscope app in order to live broadcast. • case approach. the organizing committee would expose the study cases by means of diverse tweets and an opening broadcast where the workshop would be presented. in order to link all the interactions regarding to the event, an official http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 45 http://polipapers.upv.es/index.php/muse/ hashtag would be proposed (#edureworkshop16), so that all the comments and transmissions from the rest of participating users could be easily tracked. • schedule of sessions. in order to be able to live interact with the lecturers of each broadcast, notifications with the scheduled transmission time would be published in advance. they should include an emission notification hashtag (#edureworkshop16live) so that locating the programmed sessions could be eased. • event conclusion. the event would conclude with a final broadcast from the organizing committee, commenting on the different actions carried out and extracting the main conclusions of the workshop. 3.4 assessment of the virtual workshop after finishing the event, an assessment of its functioning and the generated activity would be developed. this could serve as a way for measuring not only the results of the workshop but also the suitability and effectiveness of the proposal as a tool in order to boost the collaborative development of creative workshops¸ from a point of view either professional or educational. the assessment of the event could be based on a series of indicators that would have been defined prior to the development of the workshop, mainly focusing on the generated interventions and aspects such as: • broadcasts. number of spontaneous broadcasts from users participating in the event. • live spectators. number of spectators that have followed the event live. • deferred spectators. number of spectators who have viewed the transmissions after they ended, either until the end of the event or after that moment. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 46 http://polipapers.upv.es/index.php/muse/ • comments on the broadcasts. number of comments that have been made during each of the transmissions. • general comments on the event. number of published tweets showing any of the hashtags referring to the event. an anonymous twitter survey would also be launched after the end of the event. the aim of this action would be acquiring the anonymous opinion from the users participating in the event on its development. the survey would be based on three or four simple and direct questions about the use of twitter and periscope as tools for carrying out the workshop. each of these questions would be launched as a query-type tweet with a response span of seven days, including a likert scale of three options (dislike, don’t know, like) and the hashtag of the event. previously, during the development of the workshop, users would have been notified on the existence of this subsequent survey and on its relevance as an improvement tool for future editions of the event. so, the analysis of the indicators values, jointly with the opinion survey, would serve as a reference point for upcoming events. 4. discussion this proposal might be a revolution in the development of virtual workshops both from creative and creative-related educational points of view, such as it can be design in its various aspects (graphic, industrial, ceramic, illustration, photography, etc.) and teaching in any of these fields. the proposed organizational system, as well as the chosen tools, might allow an effective development of a virtual creative workshop including a great number of participants (spectators) with an undefined number of lecturers (sometimes spontaneous) amongst http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 47 http://polipapers.upv.es/index.php/muse/ them. the use of free applications and the users’ own smartphones permits a development of this type of events to a wide extent without any direct costs. from an educational point of view, this proposal can also be adapted to an interaction platform between diverse educational institutions to develop collaborative activities that might favour the interaction and connection between students from different fields and geographic locations, as well as enhance the generation of collective knowledge (lundvall and nielsen, 2007). this way, the cost-free status of the event is an extra added value. finally, the event assessment system may contribute to improve the proposal by means of the application of an actio-research methodology (mills, 2000). 5. conclusions technological evolutions constitute suitable opportunities for creating new ways of social creation which are based on the collaborative potential of the creative team. this paper proposes the use of live streaming tools to manage virtual creative workshops in order to favour both creation and interaction between a great number of potential receivers of the exposed contents mechanisms. similarly, this proposal allows breaking the traditional structure lecturer-spectator in the way every participant, or group of participants, can acquire a dynamic role throughout the workshop, not only by enhancing their interaction with lecturers but also by adopting that same role sometimes. virtual workshops are a great tool for boosting the active participation of the different members of a creative collective, as it easily allows the exhibition of their work proposals, the used techniques for facing concrete cases or, simply, their opinions on the topic defined as the centre of the event (banks, 2010). http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 48 http://polipapers.upv.es/index.php/muse/ moreover, these workshops are an effective educational methodology to show students a widened vision of a creative discipline, far beyond the mere academic framework (garcía garcía, galán and izquierdo, 2016; galán et al., 2014). from that approach, it could be interesting the adoption of this proposal to the current technological and pedagogical contexts, making the most of the technological resources to go one step beyond their potential extent, by moving from a local concept of workshop to a massive open online workshop (moow) concept. furthermore, the development of virtual creative workshops can contribute to increase the extent of the ple and the pln of each of the participants, thus improving their abilities for establishing quality connections and, with them, enhancing their knowledge in a concrete field. acknowledgments this work was supported by the universitat jaume i de castellón, under the grant p1· 1b2015-30 “el arte y el diseño en la nueva sociedad digital” and the gie iredise estce 009/12 "investigaciones y reflexiones sobre la enseñanza del diseño dentro del nuevo marco del espacio europeo de educación superior". references adell, j. and castañeda, l. (2010). los entornos personales de aprendizaje (ples): una nueva manera de entender el aprendizaje. in r. roig vila y m. fiorucci (eds), claves para la investigación en innovación y calidad educativas. roma: marfil. adell, j. (2012). del web 2.0 a l'aprenentatge 2.0. taller ensenyament amb el núvol. http://www.svideo.uji.es/peli.php?codi=836 accessed 1 june 2016. banks, m. (2010). craft labour and creative industries. international journal of cultural policy, 16(3) 305–321. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ http://www.svideo.uji.es/peli.php?codi=836 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 49 http://polipapers.upv.es/index.php/muse/ baser, j. a., hasan, a., sulaiman, j., and buntat, y. (2013). informal learning model for engineering technology teachers. advanced science letters, 19(12), 3730-3732. cenich, g., and santos, g. (2006). aprendizaje colaborativo online: indagación de las estrategias de funcionamiento. te & et, 1(1), 1-8. chulvi, v., mulet, e., felip, f., and garcía-garcía, c. (2016). the effect of information and communication technologies on creativity in collaborative design. research in engineering design, doi:10.1007/s00163-016-0227-2 cormier, d. (2008). the cck08 mooc connectivism course, 1/4 way. in dave’s educational blog (2 oct. 2008). http://davecormier.com/edblog/2008/10/02/the-cck08-mooc-connectivism course-14-way/ accessed 2 june 2016. couldry, n. (2012). media, society, world: social theory and digital media practice. cambridge: polity press. downes, s. (2013). mooc. the resurgence of community in online learning. blog half an hour. http://halfanhour.blogspot.be/2013/05/mooc-resurgence-of-community-in-online.html accessed 5 june 2016. galán, j., garcía-garcía, c., díaz, d., muñoz, a., and pesudo, m. c. (2014). use of videos and graphic tablets in the teaching-learning processs for handmade rendering in the degree in industrial design and product development engineering. proceedings of 8th international technology, education and development conference (inted 2014), 6860-6868. garcía-garcía, c., felip, f. and royo, m. (2016). uso del social media como estrategia global de posicionamiento para estudios de arquitectura. revista de expresión gráfica en la edificación (ege), 9. garcía-garcía, c., galán, j. and izquierdo, r. (2016). the use of social media as a tool for positioning the personal brand of students in the degree of industrial design and product development engineering. proceedings of 10th international technology, education and development conference (inted 2016), 3232-3240, doi:10.21125/inted.2016.1754 hart, j. (2011). social learning handbook: a practical guide to using social media to work and learn smarter. corsham, wiltshire: center for learning and performance technologies. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ http://davecormier.com/edblog/2008/10/02/the-cck08-mooc-connectivismhttp://halfanhour.blogspot.be/2013/05/mooc-resurgence-of-community-in-online.html multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 50 http://polipapers.upv.es/index.php/muse/ ludvall, b. a and nielsen, p. (2007). knowledge management and innovation performance. international journal of manpower, 28(3-4), 207-223. mills, g. e. (2000). action research: a guide for the teacher researcher. new jersey: prentice-hall, inc. norman, d. a. (2008). on the ple. blog of d’arcy norman, (march 5, 2008). http://darcynorman.net/2008/03/05/on-the-ple/ accessed 7 june 2016. o'reilly, t. (2006). qué es web 2.0. patrones del diseño y modelos del negocio para la siguiente generación del software. boletín de la sociedad de la información: tecnología e innovación. pérez ortega, a. (2006). cómo destacar sobre la multitud. la marca personal. in j.l. briones (ed.), marketing creativo, tomo 2 colección emprender en el siglo xxi (pp. 1-21). sevilla: emergia. ponce, i. (2012). monográfico: redes sociales. (april 17, 2012) http://recursostic.educacion.es/observatorio/web/en/internet/web-20/1043-redessociales accessed 4 june 2016. prahalad, c.k., and ramaswamy, v. (2004). co-creation experiences: the next practice in value creation. journal of interactive marketing, 18(3), 5-14. ritzer, g. (2013). prosumption: evolution, revolution, or eternal return of the same? journal of consumer culture, 14(1), 3-24. rugg, a., and burroughs, b. (2016). periscope, live-streaming and mobile video culture. in r. lobaro and j. meese (eds), geoblocking and global video culture (pp. 64-72). amsterdam: institute of network cultures. sanders, l., and simons, g. (2009). a social vision for value co-creation in design. open source business resource, december. http://timreview.ca/article/310 accessed 5 june 2016. siekkinen, m., masala, e., and kämäräinen, t. (2016). anatomy of a mobile live streaming service: the case of periscope. arxiv, preprint arxiv:1605.04270. siemens, g. (2004). connectivism: a learning theory for the digital age. elearnspace (december 12, 2004) http://www.elearnspace.org/articles/connectivism.htm accessed 4 june 2016. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ http://darcynorman.net/2008/03/05/on-the-ple/ http://recursostic.educacion.es/observatorio/web/en/internet/web-20/1043-redessociales http://timreview.ca/article/310 http://www.elearnspace.org/articles/connectivism.htm multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6347 eissn: 2341-2593 garcía-garcía et al. (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 37-51 | 51 http://polipapers.upv.es/index.php/muse/ siemens, g. (2013). massive open online courses: innovation in education?. in r. mcgreal, w. kinuthia and s. marshall (eds.), open educational resources: innovation, research and practice (pp. 5-16). vancouver: commonwealth of learning and athabasca university. stewart, d., and littau, j. (2016). up, periscope: mobile streaming video technologies, privacy in public, and the right to record. journalism and mass communication quarterly, special issue: information access and control in an age of big data, doi:10.1177/1077699016637106. telefónica (2016). informe sociedad de la información en españa 2015′ sie[15. barcelona: ariel. http://dx.doi.org/10.4995/muse.2017.6347 http://dx.doi.org/10.4995/muse.2017.6347 http://polipapers.upv.es/index.php/muse/ learning leadership from teaching: emotional competences multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 learning leadership from teaching: emotional competences a. bernal*, s. ibarrola-garcía dpto. de educación. universidad de navarra, campus universidad de navarra, edificio de bibliotecas, 31080 pamplona, spain. * corresponding author: email: abernal@unav.es; phone: + 34 948425600 ext.2466 received: 2013-12-15; accepted: 2014-08-11 abstract during the last two decades, in our societies we have sought quality education. one way to achieve this goal is to promote educational leadership. the field of educational leadership is complex: administrative, instructional, school supervision, teacher supervision. our focus is on instructional leadership and teacher leadership. there’s an extensive literature on this topic and we have selected recent publications that present a discussion of theoretical and practical implications. our theoretical study is from an educative perspective. we highlight a theme: the emotional competences of academic leaders and teachers and their influence on effective teaching. researchers insist that teachers’ emotions cannot be regarded only in their psychological aspects. they show the interconnections between teacher’s beliefs, identity, goals and emotions and teaching. the relevance of this topic drives our research on how it’s introduced in programmes of teacher education. teacher education should incorporate this vision. in particular, we analyse the most representative and current programmes in spain. we compare these programmes with the current tendency of teacher education in an international context. keywords teacher leadership; instructional leadership; emotional competences; teacher emotions; teacher training; teacher education. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 33 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 1. introduction some sets of competing contextual pressures on schools for reclaiming quality education are: constant change, new movements, new programmes and new directions; the emergence of school bureaucracies and teaching reforms; social and cultural context blocking education – the damaging impact of some mass media and new technologies, more dysfunctional families than ever before; new conceptualisations of academic organization with an emphasis on social capital, learning organizations, collective teacher efficacy and communities of professional learners. today the skills and knowledge which children require are becoming more complex and the range of other issues which schools are expected to help address is growing. leadership is itself one of the main drivers of the quality of teaching, the other is the teacher. more than school input or context, learning depends on a determined and accountable school leader. some reviews show how the world's top school systems are building their teaching leadership. there are statistically significant empirical and qualitatively high associations between heads’ educational values, qualities, and their strategic actions and improvement in school conditions leading to improvements in student outcomes. this paper is the result of a preliminary study of teacher education. the interconnection between quality education, the best teaching, academic leadership and instructional leadership inspired us to seek the necessary conditions in teacher training. a cross-theme emerged: emotions in education. the effect of emotions on teacher development, educational changes and teaching improvement are subjects of increasing importance in understanding the work and quality of school leaders and teachers. teachers’ emotions are investigated as a means of enhancing the self-awareness of the individual, understanding one’s own responses to change and how to improve, and for those in leadership roles. for this reason, the findings are interesting when teachers become leaders in their schools. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 34 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 for this purpose, it is interesting to analyse what leadership training is given, how effective it is, and the role of emotional competency in teacher training. we consider that emotions should not be reduced to technical competence. presenting emotion management as just another set of skills to be mastered, in which people can be trained, limits how we approach, understand and try to shape the emotional work that people do. emotional labour in teaching can be pleasurable and rewarding, when people are able to pursue their own purposes through it, and when they work in conditions that allow them to do their jobs well. being tactful, caring or passionate as a teacher is treated as largely a matter of personal disposition, moral commitment or private virtue, rather than of how particular ways of organizing teaching shape teachers’ emotional experiences. so we studied how emotional competence is integrated in teacher education programmes. we selected recent publications on this topic that present a discussion of theoretical and practical implications. the present theoretical study is from an educative perspective. 2. teacher leadership, instructional leadership and teaching our study is on leadership in schools. there is a consensus about the importance of school leadership. we found a growing body of evidence demonstrating the impact of effective school leadership. this evidence is consistent across a large number of countries and contexts. apart from classroom teaching, the quality of head teachers is the most important influence on achieving improvements in school standards. across the systems in the review, there is an emerging common opinion on the importance of school leadership and how to improve it (barber et al. 2010). this study review proved that the most important impact on student achievement is attributable to the effectiveness of both principal and teachers. on the other hand, the impact of social interaction on learning is evident. in view of this, the most important condition of a school’s success is the quality of relationships: academic leaders-teachers, teachers-pupils, teachers-parents, teachers-teachers. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 35 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 successful school principalship is an interactive, reciprocal and evolving process involving many players, which is influenced by and in turn influences the context in which it occurs (mulford 2010). as liberman and fiedrich (2010) explain: “teacher-leaders negotiate a variety of challenges as they work to enact a collaborative approach to leadership. these include dilemmas that arise when attempting to work collaboratively inside traditional bureaucratic structures, pedagogical challenges that emerge from teaching in any situation including other teachers, and tensions that stem from conflicts between policy mandates and values and approaches” (p. 654). teacher leadership should pay attention to the interpersonal relationships in school and to promote them. it should be noted that the teacher-leader relationship sets up one of the bases to recover learning, getting better classroom teaching and academic leadership. leadership focused on teaching, learning, and people is critical to the current and future success of schools. the research highlights both a set of practices which effective leaders share, and a common set of beliefs, core values, attitudes, and personal attributes which they possess (new leaders for new schools 2009; day et al. 2010). which of theses practices of leadership are related to teachers? we think the practices most related to teaching are: building a shared vision and sense of purpose; establishing effective teams within the school staff, and distributing leadership among the school staff; understanding and developing people; focusing on student achievement; putting children ahead of personal or political interests; being resilient and persistent in goals, but adaptable to context and people, open-minded; willing to develop a deep understanding of people and context; willing to take risks and challenge accepted beliefs and behaviours; self-awareness and the ability to learn from others. but it is really striking that high-performing principals focus more on instructional leadership and developing teachers. they see their biggest challenges as improving teaching and the curriculum, and they believe that their ability to coach others and bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 36 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 support their development is the most important skill of a good school leader. they are more likely to report that they greatly enjoy teaching. they spend more time working with the people in their school: walk the halls more, spend more time coaching teachers, interact more often with parents and external administrators, and spend more time with students. much prior research about teacher leadership has focused on teachers taking on leadership roles inside their own schools. teachers’ deep understanding of the daily workings of classrooms has long been recognised in the literature as a hallmark of their leadership (mulford 2010). classroom practice forms the foundation of these teacherleaders’ knowledge base and serves as a continued source of leadership learning. the teacher-leaders share the philosophies and research base that motivate their teaching, as well as specific practices and materials, as they work to improve education. specifically, they collaborate with their peers so that other teachers can adapt practices to best meet the needs of their students and to create their own practices. good leaders and good teachers share core values: persistent, resilient, nurturing, open, respectful, trustworthy, honest and caring. these values are experienced in their relationships. leaders influence the sense of safety and openness to collaboration in the culture of their schools and the learning conditions of their students and teachers (beatty 2011). such cultures are also reportedly more emotionally comfortable. “the nature, roles and effects of emotions in teacher development, educational change and school improvement have become subjects of increasing importance in understanding the work, quality and effectiveness of teachers and school leaders” (day and lee 2011, p. 1). school leadership preparation programmes include a social-emotional territory to develop leaders who will emphasise collaborative reflection and will take into account an emotional understanding of themselves and of teachers. they reveal the role of emotional understanding in healthy relationships and those characterised by disconnection and bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 37 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 antagonism. the studies have contributed to a growing number of efforts to explore the meaning of emotions in education and to contribute to our developing understanding of emotion’s place in leadership (grobler and conley 2014). the emotions powerfully contribute to the well-being and effectiveness of and influence on learning at all levels. the principals express a sense of duty to remain positive and cheerful to make up for the emotional baggage of teachers. the main path to understanding the emotional dimension of the relationship between leaders and teachers is the emotionality of both the leaders themselves and the teachers. the capacity for connecting with others depends heavily on the quality of the connection one has with oneself because all human relationships involve emotions. we can see the connection between the ethics of caring, moral leadership and relationships. respect/disrespect, care/lack of care and professional support/lack of professional support involve positive and negative emotions. the leader reconciles positive and negative emotions to release energy for change and address difficult problems. the relationship between respect, care, support and the teacher’s professional self, includes the ability to hope, to remain optimistic. hope helps us to find ways and resources to address difficult problems. the academic leader should attach importance to teachers’ emotions to improve teachers’ professionalism. we will discover that the main emotions and emotional competences that help leaders in their task are the same as those that help the teacher. hope and trust are claimed to make the necessary changes that we need to achieve a quality education. in this paper, we continue by studying teachers’ emotions. 3. teacher emotions teaching and learning are always emotional practices (hargreaves 2002). this affirmation is confirmed in the research about emotions, teaching and teachers. emotions provide a context for learning, students’ emotions interfere with their learning and emotions are viewed as a form of learning – emotional competences. teachers bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 38 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 experiences intense emotions in their teaching; they feel about: their pupils, their skills, their colleagues, and the structures of schooling (kelchtermans et al. 2009). job dissatisfaction, health, emotional exhaustion, stress and depression, are key components of burnout related to teachers, and are of concern to researchers. negative classroom emotions have considerable implications for student learning, the school climate and the quality of the education (corcoran and tormey 2013). teaching and learning aren’t solely emotional practices. emotion, cognition, self, context, ethical judgement and action intertwine in the complex reality of teaching and of learning. this idea illuminates the way we consider teacher training. we seek two matters in the reviewed studies: which teachers’ emotions are emphasised and what educative perspective is presented to keep teachers’ emotions in mind in teacher education. two publications serve as reference in understanding the state of research on teachers’ emotions today: advances in teacher emotion research: the impact on teachers’ lives (schutz and zembylas 2009) and new understandings of teacher's work:emotions and educational change (day and lee 2011). for the past three decades, the central issues have been: emotions in teaching, teachers’ emotions and educational change and teachers’ emotions in their lives, teachers’ emotions and relationships in school, with pupils, colleagues, parents and principals. driving teachers’ feelings is a cross-theme. denzin (2009) insists on a performance approach. teachers and students, as moral agents, enact emotions in their relationships: positive emotions – love, desire, empowerment, hope, happiness, sympathy, gratitude, admiration and contentment; negative emotions – rage, shame, despair, fear, anxiety, anger, disgust, sadness, envy, embarrassment and indignation. simultaneously, the subjects of research on teacher education are: professionalism, instructional leadership, confronting changes – cultural, social, legal variations, difficulties in work and the social demands – educational reforms. the interconnection between these fields of study is obvious. the emotions are inextricably linked to teachers’ work development and identity. for this reason, emotional education bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 39 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 of teachers joins with the educative encouragement of identity, professionalism and good teaching. teachers’ emotions are influenced by their biographies, social and working contexts, peers, phase of professional development, phase of life, sociopolitical cultures and teaching preferences. emotional education is part of moral education. the emotions in teaching are linked to matters of interest and values, to making decisions with moral consequences. teachers’ choices between pedagogical options are technical and moral, and they involve the need to do justice to children’s educational needs. “threats to ´being a proper teacher´ are so closely connected to one’s personal and professional identity; they arouse intense emotional reactions” (kelchtermans 2011, p. 76). we found proposals to enhance a passion for teaching (noddings 2011; day 2012). this passion shows other needed emotions: pleasure, joy and hope. researchers inquire into teacher development’s broader aspects containing effective phenomena: well being, effectiveness, efficacy, identity, resilience, happiness, motivation, commitment, vulnerability (day 2011; bullough 2011, macbeath 2012). these realities are interconnected; the positive emotions express the correctness of this link and strengthen teacher agency. being passionate about teaching relies on a commitment to understand and educate every learner; a conviction and hope of teachers and pupils, for themselves, to achieve the best in learning and teaching; a sense of purpose and well being connected with a sense of professional identity. these efforts can be made every day with organizational support. teachers should combine their professional competences with their personal commitment and values, but hopefulness and resilience, courage and perseverance overcome stress, fear, exhaustion, irritation, and frustration when teachers work in professional learning communities. these communities develop school cultures with common beliefs, interaction, participation and meaningful relationships. teacher effectiveness is the product of the preparation and continuing support of the head (cognition) and the heart bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 40 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 (emotion). teachers’ self-esteem, confidence and hope grow when they think their actions are consistent with their values and beliefs and they promote students’ learning. also when working with others, giving support to and asking for support from others (day and quing 2009). concluding this section, we bring into contention several implications of the research about teachers’ emotions for the training and professional development of teachers. in general, in the reviewed literature, the researchers suggest the need and benefit of being aware of the emotions; proper emotions and others’ experiences in diverse situations and in different relationships. the studies reflect a variety of ways in which emotional management is worked in the classroom, in relationships with students, teachers, and heads. changes, reforms, and demands affect teachers (neophytou, 2013). professional development could include making teachers understand their emotional life lessons and, in this way, they will be prepared and able to overcome the common obstacles of teaching. teacher education is designed from a vision of the interconnections between teachers’ identity, emotions, values, beliefs, social expectations and responsibility for social justice (pillen et al. 2013; karlsson, 2013; van uden et al. 2013). 4. teacher emotions training in spain the design of programmes aimed at the development of emotions in teachers is new. despite broad academic knowledge generated around emotions in recent years, the initiatives to promote emotional awareness among teachers and help them to develop their emotional competences are still scarce, both in pre-service teacher education and teacher practice. it is, however, possible to highlight two major programmes in spain especially given the number of teachers who have been involved. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 41 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 on the one hand, the grop group at the university of barcelona has developed a teacher-training programme based on the theoretical model of bisquerra and pérezescoda (2007) which aims to develop the following components of emotional competence: 1) emotional awareness; 2) emotional regulation; 3) personal autonomy; 4) social competence; and 5) competence for life and wellness. this training is carried out for 30 hours at teachers’ schools and is conducted for one hour a week for 9 months. on the other hand, the programme designed by the laboratory of emotions of the university of malaga, pursues teacher development within four dimensions according to the theoretical model of mayer and salovey (1997): 1) perception, appraisal and expression of emotions; 2) emotion as a facilitator of thought; 3) emotional awareness; and 4) regulation of emotions. this training requires 45 hours over 6 months (4 hours every 15 days). after the eighth session there are no classes for one month and teachers’ assistants put the theory into practice in their classrooms. they have five tutorials online. after that, they return to class and describe the application that they have conducted. initiatives in emotional teacher training are scarce and it is difficult to collect empirical evidence about their effectiveness. the experiences assessed are still few. nevertheless, some of the evidence collected highlights improvements in emotional competence, especially in two areas: emotional regulation and social competence. teacher stress levels are also reduced and the institutional climate improves (pérez-escoda et al. 2012; pérezescoda et al. 2013). muñoz (2005) evaluated a psychopedagogical intervention programme in emotional teacher training in five schools in guipúzcoa. she found that it increased the feeling of well-being of teachers, particularly over their security and in the classroom; their capacity to cope with conflicts, and they also discovered new skills. it contributed to their professional development; they became more aware of their influence on the emotional development of students. from another perspective, the programmes targeted at the emotional development of students are more numerous. for example, the intemo proyect bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 42 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 (http://emotional.intelligence.uma.es), the fosoe program (training in socioemotional skills), the “responsible education” program of the marcelino motín fundation (http://www.fundacionbotin.org/educacion_areas.htm) or the extremadura network schools emotional intelligence, are some of the initiatives that are taking place in different schools in the spanish geography. they all have something in common: they emphasise the teacher in the classroom as a reference model for the emotional development of students. emotionally competent students require teachers who are emotionally prepared. the legislative framework has opted for the integral development of the student from a competency-based approach, and this is inevitably associated with the need for such skills to be taught by a teacher team that dominates them. the teachers are a model of communication, respect, prosocial behaviour; they have the opportunity to explain, express and opine openly about issues; they can manage and guide situations of conflict; to promote or not certain desirable behaviours in their pupils and all that is associated with an optimal socioemotional classroom climate. this requires an emotional competence and resources that teachers recognise they are not ready for. the literature review and the analysis of programmes show how the emphasis is on the need to help teachers and facilitate them in the development of emotional skills to cope with their work stress. this stress is due to the challenges of the teaching-learning process and also to the curricular and organisational changes required by the educational context. so, if teachers learn to maintain positive emotional states and reduce the impact of negatives, it results in greater teacher welfare and consequently the best psychological fit with their students. in the line of positive psychology (fredrickson, 2004), it is considered that teachers who experience positive emotions (joy, interest, contentment and love) over time provide more flexible and creative thinking and they boost their psychological resilience. although it may be useful to work on certain emotional teacher competencies according to specific needs – for example, teacher behavioural deficits, only from positive values bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 43 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 can skills be routed towards personal, interpersonal and social improvement, enforcing the benefit of quality education. emotional training also requires moral training because teachers have to consider what values develop in relationships with others and their social life. otherwise, emotional teacher education responds only to a remedial approach based on the professional teacher deficit, with certain therapeutic connotations. it seems important to give greater visibility to the role of emotional competence as a key to the exercise of teacher leadership. beyond the role of emotions in the “survival” of the teacher, their well-being or the adjustment of their students, emotions are key to broadening their capabilities and the exercise of their profession. even the revised proposals claimed to adopt a broader perspective that supports the participation of a set of abilities and skills to address the social, affective and emotional problems in the life of schools and classrooms (valdivieso et al. 2013). the teacher needs to be personally aware of values and commitments, and emotional training can facilitate the development of fundamental qualities of teachers (justice, kindness, sincerity and perseverance) but it can also help them better understand the context in which they work and the demands of the teaching profession. emotional education programmes for teachers in spain will have to incorporate a holistic approach as we have seen in the previous section. on the international stage, it is possible to collect more teacher training of emotional experiences, some of them particularly relevant as jennings and greenberg (2009), nelis, quoidbach , mikolajczak and hansenne (2009) and brackett and caruso (2007) . in one of them we see how it raises the personal development of the participating teachers and educational professionals and their commitment, trust, responsibility and motivation. specifically, jennings and greenberg (2009), based on a comprehensive review of research on teacher emotional education, claims that teacher training should indispensably include four themes: 1) emotional intelligence (information on some bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 44 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 emotions that affect learning and on the identification, expression and regulation of emotions in response to common classroom situations), 2) mindfulness (as a technique that encourages the full and reflective awareness of what happens in the moment and reduces stress), 3) commitment to education, 4) developmental psychology of social and emotional development (to better understand the development and sociopersonal needs at different ages and the effect this has on the dynamics of the class). these authors, from the model “the prosocial classroom” describe how a comprehensive education that takes into account this set of factors contributes to a learning environment that promotes quality positive outcomes – academic, social and emotional – among students and also feeds the enjoyment, effectiveness and commitment of teachers to their profession. if teachers recognise the emotions of their students they will respond more effectively to their individual needs; so they will show a better effective classroom management; greater ability to use their emotional expressions and support verbally; promote enthusiasm and enjoyment of learning, guide and manage students’ behaviour and understand the dynamics of conflict and guide them properly. we find this approach interesting because the emotional competence of the teacher not only has to do with their well-being or adjustment. to this extent teachers who improve their emotional competence can also improve the methodology used, the dynamics of classes, and their enthusiasm for teaching or their relationship with students and colleagues. however, all this can be influenced by contextual factors like the leadership of the principal, the climate, school values and norms, state or local politics and even the level of stress or well-being in the personal life of the teacher or the support of another teacher in the classroom (co-teacher). if the school promotes a culture of positive engagement, a community with a shared vision of education, it generates a teacher’s sense of belonging and makes it possible for ways of coexisting with positive ways of feeling, thinking and acting. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 45 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 5. conclusions in relation to the requirements of good teachers and teaching at its best, researches have made clear the interrelation between cognition, effect and competency and the crucial part played by emotions in teachers’ capacities to exercise their professionalism. from a holistic framework, leaders can begin to better develop their own leadership practices (cradwford 2011). the context of changes, reforms and demands in which principals and teachers work challenge their moral and ethical purposes, their vision and their sense of positive identity, efficacy and agency. learning leadership and learning teaching should entail a holistic training joining cognition, emotions and skills, beliefs, values, sense of identity, agency and efficacy (meyer 2011). teachers’ capacities to sustain their commitment – to be resilient – are moderated by their identities, their professional life phases and the contexts within which they work. teacher emotions – emotional exhaustion, burnout, stress, job satisfaction, well being, joy and hope – point to how teachers experience their work and influence it. the sense of commitment facilitates emotion management; emotion management makes it easier to sustain commitment, a positive identity, and an effective competency (zembylas and schutz 2009). the need for an emotionally positive academic climate and the need for leaders’ and teachers’ wellbeing are resolved, in part, through interventions: coaching, mentoring, consultation with experienced peers and “supervisors” in contexts where leaders provide time and open cultures which build trust. such interventions will need to be made by those who themselves are emotionally attuned to the personal, professional and organisational contexts in which they work. the researchers examined have important implications for teacher education, both preand in-service. the case studies of the bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 46 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 biographies of teachers, interviews and reflective journals are qualitative methods of study which are really useful in understanding this subject. programmes of emotional teacher training in spain are scarce. also few experiences have been evaluated. the little empirical evidence that has been collected shows that after participation in a programme on emotion teachers improve their well-being, stress level, emotional regulation and classroom management. there are more initiatives on the emotional education of students which require teachers to be emotionally prepared. from the analysis of programmes and the literature review we have done in this paper, we note that it is necessary to incorporate a holistic approach that goes beyond a vision of emotional teacher training based on developing specific skills for their welfare and survival in the classroom. emotional teacher training should help teachers to exercise the values, qualities and commitments of the profession with enthusiasm and passion. 6. references barber, m., whelan, f. and clark, m. (2010). capturing the leadership premium. how the world’s top school systems are building leadership capacity for the future. (november 2010 report). resourced document. mckinsey & company in collaboration with the national college for leadership of schools and children’s services. http://mckinseyonsociety.com/capturing-the-leadership-premium/ bernal, a. (2011) tendencia educativa en auge: la «omnipresencia» de la educación emocional, en vergara, f. j., sánchez, f. y comella, b. (coords.) ideales de formación en la historia. madrid, dykinson, s.l., 769-794. bisquerra, r. and pérez-escoda, n. (2007). las competencias emocionales. educación xx1, 61-82. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 47 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 brackett, m.a. and katulak, n.a. (2006). “emotional intelligence in the classroom: skill based training for teachers and students”. en j. ciarrochi y j.d. mayer (eds.), improving emotional intelligence: a practitioner’s guide (pp.1-27). new york: psychology press taylor y francis. beatty, b. r. (2011). leadership and teacher emotions. in c. day and c. k. lee (ed.), new understandings of teacher's work: emotions and educational change (pp. 217– 242). dordrecht: springer netherlands. doi: 10.1007/978-94-007-0545-6_14 corcoran, r. p. and tormey, r. (2013). does emotional intelligence predict student teacher’s performance?. teaching and teacher education, 35(1), 34–42. doi: 10.1016/j.tate.2013.04.008 crawford, m. (2011). rationality and emotion in education leadership-enhancing our understanding. in c. day and c. k. lee (ed.), new understandings of teacher's work: emotions and educational change (pp. 205–214). dordrecht: springer netherlands. doi: 10.1007/978-94-007-0545-6_13 day, c. & qing, g. (2009). teacher emotions: well being and effectiveness. in p. a schutz and m. zembylas (ed.), advances in teacher emotion research: the impact on teachers’ lives, (pp. 15–32). boston: springer. doi: 10.1007/978-1-4419-0564-2_2 day, c., sammons, p., hopkins, d., harris, a., leithwood, k. gu, q. & brown, e. (2010). 10 strong claims about successful school leadership, 2010. resource document. national college of leadership of schools and children’s services. http://www.almaharris.co.uk/files/10strongclaims.pdf day, c. and lee, c. k. (ed.) (2011). new understandings of teacher's work:emotions and educational change. dordrecht: springer netherlands. doi: 10.1007/978-94-0070545-6 day, c. (2012). new lives of teachers. teacher education quarterly, 39(1), 7–26. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 48 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 denzin, n. m. (2009). foreword: performance, pedagogy and emotionality. in p. a schutz and m. zembylas (ed.), advances in teacher emotion research: the impact on teachers’ lives, (pp. vi–vii). boston: springer. efklides, a. y volet, s. (eds.) (2005) feelings and emotions in the learning process (monográfico). learning and instruction, 15 (5), 377-516. doi: 10.1016/j.learninstruc.2005.07.006 esteve, j. m. (2006). las emociones en el ejercicio práctico de la docencia. teoría de la educación, 18, 85-107. fredickson, b.l. (2004). the broaden-and-build theory of positive emotions. the royal society, 359(1449), 1367-1377. grobler, b. and conley, ll. (2014). the relationship between emotional competence and instructional leadership and their association with learner achievement, education as change, 17(si s1), s201-223. hargreaves, a. (2000). mixed emotions: teachers’ perceptions of their interactions with students. teaching and teacher education, 16 (6), 811-826. doi: 10.1016/s0742051x(00)00028-7 hargreaves, a. (2002). teaching in a box. emotional geographies of teaching. in sugure and c. day (ed.), development teachers and teaching practice. international research perspectives. london: routledge-falmer. jennings, p.a. and greenberg, m.t. (2009). the prosocial classroom: teacher social and emotional competence in relation to student and classroom outcome. review of educational research, 79(1), 491-525. doi: 10.3102/0034654308325693 karlsson, m. (2013). emotional identification with teacher identities in student teachers’ narrative interaction. european journal of teacher education, 36(2),133–146. doi: bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 49 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 10.1080/02619768.2012.686994 kelchtermans, g., ballet, k. and piot, l. (2009). surviving diversity in times of performativity: understanding teacher´s emotional experience of change. in p. a schutz and m. zembylas (ed.), advances in teacher emotion research: the impact on teachers’ lives, (pp. 215–232). boston: springer. doi: 10.1007/978-1-4419-0564-2_11 kelchtermans, g. (2011). vulnerability in teaching: the moral and political roots of a structural condition. in c. day & c. k. lee (ed.) new understandings of teacher's work:emotions and educational change. (pp. 65–82). dordrecht: springer netherlands. doi: 10.1007/978-94-007-0545-6_5 lieberman, a. and friedrich, l. (2010). “teacher leadership: developing the conditions for learning, support, and sustainability”. in a. hargreaves et al. (2010), second international handbook of educational change (pp. 647–668). london: springer. doi: 10.1007/978-90-481-2660-6_37 marchesi, a. (2007) sobre el bienestar de los docentes. madrid, alianza editorial. mayer, d. (2011). “but that’s the thing; who else is going to teach besides the idealist?” learning to teach in emotional contexts. in c. day & c. k. lee (ed.), new understandings of teacher's work: emotions and educational change (pp. 150–137). dordrecht: springer netherlands. doi: 10.1007/978-94-007-0545-6_9 mackinsey y co. (2010) capturing the leadership premium: how the world’s top school systems are building leadership capacity for the future. consultado el 26 de febrero de 2013. http://mckinseyonsociety.com/capturing-the-leadership-premium/. mulford, b. (2010). recent developments in the field of educational leadership: the challenge of complexity. in a. hargreaves et al. (2010), second international handbook of educational change (pp. 187–208). london: springer. doi: 10.1007/978-90-4812660-6_11 bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 50 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 macbeath, j. (2012). the future teaching profession. resource document. education international research institute. university of cambridge. faculty of education. http://download.eiie.org/docs/webdepot/ei%20study%20on%20the%20future%20of% 20teaching%20profession.pdf mayer, j.d. & salovey, p. (1997). what is emotional intelligence? en p. salovey y d. sluyter (eds.). emotional development and emotional intelligence: implications for educators (pp.3-31). new york: basic books. muñoz, m. (2005). prevención del estrés psicosocial del profesorado mediante el desarrollo de competencias emocionales: el programa pecera. revista interuniversitaria de formación del profesorado, 19(3), 115-136. nelis, d., quoidbach, j., mikolajczak, m. and hansenne, m. (2009). “increasing emotional intelligence: (how) is it possible?” personality and individual differences, 47, 36–41. doi: 10.1016/j.paid.2009.01.046 neophytou, l. (2013). emotional intelligence and educational reform. educational review, 65(2),140–154. doi: 10.1080/00131911.2011.648171 new leaders for new schools (2009). principal effectiveness: a new principal ship to drive student achievement, teacher effectiveness, and school turnarounds. report. resource document. new leaders for new schools http://www.schoolsmovingup.net/cs/smu/view/rs/24121 pérez-escoda, n., filella, g., bisquerra, r. and alegre, a. (2012). desarrollo de la competencia emocional de maestros y alumnos en contextos escolares. electronic journal of research in educational psychology, 10(3), 1183-1208. pérez-escoda, n., filella, g., soldevila, a., and fondevila, a. (2013). evaluación de un programa de educación emocional para profesorado de primaria. educación xxi, 16(1), 233-254. bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 51 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3265 social and technological sciences issn: 2341-2593 pillen, m., beijaard, d. and den brok, p. (2013). tensions in beginning teacher’s professional identity development, accompanying feelings and coping strategies. european journal of teacher education, 36(3), 240–260. doi: 10.1080/02619768.2012.696192 schutz, p. a. and zembylas, m. (ed.) (2009). advances in teacher emotion research: the impact on teachers’ lives. boston: springer. doi: 10.1007/978-1-4419-0564-2 vaello orts, j. (2009) el profesor emocionalmente competente. un puente sobre aulas turbulentas. barcelona, grao. van veen, k. y lasky, s. (eds.) (2005) emotions, teacher identity and change, emotions, teacher identity and change (monográfico). teaching and teacher education, 21 (8), 995–1006. van uden, j. m., ritzen, h. and pieters, j. m. (2013). i think i can engage my students. teachers´ perceptions of student engagement and their beliefs about being a teacher. teaching and teacher education, 32(1), 43–54. doi: 10.1016/j.tate.2013.01.004 valdivieso, a. carbonero, m.a. and martín-antón, l.j. (2013). elementary school teachers’ self-perceived instructional competence: a new questionnaire. revista de psicodidáctica, 18(1), 47-78. doi: 10.1387/revpsicodidact.5622 zembylas, m. & schutz, p. a. (2009). reserch on teachers´emotions in education: findings, practical implications and future agenda. in schutz, p. a. and zembylas, m. (ed.), advances in teacher emotion research: the impact on teachers’ lives (pp. 367– 377). boston: springer. doi: 10.1007/978-1-4419-0564-2_18 bernal and ibarrola (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 33-52 | 52 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 postural and dynamic analysis of the human body: the relevance of the functional modulator factors in the methodological design albert puig-diví1*, anna prats-puig1 1euses-school of health and sport science, university of girona, spain. av. francesc macià, 65, 17190 salt, spain. * corresponding author: email: prof.apd@euses.cat; phone: + 34 972405130 received: 2015-03-10; accepted: 2015-07-28 abstract functional studies analyze the interaction between the human body and its environment studying both, the posture and the movement. it is well known that at the universities where health science and sports studies are taught there is a need to raise academic activities involving functional studies. the main objective is to make awareness of the factors that can modulate a certain function in the human body and must be well known for university students to perform a good research projects and career. to do this, we present a detailed list of intrinsic and extrinsic functional modulator factors (fmf) of the human body and propose a new methodology to teach those skills. during both, the project data collection and data reduction, fmf such as sitting and standing postures or walking, running and jumping movements must be taken into account. the recognition of the fmf that may influence the results must be previously known and identified to succeed in the methodological design of functional studies of the human body. the novelty of our work is not to direct the attention to the lecturer but to focus it to the student’s academic support to accomplish the active work like the fieldwork presented in this paper. in conclusion, the understanding of the fmf by university students using fieldwork activities during their degree would help them to develop a critical attitude in terms of validity and scientific evidence. keywords modulating factors, function, posture, movement, methodology puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 95 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 1. introduction one of the fundamental and primary human principles is to control our environment, to achieve this, you must perform certain functions. among the most important non-vital functions of the human being is the postural control including seating, standing and the dynamic control named walking, running or jumping (cano et al. 2012). in fact, posture and movement are not different entities, understanding standing as a posture movement and movement as a quick succession between different positions (trew et al. 2006). both, movement and posture involve a huge amount of information to be properly processed by the nervous system. the integration of the information and the motor responses are very complex, this do not only require a sophisticated network of neuronal connection between the different structures of the central and peripheral nervous system, but also involve other body systems. the efficient and adapted functions performed by the central and peripheral nervous system, muscles, joints, and the skeleton, and their actions in the context of movement, is known as neuromotor control (shumway-cook et al. 2007). since the neuromotor control of postural support and locomotion is dependent on many variables that can modulate and alter the stability and progression of the human body it is essential to control these variables in all functional analysis of the human body. all these variables are functional modulator factors (fmf), which can be divided in intrinsic and extrinsic fmf (collado, 2002). intrinsic factors refer to any conditions that are specific to the subject as a unique and particular individual, whereas extrinsic factors are physical aspects that directly affect the biomechanical behavior of the human body. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 96 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 2. proposed methodology: integration of different teaching methods the proposed methodology will be developed in a biomechanics course which is offered during the second year of the physiotherapy degree. this course consists of 6 european credit transfer and accumulation system (ects credits) where contents on tissue, joint biomechanics and kinesiology biomechanics are studied. the aim of this course is to develop the following competencies in the students: 1. teamwork. 2. oral communicative skills. 3. critical skills. 4. to know and understand the sciences, models, techniques and tools in which physical therapy are built. 5. to assess the patients’ functional status. almost half of the credits of this course (2.5 ects) will be devoted to the functional study of human walking. to work these concrete contents and develop the general skills that must be achieved during the course, a set of activities are raised. these activities are oriented around fmf that can modify the function analysis of human walking and must be controlled during the study. these activities will be: 1. master class. 2. tutorials. 3. watching and hearing documents. 4. fieldwork. 5. oral presentation. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 97 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 6. problem-based learning (pbl). 7. exams. using the individual marks for each activity, the final record of the biomechanics subject will be calculated as follows: record = (20% fieldwork) + (20% oral presentation) + (20% pbl) + (40% exams) 2.1. methods with high implication of lecturer 2.1.1. master classes the main objectives are: 1) to known the different existing fmf and classify them according to their intrinsic and extrinsic nature; 2) to be able to understand how the fmf can influence the functional analysis. the main competencies to be developed by the student are: • the ability to recognize the fmf. • to be able to control this fmf in the methodological design of human functional studies.  intrinsic functional modulator factors o age there are few periods over the human development in which the behavior is different depending on the biological age of the subject. following the acquisition of walking abilities around 12 months of age the central nervous system is still in maturation. it is considered to be at the age of 5-7 years when the anthropometric and the biomechanical differences between children and adults disappears (kirtley 2006; viel et al. 2002; sutherland et al. 1988; sutherland, 1997; shumway-cook et al. 2007). it has been shown puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 98 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 that the static balance and motor synchronization improves with age in children at the age of 5 to 12 years (geuze, 2003; debrabant et al. 2012). moreover, during adolescence it does not exist a linear relationship between the spatiotemporal parameters of human walking and age because there is a peak in bone growth which unbalances the neuromotor control (shumway-cook et al. 2007; schmidt et al. 2005). finally, in adults above 65 years old, human gait decreases in speed, in stride length and joint range and increases in cadence, in bipodal support, variability in muscle activity and energy cost (perry et al. 2010; whittle, 2002; kirtley, 2006). it is still unclear whether the variation of these parameters is due to the alterations in the musculoskeletal system or as a result of the slower gait seen in the elderly population (oatis, 2009). even so, there is consensus that the quality of static and dynamic functions decreases with age (hernandez et al. 2012; huang et al. 2013). o gender although it is considered that women walk with more cadences and less length stride it has been shown that these differences disappear when it is normalized by the anthropometric data of each sex population (oatis, 2009; nerín et al. 1999). however, previous studies have revealed small gender differences in the movement of pelvis and trunk during walking; man’s trunk is moved more laterally while women preserve the verticality but moves more her hips during walking (dufour et al. 2006). in addition, men maintain the arms more static than women during walking (li et al. 2001). in relation to sports there are also some differences in postural control between genders (ku et al. 2012), women presents minor movements of postural balance during sports (greve et al. 2013). the small differences found between men and women during human gait have not been found in children (kirtley, 2006; viel et al. 2002; sánchez et al. 1999; sutherland et al. 1988; sutherland, 1997; nerín et al. 1999). o racial and cultural puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 99 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 although specific features are due to the human race and the three biotypes, there is no evidence that any of these factors could modulate differential functional behaviors. however, the cultural behaviors of a particular social group may determine the gait pattern, as in japanese culture in which giving small steps is appreciated as a value of elegance (collado, 2002). o mass according to the first law of newton, the stability of a body is conditioned by their weight. the human body is not an exception, thus postural stability improves when it has a greater mass (méndez, 2006). when a disturbance destabilizes a heavy body in standing, a greater muscular work is required in order to control the center of mass and maintain the postural stability (méndez, 2006; shumway-cook et al. 2007). it is also known that a given disturbance destabilizes more easily light bodies than heavy ones (joshi et al. 2008). furthermore, it has been shown a worse postural control in children with obesity due to the mismatch between the rate of fat and muscle mass in terms of force generation (d'hondt et al. 2008; mcgraw et al. 2000). o height body height modifies the baseline parameters of human walking (perry et al. 2010; kirtley, 2006; whittle, 2002; vaughan et al. 1999; viladot, 2000; viel et al. 2002). a shorter human body walking at the same speed than a taller one, will show a higher cadence to compensate the reduced stride length. referring to the angular level, an increased rotation of the shoulder and pelvic girdles in anthropometrical shorter subjects will also be appreciated (collado, 2002). o musculoskeletal system together with the visual, vestibular and somatosensory system, the mechanical state of the musculoskeletal system can affect the neuromotor control of the posture and puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 100 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 movement (westcott et al. 1997). the main musculoskeletal problems can be grouped into muscle weakness alterations, loss of range of joint mobility and postural and articular misalignment (shumway-cook et al. 2007). o fatigue the muscles motor activity and the concomitant generated force are conditioned by the availability of energy. when this energy is deficient fatigue starts. fatigue is a multifactorial state that must be controlled to avoid an asymmetric posture (paillard et al. 2013; viner et al. 2008), fatigue, cannot be controlled with only one variable (suay et al. 1998). o nervous system to ensure a good body stability and orientation of all his segmental components during an activity by the neuromotor control (woollacott et al. 2005; woollacott et al. 1998; rama lópez et al. 2003), an adequate motor strategy together with an integration of the visual, vestibular and somatosensory system is required (alemán et al. 2003; martín et al. 2004; rama lópez et al. 2003). the integrity of the nervous information ensures an appropriate movement pattern and postural stability to guarantee an appropriate adaptation of the body to the environment (shumway-cook et al. 2007). it is also required a healthy peripheral nerves which can transport the somatosensory afferent information to the spinal cord and to higher centers, and later on to transmit the efferent or motor impulses to the relevant muscular system (wiebers et al. 2007). o cognitive-perceptual dimension the gait functionality is governed by technical and biomechanical constraints but is heavily managed and influenced by mental parameters such as fear of falling or other feelings of insecurity (shumway-cook et al. 2007). the individual psychological conditions influences the neuromotor control in general and the postural control in puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 101 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 particular (trew et al. 2006). zok et al. (2008) have previously shown how the posturographical records were altered in subjects with a simple change in the instructions, showing less stability when participants were asked to "to stay standing quietly" versus a the second statement which was "to stand as still as possible". personality and mood varies between subjects and influences directly to the human gait (collado, 2002 and shumway-cook et al. 2007). o attention the human being has a very low level of awareness of the movements performed daily to carry out various human functions successfully. movements that initially have been learned with a high level of willingness, end with an automatic and unconscious execution (trew et al. 2006). controlling the standing and walking are considered automatic activities, nevertheless, numerous investigators have demonstrated that it can be highly influenced by the conscious part of the brain (blanchard et al. 2005; tiernan et al. 2008; zaino et al. 2008; rankin et al. 2000; brown et al. 1999; tomita et al. 2010). o experience and motor learning motor learning is an adaptive and an internal processes that associated with the practice and experience can lead to a relatively permanent changes in the ability to produce motor activities and behavior (abernethy et al. 2005; schmidt et al. 2005). there are different types of learning: the simplest way is the habituation or non-associative learning, which consist in the integration of a single action (bertoti, 2004), the complex way is the associative learning which consist in learning a new task that requires its repetitive practice (shumway-cook et al. 2007). in conclusion, the reproducibility and repeatability of a particular task or function is conditioned by the prior and practical notion we have of it. o pain puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 102 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 pain is an unpleasant sensory and emotional experience associated to a present or potential injury (ibarra, 2006). the physical pain response is immobilization and protection of the painful area to avoid more stress to the involved tissues (trew et al. 2006). during a painful walk it can be seen a reduction of the load intensity, the time of support and an inactivation of the joint mobilization, stretching and muscle activation (sánchez et al. 1999; perry et al. 2010). since the presence of pain is not always verbalized by the subject, especially by children, it is recommended to ask explicitly about the presence or absence of pain (hicks et al. 2001). although pain involves a subjective perception is convenient to quantify it using numeric, visual or analog scales (quiles et al. 2004).  extrinsic functional modulator factors o ground firstly, ground type influences the reaction force so the impacts increase with increasing pavement hardness. moreover, the inclination of soil influences human posture and gait. secondly, when walking on a flat ground one leg has an accelerator role and the contralateral leg has a decelerator role interspersed with steps, while walking on slops both legs involved in the same function. also, the stride length is inversely proportional to the slope. specifically, when the slope is ascending the body leans forward and foot plays a predominant propulsive function in the lower extremity. in contrast, when the slope is downward body leans back and foot is in equines playing a braking role in the lower extremity. in the event that the slope inclination is lateralized, body leans towards the higher side, placing the foot on that side in eversion and the most fallen in supination (collado, 2002). o footwear puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 103 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 there are many different kinds of shoes, according to its material and design, or according to the sport or activity carried with them. shoes can produce either beneficial biomechanical effects (cushioning and protection) or undesired effects (articular or postural alterations). the use of heels in standing, shifts the subject’s mass center forward flexion and increases across the lower extremity, pelvic anteversion and lumbar lordosis, while in walking heels reduces step length and gait speed (rueda, 2004; sánchez et al. 1999). o clothing apart from the convenience of using lightweight and tight clothes for gait observation (perry et al. 2010; kirtley, 2006), clothing can influence gait pattern. for example, a heavy coat induce the subject to walk hunched forward or a narrow skirt forced to do a shorter steps and a higher cadence (collado, 2002). o base of support every solid body not in suspension is held by support points delimiting a surface on the floor; this is the base of support (bos) (lópez et al. 2003; oatis, 2009). the spatial relationship between the body base and its base of support set the stability of the body. in order to study this ratio the body center of mass (com) should be determined and the vertical line of force should be projected, the balance will be found when the force falls within the center of the polygon (lópez et al. 2003; ferdjallah et al. 2002). o center of mass when the position of the center of mass descends being closer to the ground, the stability of the body increases, and conversely (méndez, 2006). the stability of a posture like standing and the dynamic control of a skill like driving, are related to the relationship between the position of the center of mass and central limits of bos (de graaf-peters et al. 2007; peydro de moya et al. 2005; baydal-bertomeu et al. 2004). puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 104 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 o segmentation the stability of a column consisting of different parts or segments is more complex than others formed by a single piece (méndez, 2006). the stability of a multi-segmented column such as the human vertebral column, can improve when the segments are positioned into the vertical (méndez, 2006; lópez et al. 2003; dufour et al. 2006). o speed the walking speed increases with the increase of: 1) the cadence; 2) the stride length; 3) the variability in the steps amplitude; 4) the joint range of joints; 5) the ground reaction forces; 6) the intensity of muscle activity; 7) work and power in all joints of the locomotive unit; 8) the extent of vertical and horizontal displacement of the center of mass (oatis, 2009; grieve et al. 1966; damiano et al. 2006; johnston et al. 2011; perry et al. 2010; kirtley, 2006; whittle, 2002; viel et al. 2002; sánchez et al. 1999; gage et al. 2009; vaughan et al. 1999). o friction a good firmness between tangential forces of the support areas and the contact surfaces, improves the stability of the body, in the way that, when friction increases, stability increases (méndez, 2006; hewitt, 2004). o loads during daily routines we often loads different weights that are distributed and grasped in different ways (collado, 2002). when the human body interacts with a heavy load the center of mass is shifted towards the load to stabilize the new disturbance (horak et al. 1986). for example, when we are carrying a loaded backpack onto the back, the oscillations of the higher center of masses are minimized, reducing the shoulder movement, the pelvic girdle and the arms swing while walking (collado, 2002). puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 105 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 2.2.2. tutoring tutorials are optional and can be performed individually or in groups not exceeding 10 students. students must attend with concrete and properly prepared doubts according to the content of the previous lectures. therefore, the main objective for both the teacher and the student is to clarify any doubts or complete any information gaps. 2.2. methods with high implication of the student. independent work. 2.2.1. document viewing and listening students should watch certain documents that are linked to class content in websites, blogs and videos recommended by the teacher in addition to the master class assistance. 2.2.2. fieldwork activity fieldwork will be done by 3 or 4 students as teamwork. it will consist in recording the human gait of 4 subjects who will be walking freely following a scientific validated methodology in terms of technique and biomechanics. the objective of this activity is to consolidate the theoretical knowledge of all fmf studied in class in a real human walking kinematic analysis example. each team can choose a specific fmf and perform a functional analysis analyzing variations in this factor. for example, if the variable "speed" is chosen, 4 subjects will be recorded walking firstly at a slower speed and then at a faster speed, afterwards the team will have to analyze the kinematic parameters and study how are they altered by changes in speed. the work will be presented on cd or dvd with the following documents: 1) the informed consent of the 4 participants; 2) recorded and edited videos (3 replicates for each condition and participant); 3) statistical analysis; 4) a detailed document containing the hypotheses, the study design, the results and the main conclusion; 5) the oral presentation file. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 106 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 in this project, the students will be offered the possibility to record the video in a cinematic laboratory whit the researchers support. this will allow us to have both: the experimental group ⇒ university students who performs the records in the kinematic laboratory. the control group ⇒ students who record the video elsewhere without additional educational support. 2.2.3. oral presentation all the team members must actively participate on the oral presentation of their own fieldwork. the maximum exposure time is 5 minutes and the use of english will be positively evaluated. 2.2.4. problem based learning (pbl) it consists in solving 10 practical problems or case studies in small groups and with the teachers’ support. the aim is to actively work on the theoretical knowledge taught in the master class. finally, these pbl will be interactively solved with the teacher. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 107 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 2.2.5. exams five on-line type test exams will be set to perform with the help of students’ notes. the aims of these written tests are: 1) to evaluate the knowledge acquired by students and 2) to stimulate a continuous students learning. results in this study there were 133 students included in the control group and 48 students included in the experimental group. qualification records for each activity according groups are shown in table 1 and table 2 for all the studied subjects (n=188). table 1: qualifications of the studied subjects. all students control group experimental group p-value n 181 133 48 - pbl 6.78 ± 1.89 6.91 ± 1.51 7.38 ± 1.11 0.058 oral presentation 7.09 ± 2.17 7.42 ± 1.50 7.88 ± 1.10 0.071 fieldwork activity 7.54 ± 2.02 7.72 ± 0.94 8.63 ± 0.79 p<0.0001 final exam 6.76 ± 1.91 6.98 ± 1.45 7.33 ± 1.18 0.172 final record 7.00 ± 1.73 7.20 ± 1.00 7.71 ± 0.75 0.001 data are shown as mean and standard deviation. pbl: problem based learning. p values are from mannwhitney u test. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 108 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 table 2: fieldwork activity qualifications of the studied subjects. all students control group experimental group p-value n 181 133 48 - informed consent collection 0.91 ± 0.22 0.89 ± 0.24 0.96 ± 0.14 0.081 work presentation 0.88 ± 0.17 0.88 ± 0.17 0.88 ± 0.17 0.975 bibliography used 0.33 ± 0.39 0.29 ± 0.38 0.44 ± 0.39 0.016 introduction 0.42 ± 0.37 0.42 ± 0.34 0.41 ± 0.43 0.851 hypothesis and objectives 0.95 ± 0.12 0.95 ± 0.12 0.95 ± 0.09 0.649 data collection quality 3.97 ± 0.86 3.62 ± 0.72 4.96 ± 0.14 p<0.0001 data reduction 2.49 ± 0.52 2.42 ± 0.52 2.67 ± 0.44 0.003 data presentation 0.89 ± 0.28 0.96 ± 0.11 0.71 ± 0.46 0.005 data interpretation 0.97 ± 0.10 0.96 ± 0.11 0.98 ± 0.05 0.402 final work 4.12 ± 0.69 4.05 ± 0.71 4.30 ± 0.63 0.033 data are shown as mean and standard deviation. p values are from mann-whitney u test. students included in the experimental group showed 1 more point in average of the fieldwork activity (p<0.0001) and 0.5 more points in average of the final record (p=0.001) than the students included in the control group. no differences were seen in the problem based learning activities, the oral presentation or the final exam records (table 1; figure 1). puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 109 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 figure 1: qualifications of studied subjects according to groups (n=181). graphs represent error bars. p values are from mann-whitney u test. taking into account only the records of the fieldwork activity, four factors 1) the bibliography used (p=0.016); 2) the data collection quality (p<0.0001); 3) the data reduction (p=0.003) and 4) the data presentation (p=0.005) significantly differed between groups, and concomitantly, the final work qualification (p=0.033). no differences were seen in the records obtained for the informed consent collection, the work presentation, the introduction, hypothesis and objectives and finally, the data interpretation (table 2; figure 2). puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 110 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 figure 2: fieldwork activity qualifications of studied subjects according to groups (n=181). graphs represent error bars. p values are from mann-whitney u test. 4. conclusions the academic results of the biomechanics course were satisfactory in general but shown better performance in the group of students who have enjoyed greater academic support during the data collection in the fieldwork activity. while there are other works that have their own indicators of superior academic performance and in which the need to increase financial resources for the improvement of the academic performance is reported (garbanzo-vargas, 2007; jara et al., 2008) the puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 111 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 novelty of our work is not to direct the attention to the lecturer actions but to focus the academic support to the active work of the students like the fieldwork we have presented in this paper. moreover we suggest that academic reinforcement could be done by fellows who have previously participated as student in the fieldwork and to set strategies to develop a constructive and cooperative approach for the active learning of the student. although the lecturer and principal author of this study is who has evaluated the student’s activities, the academic support was offered by a second lecturer who not corrected the fieldwork activities nor participated in the preparations of this paper. 5. acknowledgments euses-school of health and sport science, university of girona, spain. 6. references abernethy, b., hanrahan, s. j., kippers, v., mackinnon, l. t., & pandy, m. g. (2005). the biophysical foundations of human movement. (2ª ed.) leeds: human kinetics. alemán, o., pérez, n., & sánchez, n. (2003). rehabilitación vestibular. rev med univ navarra, 47, 72-76. baydal-bertomeu, j. m., guillem, r., soler-gracia, c., peydro de moya, m. f., prat, j. m., & barona de, g. r. (2004). determinación de los patrones de comportamiento postural en población sana española. acta otorrinolaringol.esp., 55, 260-269. http://dx.doi.org/10.1016/s00016519(04)78520-9 puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 112 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/s0001-6519(04)78520-9 http://dx.doi.org/10.1016/s0001-6519(04)78520-9 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 bertoti, d. b. (2004). functional neurorehabilitation through the life span. philadelphia: f.a. davis co. blanchard, y., carey, s., coffey, j., cohen, a., harris, t., michlik, s. et al. (2005). the influence of concurrent cognitive tasks on postural sway in children. pediatr.phys ther., 17, 189-193. http://dx.doi.org/10.1097/01.pep.0000176578.57147.5d brown, l. a., shumway-cook, a., & woollacott, m. h. (1999). attentional demands and postural recovery: the effects of aging. j.gerontol.a biol.sci.med sci., 54, m165-m171. http://dx.doi.org/10.1093/gerona/54.4.m165 cano, r. & collado, s. (2012). neurorrehabilitación. métodos específicos de valoración y tratamiento. madrid: médica panamericana. collado, s. (2002). análisis de la marcha humana con plataformas dinamométricas. influencia del transporte de carga. [tesis doctoral]. madrid. d'hondt, e., deforche, b., de, b., i, & lenoir, m. (2008). childhood obesity affects fine motor skill performance under different postural constraints. neurosci.lett., 440, 72-75. http://dx.doi.org/10.1016/j.neulet.2008.05.056 damiano, d. l., laws, e., carmines, d. v., & abel, m. f. (2006). relationship of spasticity to knee angular velocity and motion during gait in cerebral palsy. gait posture, 23, 1-8. http://dx.doi.org/10.1016/j.gaitpost.2004.10.007 de graaf-peters, v. b., blauw-hospers, c. h., dirks, t., bakker, h., bos, a. f., & hadders-algra, m. (2007). development of postural control in typically developing children and children with puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 113 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1097/01.pep.0000176578.57147.5d http://dx.doi.org/10.1093/gerona/54.4.m165 http://dx.doi.org/10.1016/j.neulet.2008.05.056 http://dx.doi.org/10.1016/j.gaitpost.2004.10.007 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 cerebral palsy: possibilities for intervention? neurosci.biobehav.rev, 31, 1191-1200. http://dx.doi.org/10.1016/j.neubiorev.2007.04.008 debrabant, j., gheysen, f., vingerhoets, g., & van, w. h. (2012). age-related differences in predictive response timing in children: evidence from regularly relative to irregularly paced reaction time performance. hum mov sci, 31, 801-810. http://dx.doi.org/10.1016/j.humov.2011.09.006 dufour, m. & pillu, m. (2006). biomecánica funcional. barcelona: masson elsevier. ferdjallah, m., harris, g. f., smith, p., & wertsch, j. j. (2002). analysis of postural control synergies during quiet standing in healthy children and children with cerebral palsy. clin.biomech., 17, 203-210. http://dx.doi.org/10.1016/s0268-0033(01)00121-8 gage, j. r., schwartz, m. h., koop, s. e., & novacheck, t. f. (2009). the identification and treatment of gait problems in cerebral palsy. (2ª ed.) london: mac keith press. garbanzo-vargas, g. (2007). factores asociados al rendimiento académico en estudiantes universitarios, una reflexión desde la calidad de la educación superior pública. revista educación, 31, 43-63. http://dx.doi.org/10.15517/revedu.v31i1.1252 geuze, r. h. (2003). static balance and developmental coordination disorder. hum mov sci, 22, 527-548. http://dx.doi.org/10.1016/j.humov.2003.09.008 greve, j. m., cug, m., dulgeroglu, d., brech, g. c., & alonso, a. c. (2013). relationship between anthropometric factors, gender, and balance under unstable conditions in young adults. biomed.res.int., 2013, 850424. http://dx.doi.org/10.1155/2013/850424 puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 114 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/j.neubiorev.2007.04.008 http://dx.doi.org/10.1016/j.humov.2011.09.006 http://dx.doi.org/10.1016/s0268-0033(01)00121-8 http://dx.doi.org/10.15517/revedu.v31i1.1252 http://dx.doi.org/10.1016/j.humov.2003.09.008 http://dx.doi.org/10.1155/2013/850424 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 grieve, d. w. & gear, r. j. (1966). the relationship between length of stride, step frequency, time of swing and speed of walking for children and adults. ergonomics, 9, 379-399. http://dx.doi.org/10.1080/00140136608964399 hernandez, m. e., ashton-miller, j. a., & alexander, n. b. (2012). the effect of age, movement direction, and target size on the maximum speed of targeted cop movements in healthy women. hum mov sci, 31, 1213-1223. http://dx.doi.org/10.1016/j.humov.2011.11.002 hewitt, p. g. (2004). física conceptual. (9ª ed.) méxico. hicks, c. l., von baeyer, c. l., spafford, p. a., van, k., i, & goodenough, b. (2001). the faces pain scale-revised: toward a common metric in pediatric pain measurement. pain, 93, 173-183. http://dx.doi.org/10.1016/s0304-3959(01)00314-1 horak, f. b. & nashner, l. m. (1986). central programming of postural movements: adaptation to altered support-surface configurations. j neurophysiol., 55, 1369-1381. huang, m. h. & brown, s. h. (2013). age differences in the control of postural stability during reaching tasks. gait.posture., 38, 837-842. http://dx.doi.org/10.1016/j.gaitpost.2013.04.004 ibarra, e. (2006). una nueva definición de "dolor". un imperativo de nuestros días. rev.soc.esp.dolor, 13, 65-72. jara, d., velarde, h., gordillo, g., guerra, g., león, i., arroyo, c. et al. (2008). factores influyentes en el rendimiento académico de estudiantes del primer año de medicina. an fac med, 69, 193-197 puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 115 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1080/00140136608964399 http://dx.doi.org/10.1016/j.humov.2011.11.002 http://dx.doi.org/10.1016/s0304-3959(01)00314-1 http://dx.doi.org/10.1016/j.gaitpost.2013.04.004 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 johnston, t. e., watson, k. e., ross, s. a., gates, p. e., gaughan, j. p., lauer, r. t. et al. (2011). effects of a supported speed treadmill training exercise program on impairment and function for children with cerebral palsy. dev.med child neurol., 53, 742-750. http://dx.doi.org/10.1111/j.1469-8749.2011.03990.x joshi, j. & kotwal, p. (2008). essentials of orthopaedics and applied physiotherapy. elsevier. kirtley, c. (2006). clinical gait analysis: theory and practice. london: churchill livingstone elsevier. ku, p. x., abu osman, n. a., yusof, a., & wan abas, w. a. (2012). biomechanical evaluation of the relationship between postural control and body mass index. j.biomech., 45, 1638-1642. http://dx.doi.org/10.1016/j.jbiomech.2012.03.029 li, y., wang, w., crompton, r. h., & gunther, m. m. (2001). free vertical moments and transverse forces in human walking and their role in relation to arm-swing. j.exp.biol., 204, 4758. lópez, a. & lópez, e. (2003). biofísica aplicada a la biomecánica del cuerpo humano. madrid. martín, e., barona, r., comeche, c., & baydal, j. m. (2004). análisis de la interacción visuovestibular y la influencia visual en el control postural. acta otorrinolaringol esp, 55, 9-16. http://dx.doi.org/10.1016/s0001-6519(04)78476-9 mcgraw, b., mcclenaghan, b. a., williams, h. g., dickerson, j., & ward, d. s. (2000). gait and postural stability in obese and nonobese prepubertal boys. arch.phys med rehabil., 81, 484-489. http://dx.doi.org/10.1053/mr.2000.3782 puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 116 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1111/j.1469-8749.2011.03990.x http://dx.doi.org/10.1016/j.jbiomech.2012.03.029 http://dx.doi.org/10.1016/s0001-6519(04)78476-9 http://dx.doi.org/10.1053/mr.2000.3782 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 méndez, r. (2006). evaluación y análisis de la influencia de la manipulación global de la pelvis: estudio baropodométrico y estabilométrico. [tesis doctoral]. madrid. scientific european federation of osteopaths-escuela de osteopatía de madrid. nerín, s., villarroya, a., cepero, e., gómez, j., marco, c., & miguelena, m. (1999). diferencias en la marcha según el sexo. biomecánica, 7, 27-38. oatis, c. a. (2009). kinesiology: the mechanics and pathomechanics of human movement. (2ª ed.) philadelphia: lippincott williams & wilkins. paillard, t. & borel, l. (2013). unilateral and bilateral fatiguing contractions similarly alter postural stability but differently modify postural position on bipedal stance. hum mov sci, 32, 353-362. http://dx.doi.org/10.1016/j.humov.2012.12.001 perry, j. & burnfield, j. m. (2010). gait analysis: normal and pathological function. (2ª ed.) thorofare, nj (usa): slack incorporated. peydro de moya, m. f., baydal-bertomeu, j. m., & vivas broseta, m. j. (2005). evaluación y rehabilitación del equilibrio mediante posturografía. rehabilitación, 39, 315-323. http://dx.doi.org/10.1016/s0048-7120(05)74365-6 quiles, m. j., van-der hofstadt, c. j., & quiles, y. (2004). instrumentos de evaluación del dolor en pacientes pediátricos: una revisión. rev.soc.esp.dolor, 11, 360-369. rama lópez, j. & pérez fernández, n. (2003). pruebas vestibulares y posturografía. rev med univ navarra, 47, 21-28. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 117 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/j.humov.2012.12.001 http://dx.doi.org/10.1016/s0048-7120(05)74365-6 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 rankin, j. k., woollacott, m. h., shumway-cook, a., & brown, l. a. (2000). cognitive influence on postural stability: a neuromuscular analysis in young and older adults. j gerontol.a biol.sci med sci, 55, m112-m119. http://dx.doi.org/10.1093/gerona/55.3.m112 rueda, m. (2004). podología. los desequilibrios del pie. barcelona: paidotribo. sánchez, j. j., prat, j. m., hoyos, j. v., viosca, e., soler, c., comín, m. et al. (1999). biomecánica de la marcha humana normal y patológica. valencia: publicaciones instituto biomecánico de valencia. schmidt, r. a. & lee, t. d. (2005). motor control and learning: a behavioural emphasis. (4ª ed.) leeds: human kinetics. shumway-cook, a. & woollacott, m. h. (2007). motor control. translating research into clinical practice. (3ª ed.) philadelphia: lippincott williams & wilkins. suay, f., ricarte, j., & salvador, a. (1998). indicadores psicológicos del sobreentrenamiento y agotamiento. rev psicol dep, 7-27. sutherland, d. (1997). the development of mature gait. gait & posture, 6, 163-170. http://dx.doi.org/10.1016/s0966-6362(97)00029-5 sutherland, d. h., olshen, r. a., biden, e. n., & wyatt, m. p. (1988). the development of mature walking. london: mac keith press. puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 118 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1093/gerona/55.3.m112 http://dx.doi.org/10.1016/s0966-6362(97)00029-5 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 tiernan, c. w. & angulo-barroso, r. m. (2008). constrained motor-perceptual task in infancy: effects of sensory modality. j.mot.behav., 40, 133-142. http://dx.doi.org/10.3200/jmbr.40.2.133142 tomita, h., fukaya, y., honma, s., ueda, t., yamamoto, y., & shionoya, k. (2010). anticipatory postural muscle activity associated with bilateral arm flexion while standing in individuals with spastic diplegic cerebral palsy: a pilot study. neurosci.lett., 479, 166-170. http://dx.doi.org/10.1016/j.neulet.2010.05.059 trew, m. & everett, t. (2006). fundamentos del movimiento humano. (5ª ed.) barcelona: masson & elsevier. vaughan, c. l., davis, b. l., & o'connor, j. c. (1999). dynamics of human gait. (2ª ed.) western cape: human kinetics publishers. viel, é., asencio, g., blanc, y., casillas, j.-m., esnault, m., laassel, e.-m. et al. (2002). la marcha humana, la carrera y el salto: biomecánica, exploraciones, normas y alteraciones. barcelona: masson. viladot, a. (2000). lecciones básicas de biomecánica del aparato locomotor. barcelona: springer-verlag iberica. viner, r. m., clark, c., taylor, s. j., bhui, k., klineberg, e., head, j. et al. (2008). longitudinal risk factors for persistent fatigue in adolescents. arch.pediatr.adolesc.med, 162, 469-475. http://dx.doi.org/10.1001/archpedi.162.5.469 puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 119 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.3200/jmbr.40.2.133-142 http://dx.doi.org/10.3200/jmbr.40.2.133-142 http://dx.doi.org/10.1016/j.neulet.2010.05.059 http://dx.doi.org/10.1001/archpedi.162.5.469 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3754 social and technological sciences issn: 2341-2593 westcott, s. l., lowes, l. p., & richardson, p. k. (1997). evaluation of postural stability in children: current theories and assessment tools. phys.ther., 77, 629-645. whittle, m. w. (2002). gait analysis: an introduction. (3ª ed.) london: elsevier health sciences. wiebers, d. o., dale, a. j., kokmen, e., & swanson, j. w. (2007). examination in neurology. (7ª ed.) barcelona: mayo foundation. woollacott, m. h., burtner, p., jensen, j., jasiewicz, j., roncesvalles, n., & sveistrup, h. (1998). development of postural responses during standing in healthy children and children with spastic diplegia. neurosci.biobehav.rev, 22, 583-589. http://dx.doi.org/10.1016/s0149-7634(97)00048-1 woollacott, m. h. & shumway-cook, a. (2005). postural dysfunction during standing and walking in children with cerebral palsy: what are the underlying problems and what new therapies might improve balance? neural plast., 12, 211-219. http://dx.doi.org/10.1155/np.2005.211 zaino, c. a. & mccoy, s. w. (2008). reliability and comparison of electromyographic and kinetic measurements during a standing reach task in children with and without cerebral palsy. gait.posture., 27, 128-137. http://dx.doi.org/10.1016/j.gaitpost.2007.03.003 zok, m., mazza, c., & cappozzo, a. (2008). should the instructions issued to the subject in traditional static posturography be standardised? med eng phys., 30, 913-916. http://dx.doi.org/10.1016/j.medengphy.2007.12.002 puig-diví & prats-puig (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 95-120 | 120 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1016/s0149-7634(97)00048-1 http://dx.doi.org/10.1155/np.2005.211 http://dx.doi.org/10.1016/j.gaitpost.2007.03.003 http://dx.doi.org/10.1016/j.medengphy.2007.12.002 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 digital literacy and metaphorical models c. girón-garcía*, i. navarro i ferrando dpto. de estudios ingleses. universitat jaume i de castelló, av. sos baynat s/n, 12071 castelló de la plana, spain. * corresponding author: email: gironc@uji.es; phone: + 34 964729621 received: 2013-12-31; accepted: 2014-08-11 abstract it is an acknowledged fact that the appearance of new genres in cyberspace has shifted the main focus of instruction strategies nowadays. learners of any field are challenged by the acquisition of a new type of literacy, digital literacy –how to read and write, or how to interact, in and through the internet. in this line, websites often show expressions like "home", "visit", "down-load", "link", etc. which are used in a new sense that did not exist before the digital era. such expressions constitute the manifestation of mental models that have been transferred from traditional conceptual domains onto the new knowledge domain of the internet. these conceptual metaphors are some of the cognitive models that help in the conceptualization of new cybergenres. this paper points at describing how these cognitive models build our notion of diverse cybergenres in english – e.g. the weblog, the social network, the cybertask. our aim here consists in detecting these metaphorical models as well as describing and classifying their conceptual mappings between domains. with that purpose, some digital materials are analyzed, so as to test the hypothesis that such mappings and models guide the user's representation of the genre, as a coherent structure. the results make evident some implications on the relevance of digital literacy in educational contexts. keywords digital literacy; spontaneous digital literacy; idealized cognitive models; metaphor; learning styles girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 160 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 1. introduction toms & campbell (1999: 3) point out that present day digital reality has prompted researchers to explore users’ interaction with online genres. on the other hand, shepherd & watters (1998: 98) remark the crucial role of genre in the development of the notion of interface in the internet. some recent work also suggests that digital navigation patterns may be guided by previous knowledge about spatial models such as house, site, or journey (navarro 2008; navarro et al. 2008; navarro & silvestre 2009). the aim of this paper is to show some evidence that metaphorical models play a role in the comprehension and production of texts, particularly digital texts in the internet. moreover, our research aims at characterizing the role of metaphorical models in the construction of digital genres as different from other genres as far as content organization and structure are concerned. the fact that metaphorical conceptualization plays a role in the comprehension and production of texts as context-bound and genre-bound communicative events has been often pointed out (caballero 2003, 2005; ponterotto 2000, 2005). in relation to online interaction, kress (2004) develops the notion of trajectory in the use of digital environments. our intention is to show some ways in which metaphorical models provide a guide for internet readers to construct meaning throughout their trajectories across hypertexts. consequently, the concrete purpose of this work is to identify the nature of some metaphorical idealized cognitive models (henceforth icms) that give coherence to the design and organization of websites. the notion of icm was developed by lakoff (1987) in order to systematize cultural knowledge into shared packages or configurations of knowledge that are used by community members so as to interpret their social experiences coherently. in this line, expressions such as “home” or “visit” are used in new digital contexts –cybergenres– by virtue of a projection or mapping from the social domain onto the digital domain. in this context, we aim at disclosing the connection between metaphorical models and ‘digital literacy’. accordingly, the reading of cybergenres turns out to be a cognitive process where users give meaning to digital environments through the implementation of girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 161 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 previously well-entrenched cognitive models. in addition, the study of the productive and processual mechanisms that govern interaction with online genres can be fundamental to the teaching and development of autonomous skills for language learning in hypermedia environments. 2. digital literacy the appearance and wide use of computers, as well as the internet, offers the opportunity to devise changes in our conception of literacy, allowing for a more precise skill notion, i.e. ‘digital literacy’ (girón-garcía 2013; girón-garcía & gaspar 2012). kern (2000: 16-17) highlighted the fact that students practising literacy in a non-native language would learn more about not only vocabulary and grammar, but also about discourse, that is, constructing meaning through structuring their thoughts. in addition, kress (2004) argued that the concept of ‘literacy’ could be understood from both the linguistic and cognitive dimensions. understanding ‘literacy’ from the linguistic dimension has to do with the way people use their language, in particular, the written language. on the other hand, from the cognitive perspective, ‘literacy’ is related to an active participation at the cognitive level. thus, digital reading may be seen as a cognitive process through which users establish relationships between what they perceive on the screen and their previous knowledge configurations in order to associate meaning to a digital environment. thus, schmar-dobler (2003) points out that the appearance of computers with linked information (internet) provokes changes in literacy. as a result, the screen has become the main channel of communication where the combination of reading and technological skills has created a new form of literacy for the management of the web and the internet. thus, digital literacy demands that users learn how to use technological resources, as well as being able to surf the web, use e-mail, evaluate information, and the like (labbo et al. 2003; schmar-dobler 2003). girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 162 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 in this line, girón-garcía (2013) suggests that the association of the required technology skills together with previously shared generic models gives rise to a new literacy that not necessarily needs being learnt through instruction, but that new generations may acquire spontaneously. in such context, the term ‘new literacy’ as understood in the literature is conceived as a new way of reading on the web. traditionally, we used to read on paper format (books, articles, reviews…) but with the arrival of the internet, a new way of reading appeared (‘digital texts’). in addition to this broad notion of new literacy, we suggest the more specific notion of ‘spontaneous digital literacy’, which implies individuals who surf the net in order to gather information with the condition that these users have not received any previous or specific instruction or training on how to do it. this spontaneous literacy incorporates the management of ‘new technology skills’, which can be defined as the ability to use and combine internet resources in order to solve problems, activities, tasks, or simply to satisfy personal needs. by the present time, the world wide web is already used steadily as a huge resource of information. as web documents can be manipulated like any ordinary computer file, users can cut and paste text, graphics, sound, and video into their own personal documents. thus, in an era of technological revolution and new technologies the need arises to develop new forms of media literacy, computer literacy, and multimedia literacy. in this new environment, traditional print literacy preserves a fundamental importance in the process of developing new reading and writing abilities in the computer-mediated cyberspace. since media are a central part of our cultural experience, training in media literacy should begin in early childhood and continue into adulthood, as “new technologies are constantly creating new media and new genres, technical innovations and conventions are constantly emerging” (kellner 1997: 5). educators need to pursue the challenge to teach digital literacy while using media materials that contribute to advancing multicultural education. thus and thus, many teachers have discovered that media materials can be valuable in a variety of instructional tasks, helping to make complex matters accessible and engaging. however, media cannot girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 163 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 be a substitute for print material and classroom teaching; rather they should be seen as a supplement to traditional materials. an example of instructional tasks would be the design of language learning ‘cybertasks’ which help students develop the ability to read online texts and construct their own texts, what luzón & ruiz-madrid (2008) call an autonomising ‘wreading’ competence, i.e.: “the ability to understand the pragmatic, discursive and semiotic features of online texts, harness their affordances and interact with them in various ways, find relevant information in different semiotic modes within and across these texts, and relate and meaningfully use such information in order to achieve a specific purpose, complete a task or produce an output” (luzón & ruiz-madrid 2008: 28). in this particular respect, students may receive the benefit of instruction in digital literacies in order to foster the use of a new kind of abilities related to both the management of ict’s and the comprehension of digital environments. accordingly, new types of tasks require that learners are provided with a virtual environment that complements the traditional face-to-face teaching activities and materials (girón-garcía & ruiz-madrid in press). the design of these virtual environments as a new reality cannot escape, on the one hand, the existence of processual strategic tendencies in students, also called “learning styles”, and on the other hand, the use of cognitive models which are already present in the culture. as villanueva (1997) points out, learning styles are characterized according to the more or less frequent use of certain cognitive and pragmatic strategies related to different learning procedures (such as being teacher-dependent, individual, inductive, positive attitude towards ict’s, among others). it is an extended idea that “learning styles are the biologically and developmentally imposed set of characteristics that make the same teaching method wonderful for some and terrible for others” (dunn & griggs, 1988: 3). girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 164 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 although individual tendencies exist, we should avoid thinking that a particular learning style provides a global characterization of each individual learner. thus, a learner may employ various strategies belonging to different learning styles if s/he has to face different tasks or experiences. more precisely, in our view learning styles are not to be conceived as fixed behavioural schemes that determine individual behaviour. in this sense, styles are characterized as a bundle of learning strategies correlated in a meaningful manner, that is, whose appearance frequency does not mean that different learning traits belong to different learning styles used by the same individual (giróngarcía & gaspar 2012: 144-145). however, girón-garcía (2013) suggests that the notion of learning style seems to be too broad because a student may manifest traits in one style or another at different stages of the same task. for this reason, we avoid talking about ‘learning style’ in absolute terms, but we prefer to do it rather in terms of ‘learning style traits’, which in combination with other traits may constitute complex learning profiles or ‘style-blendings’ (girón-garcía 2013: 435-436). in other words, this combination of strategies (i.e. style-blending) is what we call blended learning profile, which has an influence on how students solve a given task. ‘style-blending’ can be defined as a combination, mixture or synthesis of strategies that characterize a particular student’s learning profile required at a particular stage along a complex task. the design of cybertasks and other digital environments is expected to meet the needs of diverse learners’ profiles, and therefore, their makeups may also incorporate diverse cognitive models. in turn, the presence of different cognitive models in the same digital environment makes it possible for users to switch from one learning profile to another (style-switching). furthermore, digital tasks might demand the successive activation of different profiles. all in all, the users’ management of their own learning traits as well as their knowledge of cultural cognitive models provides better conditions for the acquisition of effective digital literacy. 3. cognitive models and digital literacy girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 165 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 digital users’ linguistic background includes both social and individual experience. the reader’s interpretive process needs generic conventions as management mechanisms that assign texts a global organization, a content and a context. consequently, genres raise expectations that underlie textual comprehension processes. in addressing a text, users turn to previously known genre comprehension skills. we hypothesize that metaphorical models help users in genre identification, by contributing cognitively to the delineation of the text macrostructure (van dijk & kintsch 1983) as a knowledge structure. thus, website readers resort to their previous knowledge of other genres. moreover, adaptation to digital media requires supplementary new reading strategies to cope with links, menus, etc. in order to grasp the content, purpose, organization, and function of a digital document, and consequently use it effectively, users should be able to identify the features that make it different to other kinds of documents. as toms & campbell (1999: 2) suggest, genre can work as an organizing metaphor that helps users in both the identification process and the subsequent interaction process. in identifying a genre, the user activates a mental model that brings about a set of expectations and inference pathways, which facilitate textual interpretation and use. as far as each user is able to activate these pathways, we may describe processes of spontaneous digital literacy. this research pursues the characterization of digital environments in the internet medium, given that new technologies make it possible for new generic structures and organizational patterns to appear. as villanueva et al. (2008) point out, “hypertextual technology offers the possibility of rhizome-based designs for hypermedia texts by providing users/readers with multiple choices through hypertexts. each user may find their way through hypertexts as long as spontaneous digital literacy is activated. as suggested above, a crucial component of that spontaneous digital literacy is constituted by the user’s ability to shift and blend learning-style traits in the process of adaptation to digital environments. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 166 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 below, we show how metaphorical models provide a conceptual guide for users to construct meaning throughout their trajectories across hypertexts. our aim is to identify the nature of some metaphorical models that give coherence to the design and organization of websites, provide an anchor to previous genre knowledge and allow for learning-style shifts and blends. we understand metaphorical models as mappings of idealized cognitive models (icms), onto the digital domain. lakoff (1987) defined icms as knowledge structures, which are idealized in the sense that they do not necessarily reflect real objective situations in nature, they are cognitive because they are part of cultural knowledge, and they are models because they provide coherent configurations that are used to interpret and make sense of social experience. as an example, the week in the western calendar provides an icm for our understanding of time cycles. so, tuesday is a category, which acquires its sense within and by virtue of the model (lakoff 1987: 68-69). in our view, metaphor is conceived as a conceptual mechanism by which an icm – that is, its structural elements and their operational relations – is projected or mapped onto a more abstract cognitive domain as target. in the process, the source icm provides a configuration and coherence for the understanding of the target domain (lakoff 1993). for example, in the metaphor reading a website is visiting a location, the icm that our culture offers for our understanding of visiting locations provides the elements and relations that allow users to configure their knowledge about how to read a website. thus, our claim goes that some metaphorical models play a significant rhetorical role in displaying an interactive hyperdiscourse in websites. according to toms & campbell (1999: 2): system designers use a metaphor at the point of interaction to teach the user how to manipulate the interface. the user ‘loads’ the metaphor into the working memory […], and the similarity between the structure of the girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 167 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 metaphoric image and the structure of the interface enables the user to exploit prior knowledge to understand the system and work with it. in the first place, toms & campbell refer to prior knowledge rather vaguely; secondly, these authors consider that the user loads the metaphor introduced by the system designer; and finally, they point at similarity between domains as the factor that enables users to exploit prior knowledge. in our view, even though the designer triggers off the use of a particular metaphor, previous knowledge about that metaphor source domain is previously shared by designers and users. shared knowledge is well structured into conventional models based on social and individual experience. moreover, rather than similarity between the structure of the interface and prior knowledge schemas, in our view prior knowledge is mapped onto the conceptualization of the interface structure, as a new abstract cognitive domain. in addition to images, visual layout, page frames, among other multimodal resources, linguistic expressions like bookmark, link, site map, or visitor show the existence of subjacent metaphorical models that offer users a scheme for conceptualizing, structuring and giving coherence to website discursive configuration. these linguistic forms constitute clear evidence of the existence of particular models in the designers’ and users’ conceptual systems. these models guide the users’ representations of websites and, in turn, their reading process and their decisions along navigation. we assume, therefore, that users turn to their conceptual background in their decision-making along the reading process. our procedure begins at looking for metaphorical expressions, images, frames, information layout, etc. and goes on to identify these formal features with the conceptual models (icms) they express. then, we analyse the configuration of such cognitive models. in a future experimental phase we may test to what extent users’ navigation is guided by such models, so as to determine to what extent they contribute to the users’ representation of their own navigation, and in turn to their digital literacy. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 168 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 for instance, if we categorize the expression ‘visit our site’ as a linguistic manifestation of the metaphor “reading a website is visiting a location”, then we will probably find more expressions pertaining to the same model, like come in, exit, go ahead, go back to… etc. thus, to what extent is the metaphor “reading a website is visiting a location” part of the users’ previous knowledge? or, to what extent is it ingrained in the users’ conceptualisation of their own reading? finally, how does it help users to make decisions on their reading or navigating process – consciously or unconsciously? in the following, we identify metaphorical expressions and the metaphors they express, and we describe their conceptual mappings from source domains onto target domains. the metaphorical models that result may be deemed as facets of the website cybergenre, since they contribute to hypertextual coherence and architecture, they are idiosyncratic of cybergenres, and presumably, may characterize reading strategies and modes. 4. metaphorical models in websites in the present section we characterize source domains that map onto the target domain “website”, a very recent domain in our conceptual system. secondly, we analyse the mappings. finally, we investigate the entailments of the models for the users’ understanding of the target domain. users are not aware of the systematic mappings between source and target domains. on the contrary, speakers’ knowledge of such mappings is largely unconscious, and it is only for the purpose of analysis that domain maps are brought into awareness. as kövecses (2010) points out, it is the occurrence of metaphorical linguistic expressions that reveals the existence of the conceptual metaphors, i.e. the forms of language provide evidence of the existence of ways of thinking. in fact, it is rather difficult to speak about the domain of websites without turning to the usual linguistic expressions – site, link, navigate, home, etc. – which are the manifestation of the metaphorical models that constitute our object of study. however, in order to understand the mapping we are compelled to build a sort of girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 169 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 literal meta-language that serves to describe the domain as literally as possible. thus, if we think of our experience of the domain “website” literally what we find is the following: a designer elaborates a software program and stores it into a computer server at a real physical location. we – probably sitting on a chair – face a computer screen. we switch on the computer that is linked to the server by means of a telephonic connection. we move the pointer of our electronic device (mouse), for it to approach and cover a section on the screen; then, we press a button of our device to activate another screen; a new layout appears where we recognize labels, pictures or photographs, and most probably a written text. we read labels and paragraphs, and see that certain words or expressions are coloured or underlined. we understand that these marked expressions can be used to activate new screens; we decide to place the pointer again on a label or marked expression so as to activate another screen which appears with yet a different layout, written texts, labels and marked expressions. so, apparently we interact physically with the electronic device under our hand that causes the activation of screens, and we also read the linguistic expressions and paragraphs on the screen that a designer has created for us. actually, we do not know what is going to appear next on the screen as we click on a marked expression. the real makeup of the available material is hidden, and we have no physical access to it. thus, if we were to interact with the internet only in terms of the conceptualisation and representation of what we are actually physically doing, it would be rather difficult for us to make decisions on what we want to activate and, consequently, read or visualise. interaction with our computer would turn out to be rather weird and would most probably go awry. that is why human beings resort to metaphorical models –conceptual metaphors – that provide a coherent structure for previously unknown realities, domains that have not been experienced previously or do not allow for physical direct experience (abstract domains). the unknown, new or abstract domain is called target domain, whereas the previously known, and well-understood one, is called source domain because it is the conceptual source for understanding the target domain (kövecses 2010; lakoff 1993). girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 170 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 in fact, we fairly interact with websites, make decisions as we activate screens, and use internet materials and resources. some cognitive domains that we draw from our previous experience help us in understanding our interaction with the computer. navarro & silvestre (2009) illustrate five source domains that map onto the abstract domain that we usually know as ‘website’. these domains are: house, site, journey, book, and net. • the house model the model is conceived as a house that is visited by people mainly because it hosts some social activities, events or objects. thus, when we enter the house we may be required to give a password, if access is limited, or sign in a reception book so that our visit gets logged. there may be someone, the house master, who welcomes visitors and invites them to come in. once in the house we go around and may go back and forth visiting different rooms. within the rooms we may browse among the objects or materials, or even we may be allowed to use diverse devices or appliances, for instance a toolbox or a message board where we can post messages for other people. there may be a visitors’ book, where visitors are invited to write down their comments. further elements may be added as long as the general logic of the model is maintained, for example special rooms devoted to particular purposes, like a chat room. • the site model in this model, a site master welcomes and invites visitors to go around, and they may be requested to sign in a reception book so that their visit gets logged, or they may move back and forth a path. visitors arrive at a site that may be a rather large area, and therefore they might need a sitemap, and possibly some directions, to find their way in the site, for instance a notice including you-are-here indications. some sites may have a shop, and an info desk where visitors get answers to frequently asked questions. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 171 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 • the journey model the journey model is active through the concept ‘home’, which recalls the location where a journey begins and ends. the home is also the place that indicates the point of departure to any destination. once en route the traveller (surfer) uses instruments containing choice menus for navigation, i.e. finding the way to a destination. destinations are not limited or predetermined by the master or by maps, rather users may freely find their own way and choose their own course. links make it possible for travellers to reach further locations. • the book model the book model is reminiscent of the traditional representation and conceptualisation of the reading process as associated to paper formats. thus, the website consists of pages that users may browse through. one can go from one page to the next page and back to previous pages. there is an index or a contents table, and we can bookmark an interesting page. • the net model the net model is activated by the expression link, as it is used to refer to activation labels and marked expressions that lead to new screens. nets are extended works where all nodes are linked to each other, so that the unity of the whole is guaranteed. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 172 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 site house journey book net site house home main page net sign in sign in navigation logging logging navigate “you are here” panel directory links contents links invitation enter start now browse map board navigation menu index external links follow us come in search find visit visit go to bookmark connect welcome welcome explore image of a book back to …. back to… back to… area room location page visitors’ book visitor’s book visitors visitors readers password toolbox housemaster table 1. source domains and their linguistic expression (adapted from navarro & silvestre 2009) 5. websites and metaphorical expressions in this section, we show a series of examples of how source domain expressions may appear in actual websites. for the purpose of illustration we have selected a few websites and we have traced the occurrence of linguistic expressions that hint to the identification of metaphorical models. our analytical procedure is of an inductive character. we have looked into websites from different parts of the world. the linguistic expressions of source domain concepts mostly occur as labels on a menu. nevertheless, it is interesting to look at them in other contexts because contexts make it apparent that the thinking process is grounded on the source domain rather than the target domain. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 173 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 for the purpose of the present work, we have considered the following websites for analysis: [1] http://www.harrods.com/ [2] www.webquest.org [3] http://rubistar.4teachers.org/ [4] www.facebook.com [5] http://houseoffairytales.org/ [6] www.wikipedia.com [7] https://en.wikisource.org/wiki/main_page [8] http://www.bmwusa.com/ • the site model as for the site model, the most usual expressions are the words site (which occurs explicitly) and the expression site map or, alternatively, sitemap. another idiosyncratic element is an indication of the already covered route in the form of a poster including the sentence you are here [1] 1. the metaphor is fully deployed in [1], where the master welcomes visitors, and his text unfolds its coherence according to the metaphorical model: (1) welcome! your opinion is important to us. after your visit, would you take a moment to answer a few simple questions about your experience? in this extract, several structural elements from the site model reinforce its presence in the discourse. visitors are welcome, invited to enjoy and to recommend the experience to 1 numbers in square brackets refer to the list of websites above. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 174 http://www.harrods.com/ http://www.webquest.org/ http://rubistar.4teachers.org/ http://www.facebook.com/ http://houseoffairytales.org/ http://www.wikipedia.com/ https://en.wikisource.org/wiki/main_page http://www.bmwusa.com/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 other potential visitors. other expressions that reinforce the model are “visiting”, “store guide”, “follow us!” concerning the site model, expressions such as welcome, you’ve arrived at, or you’ll find something here are the most commonly employed. in this fashion, website [2] welcomes visitors, and the text coheres as follows: (2) welcome you’ve arrived at the most complete and current source… …you’ll find something here to meet your needs. in this extract, several structural elements of the site model reinforce its presence in the argumentative discourse. accordingly, visitors arrive at a place where they will find something that would satisfy their needs. words or expressions such as welcome, visit, login, sign up, and link are found in website [3]. (3) welcome to rubistar! visit rubistar's inspiration page… registration and use of this tool is free, so click the register link in the login area… in extract (3) we find several expressions of elements belonging to the site model, the website invitation, the conception of a place, and the possibility to register. there is an area for registration and someone who meets and welcomes visitors. the following examples also show expressions from website [4], which also represents the site model. thus, expressions such as “sign up”, “log in”, “password” provide connotations of registration (sign up, log in), and access to a place. • the house model many internet locations are straightforwardly denominated “house”, a fact that immediately triggers the house cognitive model in the users’ minds, as in [5]. in that location the first screen shows the following text that makes the house model evident by means of linguistic form: girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 175 http://www.facebook.com/ multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 (5) we will be moving to a brand new home, our new exciting website thus, actions like moving into a new home are linguistically expressed to communicate a change of website. the linguistic expression activates the cultural icm in the user’s mind. • the book model the wikipedia [6] and analogous sites incorporate the book model, since they use expressions like “contents” in the menu where all the possible links can be clicked upon at the main page. accordingly the initial screen is not called “home”, but “main page”. other expressions that are reminiscent of the book model are link labels such as “contact page” or “create a book”. the dynamics of the interaction calls for a cognitive model where the reader browses through the pages of a book and can even put together a bunch of pages to compile a book. in [7] we find further expressions like “index” or “subject index”, and diverse links that use the expression “page”, such as “help page”, “special page”, “page information”, which activates the book model in several occasions. • the journey model the journey model is activated by expressions like “start now”, “explore” or “search” as it is the case in [8]. • the net model the net model is activated by expressions like “links”, “net”, “external links” and “connect” as in the case of [3]. 6. conclusions in the light of the data expounded in previous sections, we may suggest that digital literacy is probably very much influenced by previous cultural knowledge (cognitive models). we have shown how some source domains allow for metaphorical cognitive models used in internet genres. these icms provide coherence to genres in the internet girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 176 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 because they are previously ingrained in both users’ and designers’ minds. that entrenchment constitutes, therefore, a crucial factor that fosters spontaneous digital literacy. to what extent do users connect or map previous experiential models to the digital environment? in that respect, users will be able to use digital environments more fluently and dynamically to the extent that they are able to perform those mappings. as a consequence, the identification and description of metaphorical mappings in cybergenres may help elucidate the connection between spontaneous digital literacy and culture. our data make evident that the metaphors described in this paper are used in the englishspeaking world. since we have not searched websites in other languages we may not affirm that the same models are activated to the same extent in users who are not native in the english speaking culture. it will be therefore a goal in further research to check whether the same models are also active in other languages and cultures. 7. final considerations regarding digital literacy, education and learning as suggested above, new genres in cyberspace have changed the main focus of instruction strategies. thus, learners are challenged by the acquisition of new digital literacies (read and write in the context of cybergenres) through the internet. nevertheless, not only students are challenged by new literacy acquisition. teachers also need to face a new kind of instruction procedures that integrate the employment of icts in the teaching-learning arena (girón-garcía & gaspar 2012). along this line, the management of icts alone is not the only factor that students need to be proficient in. furthermore, students’ face the challenge of becoming aware and managing their own learning styles because these can determine to what extent they will be able to manage the new technologies with adequate proficiency. we are aware of the fact that learning styles and all the concepts derived from them are not the sole factors to determine the level of success in students’ management of digital literacy. however, we believe that they constitute an important variable. finally, we view the possibility to further explore to what extent the concepts ‘style-blending’ and ‘style girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 177 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 switching’ are promoted by metaphorical competence, i.e the ability of becoming aware of icms. 8. references caballero, r. (2003). metaphor and genre: the presence and role of metaphor in the building review. applied linguistics. 24/2, 145-167. doi: 10.1093/applin/24.2.145 caballero, r. (2005). the rhetoric of metaphor in genre settings: the case of building reviews. in maalej, zouhair (ed) metaphor, cognition and culture: selected papers from the fourth international conference on researching and applying metaphor (raam iv). manouba-tunis, tunisia: faculty of letters, arts and humanities, 3759. dunn, r., and s. griggs. (1988). learning styles: quiet revolution in american sec ondary schools. reston, va.: national association of secondary school principals. girón-garcía, c. (2013). learning styles and reading modes in the development of language learning autonomy through “cybertasks", ph.d. universitat jaume i. retrieved from http://hdl.handle.net/10803/125440 girón-garcía, c. and ruiz-madrid, m. n. (2014). design and development of a webquest for the course ‘english applied linguistics: an introduction’ (ea0914). evaluation of its educational role in the classroom. girón-garcía, c. and gaspar-verdú, m. v. (2012). foreign language learning in ict contexts: a study of learning styles and reading modes in english, french and german. edulearn12 proceedings, 143-148. kern, r. (2000). literacy and language teaching, oxford university press, 2000. kress, g. (2004). literacy in the new media age. london: routledge. kellner, d. (1997). "intellectuals, the new public spheres, and technopolitics," new political science 41-42 (fall): 169-188. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 178 http://hdl.handle.net/10803/125440 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 kövecses, z. (2010). metaphor. a practical introduction, oxford: oxford university press. labbo et al. (2003). teacher wisdom stories: cautions and recommendations for using computer-related technology for literacy instruction [electronic version]. reading teacher, 57 (3), 300-304. lakoff, g. (1987). women, fire and dangerous things. what categories reveal about the mind. the university of chicago press. doi: 10.7208/chicago/9780226471013.001.0001 lakoff, g. (1993). the contemporary theory of metaphor. in a. ortony (ed.) metaphor and thought. cambridge university press. doi: 10.1017/cbo9781139173865.013 luzón, m.j. and ruiz-madrid, n. (2008). learning to learn in a digital context: language learning webtasks for an autonomising “wreading” competence. corell. luzón, m. j., ruiz-madrid, m. n. and villanueva, m. l. (2009). students´ perceptions of language learning webtasks to develop electronic literacy. in helping people to learn foreign languages: teach-niques and teach-nologies. universidad católica san antonio de murcia. 156-173. navarro, i. (2008) metaphorical icms in cybergenre representation, vol 7, studia universitatis petru maior. series philologia, targu mures.167-177. navarro, i. and a. j. silvestre. (2009). the role of icms in cybergenre representation and reading modes. in m. navarro coy (ed.) practical approaches to foreign language teaching and learning. linguistic insights 78. berlin/ny: peter lang. 269-292. navarro, i., m. l. villanueva, c. girón-garcía and a. j. silvestre (2008). cybergenres and autonomous language learning: pragmatic strategies and cognitive models in the production and processing of digital genres, inted2008 proceedings, international association for technology, education and development, iated publications.777-786. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 179 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.2991 social and technological sciences eissn: 2341-2593 ponterotto, d. (2000). the cohesive role of cognitive metaphor in discourse and conversation. in barcelona, antonio (ed) metaphor and metonymy at the crossroads: a cognitive perspective. berlin-new york: mouton de gruyter, 283298. ponterotto, d. (2005). text, context and cognitive metaphor. in baicchi, annalisa et al. (eds) modelling thought and constructing meaning. cognitive models in interactions. milano: franco angeli, 156169. schmar-dobler, e. (2003). reading on the internet: the link between literacy and technology. journal of adolescent and adult literacy, 47(1), 80-85. shepherd, m. a., watters, c. r. and kennedy, a. (2004). cybergenre: automatic identification of home pages on the web. j. web eng., 3(3-4), 236-251. shepherd, m. a. and watters, c. (1998). the evolution of cybergenres. proceedings of the 31st hawaii international conference on system sciences. maui, hawaii: ieee, 2, 97109. toms, e. g. and campbell, g. (1999). genre as interface metaphor: exploiting form and function in digital environments. proceedings of the 32nd hawaii international conference on system sciences. maui, hawaii ieee press, 2, 9pp. villanueva, m.l. and navarro, i. (1997). los estilos de aprendizaje de lenguas. castellón: publicacions de la universitat jaume i, col.lecció summa, sèrie filològia 6. villanueva, m. l. et. al. (2008). understanding cybergenres as semiotic artefacts: meaning and cognition beyond standardized genres. computers and composition on line. amsterdam: elsevier publishing. van dijk, t.a. and kintsch, w. (1983) strategies of discourse comprehension. new york. academic press. girón-garcía and navarro (2014) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 1 nº 2 (2014): 160-180 | 180  the house model  the site model  the journey model  the book model  the net model multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 an interdisciplinary complex problem as a starting point for learning: impact of the pbl method in second-year environmental engineering students e. sáez de cámara1*, a. lopez-urionabarrenechea1, m.n. azpiazu1, p. ruiz de arbulo2, g. insunza2 faculty of engineering, university of the basque country upv/ehu (1) department of chemical and environmental engineering (2) department of business administration alameda de urquijo s/n, 48013 bilbao, spain * corresponding author: email: estibaliz.saezdecamara@ehu.eus; phone: + 34 946018212 received: 2015-04-03; accepted: 2015-05-08 abstract three courses of the second year degree in environmental engineering (geology and pedology, ecology and economics and business administration) have been remodeled using the problem-based learning methodology. the proposed problem is a real-life and integrative problem related to their specialization which must be solved in these three courses at the same time. the results reveal that during this experience students were considerably more active, cooperative and involved, and the success rate doubled that of similar engineering courses of the faculty. regarding students’ opinion, it should be emphasized that they perceive that this method is functional and encouraging. a high percentage of the students describe the experience as positive or very positive. additionally, they stated that the problem-based learning promoted the development of skills that, in their own view, are essential for their career, such as teamwork and communication. keywords problem-based learning; environmental engineering; interdisciplinary approach; life-long learning; intrinsic motivation. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 153 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 1. introduction higher education in environmental engineering was implemented in spain in 2011 (ministerio de educación, cultura y deporte, 2011), at the same time as the modification in curricula for harmonization with the european education system. until 2011, the engineering solutions applied to pollution-related problems were designed by civil engineers, industrial engineers, forest engineers, chemists or biologists, leading to partial solutions focused on their respective field of knowledge. therefore, the reason behind the implementation of the new bachelor`s degree in environmental engineering was to train engineers with an integrated view of the environment, able to address complex environmental problems: environmental impacts of population growth, resource depletion, climate change, water scarcity and the fate and hazards associated with pollutants, among others. the degree in environmental engineering at the faculty of engineering of bilbao of the university of the basque country (upv/ehu) implies four years of study to obtain the bachelor qualification. in the first two years students learn core fundamental sciences (such as chemistry, physics, mathematics, mechanics, thermodynamics, geology, ecology, and business), some of them together with the students of some of the other degrees awarded at the faculty: industrial technology engineering and industrial organization engineering. conversely, the last two years are devoted to the application of scientific knowledge to solve practical problems, especially in industry. this structure, quite similar to the pre-bologna process one, leads to (i) unmotivated first and second year-students, which results in high dropout rates, and (ii) severe difficulties to integrate theory and practice in the third and fourth year-courses as well as in their future career. furthermore, the results of surveys of industry perceptions of engineering graduates of how well they were prepared for work showed gaps in other attributes that play an important role in effective performance in the workplace: communication and teamwork skills together with broad perspective of other disciplines (markes, 2006; crawley et al., sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 154 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 2007; richter, 2009). the last attribute, a holistic and an integrative approach, is essential in for our graduates since environmental engineering is defined as that branch of engineering concerned with the application of scientific and engineering principles for (i) the protection of human populations from the effects of adverse environmental factors, (ii) protection of environments, both local and global from the potentially deleterious effects of natural and human activities and (iii) improvement of environmental quality (aaee, 2008; abet, 2013). introducing and integrating these non-technical aspects in engineering education is not an easy task. innovative approaches based on active and cooperative methodologies can provide a solution to the abovementioned challenges. worldwide, an increasing number of institutions have changed, or they are in the process of changing, from traditional learning-teaching methods and moving towards these kind of approaches, as it has been demonstrated that they are more effective making graduates critical thinkers, problem solvers, life-long learners, effective communicators, team player and technically skilled. according to kolmos (1996), hmelo-silver (2004) and walker and leary (2009), the main advantages of using innovative methodologies are: 1) increase of the student’s intrinsic motivation, interest and implication, 2) reduction of the time to finish the bachelor degrees and a lower dropout rate, 3) augment of the ability to memorize concepts and apply the acquired knowledge, 4) development and strengthening of professional skills and 5) amplification of the capability to connect theory and practice. taking into account that such advantages cover the gaps identified in students of the degree in environmental engineering, an innovative and collaborative approach, problem-based learning (pbl) methodology, was designed and implemented in three courses of these new degree. the pbl method uses relevant problems presented at the beginning of the learning cycle and, employed to provide the context of the courses. together with the development of the curricula of the courses, the students, working in teams, go solving the problem by means of the application of theory, which concerns the sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 155 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 student about the direct relationship between the course and the real-life applications. teachers act as coaches and facilitators throughout the learning cycle. although the pbl evolved from medical education, it is regarded by some experts as the most suitable active and cooperative methodology for the simulation of tasks that are as close as possible to those carried out by engineers in their practice (kolmos, 1996; de graaf and kolmos, 2003; mills and treagust, 2003). the pbl experiment presented in this paper was designed and applied to three courses during the academic year 2013/14 in order to reduce the dropout rates of the first courses of the degree in environmental engineering, to increase the student’s implication and motivation, and to prepare the future graduates for research and for employment in industry by developing the inherent skills of the environmental engineers. it is divided into five main sections. section 2 provides a brief overview of the methodology. students’ opinions and academic results are presented in section 3. finally, section 5 summarizes the results and discusses the avenues for the future. 2. proposed methodology: problem based learning three courses of the 2nd semester of the 2nd year of the degree in environmental engineering (geology and pedology, ecology and economics and business administration) were remodeled in the academic year 2013/2014 using the innovative pbl approach. an additional originality of our work lies on the fact that the pbl was applied simultaneously in these three dissimilar courses, since the problem to be solved was the same for the three of them and students need concepts and skills which are developed in these three courses in order to solve entirely the problem. for this to be possible, a real-life and integrative problem related to their specialization was proposed. additionally, we formulated a contemporary problem close to real life, for the purpose of helping students to see the connections between the academic environment sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 156 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 and the real world. in particular, the dilemma was the selection of the most appropriate solution for managing and treating the organic fraction of the urban waste generated by a community. the proposal, presented figure 1, was a complex and open-ended problem. the students, working in small teams, had to choose and select, and then justify, the most appropriate solution. “_____________________ is a graduate in environmental engineering and an advisor for the city hall since a few years ago. in this town, as a result of the construction and occupation of new single-family houses, the municipal solid waste (msw) production rate has grown exponentially in the last ten years. since the landfill where they are actually dumped is exhausting its usable life and it will have to be closed in a short term and, at the same time with the aim of advancing towards the goals marked by the european legislation, different options are being considered for the treatment of the most abundant fraction in the msw: the organic matter. the city hall has several parcels of land that could be used for the construction of these treatment facilities. these parcels were acquired in 2007 to build an incineration plant and the auxiliary structures, but the pressure of the activists of several platforms and the neighbors’ rejection stopped the project. in the plenary session of the past 22 of january the council decided to ask the advisor ____________________ to analyze the technical, legal, social and economic viability of this solution, the measures and resources necessary to carry it out and to match the benefits compared with actual wastes management. the study should include a brief processes and installation descriptions, and a tentative target of economic analysis. in order to do so, the advisor will have access to all available information regarding this issue as well as to all necessary resources. the advisor ____________________ has three months to elaborate the document, which will be presented in the city hall meeting the first fortnight of may” figure 1. the problem proposed to environmental engineering second-year students as a starting point for learning this macro problem was divided into a number of partial problems dealing with contents, topics and methods of the three courses involved: (1) through the knowledge of geology and pedology’s concepts and calculations, students had to choose the most suitable plot to carry out the treatment process; (2) through the knowledge of ecology’s concepts and calculations, students had to described the selected waste decomposition process; and (3) sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 157 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 through the knowledge of economics and business administration’s concepts and calculations, students had to make an economic balance of the selected solution. both, the macro problem and the partial problems, were specifically selected and edited to meet the educational learning outcomes, competencies and criteria of these three 6-ects compulsory courses. as most of the students had no previous pbl experience, the learning process also concerned a series of activities related to introduce concepts and techniques. as recommended by experts (felder and brent, 1994; de graaf and christensen, 2004; hmelo-silver, 2004; borrego and newswander, 2007), activities involved understanding, research, decision-making and writing. figure 2 shows three examples, one of each course. e co no m y an d b us in es s a dm in is tr at io n the balance sheet by ekin s.a. in december 31, 2013 shows the following elements. assets liability building……………………………………..100.000 € building´s accumulated amortization...…… (30.000 €) customers……………………………………...5.000 € banks……………………..………………......20.000 € cash..…………………………………………..5.000 € equity……………..………………………….50.000 € long term debt……..………………………..30.000 € short term debt………………..……………...20.000 € during 2014 ekin s.a. has carried out the following transactions. • in january the first, ekin s.a. bought a van worth 15.000 €. it´s estimated life is 10 years and it´s residual vale 1.000 €. (paid through bank account) • the services offered by ekin s.a. during 2014 were worth 65.000 €, from which 50.000 € were collected through the bank account, leaving the rest for future payment. • during 2014 ekin s.a. faced the following expenses: • salaries: 40.000 € (paid through bank account) • combustibles: 1.500 € (paid through bank account) • insurances: 2.000 € (paid through bank account) • office materials: 500 € (paid through bank account) a) what´s the amortization rate for the van? b) build up the profit and loss account for period 2014-2015 c) build up the balance sheet for period 2014-2015 sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 158 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 e co lo gy decomposition process a) taking into account the availability of oxygen, how many types of decomposition processes are there? describe their main stages. b) what are the main organisms involved in decomposition? does it depend on the decomposition process? give an example of a decomposer organism. c) what are the main chemicals released during decomposition? does it depend on the decomposition process? provide the chemical names and the state (gas, liquid, solid, and dissolved) d) what are the main differences between the natural (ecological) decomposition of organic matter and the industrial decomposition of organic waste? g eo lo gy a nd p ed ol og y the analysis of two soil samples carried out in the laboratory reveal the following results regarding the solid-phase components and the cation exchange capacity (cec): sample 01 60 % clay, 3% organic matter cec = 76 cmolc∙kg -1 sample 02 60 % clay, 2% organic matter and cec = 30 cmolc∙kg -1 a) state which is main the type of clay in each soil sample b) calculate the maximum amount of nickel retained by a soil layer of 27 cm and 1 hectare of each soil sample. the density is approximately of 1350 kg∙m-3 for both of them. give the results in kilograms. c) if an accident would happen, which of the soils would be able to retain more efficiently pollutants avoiding aquifers’ contamination? figure 2. three examples of activities the macro problem was presented to the students at the end of the second week of the semester and a maximum of ten weeks were given for solving it. each week each course takes up approximately four hours in the timetable. working in faculty-supervised groups of not more than four members, as suggested by oakley et al. (2004), students solved the partial problems. simultaneously, students re-formulated the macro problem in terms that they could understand, devised an approach to a solution, identified what they knew and what they require to advance towards it, constructed new knowledge based on their previous knowledge and worked together to find a solution. the supervision plan designed by the lecturers consisted of three personalized meetings distributed along the ten weeks. in such meetings, the students presented the lecturers their progress on the partial problems concerning each specific course and the macro sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 159 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 problem and, they discussed about general issues too. these tutorials were not mandatory and they were performed on students’ demand. additionally, students could also ask explicit questions to a fourth-course student, who also took part in the pbl project as an advisor. the pbl project was assessed through an oral examination at the end of the semester as well as on the basis of a written report. the oral examination committee was composed of the three lecturers involved in the pbl project. this oral presentation took the 30% of the pbl project score. the score of the oral presentation of each group was agreed by the committee. the remaining 70% of the pbl project score corresponded to the written report, divided into two parts: general approach (15%, agreed by the lecturers) and the solution to the partial problems (45%, each lecturer evaluated issues regarding his/her course). the pbl project counts for 15 to 20 percent of the total score in the courses involved in the project. students’ assessment in each course was completed with their participation in classroom activities, written reports, laboratory sessions, computer labs and a final written test (gibbs and simpson, 2005). finally, students’ perception about this innovative experience was compiled using two opinion polls: one before the pbl experience and another one, after it. they included 10-15 questions and the chance to add comments. the questions were related to their knowledge and interest in these three courses, skills and the pbl methodology. below, the effects of the pbl approach on the students’ attitude towards the learning process together with the impact on the students’ performance and success rates are presented. 3. results and discussion our pbl experience has been compared to that already used in other courses that are taught in the first and the second year of the degree in environmental engineering as well as to these courses when the traditional (lecture-based) methodology was used. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 160 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 3.1. students’ opinion on the pbl initiative our investigation reveals that during this experience the second year engineering students were considerable more active, cooperative and involved than with the traditional methodology: the class attendance was close to a 100% and they devoted more hours to study these three courses. another point worth noting of this innovative experience was the good classroom atmosphere: students participated continuously and enthusiastically (frequently, one intervention per minute). this attitude was unusual with the traditional methodology. hence, the pbl approach seems to be more effective in terms of capturing students’ interest and sustaining motivation. these are some of the strengths reported in literature (major and palmer, 2001; macandrew et al., 2008). from the survey conducted before applying the pbl methodology, it was deduced that the students’ interest towards the utility of the three subjects involved in the project for their future professional practice, was medium-high. they showed the same degree level expectations in the opportunity of developing communication skills, decision making, autonomy to learn and ability to work in groups. lastly, the possibility of obtaining a remarkable final score in the courses involved in this multidisciplinary project was considered medium. figure 3 shows part of the results obtained in the opinion poll conducted at the end of the semester, after the oral presentation to the committee. the 72% of students rated the experience as quite satisfactory and the 7% as very satisfactory (figure 3, left). none of the students claimed they have learned less; moreover, the 72% stated that they have learned more and the 7% believe to have learned even much more with this approach than with the traditional methodology (figure 3, middle). as illustrated in the left graph, the 75% will repeat again this experience. the most remarkable result was that all of the students felt that the assessment system was adequate (not shown); the reverse is actually the case of most pbl experiences. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 161 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 figure 3. summary of the results of opinion polls about the pbl experience regarding the contribution of pbl to developing skills for their future professional career, it should be stressed that the 79% of students considered it quite an adequate methodology for learning and the 64% said that it has been quite useful to understand better theoretical contents of the disciplines involved in the pbl experience as well as to investigate with more independently. lastly, the 57% of the students think that the pbl approach helped them to take decisions on their own in the proposal of solutions to reallife problems and they also value the contributions of multiple fields to complex problems. table 1 presents the answers of improvement in certain abilities after the pbl experience. students were asked to rate their level of expertness before and after the pbl methodology on a scale from 1 (very low) to 6 (very high). the coefficients of the column improvement have been calculated as the mean of the differences between results before and after the application of the innovative methodology. results reveal that they think that there has been an improvement in all the evaluated abilities (the difference after-before is positive for all the evaluated competences) being higher in the following: diagnosis of environmental problems (+1.14), information search and analysis (+1.14), and preparation of technical reports (+1.07). in view of the obtained results, we can say that problem-based learning promotes development of skills that are essential in the modern engineering profession as they are in demand by future employers. this perception agrees with the study by akinoglu and tandogan (2007) suggesting that pbl sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 162 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 causes positive changes in social tendencies, such as making decisions with group members and team spirit. table 1. students´ opinion on the usefulness of the pbl approach to help the development of several abilities skill improvement information search and analysis 1.14 effective management of the time to study 0.07 ability to learn by oneself 0.64 diagnosis of environmental problems 1.14 teamwork 0.64 argumentation and critical analysis 0.43 oral communication 0.36 technical reports preparation 1.07 on the other hand, some of the students complain about the extensive work time; in their view, pbl was too time-consuming. the pbl did not “work” well with those students who did not attend to one of the three courses implied in the project. similar drawbacks have been reported by other instructors and researchers (kolari et al., 2008). 3.2 impact of pbl on the academic results. the improvement in success rates (approved/submitted), performance rates (approved/enrolled) and other ratios such as attendance, active participation and delivery of reports was remarkable for the 2013/2014 academic year. figure 3 shows the evolution in the percentage of students who took the test over the enrolled students. these data correspond to the last three academic years, which gather all the years in which the second course of the degree in environmental engineering has been active. the data show that this rate has increased considerably in the three courses involved in the pbl experience: 93.6% in geology and pedology (+5.6%), 86.4% in sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 163 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 ecology (+15.9%) and 100% in economy and business administration (+11.8%). these data, together with the results obtained in the opinion poll, corroborate that the experience has motivated students in a significant way. figure 4. evolution of the percentage of students who took the final test over the enrolled students in geology and pedology, ecology and economy and business administration (it includes may and july calls) the evolution of the average rates of submitted over enrolled for the three courses involved in pbl experiment versus the other six courses in the second year of the degree in environmental engineering is illustrated in figure 4. it shows that while the rate of submitted over enrolled was greater for the rest of the courses in the academic year 20112012, in the academic year 2013-2014 (and also in 2012-2013) the rate of the students submitted over enrolled reach the 93.4% for the courses involved in the pbl experience; a rate that is considerably above the average for the rest of the courses. as a matter of fact, the rate of submitted over enrolled for the academic year 2013-2014 in the courses involved in pbl experience was greater than the average rate of the rest of courses in any year (first, second, third and fourth year), and even the average rate of all the courses belonging the second course of the degree. 2011-2012 academic year 2012-2013 academic year 2013-2014 academic year geology and pedology 87,50 88,89 93,75 ecology 64,71 76,19 86,36 econom y and business adm inistration 87,50 88,89 100,00 60 70 80 90 100 sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 164 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 figure 5. evolution of the average rates of submitted over enrolled for the courses of the second year of the degree in environmental engineering. the courses applying the pbl approach (in red) are compared with the other ones, using lecture-based methods (in blue). figure 6 shows the evolution of the success rate, that is, the percentage of students who passed the test over students who took the test for the three courses involved in the pbl experience. results reveal that in ecology and economy and business administration the success rates improved considerably in the academic year 2013-2014 compared to previous years: 81.8% (+10.8%) and 94.5% (+12.1%), respectively. these data are even more valuable if we consider that the rate of student who took the test over the enrolled students increased too. the course geology and pedology employed the pbl approach since the degree in started. this fact explains the absence of an improvement in the academic year 2013-2014, which in any case, were outstanding: an average of 85.9%. 2011-2012 2012-2013 2013-2014 courses traditional approach 90,29 80,55 87,84 pbl courses 79,9 84,66 93,37 60 70 80 90 100 % students took test / enrolled sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 165 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 figure 6. evolution in the percentage of students who passed the test over the ones who took the test in geology and pedology, ecology and economy and business administration in the last three academic years. the evolution of the average success rates for the three courses involved in pbl versus the rest of the courses in the second course of the degree in environmental engineering is displayed in figure 7. results expose that success rates were greater in the courses involved in pbl project that for the rest of the courses. nevertheless, a small decline of the average success rates for the courses involved in the project can be identified: from 95.3% to 93.8%. this is due to the considerable downturn in geology and pedology; it causes a slight downturn in the average rate besides the improvements in economy and business administration and ecology, which join the pbl methodology in academic year 2013-2014. 2011-2012 academic year 2012-2013 academic year 2013-2014 academic year geology and pedology 87,50 88,89 81,25 ecology 70,59 71,43 81,82 econom y and business adm inistration 87,50 77,78 94,74 60 70 80 90 100 sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 166 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 figure 7. same as figure 5 for average success rates. the evolution of the performance rate, that is, the percentage of students who passed the course, over the enrolled is displayed in figure 8. in economy and business administration and ecology courses, the performance rates obtained in academic year 2013-2014 were higher than the ones obtained in previous years: 81.8% (+10.8%) and 94.7% (+12.1%). as in figure 6, the absence of improvement in geology and pedology is explained by the fact that the pbl methodology was introduced in this course in academic year 2011-2012 and has continued since then. 2011-2012 2012-2013 2013-2014 courses traditional approach 90,94 85,75 87,95 pbl courses 96,97 95,83 93,80 70 80 90 100 sucess rate 2011-2012 academic year 2012-2013 academic year 2013-2014 academic year geology and pedology 87,50 88,89 81,25 ecology 70,59 71,43 81,82 econom y and business adm inistration 87,50 77,78 94,74 60 70 80 90 100 sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 167 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 figure 8. evolution of the performance rates in geology and pedology, ecology and economy and business administration in the last three academic years. figure 8 shows the evolution of the average performance rates for the three courses involved in pbl versus the rest of the courses in the second year of the degree. while in the academic year 2011-2012 the performance rate was greater for the rest of the courses (82.31%), in 2013-2014 the situation reversed: 87.5% vs. 77.7%. it is also remarkable that during the studied period the average efficiency rate for the three courses involved in pbl experience has increased nearly a 10 %. figure 9. same as figure 5 for the average performance rates the final grades of the courses involved in the pbl experience for the last three academic years are detailed in table 2.the most remarkable result is the downturn in the percentage of students not taking the final test: 6.3% in geology and pedology (-5.6%), 13.6% in ecology (-16.0%) and 0% in economy and business administration (-11.8%). moreover, there was an improvement in the final scores in economy and ecology compared to the previous academic year, looking more like the scores in year 2011-2012 (first promotion of the degree). it is also remarkable the increase in the rate of students who reach an outstanding score in ecology in the academic year 2013/2014: 45.5%. we have also 2011-2012 2012-2013 2013-2014 courses traditional approach 82,31 70,21 77,71 pbl courses 77,94 80,95 87,45 60 70 80 90 100 performance rate sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 168 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 detected that the rate of students who reach an outstanding score increased in economy and business administration (31.6%) and geology and pedology (37.5%) compared to year 2013/2014, getting closer to levels obtained in the academic year 2011/2013. table 2. scores obtained in geology and pedology, ecology and economy and business administration in the last three academic years. geology and pedology ecology economy and business administration 20112012 20122013 20132014 20112012 20122013 20132014 20112012 20122013 20132014 not submitted 12,50 11,11 6,25 35,29 23,81 13,64 12,50 11,11 0,00 failed 0,00 0,00 12,50 5,88 4,76 0,00 12,50 11,11 5,26 pass (5.06.9) 37,50 66,67 43,75 41,18 71,43 40,91 31,25 72,22 63,16 outstanding (7.0 8.4) 43,75 22,22 37,50 11,76 0,00 45,45 37,50 5,56 31,58 excellent (8.5-10) 0,00 0,00 0,00 5,88 0,00 0,00 6,25 0,00 0,00 honor (10.0 +) 6,25 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 the following tables show the preliminary results of the hypothesis contrasts designed to verify if the average scores in economy and ecology in 2013/2014 are greater than the ones in academic years 2011/2012 and 2011/2013. the same contrast is done for geology, but we expect the average score to be similar throughout the three academic years for the aforementioned reasons. table 3 shows the results of the basic statistics. • economy and business administration. the average grade in 2011/2012 was 6.35. in 2012/2013 the final scores decreased, while in 2013/2014, when the pbl method was introduced, it increased reaching a 5.43 average final score. • ecology. the average score in 2011/2012 was 5.46. in academic year 2012/2013 the average score decreased, and in 2013/2014, the year in which the pbl was introduced, it increased reaching an average score of 6.32. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 169 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 • geology and pedology. as we noted before, the pbl method was introduced in the academic year 2011/2012. therefore, as expected, the average grade is very similar through the three academic years, between 6.1 and 6.6. table 3. final scores and statistics in the three courses involved in the pbl experience for the last academic years course academic year n average standard error average standard deviation economy and business administration 2011-12 2012-13 2013-14 17 19 26 6,3529 4,9158 5,4385 0,46795 0,26205 0,32430 1,92942 1,14225 1,65362 ecology 2011-12 2012-13 2013-14 16 22 23 5,4625 5,0136 6,3174 0,50024 0,19141 0,24773 2,00096 0,89777 1,18807 geology and pedology 2011-12 2012-13 2013-14 17 18 16 6,5471 6,0056 6,0875 0,39380 0,29885 0,37248 1,62370 1,26792 1,48991 table 4 shows the results of comparing the scores of course 2012/2013 with the ones obtained in course 2013/2014 using levene and t test. in our case the samples are independent, thus, we used the t test for independent samples. • economy and business administration. we deduce that the variance are different since p-value= 0.028 < 0.05 so we used the information corresponding that assumption. since p-value= 0.217 > 0.05 we cannot reject the hypothesis of average equality. • ecology. we deduce that the variance are different since p-value= 0.012 < 0.05 so we used the information corresponding that assumption. since p-value= 0,140 > 0.05 we cannot reject the hypothesis of equality (confidence level of 95%). • geology and pedology. in this case, there are no reasons to believe that the variances are different since p-value=0.363 > 0.05. assuming equality in sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 170 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 variances we find that we can’t reject the hypothesis of average equality (confidence level of 95%) since p-value = 0.864 > 0.05. table 4. results of statistical tests for the three courses involved in the pbl experience for the last academic years levene’s test analysis of variances t test analysis of the averages f significance t degrees of freedom significance (bilateral) economy and business administration equal variances not equal variances 5,194 0,028 1,185 1,254 43 42,904 0,243 0,217 economy equal variances not equal variances 7,037 0,012 1,673 1,531 37 22,343 0,103 0,140 geology and pedology equal variances not equal variances 0,853 0,363 0,173 0,172 32 29,677 0,864 0,865 4. conclusions we have presented a case study of the pbl approach in the second year of the degree in environmental engineering. the experiment was applied in the academic year 20132014. introducing the pbl to three courses was motivated by the desire to engender a learning context favoring enthusiasm, the establishment of links between concepts and their applications together with the proposition of integral a cross-disciplinary solutions to environmental problems. our findings clearly demonstrate that the students’ attitude improved significantly from previous academic years or respect to other courses using traditional (lecture-based) methods. the students’ participation and the interaction (both student-student and instructor-student) were higher compared to other non-pbl courses. our investigation also reveals that this positive attitude and implication led to better academic results. moreover, the pbl helped undergraduates to develop of a number of skills that are sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 171 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 critical for their career, such as the ability to function on multidisciplinary teams and to understand the impact of engineering solutions in a global, economic, environmental and societal context. finally, it should be emphasized that we have encountered some drawbacks and difficulties (such as an inconsistency between the new educational format and the form or examination or the harmful effects of a harder work in these courses on the results obtained in the other ones) that offer opportunity for further research and adjustments. 5. references accreditation board for engineering and technology (abet). engineering accreditation commission (2013). criteria for accrediting engineering programs. http://www.abet.org/displaytemplates/docshandbook.aspx akinoglu o. and tandogan r.o. (2007). the effects of problem-based active learning in science education on students’ academic achievement, attitude and concept learning. eurasian journal of mathematics, science & technology education 3 (1), 71– 81. american academy of environmental engineers (aaee) environmental engineering body of knowledge working group (2008). environmental engineering body of knowledge: summary report. environmental engineer environmental engineer: applied research & practice, 6. http://www.aaees.org/ borrego m. and newswander l.k. (2007). characteristics of successful crossdisciplinary engineering education collaborations. journal of engineering education, 97 (2), 123-134. http://dx.doi.org/10.1002/j.2168-9830.2008.tb00962.x crawley e.f., malmqvist j., ostlunf s. and brodeur d. (2007). rethinking engineering education. the cdio approach. 286 pp. isbn: 978-0-387-38287-6. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 172 http://polipapers.upv.es/index.php/muse http://www.abet.org/displaytemplates/docshandbook.aspx http://www.aaees.org/ http://dx.doi.org/10.1002/j.2168-9830.2008.tb00962.x multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 de graaf e. and christensen c.h. (2004). editorial: theme issue on active learning in engineering education. european journal of engineering education, 29 (4), 461-463. http://dx.doi.org/10.1080/03043790410001716310 de graaf e. and kolmos a. (2003). characteristics of problem-based learning. international journal of engineering education, 19 (5), 657-662. felder r.m. and brent r. (1994). cooperative learning in technical courses. a monograph presenting suggestions for effective implementation of cooperative learning in engineering and science courses. eric document reproduction service, ed 377038. http://www4.ncsu.edu/unity/lockers/users/f/felder/public/papers/education_paperschronological.html gibbs g. and simpson c. (2005). conditions under which assessment supports student’s learning. learning and teaching in higher education, 1, 3-31. hmelo-silver c.e. (2004). problem-based learning: what and how do students learn? educational psychology review, 16(3), 235-266. http://dx.doi.org/10.1023/b:edpr.0000034022.16470.f3 kolari s., savander-ranne c. and viskeri, e.j. (2008). learning needs time and effort: a time-use study of engineering students. european journal of engineering education, 33(5-6), 483-498. http://dx.doi.org/10.1080/03043790802564046 kolmos a. (1996). reflections on project work and problem-based learning. european journal of engineering education, 21(2), 141-148. http://dx.doi.org/10.1080/03043799608923397 macandrew s., walsh l., oduyemi k. and lantz c. (2008). problem based learning in practice: listening to the lecturers. an investigation of academics’ perceptions and practice concerning problem based learning. scottish higher education enhancement research (2) final report. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 173 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1080/03043790410001716310 http://dx.doi.org/10.1023/b:edpr.0000034022.16470.f3 http://dx.doi.org/10.1080/03043790802564046 http://dx.doi.org/10.1080/03043799608923397 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 major c.h. and palmer b. (2001). assessing the effectiveness of problem-based learning in higher education: lessons from the literature. academic exchange quarterly, 5(1), 4-9. markes i. (2006). a review of literature on employability skill needs in engineering. european journal on engineering education, 31(6), 637-650. http://dx.doi.org/10.1080/03043790600911704 mills j.e. and treagust d.f. (2003). engineering education is problem-based or projectbased learning the answer? australian journal of engineering education, 1-16. ministerio de educación, cultura y deporte (2011). resolución de 20 de diciembre de 2010, de la universidad del país vasco, por la que se publica el plan de estudios de graduado en ingeniería ambiental. boletín oficial del estado, 26, 31 de enero de 2011, sec. iii, p. 10782. oakley b., felder r.m., brent r. and elhajj i. (2004). turning students into effective teams. journal of student centered learning, 2(1), 9-34. richter d.m. and paretti m.c. (2009). identifying barriers to and outcomes of interdisciplinarity in the engineering classroom. european journal of engineering education, 34(1), 29-45. http://dx.doi.org/10.1080/03043790802710185 walker a. and leary h. (2009). a problem based learning meta analysis: differences across problem types, implementation types, disciplines, and assessment levels. interdisciplinary journal of problem-based learning, 3(1), 6-28. http://dx.doi.org/10.7771/1541-5015.1061 sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 174 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1080/03043790600911704 http://dx.doi.org/10.1080/03043790802710185 http://dx.doi.org/10.7771/1541-5015.1061 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2015.3697 social and technological sciences eissn: 2341-2593 acknowledgements this research study has been funded by the upv/ehu through the educational innovation project entitled “environmental problem based learning: ∫ disciplines”. grateful acknowledgement is also made to eragin (encourage in basque) and behatu (observe in basque) active learning professional development programmes. sáez de cámara et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 153-175 | 175 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 65 canvas framework for performing systematic reviews analysis fernando almeida university of porto, inesc tec, and ispgaya; portugal email: almd@fe.up.pt received: 2018-01-15; accepted: 2018-03-24 abstract the systematic review of the literature is a fundamental methodology for analyzing critically the existing literature on a given research theme. they are designed to be methodical, replicable and guide the author in identifying the main lines of investigation and conclusions in each scientific domain and, in addition, help them in the identification of new directions of research. however, the systematic review process is typically viewed as too heterogeneous, complex and time-consuming. in this sense, it is pertinent to propose a new approach for conducting systematic reviews that may be more agile, not only in terms of development, but also in the analysis of the results of a systematic review process. this article presents a canvas framework for conducting a systematic review composed of nine blocks and based on a set of identified good practices found in the literature, in which it is possible to easily identify all the steps of the process, options taken, and main results. keywords systematic review, scientific methodology, model canvas, literature review, narrative review https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 66 1. introduction the bibliographic review is a fundamental pillar that sustains any scientific research. it is indispensable for the delimitation of the problem in a research project and for obtaining an accurate idea of the current state of knowledge about a theme, about its gaps and in the identification of the contribution of research to the development of knowledge. in addition, it helps defining the objectives of a scientific research and it also contributes to the theoretical constructions, comparisons and validation of results obtained in a project. the literature review should not focus exclusively on a collection of abstracts. in contrast, it should be a critical discussion of what was found and related it to the problem. in this sense, it is a component that helps to choose the appropriate methodology for data collection. finally, it also aims to produce new ideas, both for solving a problem and for providing new sources of research. two main categories of literature review can be found in the literature. narrative reviews and systematic reviews, which share the designation of review, have different characteristics and objectives. in a first phase, this manuscript intends to perform a critical and comparative analysis of the two approaches of review. subsequently, a framework for conducting systematic literature review analysis is proposed. the framework intends to be sufficiently complete and flexible to offer a robust yet simple method of conducting this process, by identifying the fundamental and optional elements that constitute a systematic literature review. the paper is structured as follows: first, we perform a review on related work by discussing the differences between a narrative review and a systematic review. then, we present the adopted methodology, followed by the presentation and discussion of the proposed framework and its phases. finally, we draw the conclusions of our work. https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 67 2. related work literature review is the process of searching, analyzing and describing a given theme. baker (2016) identifies the following objectives for a literature review:  provide a theoretical framework on a topic under study;  highlight a set of fundamental elements that allow to characterize a given area of knowledge;  identify the research methodologies used by existing studies;  justify the options taken in relation to the methodologies adopted in the study proposed by the authors;  demonstrate the gap in the literature (evidence of what has already been done in a given area compared to what still needs to be done). cronin, ryan, and coughlan (2008) identify five steps for performing a literature review process: (i) selecting a review topic; (ii) searching the literature; (iii) gathering, reading and analyzing the literature; (iv) writing the review; and (v) references. galvan (2006) adds a new step entitled “developing a coherent essay” which is fundamental when the review is long. pautasso (2013) suggests ten simple rules for writing a literature review, respectively: (i) define a topic and audience; (ii) search and re-search the literature; (iii) take notes while reading the documentation; (iv) choose the type of review; (v) keep the review focus, but simultaneously broad of interest; (vi) be critical and consistent; (vii) define a logical structure; (viii) make use of feedback; (ix) be objective when performing the review; and (x) include up-to-date studies but don’t forget relevant older studies. narrative literature review and systematic literature review are two common approaches for performing a literature review (rother, 2007). however, four other forms of performing a literature review can be found, such as evidence mapping, scoping review, rapid review and umbrella review. evidence mapping adds explicit questions, systematic search for evidence, and tabular summaries of the nature and findings of the studies https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 68 (dijkers, 2015); scoping reviews adds a narrative integration of the relevant evidence (dijkers, 2015); rapid review assesses what is already known about a policy or practice issue, by using systematic review methods to search and critically evaluate existing literature (grant and booth, 2009); while umbrella reviews compiles evidence from multiple reviews into one accessible and usable document (grant and booth, 2009). however, within the scope of this article we focus the literature review exclusively on narrative literature and systematic reviews, since they are the two most adopted strategies for conducting a literature review and enable us to highlight the distinctive and antagonistic aspects of the two approaches. narrative literature review is the most traditional way of conducting a bibliographic analysis on a given subject, from a theoretical or contextual point of view. narrative reviews do not describe the methodology used to search for references, or the criteria adopted in the evaluation and selection of the studies. their analysis is built on the critical and personal interpretation of various bibliographic items, such as books, papers published in scientific journals, and articles published in conference proceedings. narrative reviews play a fundamental role in the scientific production of knowledge, since they allow the reader to acquire and update knowledge about a specific theme in a short time. however, they don’t adopt a methodology that allows the reproduction of the data nor do they provide quantitative answers to specific questions. randolph (2009) states that narrative reviews tend to be significantly affected by the reviewer’s subjectivity. in fact, the absence of an objective and systematic review process of bibliography results in a number of methodological shortcomings leading to clear bias of the author's interpretation and conclusions (pae, 2015). additionally, narrative reviews become less feasible as the number of included studies increases (gifford, 2016). narrative reviews don’t have a standard structure. ferrari (2015) states that they can be structured in four sections: (i) introduction; (ii) methods; (iii) results; and (iv) discussion. green, johnson, and adams (2006) introduce three new elements: objective, background https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 69 and conclusion. however, the first two elements can be merged as an introduction to the study. other sections can also appear in narrative reviews, such as keywords, acknowledgements, tables and figures. in both approaches, all manuscripts must be a title, abstract and references. however, green et al. (2006) argue that the use of a structured abstract is more desirable. additionally, byrne (2016) suggests the adoption of peer reviews to improve the reliability and accessibility of narrative review articles. on the other hand, the systematic review of literature is a planned review to answer a specific question. it uses explicit and systematic methods to critically identify, select and evaluate the studies, and to collect and analyze data from these studies included in the review. in this way, systematic review studies have methodological rigor. other advantages may be associated with systematic reviews, such as power of synthesis, objectivity, balancing, replication, dynamism and communication (mallett, hagenzanker, slater, and duvendack, 2012; gopalakrishnan and ganeshkumar, 2013). however, it should be mentioned that a systematic literature review is typically a complex, long and resource-intensive process, which involves a significant number of practical challenges (mallet, hagen-zanker, slater, and duvendack, 2012). ferrari (2015) states that the main objective of a systematic literature review is to formulate a research question, which can later be validated using a quantitative or qualitative analysis, followed or not by a meta-analysis. gough, oliver, and thomas (2013) argue that systematic reviews are a form of secondary research and allow that the results of the review accountable and open to criticism and debate. fink (2014) advocates that a rigorous stand-alone literature review should be systematic in terms of its approach, explicit in defining its procedures, comprehensive in its scope, and yet reproducible so that the process can be replicated by other researchers. khan (2003) defines five steps to performing a systematic review: (i) framing questions for a review; (ii) identifying relevant work; (iii) assessing the quality of studies; (iv) summarizing the evidence; and (v) interpreting the findings. one year later, kitchenham https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 70 (2004) establishes a guideline composed of six steps to perform a systematic review method, which includes: (i) define research questions; (ii) define the search process; (iii) establish the inclusion and exclusion criteria; (iv) quality assessment; (v) data collection process; and (vi) data analysis process. there is also an additional step that should be included, when needed, related to deviations from protocol. cochrane handbook has become a central reference for planning and carrying out a systematic review. the cochrane handbook was proposed by higgins and green (2008) and establishes eight phases: (i) defining the review questions and developing criteria for including studies; (ii) searching for studies; (iii) selecting studies and collecting data; (iv) assessing risk of bias in included studies; (v) analyzing data and undertaking meta-analysis; (vi) addressing reporting biases; (vii) presenting results and summary of findings; and (viii) interpreting results and drawing conclusions. okoli and schabram (2010) propose also an eight-step guide to conducting a systematic literature review. the number of phases is identical to presented in the cochrane handbook, but there are slight differences. the proposed phases include: (i) purpose of the literature review; (ii) protocol and training; (iii) searching for the literature; (iv) practical screen; (v) quality appraisal; (vi) data extraction; (vii) synthesis of studies; and (viii) writing the review. there are also authors that propose a set of good practices for the execution of a systematic literature review. baker (2016) argues that for replication purposes of the study, it is fundamental to present a detailed list of databases searched, keywords, and adopted time frame. chairmani, caldwell, li, higgins, and salanti (2017) emphasize the importance of the systematic review protocol, arguing that it should be public and should exhaustively describe the rationale, hypothesis, and planned methods of the review. they also suggest the use of the prospero platform to register the protocol. liberati, altman, tetzlaff, mulrow, gotzsche, ioannidis, clarke, devereaux, kleijnen, and moher (2009) suggest the adoption of evaluation grids to assess and verify the implementation of the protocol. in this sense, the use of prisma is recommended. https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 71 finally, a comparative analysis of the narrative reviews vs. systematic reviews is made in table 1. for this purpose, five criteria were considered: (i) research question; (ii) data selection; (iii) synthesis; (iv) evaluation; and (v) limitations. table 1. comparative analysis between narrative and systematic reviews (adapted from: yuan and hunt, 2009; ferrari, 2015) criteria narrative reviews systematic reviews research question broad with not specified limits specific research areas data selection frequently non-specific and according to a subjective analysis made by the researcher selection based on comprehensive sources with an explicit and replicable search strategy synthesis qualitative quantitative and adoption of meta-analysis evaluation variable and subjective detailed and reproducible limitations assumptions and protocol are not specified. selection and evaluation biases not known. not reproducible. the scope is limited by the defined query, search terms, and the selection criteria. very time consuming. 3. methodology the adopted methodology is composed by three phases as depicted in figure 1. in the preliminary stage, we identify the main systematic review (sr) approaches and good practices found in the literature. this step is fundamental to understanding which approaches and best practices are best known and used in scientific research studies. then, in a second stage, we perform a comparative analysis of those approaches and good practices. this step allows us to perform a critical and comparative analysis of the https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 72 elements identified in the preliminary stage. finally, in the last phase we present a proposal of a sr canvas approach. the suggested approach is innovative and seeks to bridge the gaps associated with the traditional processes of undertaking a systematic review. figure 1. overview of the adopted methodology the systematic review (sr) canvas is inspired in the popular and well-known business model canvas designed by osterwalder and pigneur (2010). this model is known for promoting the innovation, prototyping and co-creation, using concepts of design thinking (teece, 2010; gavrilova, alsufyey, and yanson, 2014; oyedele, 2016). in our approach, the sr canvas also adopts the principles of design thinking to construct a graphical model, which allows representing all the steps and good practices of a systematic review process. 4. analysis and discussion of results we initially start by comparing the main systematic review approaches. to do this, we identify the main steps of each approach and perform a comparative analysis in table 2. the following evaluation was done: yes – steps are common in both approaches; partial – steps are mentioned in both approaches, although there is no complete in the process; no – steps are not common in both approaches. it is possible to conclude that most of the https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 73 steps are common in the four systematic reviews methodologies. the major differences occur in three phases: (i) establish the inclusion and exclusion criteria; (ii) purpose of the literature review; and (iii) writing the review. all considered methodologies establish that it is relevant to define in detail the criteria of inclusion and exclusion of studies in the systematic analysis, but only kitchenham (2004) dedicates an autonomous step to this process. the other authors consider that these elements are defined in the previous step, that is, when defining the research process. on the other hand, writing the review is considered fundamental only by two studies; others consider that the summary of findings is sufficient to interpret the results of the research process. finally, only okoli and schabram (2010) state that it becomes necessary to specify the purpose of the literature review. the other authors consider that this situation becomes clear when defining the question under investigation. table 2. comparative analysis of systematic review approaches step khan (2003) kitchenham (2004) higgins and green (2008) okoli and schabram (2010) purpose of the literature review no no no yes define research question yes yes yes yes define the search process yes yes yes yes establish the inclusion and exclusion criteria partial yes partial partial collecting data partial yes yes yes quality assessment yes yes yes partial analyzing data yes yes yes yes https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 74 summary of findings yes partial yes yes writing the review no no yes yes then, in table 3, we perform a comparative analysis of the set of good practices identified in the literature review, considering each of the four methodologies. we found that all approaches considered essential the indication of the databases searched. the list of keywords is a fundamental element for the replication of a systematic review process, but is not explicitly mentioned by khan (2003) and kitchenham (2004). only the cochrane handbook developed by higgins and green (2008) is explicit in the use of prospero and prisma. not being two compulsory elements, we can find in the literature several examples of systematic reviews of literature that adopt these two platforms, particularly in the field of health sciences (toews, 2016; zhang, huang, and du, 2017; tursunbayeva, bunduchi, franco, and pagliari, 2017). table 3. comparative analysis of systematic review good practices good practice khan (2003) kitchenham (2004) higgins and green (2008) okoli and schabram (2010) databases searched yes yes yes yes list of keywords partial partial yes yes adopted time frame no yes yes no use of prospero no no yes no use of prisma no no yes no https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 75 finally, we propose the adoption of a sr model canvas composed of nine blocks to represent the various phases of a systematic review (figure 2). only two steps provided in table ii were not incorporated in the sr model canvas. the initial stage was considered unnecessary, since the purpose of the literature review is already clear when defining the research question. on the other hand, the last step “writing the review” was also eliminated, because it has become redundant with the “summary of findings” block. finally, all identified good practices in table 3 were incorporated into sr model canvas. the first three elements were incorporated as processes into the “search process” block and the last two good practices as two individual blocks. figure 2. sr model canvas in the research question phase, the researcher must explicitly submit clear, unambiguous and structured questions before beginning the review. all methodological proposals considered it a fundamental initial step in a systematic review approach. according to higgins and green (2008), the questions addressed by a review may be broad or narrow in scope. the process of choosing the scope of a review question is dependent on https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 76 multiple factors, such as its relevance and potential impact, theoretical support, generalization potential, available resources and the validity of the answers. the search process phase is one of the most time-consuming activities. in order to optimize and speed up the search process, the use of digital libraries is recommended. to perform this activity, the author must register the name of the database, document the search strategy for the database and register the date of search. kitchenham (2004) establishes that for each journal or conference the following elements should be extracted:  name of journal or title of proceedings;  years covered by the search;  any issues not searched (in case of journals);  journal name, if published as part of a journal (in the case of conference proceedings). the use of journals and conferences should be prioritized. however, it is also possible to consider other sources of information, such as technical reports, unpublished studies or web sites. in this situation, the author should always identify the publication's origin, search date and url. identify the main and unique keywords in a systematic review process is a fundamental step. keywords and index terms can be used simultaneously. keywords can be utilized to broaden the search results and indexed terms help to focus the results. keywords can be applied to journal tittles, article titles and article abstracts. on the other hand, index terms perform their search using the keywords supplied by an indexer. additionally, the “or” and “and” boolean operators can be used for the purpose of broadening or restricting the search process. the “or” operator allows the researcher to find documents in which at least one of the search terms exists; on the other hand, the “and” operator only returns results where all search terms exist together. https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 77 it is advisable the adoption of software that allows to manage references. three platforms emerge in the market due to their wide use in academia: mendeley , endnote and zotero. according to sungur and seyhan (2013) the main criteria for choosing a reference management tool are: (i) cost; (ii) storage space; (iii) compatibility; (iv) platform matching; (v) electronic library research; (vi) degree of cooperation; (vii) adopted technology; (viii) metadata search; and (ix) journal writing style. due to its great relevance we consider that the criteria for inclusion and exclusion of studies should be an individual block, whose content influences the research process. thus, in inclusion and exclusion criteria phase the researcher must identify the frame date in which the documentation research process was carried out, the types of studies considered (e.g., journals and conferences proceedings, inclusion of web sites or newspapers, etc.) and the removal of duplicate studies, describing how this detection process was performed. after that, appears the collecting data phase, in which two sub-steps emerge: (i) the adoption of a data collection form; and (ii) the reliability check. the use of the data collection form allows systematizing the data collection process. higgins and green (2008) recommend that the following elements be recorded: (i) inclusion the title of the review; (ii) inclusion of a revision date; (iii) record the name of the researcher who is completing the form; (iv) inclusion of a unique study id, which is relevant when we have multiple reports of the same study; (v) assessment of the study; and (vi) identify the nature of the study (e.g., literature review, qualitative study, quantitative study or mixed methods). at the level of reliability check the quality of the studies considered in a systematic review should be ensured. there are sometimes discrepancies in whether a given study is relevant enough to be included in a systematic review. to avoid these situations, the kappa statistics proposed by mchugh (2012) can be adopted. one phase that has gained more prominence in recent years is the quality assessment. although the systematic review is considered a robust methodology, not all of them are https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 78 performed with the same quality. the researchers tend to adopt different methods and criteria to identify, analyze and synthesize the data, causing a great methodological variability in the systematic reviews. in this sense, the need to evaluate the methodological quality of the studies emerges as a crucial point, since the degree of confidence in the data affects the quality of the review. a bias is defined by higgins and green (2008) as “a systematic error, or deviation from the truth, in results or inferences”. the effects of bias can be only small and trivial when compared to the sample size, but it can also be substantial which can lead to underestimation or overestimation of the study findings. in the literature three main sources of bias can be identified (i) selection; (ii) calibration; and (iii) confounding (turner, boutron, hróbjartsson, altman, and moher, 2013; klamer, bakker, and gruis, 2017). katikireddi, egan, and petticrew (2014) suggest the adoption of the following methods to reduce the risk of bias into the synthesis process: (i) sensitivity analysis; (ii) narrative assessment; and (iii) restricting the synthesis to studies at a lower risk of bias. analyzing data is another fundamental and time-consuming phase of a systematic review study. the use of a meta-analysis technique is fundamental to combine data from multiple studies in a systematic review and to guarantee the validity of the conclusions. at this stage, the researcher must choose the statistical methods and techniques that will be used to integrate the search results. several statistical techniques can be used at this stage, namely the adoption of descriptive data analysis, correlational analysis and hypothesis testing. heterogeneity investigation is also proposed by higgins and green (2008) to increase the quality of this process. they suggest the use of subgroup analysis, which involves splitting the data into subgroups, typically to perform comparisons between them. finally, the summary of findings is presented. the elements presented here result from the data analysis work carried out in the immediately previous phase. higgins and green (2008) argue that the summary of findings should be presented in a table with the https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 79 following elements: (i) important outcomes; (ii) identification of risks; (ii) magnitude of effects; (iv) number of participants; (v) quality of the evidence; and (vi) comments. after this phase, it is important to interpret the results based on the previous statistical analysis performed previously and to draw the appropriate conclusions, evidencing the implications for practice and research. in the sr model canvas appear two additional blocks that result from the identification of good practices. due to their importance in the process of conducting a systematic review we consider these elements as two autonomous blocks. prospero is a platform that allows the registration of the systematic review protocol. the platform allows registering the researcher’s progress throughout his work. it also makes easier to detect and avoid duplication, which will help the researcher to increase the performance of his/her work. for other hand, prisma offers a checklist of 27 items and a four-step flow diagram to document the systematic review process. the goal of prisma is to help researchers to report on systematic reviews and meta-analyzes. according to swartz (2011) a collateral effect of the prisma statement is the improvement of the transparency and the scientific merit of a systematic review or meta-analysis. this approach has been recommended by several journals particularly in the area of health sciences, in which can be found several studies that adopt this guideline (rivero, nuñez, pires, and bueno, 2015; welch, petticrew, petkovic, moher, waters, and white, 2016; cullis, gudlaugsdottir, and andrews, 2017). the checklist of prisma is structured into seven sections: (i) title, which is used to identify the report; (ii) abstract that provides a traditional structured summary; (iii) introduction that is used to describe the rationale for the review and to provide and explicit statement of question being addressed in the review; (iv) methods, which is the longest section, that is used to describe the protocol and registration, the eligibility criteria, information sources, search, study selection, data collection process, data items, risk of bias in individual studies, summary of measures and results, risk of bias across https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 80 studies, and additional analyses; (v) results, which is the second longest topic after the previous section, that describes the study selection, study characteristics, risk of bias within studies, results of individual studies, synthesis o results, risk of bias across studies, and additional analysis; (vi) discussion, which presents summary of evidence, limitations, and conclusions; and (vii) funding that presents eventual sources of funding for the systematic review and other financial research aids. the flow diagram of prisma is divided into four steps as proposed by moher, liberati, tetzlaff, and altman (2009). in the identification phase (step 1), the researcher must identifies the total number of records found through the database searching and other sources of information; in the screening phase (step 2), the researcher must indicate the number of records removed after identification of duplicated items, the total number of screened records, and the number of excluded records; in the eligibility phase (step 3), must be specified the number of full-text articles assessed for eligibility and the number of full-text articles excluded, describing the reasons for that; and, finally, in the included phase (step 4), the researcher must indicate the number of studies included in the qualitative and quantitative synthesis. 5. conclusions the systematic review aims to summarize all the existing information about a phenomenon in an impartial and complete way. in contrast to the non-systematic process, the systematic review is performed in a formal and meticulous manner, in which we follow plan defined in the review protocol. a systematic review ensures greater coverage of relevant publications and enables audits to be audited, replicated and continued. several authors have suggested a set of steps to conduct a systematic review. in this article we identify the common aspects that we can find in these approaches and also a set of good practices that are mentioned in the literature to formulate a systematic review canvas that can, in a simple, graphic and appealing way, synthesize the various steps and https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 81 decisions taken in the realization of a systematic review. the sr model canvas is composed by nine blocks, respectively: (i) research question; (ii) search process; (iii) inclusion and exclusion criteria; (iv) collecting data; (v) quality assessment; (vi) analyzing data; (vii) summary of findings; (viii) prospero; and (ix) prisma. as future work we intend to apply the sr model canvas to several scientific areas (e.g., medicine, engineering, social sciences, etc.) in a systematic review. the idea would be to evaluate the impact of the application of the sr model canvas and to analyze if its adoption facilitated the process of conducting systematic conduction at various levels, namely in terms of process complexity, development time and perception of results among the scientific community. 6. references baker, j. (2016). the purpose, process, and methods of writing a literature review. aorn journal, 103(3), 265-269. doi: 10.1016/j.aorn.2016.01.016 byrne, j. (2016). improving the peer review of narrative literature reviews. research integrity and peer review, 1(12), 1-4. doi: 10.1186/s41073-016-0019-2 chairmani, a., caldwell, d., li, t., higgins, j., and salanti, g. (2017). additional considerations are required when preparing a protocol for a systematic review with multiple interventions. journal of clinical epidemiology, 83(3), 65-74. doi:10.1016/j.jclinepi.2016.11.015 cronin, p., ryan, f., and coughlan, m. (2008). undertaking a literature review: a step-bystep approach. british journal of nursing, 17(1), 38-43. doi: 10.12968/bjon.2008.17.1.28059 cullis, p, gudlaugsdottir, k., and andrews, j. (2017). a systematic review of the quality of conduct and reporting of systematic reviews and meta-analyses in paediatric surgery. https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 82 plos one, 12(4): e0175213. doi: 10.1371/journal.pone.0175213 dijkers, m. (2015). what is a scoping review? kt update, 4(1), 1-5. ferrari, r. (2015). writing narrative style literature reviews. medical writing, 24(4), 230-235. doi: 10.1179/2047480615z.000000000329 fink, a. (2014). conducting research literature reviews. thousand oaks, ca: sage publications. galvan, j. (2006). writing literature reviews: a guide for students of the behavioral sciences. glendale, ca: pyrczak publishing. gavrilova, t., alsufyev, a., and yanson, a. (2014). modern notation of business models: visual trend. foresight-russia, 8(2), 56-70. gilford, r. (2016). research methods for environmental psychology. new jersey: wiley. gopalakrishnan, s., and ganeshkumar, p. (2013). systematic reviews and meta-analysis: understanding the best evidence in primary healthcare. journal of family medicine and primary care, 2(1), 9-14. doi: 10.4103/2249-4863.109934 gough, d., oliver, s., and thomas, j. (2013). learning from research: systematic reviews for informing policy decisions. alliance for useful evicence. available online at: http://www.alliance4usefulevidence.org/publication/learning-from-research/ (accessed on 14th november 2017). grant, m., and booth, a. (2009). a typology of reviews: an analysis of 14 review types and associated methodologies. health information and libraries journal, 26, 91-108. doi: 10.1111/j.1471-1842.2009.00848.x green, b., johnson, c., and adams, a. (2006). writing narrative literature reviews for peer-reviewed journals: secrets of the trade. journal of chiropractic medicine, 5(3), https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 83 101-117. doi: 10.1016/s0899-3467(07)60142-6 higgins, j., and green, s. (2008). cochrane handbook for systematic reviews of interventions. new jersey: wiley. katikireddi, s., egan, m., and petticrew, m. (2014). how do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? a methodological study. journal of epidemiol community health, 12, 1-7. doi: 10.1136/jech-2014-204711 khan, k., kunz, r., kleijnen, j., and antes, g. (2003). five steps to conducting a systematic review. journal of the royal society of medicine, 96(3), 118-121. kitchenham, b. (2004). procedures for undertaking systematic reviews. joint technical report, computer science department, keele university (tr/se0401) and national ict australia ltd. (0400011t.1). klamer, p., bakker, c., and gruis, v. (2017). research bias in judgement bias studies – a systematic review of valuation judgement literature. journal of property research, 2(79), 1-19. doi: 10.1080/09599916.2017.1379552 liberati, a., altman, d., tetzlaff, j., mulrow, c., gotzsche, p., ioannidis, j., clarke, m., devereaux, p., kleijnen, j., moher, d. (2009). the prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. bmj, 339. doi: 10.1136/bmj.b2700 mallett, r., hagen.zanker, j., slater, r., duvendack, m. (2012). the benefits and challenges of using systematic reviews in international development research. journal of development effectiveness, 4(3), 445-455. doi: 10.1080/19439342.2012.711342 mchugh, m. (2012). interrater reliability: the kappa statistic. biochemia medica, 22(3), 276-282. moher, d., liberati, a., telzlaff, j., and altman, d. (2009). preferred reporting items for https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 84 systematic reviews and meta-analyses: the prisma statement, plos med, 6(7): e1000097. doi: 10.1371/journal.pmed.1000097 okoli, c., and schabram, k. (2010). a guide to conducting a systematic literature review of information systems research. sprouts: working papers on information systems, 10(26), 1-49. osterwalder, a., and pigneur, y. (2010). business model generation: a handbook for visionaries, game changers, and challengers. new jersey: john wiley and sons. oyedele, a. (2016). emerging market global business model innovatio. journal of research in marketing and entrepreneurship, 18(1), 53-62. doi: 10.1108/jrme-11-20150050 pae, c. (2015). why systematic review rather than narrative review? psychiatry investigation, 12(3), 417-419. doi: 10.4306/pi.2015.12.3.417 pautasso, m. (2013). ten simple rules for writing a literature review. plos computational biology, 9(7), 1-6. doi: 10.1371/journal.pcbi.1003149 randolph, j. (2009). a guide to writing the dissertation literature review. practical assessment, research & evaluation, 14(13), 1-13. rivero, t., nuñez, l., pires, e., and bueno, o. (2015). adhd rehabilitation through video gaming: a systematic review using prisma guidelines of the current findings and the associated risk of bias. frontiers psychiatry, 6(151), 1-16. doi: 10.3389/fpsyt.2015.00151 rother, e. (2007). systematic literature review x narrative review. acta paulista de enfermagem, 20(2), 1-2. doi: 10.1590/s0103-21002007000200001 sungur, m., and seyhan, t. (2013). writing references and using citation management software. turkish journal of urology, 39, 25-32. 10.5152/tud.2013.050 https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9832 social and technological sciences issn: 2341-2593 almeida (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 65-85 | 85 swartz, m. (2011). the prisma statement: a guideline for systematic reviews and meta-analyses. journal of pediatry health care, 25, 1-2. doi: 10.1016/j.pedhc.2010.09.006 teece, d. (2010). business models, business strategu and innovation. long range planning, 43, 172-194. doi: 10.1016/j.lrp.2009.07.003 toews, l. (2016). compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (prisma) literature search reporting guidelines. journal of the medical library association, 105(3), 233-239. doi: 10.5195/jmla.2017.246 turner, l., boutron, i., hróbjartsson, a., altman, d., and moher, d. (2013). the evolution of assessing bias in cochrane systematic reviews of interventions: celebrating methodological contributions of the cochrane collaboration. systematic reviews, 2(79), 1-8. doi: 10.1186/2046-4053-2-79 tursunbayeva, a., bunduchi, r., franco, m., and pagliari, c. (2017). human resource information systems in health care: a systematic evidence review. journal of the american medical informatics association, 24(3), 633-654. doi: 10.1093/jamia/ocw141 welch, v., petticrew, m., petkovic, j., moher, d., waters, e., and white, h. (2016). extending the prisma statement to equity-focused systematic reviews (prisma-e 2012): explanation and elaboration. journal of development effectiveness, 8(2), 287-324. doi: 10.1186/s12939-015-0219-2 yuan, y., and hunt, r. (2009). systematic reviews: the good, the bad, and the ugly. am j gastroenterol, 104(5), 1086-1092. doi: 10.1038/ajg.2009.118 zhang, y., huang, j., and du, l. (2017). the top-cited systematic reviews/meta-analyses in tuberculosis research: a prisma-compliant systematic literature review and bibliometric analysis. medicine, 96(6), 1-5. doi: 10.1097/md.0000000000004822 https://doi.org/10.4995/muse.2018.9832 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 positive institutions and their relationship with transformational leadership, empathy and team performance solares menegazzo, jéssica1*, cruz-ortiz, valeria1, ortega-maldonado, alberto1, salanova, marisa1. affiliation author1 (universitat jaume i de castelló, wont research team) *corresponding author: universitat jaume i, departamento de psicología evolutiva, educativa, social y metodología. av. de vicent sos baynat, s/n, 12071, castellón de la plana: castellón (spain).email: solares@uji.es. phone: +34 964 729 955 received: 2015-02-03; accepted: 2015-06-12 abstract the current study tests the relationship between transformational leadership, empathy and excellent team performance, based on the healthy & resilient organizations (hero) model (salanova, llorens, cifre & martínez, 2012) in positive institutions, the third pillar of positive psychology (seligman & csikszentmihalyi, 2000). the study of empathy and its role in companies is important because it is not yet widely known the benefits that can provide to any institution. the study attempt to show a full mediating role of empathy in this relationship. the sample consist on 69 work teams, from 7 small -and mediumsized enterprises (smes) including 4 educational institutions and 3 institutions of medical services from spain. the interclass correlation coefficients (icc1 and icc2) and the average deviation index (adm(j)) supports data aggregation at the team level. the results using sem, through work teams, supported the hypothesis, that is, the empathy plays a full mediating role between transformational leadership and excellent team performance. finally, theoretical and practical implications of the results are discussed. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 38 http://polipapers.upv.es/index.php/muse mailto:solares@uji.es multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 keywords transformational leadership, empathy, excellent performance, positive institutions. positive psychology is defined as "the scientific study of optimal human functioning" (seligman, 1999) and includes the study of three pillars: (1) positive emotions experience, (2) positive traits, and (3) positive institutions (seligman & csikszentmihalyi, 2000; peterson, 2006). moreover, the field of study of this psychological perspective focuses both on an individual level, which studies including the positive characteristics of the person or the development of the virtues and character strengths (peterson & seligman, 2004; peterson, ruch, beermann, park & seligman, 2007) and at the group level, which focuses on the characteristics of the groups and institutions that allow the development of better citizens (seligman & csikszentmihalyi, 2000). thus, positive institutions are seen as a key pillar through which society contributes to the personal flourishing (seligman, steen, park & peterson, 2005) establishing its objectives taking into account the rituals and social practices that cultivate the virtues of society. regarding to donaldson & dollwet (2013) this third pillar of positive psychology includes families, communities, and societies; but organizations as businesses, schools or hospitals are considered positive institutions too. in this sense, it is important to consider that there are some organizations which offer main services to society (as education or health care) becoming a key resource for collectivity. furthermore, given this special condition, it is important for these organizations, that teams obtain excellent performance, understanding this concept as an added value in the organization, given by a set of teams behaviors, contributing directly and indirectly to the organization goals (borman & motowidlo, 1997). solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 39 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 for instance, when we go to a hospital, we expect to receive the best care because our health depends on the people who work there. furthermore and consistent with this, lyubomirsky, king and diener (2005) documented the relationship between indicators in healthy employees and organizational results, showing that these results are reflected in products and / or services of excellence which generates customer’s loyalty. in this line, there has been recently growing interest in research organizations and their behavior from a positive perspective (culbertson, fullagar & mills 2010) to try to describe, explain and predict the optimal functioning in these contexts, as well as amplify and enhance psychosocial well-being and quality of work life and organization (salanova, martínez & llorens, 2005; llorens, salanova & martinez, 2008). in this way have been proposed and validated several explanatory models of psychosocial well-being in the workplace, such as demand-control model of occupational stress (karasek, 1979; karasek & theorell, 1990), the job demands-resources model (demerouti, bakker , nachreiner, & schaufeli, 2001; schaufeli & bakker, 2004), the spiral dual occupational health model (salanova, cifre, martinez & llorens, 2007), and the healthy & resilient organizations (hero) model (salanova, llorens, cifre & martinez, 2012). salanova and colleagues (2012), proposed in their model of healthy organizations three important areas that are interrelated to contribute their common goal. the first one is called healthy practices or organizational resources, (e.g. team empathy, transformational leadership). the second area is the psychosocial wellbeing of employees (healthy employees) (e.g. team efficacy, team resilience, etc.). the third and final area is the healthy organizational outcomes (e.g. excellent team performance and quality service). thus, in the model, each of them is interrelated with the others. therefore considering this approach, it possible to understand that healthy organizational outcomes (such as performance excellence team) are related to healthy practices and organizational resources. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 40 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 organizational resources one of the most important organizational resources studied in positive organizational psychology is leadership because leaders can influence their teams (donaldson & dollwet, 2013). in addition positive leadership style is associated with followers´ psychological capital and positive work environment (woolley, caza & levy, 2011). following hero model, positive leadership style is defined as transformational leadership. in this regard, salanova in 2008 argued that a transformational leader is the one that leads the follower through a shared goal and achieve the commitment of the team members and the organization, and molero (2011) exposed that a transformational leader, motivates the followers to give beyond than expected. besides this, more and more is being investigated on transformational leadership, because of the benefits that produce in the organizations. transformational leader has shown to have subordinates that report greater satisfaction, have higher performing work teams and receive higher rating of effectiveness and performance (bryman, 1992; bass, 1995). but what characteristics make the transformational leader lead the subordinates to accomplish the goals and also challenge them to give beyond expected? in this sense the big five structure of personality framework gives the opportunity to integrate commonalities among diverse approaches to personality (john & srivastava, 1999). according to judge and bono (2000), extraversion and openness to experience correlate with a transformational leader, neither neuroticism nor conscientiousness displayed any significant relationships with transformational leadership, and agreeableness displayed the strongest relationship with transformational leadership. this can be explained because, to mentor successfully, empathy is required and transformational leaders give special attention to neglected group members, treat each subordinate as an individual, and express appreciation for a job well done (bass, 1985). solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 41 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 according to studies mentioned before empathy and support (the fourth dimension of transformational leadership) are important elements of a transformational leader. empathy is defined as people's ability to understand others feelings and re-experience those feelings in the team (huy, 1999). barbuto & burbach (2004) found that empathy was related to transformational leadership. moreover the leadership literature is beginning to recognize that the ability to extend empathy contributes to leadership success (cooper & sawaf, 1997; yukl, 1998). some studies also show the importance of an empathic leader. george (2000) and lewis (2000) exposed that high quality relationships derived from empathy tend to enhance perceptions of a leader’s integrity or credibility, and tend to engender cooperation and trust. the authors also manifested that the knowledge and understanding gained from their sense of empathy, may enable leaders to influence follower’s emotions and attitudes. in this sense, social psychology display several studies showing how common beliefs and affective experiences that emerge from people working together, tend to show similar patterns of behavior and feel collective emotions (barsade, 2002; gonzález-romá, peiró, subirats & mañas, 2000). this might explain why the followers replay with their coworkers some of the leader attitudes such as empathy. but empathic attitudes between coworkers is not the only a possible positive outcome of a transformational leader. studies have shown that transformational leadership influences in higher levels of individual, group, and organizational performances (bass & avolio, 1994) and liao & chuang,(2007) have found that a transformational leader was positively related to employee service performance. other research has focused on identifying the effect of variables mediators in the leadership-performance ratio; such as the mediating effect of the group potency (schaubroeck, lam & cha, 2007). also the relationship between transformational leader role with the extra performance is mediated efficacy beliefs and engagement (salanova, lorente, chambel & martínez, 2011). other solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 42 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 studies show the relationship of transformational leader and group performance is mediated by the engagement (cruz-ortiz, salanova& martínez, 2013) thus, we undersand that on one hand there are a relationship between resources such as transformational leadership and empathy with the result of excellent performance, and on the other hand exist a need to deepen the knowledge about what processes are involved in the role played by transformational leaders in the excellent team performance. based on the hero model (salanova et al., 2012) in which, practices and positive resources are vital for healthy employees in order to get healthy and positive outcomes, this paper tests how organizational resources such as transformational leadership and personal resources as empathy, generate desired and excellent performance on this type of positive institutions. we propose that empathy plays a fundamental and mediating role in the relationship between transformational leadership and excellent performance. thereby this study about positive schools and health institutions, and the role of transformational leadership and empathy in their excellent results, should contribute to the development of team management for organizations to take into account the proper use of practices and resources for healthy results to the flourishing of the community. transformational leadership bass (1985) model of transformational leadership has been embraced by scholars and practitioners alike as one way in which organizations can encourage employees to perform beyond expectations. some studies considered the role of transformational leadership in the motivational process of his followers by transforming their attitudes and values as well as increased performance (molero, cuadrado, navas & morales, 2007). in terms of the operationalization of the construct there are several proposals exposing that transformational leadership is composed of dimensions. one of the earliest and most extended is the multifactor leadership questionnaire (mlq) designed by bass solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 43 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 and his colleagues (bass, 1985; bass & avolio, 1990) that included both transactional and transformational leadership, and proposed that transformational leadership consisted of four dimensions: charisma, inspiration, intellectual stimulation and individualized consideration. rafferty and griffin (2004), based on a review of the mlq and the research generated by this instrument, proposed a redefinition of a model of leadership with five dimensions: (1) vision, defined as an expression of an idealized picture of the future based on the values of the organization; (2) inspirational communication, are those positive messages about the organization, that build motivation and trust; (3) intellectual stimulation, promotes the interest of employees to think the problem in new ways; (4) support, expressing concern for followers and taking account of their individual needs, ashkanasy and tse (2000) also commented that “transformational leaders are sensitive to followers needs. . . they show empathy to followers, making them understand how others feel” (pag.232); and (5) personal recognition, the provision of rewards such as praise and acknowledgement of effort for achievement of specified goals. the authors show empirical evidence of the factorial structure of the construct attending to these five dimensions. following this proposal, transformational leadership has been operationalized based on these five dimensions. empathy holling, (1994) referred to empathy as the ability to see the world, including our own behavior, from the point of view of others. but as mentioned before huy (1999), defined empathy in the organizational context as people's ability to understand others feelings and re-experience those feelings in the organization. muller (2014) went a step further and talk about collective empathy referring to it as the collectively shared desire to help others in need. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 44 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 the study of empathy in the organizational environment has been limited. however, jarrard (1956) noted that the implementation of empathy in industrial and organizational settings had as major concern assessing empathic abilities of leaders, management personnel and employees. subsequent studies by eisenberger, huntintong, hutchinson and sowa (1984) reported that empathy in organizations generates less absenteeism, more commitment and more satisfied employees. but some other studies showed the importance of empathy between team members, for example nadler y liviatan (2006) exposed that empathy produce trust and also makes people more willing to reconcile. another construct related with empathy is performance, for example, roberge (2013) argued that collective empathy works as a moderator in team diversity and performance. also akgün & dogan (2014) exposed that “the existence of group norms collective empathy becomes a resource projects for performance improvements” and their study confirm that collective empathy has a significant effect on the performance of software development projects. this idea is supported by hero model, because this model explains how resources are used to produce results. group performance (in role, extra role and service quality) goodman y svyantek (1999) proposed two dimensions of performance: (1) intra role, defined as those activities that contribute directly or indirectly to the technical base of the organization and vary between different jobs within the same organization and; (2) extra role, defined as those activities that are not formally part of the work and employees perform them voluntarily. moreover service quality can be explained as customer perception about employee performance (salanova, agut & peiró, 2005). some studies show the relation between performance and transformational leadership. dionne, yammarino, atwater y spangler, (2004) exposed that the leader, through idealized influence, inspirational motivation and intellectual stimulation, can solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 45 http://polipapers.upv.es/index.php/muse http://www.sciencedirect.com/science/article/pii/s037872061400144x multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 promote the performance of work teams. some other studies manifested that the transformational leadership increases the motivation of the teams. it can be directly through motivational strategies, but also indirectly through the dimensions that make up the leadership (zaccaro, rittman, and marks, 2001). other studies related performance and empathy. roberge (2013) suggested that both individual-level and team-level empathy are necessary mechanisms to explain how people may work harmoniously together and increase the overall team performance. ensari and miller (2006) suggested that empathy increase effectiveness and productivity. according to the literature and based on hero model (salanova et al., 2012) this paper proposes to study the relationship between transformational leadership, empathy and performance (in role ,extra role and service quality) by aggregating data at the team level. on this basis we expect that empathy fully mediates the relationship between transformational leadership and team performance (in role, extra role and service quality). solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 46 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 figure 1. theoretical research model and hypothesis method sample and procedure a convenience sample consisting of 392 employees, nested within 69 teams, from seven small -and mediumsized enterprises (smes) including four educational institutions and three institutions of medical services from spain. moreover, 60% were women, 65% had a tenured contract, 27% had a temporary, and 8% were self-employed contract. the empathy transformational leadership excellent performance vision inspirational communication intellectual stimulation support personal recognition empathy in-role performance extra-role performance service quality solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 47 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 average job tenure was 95 months (sd = 89.3). after reaching an agreement about the company’s participation in the study, questionnaires were administered to the participants, who were asked to take part voluntarily. to lead respondents’ attention away from the individual level to the team level, all items focused on team perceptions as stipulated in the hero (healthy and resilient organizations) questionnaire (salanova et al., 2012). the confidentiality of the answers was guaranteed. instruments transformational leadership resources were assessed by 15 items in five different scales, we used the questionnaire of rafferty & griffin (2004): vision (three items; e.g., “our supervisor understand perfectly which the objectives of the group are”; alpha = .90), inspirational communication (three items; e.g., “our supervisor say positive thinks about the department; alpha = .94), intellectual stimulation (three items; e.g., “our supervisor has ideas that stimulate us to rethink about questions that never we had thought before”; alpha= .95), support (three items; e.g., “our supervisor thinks about our personal needs”; alpha=. 95), and personal recognition (three items; e.g., “our supervisor congratulate us personally when we do an excellent work”; alpha=. 97). empathy was assessed by three items we used the questionnaire from salanova, et. al (2012hero, healthy and resilient organizations) (three items; e.g., “during the interpersonal relationships with others we should express emotions that not coincide with our truly feelings; alpha=.88). excellent performance, we tested three dimensions: two different scales were considered: in-role performance (three items; e.g., “my work unit do all the functions and tasks demands by the job”; alpha = .73) and extrarole performance (three items; e.g., “in my work unit there are a high level of trust in the direction and in employees; alpha = .81), adapted from the goodman and svyantek scale (1999) , and service quality (price, arnould & tierney, 1995; parasuraman, zeithaml & berry, 1988) (sevens items; e.g., “in this organization we can share our ideas, emotions and hopes; alpha = .88). solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 48 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 all scales were included in the questionnaire hero (healthy and resilient organizations) (salanova et al., 2012). respondents answered using a 7-point likert-type scale ranging from 0 (never) to 6 (always). all items had as reference the work teams. later was conducted aggregation of data at the team level, considering the scores averages of items answered. data aggregation firstly, the harman’s single factor test (e.g., podsakoff, mackenzie, lee, & podsakoff, 2003) was carried out using amos 18.0 (arbuckle, 2009) for the variables assessed by the employees. secondly, the agreement of employee perceptions in teams was checked using various indices: following a consistency-based approach, both icc(1) and icc(2) indices were calculated. values greater than .05 for icc(1) indicate an adequate level of within-unit agreement (blease, 2000). for the icc(2), values greater than .60 support aggregations (glick, 1985). from a consensus-based approach, the average deviation index was computed (adm(j)) (burke, finkelstein, & dusig, 1999), whereby team agreement was concluded when adm(j) was equal to or less than 1 (burke et al., 1999). finally, analyses of variance (anova) were computed in order to ascertain whether there was significant between-group discrimination for the measures (kenny & la voie, 1985). data fit we used structural equation modeling (sem) by amos 22.0 (arbuckle, 2009). three competitive models were compared: m0, the independence model; m1, the fully mediated model; and m2, the partially mediated model. maximum likelihood estimation methods were used by computing the absolute goodness-of-fit indices were assessed: (1) the χ 2 goodness-of-fit statistic; and (2) the root mean square error of approximation (rmsea). (3) the normed fit index (nfi); (4) the tucker-lewis index (tli); (5) solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 49 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 comparative fit index (cfi); and (6) the incremental fit index (ifi). values smaller than .05 are indicative of an excellent fit for rmsea (brown & cudeck, 1993) and values higher than .95 are indicative of an excellent fit for the relative indices (hoyle, 1995). results descriptive and aggregation analyses firstly, the results of the harman’s single factor test (podsakoff et al., 2003) revealed a poor fit to the data: χ 2 (28) = 395.10, p < .000, rmsea = .43, nfi = .37, tli = .20, ifi = .39 and cfi = .38. results also showed that the model considering three latent factors (i.e., transformational leadership, empathy and excellent performance) fit the data well: χ 2 (25) = 66.68, p < .000, rmsea = .15, nfi = .90, tli = .90, ifi =. 93 and cfi = .93. the difference between both models is also significant in favor of the model with two latent factors, delta χ2 (3) = 328.42, p < .000. consequently, common method variance is not a serious deficiency in these data (conger, kanungo, & menon, 2000). table 1 shows means, standard deviations, intercorrelations, and aggregation indices of all the study variables. icc (1), icc(2) and adm(j) indices ranged from .16 to .45, from .13to .83, and from .46 to .92, respectively. results for these indices were modest in the case icc(2) for empathy (icc(2)=.53) and for quality service (icc(2)=.47). however, one-way anova results showed statistically significant between-group discrimination. in conclusion, overall aggregation results indicated within-group agreement in the teams so that unit members’ perceptions can be aggregated. the database was constructed aggregate team mean scores. aggregate data (table i), the positive and significant correlation was found between the dimensions of the constructs (between .13 and .84; p <.001). solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 50 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 tabla 1 means, standard deviations, intercorrelations, and aggregation indices for the study variables (aggregate measures; n= 69 teams) variables means sd icc1 icc2 f adm(j) 1 2 3 4 5 6 7 8 9 1. lid. vision 4.32 1.50 .38 .78 5.932*** .73 _ .80** .74** .70** .66** .37** .18** .37** .34** 2. lid. inspirational communication 4.06 1.70 .45 83 5.323*** .84 .84** _ .84** .78** .76** .33** .11* .28** 27** 3. lid. intellectual stimulation 3.73 1.69 .41 .80 5.138*** .83 .83** .90** _ .76** .73** .32** .07** .26** .28** 4. lid. support 3.94 1.76 .33 .74 3.983*** .91 .81** .89** .87** _ .82** .37** .08** .26** .25** 5. lid. personal recognition 4.02 1.83 .34 .75 4.058*** .92 .77** .85** .80** .89** _ .37** .13** .22** .27** 6. empathy 4.63 1.29 .16 .53 2.131*** .75 .54** .50** .40** .48** .51** _ .70** .60** .44** 7. in-role performance 4.71 5.16 .23 .64 2.806*** .46 .33** .20** .12** .71** .22** .57** _ .49** .58** 8. extra-role performance 5.05 .97 .25 .66 3.074*** .54 .57** .50** .43** .45** .41** .53** .57** _ .50** 9. service quality 4.88 .83 .23 .13 1.938*** .47 .54** .46** .46** .37** .39** .39** .58** .72** _ notes: correlations are preseted at the individual-level (n= 392, below the diagonal) and the team-level (n= 69, above de diagonal). * p < .05,**p < .01; ***p < .001; solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 51 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 model fit: structural equation modeling as stated by brown (2006), in cases in which it may be necessary to use single indicators in a sem,measurement error can be readily incorporated into a dimensional indicator by fixing its unstandaridized error to some non-zero, calculate on the basis of measure`s sample variance estimate and know psychometric information. thus, we fixed the unstandaridized error of the indicator of resilience with the formula variance* (1-α). to compute sem, we used the aggregated database that included transformational leadership, empathy and excellent performance (n = 69). table 2 shows the results of the sem analysis indicating that the proposed partially mediated model fits the data well, with all fit indices satisfying their corresponding criteria. the chi-square difference test between m1 (the fully mediated model) and m0 (the independence model) shows a significant difference between the two models in favor of m1, delta χ 2 (12) = -36, p < .001. the chisquare difference test between m1 (the fully mediated model) and m2 (the partially mediated model) shows a non-significant difference between the two models, delta χ2 (1) = .28, which is to be interpreted in favor of the most parsimonious one, namely m1. tabla 2 indices del modelo de ecuaciones estructurales (n= 69 grupos) modelos χ2 df cfi nfi tli ifi rmsea ∆c2 ∆df m0 632.49 36 .00 .00 .00 .00 .49 m1 54.56 25 .95 .91 .93 .95 .13 ∆m0-m1 577,53*** 12 m2 54,28 24 .95 .91 .92 .95 .13 ∆m1-m2 .28 ns 1 notes. χ2 = chi-square; df = degrees of freedom; rmsea= root mean square error of approximation; nfi = normed fit index; tli = tucker-lewis index; ifi = incremental fit index; ***p < .001, non-significant solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 52 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 to assess the mediation effect, the sobel test (sobel, 1988) was conducted, which showed significant results (sobel t = 2.57, p <.005). however, further analyses were conducted using the approach developed by baron and kenny (1986): (1) transformational leadership were positively and significantly related to excellent performance (β = .33, p = .000); (2) transformational leadership was positively and significantly related to empathy (β = .43, p = .000); (3) empathy was positively and significantly related to excellent performance, controlling for transformational leadership (β = .82, p = .003); and finally (4) the effect of transformational leadership on excellent performance is reduced to nonsignificance when empathy`s effect on excellent performance is taken into account (β = .10, p = .55 n.s.). the fact that the relationship between transformational leadership and excellent performance became significant suggests that empathy full mediated the relationship between transformational leadership and excellent performance. in conclusion, previous results using sem and mediation analyses provide some evidence for m2, that is, the partially mediated model. the final model is depicted in figure 2. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 53 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 figure 2. the final model with standardized coefficients (n = 69). all coefficients are significant at ***p < .001 .72 .88 .82 .88 .67 .92 .94 .96 .88 .80 .71 empathy transformational leadership excellent performance vision inspirational communication intellectual stimulation support personal recognition in-role performance extra-role performance service quality empathy solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 54 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 discussion the objective of this paper was to analyze the relationship of transformational leadership with group excellent performance (in role performance, extra role performance and service quality) and the role that empathy plays in this relationship. the study tested the full mediation of empathy in the relationship between transformational leadership and performance in a sample of 69 units of teams from different pymes distributed nationwide. earlier in this paper we discuss that the role of empathy in the organizational context has not been extensively studied, for this reason we propose to study the effects of the empathy as a full mediator between the transformational leadership and team excellent performance. showing that, although transformational leadership has effects on the excellent performance of team members is not enough and empathy remains as an important factor for the expected results. the result shows the importance to develop empathy between team members, because although the characteristics of a transformational leader are important for the performance, empathy plays a vital role for excellent results. moreover, the results support the hypothesis proposed in this research and can indicate that the aim of this study has been achieved. this is important because it shows the value of the role played by empathy as an organizational and social resource to generate desired results. theoretical and practical implications the present study shows different theoretical and practical implications. theoretically, it expands the study of empathy in work teams. in addition provides evidence to hero model (salanova, 2008; salanova et al, 2009; 2012.) analyzing the interaction of health resources and organizational practices (e.g., empathy and transformational leadership) and health outcomes (e.g., in role and extra role performance) using superior levels of analysis (i.e., teams).furthermore, the results contribute to research, showing the benefits of promoting the positive aspects in work contexts, in this case the role of empathy and its important role in the group performance. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 55 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 from the point of view of organizational practices, this research interests human resources professionals toward implementing practices and resources that improve outcomes in teams work. we can say that it is important to conduct practices that increase empathy among employees and to enhance the wellbeing of teams, as this will have a positive relationship in how they carry out their tasks and generate expected results. limitations and future research one possible limitation of this study is that data was obtained through self-report measures. however, the data was not treated on an individual level but aggregated perceptions of teams to empathy, transformational leadership and team performance. as a consequence, using aggregate-level team data can increase the validity of the scores, considering that we are dealing with "shared intersubjectivities" with shared and integrated mental models among team members, and not individual subjectivities. future studies can be directed to unravel the causal pathways by using longitudinal studies to observe how this mediation evolves over time. the use of multilevel methodology is also recommended to explore longitudinal studies in which the organizational level and lower-level variables are related. finally, in conclusion, this paper shows that empathy plays a full mediating role between transformational leadership and excellent team performance, so this relationship may be very important for the development of positive institutions. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 56 http://polipapers.upv.es/index.php/muse multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 references akgün, a. e., keskin, h., cebecioglu, a. y., & dogan, d. (2014). antecedents and consequences of collective empathy in software development project teams. information & management. ikarus arbuckle, j. l. (1997). amos users´ guide versión 4.0. chicago, il: smallwaters corporation. ashkanasy, n. m., & tse, b. (2000). transformational leadership as management of emotion: a conceptual review. in n. m. ashkanasy, c. e. j. härtel, & w. j. zerbe (eds.), emotions in the workplace: research theory, and practice (pp. 221–235). westport, ct: quorum books. barbuto jr. j. e., & burbach, m. e. (2004). the emotional intelligence of transformational leaders: a field study. paper presented at the gallup leadership institute inaugural summit, omaha, ne. barsade, s. g. (2002). the ripple effect: emotional contagion and its influence on group behavior. administrative science quarterly, 47, 644-675. http://dx.doi.org/10.2307/3094912 bass, b. m. & avolio, b. j. (1990). transformational leadership development: manual for the multifactor leadership questionnaire. palo alto, ca: consulting psychologists press inc. bass, b. m. (1985). leadership and performance beyond expectations. nueva york: the free press. bass, b. m. (1996). theory of transformational leadership redux. the leadership quarterly, 6, 463-478. http://dx.doi.org/10.1016/1048-9843(95)90021-7 bass, b. m., & avolio, b. j. (1994). improving organizational effectiveness through transformational leadership, sage (pp. 26–47), thousand oaks, c.a. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 57 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.2307/3094912 http://dx.doi.org/10.1016/1048-9843(95)90021-7 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 baron, r. m., & kenny, d. a. (1986). the moderatormediator variable distinction in social psychological research: conceptual, strategic and statistical consideration. journal of personality and social psychology, 51, 1173-1182. http://dx.doi.org/10.1037/0022-3514.51.6.1173 borman, w. c., & motowidlo, s. j. (1997). task performance and contextual performance: the meaning for personnel selection research. human performance, 10, 99-109. http://dx.doi.org/10.1207/s15327043hup1002_3 bliese, p. d. (2000). within-group agreement, non-independence, and reliability. implications for data aggregation and analysis. in k.j. klein, & s.w.j. kozlowski (eds.), multilevel theory, research, and, methods in organization (pp. 349-381). san francisco: jossey-bass. brown g. d. (2006) dectin-1: a signalling non-tlr pattern-recognition receptor. nat rev immunol 6: 33–43. http://dx.doi.org/10.1038/nri1745 burke, m. j., & dunlap, w.p. (2002). estimating interrater agreement with the average deviation index: a user’s guide. organizational research methods, 5, 159-172. http://dx.doi.org/10.1177/1094428102005002002 glick, w. h. (1988). response: organizations are not central tendencies: shadowboxing in the dark, round 2. academy of management review, 13, 133-137. bryman, a. (1992). charisma and leadership in organizations. newbury park, ca: sage pubns. conger, j. a., kanungo, r. n., & menon, s. t. (2000). charismatic leadership and follower effects. journal of organizational behavior, 21, 747-767. http://dx.doi.org/10.1002/1099-1379(200011)21:7<747::aid-job46>3.0.co;2-j cooper, r. k., & sawaf, a. (1997). executive eq: emotional intelligence in leadership and organizations. new york: grosset/putman. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 58 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1037/0022-3514.51.6.1173 http://dx.doi.org/10.1207/s15327043hup1002_3 http://dx.doi.org/10.1038/nri1745 http://dx.doi.org/10.1177/1094428102005002002 http://dx.doi.org/10.1002/1099-1379(200011)21:7%3c747::aid-job46%3e3.0.co;2-j multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 cruz-ortiz, v., salanova, m., & martínez, i. m. (2013). liderazgo transformacional y desempeño grupal: unidos por el engagement grupal. revista de psicología social, 28, 183-196. http://dx.doi.org/10.1174/021347413806196762 culbertson, s. s., fullagar, c.j. & mills, m. j. (2010). feeling good and doing great: the relationship between psychological capital and well-being. journal of occupational health psychology, 15,421-433. http://dx.doi.org/10.1037/a0020720 demerouti, e., bakker, a. b., nachreiner, f. y schaufeli, w. b. (2001). the job demands-resources model of burnout. journal of applied psychology, 86, 499-512. http://dx.doi.org/10.1037/0021-9010.86.3.499 dionne, s. d., yammarino, f. j., atwater, l. e. & spangler, w. d. (2004). transformational leadership and team performance. journal of organizational change management, 17, 177-193. http://dx.doi.org/10.1108/09534810410530601 donaldson, s.i., & dollwet, m. (2013). taming de waves and wild horses of positive organizational psychology. in a.b. bakker (ed.), advances in posotive organizational psychlogy. (pp. 1–22). bingley, uk: emerald. http://dx.doi.org/10.1108/s2046410x(2013)0000001003 eisenberg, r. huntington, r, hutchinson, s., & sowa, d. (1986). perceived organizational support. journal of applied psychology, 71, 500-507. http://dx.doi.org/10.1037/0021-9010.71.3.500 ensari, n. k., & miller, n. (2006). the application of the personalization model in diversity management. group processes and intergroup relations, 9, 589-607. http://dx.doi.org/10.1177/1368430206067679 george, j. m. (2000). emotions and leadership: the role of emotional intelligence. human relations, 53, 1027–1055. http://dx.doi.org/10.1177/0018726700538001 gonzález-romá, v., peiró, j. m., subirats, m. & mañas, m. a. (2000). the validity of affective work team climates. in m. vartiainen, f. avallone, & n. anderson (eds.), solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 59 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1174/021347413806196762 http://dx.doi.org/10.1037/a0020720 http://dx.doi.org/10.1037/0021-9010.86.3.499 http://dx.doi.org/10.1108/09534810410530601 http://dx.doi.org/10.1108/s2046-410x(2013)0000001003 http://dx.doi.org/10.1108/s2046-410x(2013)0000001003 http://dx.doi.org/10.1037/0021-9010.71.3.500 http://dx.doi.org/10.1177/1368430206067679 http://dx.doi.org/10.1177/0018726700538001 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 innovative theories, tools and practices in work and organizational psychology (pp. 97109). götting: hogrefe & huber. goodman, s. a., & svyantek, d. j. (1999). person-organization fit and contextual performance: do shared values matter. journal of vocational behavior, 55, 254-275. http://dx.doi.org/10.1006/jvbe.1998.1682 hoyle, r. h. (1995). structural equation modeling: concepts, issues, and applications. (ed.). sage publications. holling, c. s. (1994) new science and new investments for a sustainable biosphere. in a. m. jansson, m. hammer, c. folke and r. costanza, editors. investing in natural capital . (pp. 57-73). island press, washington, d.c huy, q. h. (1999). emotional capability, emotional intelligence, and radical change. academy management review, 24, 325–345. jarrard, l. e. (1956). empathy: the concept and industrial applications. personnel psychology, 9, 157-167. doi: 10.1111/j.1744-6570.1956.tb01059. john, o. p., & srivastava, s. (1999). the big five trait taxonomy: history, measurement, and theoretical perspectives. in e. pervin & o. john (eds.), handbook of personality (pp. 102–138). new york: guilford press. judge, t. a., & bono, j. e. (2000). five-factor model of personality and transformational leadership. journal of applied psychology, 85, 751-765. http://dx.doi.org/10.1037/0021-9010.85.5.751 karasek, r. a. (1979). job demands, job decision latitude and mental strain. implications for job redesign. administrative science quarterly, 24, 285-308. http://dx.doi.org/10.2307/2392498 karasek, r .a. y theorell, t. (1990). healthy work stress, productivity and the reconstruction of working life. new york: basic books. solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 60 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1006/jvbe.1998.1682 http://dx.doi.org/10.1037/0021-9010.85.5.751 http://dx.doi.org/10.2307/2392498 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 kenny, d. a., & la voie, l. (1985). separating individual and group effects.journal of personality and social psychology, 48, 339-348. http://dx.doi.org/10.1037/00223514.48.2.339 lewis, k. m. (2000). when leaders display emotion: how followers respond to negative emotional expression of male and female leaders. journal of organizational behavior, 21, 221–224. http://dx.doi.org/10.1002/(sici)10991379(200003)21:2<221::aid-job36>3.0.co;2-0 liao, h., & chuang, a. (2007). transforming service employees and climate: a multilevel, multisource examination of transformational leadership in building longterm service relationships. journal of applied psychology, 92, 1006-1019 http://dx.doi.org/10.1037/0021-9010.92.4.1006 llorens, s., salanova, m., & martínez, i. m. (2008). psicología ocupacional positiva: concepto y metodología para su evaluación. in j. tous, m. a. carrión y lópez, f. (eds.), promoción de la salud ocupacional. colección psicosociología de la salud ocupacional (pp. 88-108). mollet del vallés, barcelona: aepa. lyubomirsky, s., king, l., & diener, e. (2005). the benefits of frequent positive affect: does happiness lead to success? psychological bulletin, 131, 803-855. http://dx.doi.org/10.1037/0033-2909.131.6.803 molero, f. (2011). liderazgo transformacional y carismático. in f. molero, & j.f. morales (eds.), liderazgo hecho y ficción. visiones actuales (pp. 117-143). madrid: alianza. molero, f., cuadrado, i., navas, m. & morales, f. (2007). relations and effects of transformational leadership: a comparative analysis with traditional leadership styles. the spanish journal of psychology, 10, 358-368. http://dx.doi.org/10.1017/s1138741600006624 solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 61 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1037/0022-3514.48.2.339 http://dx.doi.org/10.1037/0022-3514.48.2.339 http://dx.doi.org/10.1002/(sici)1099-1379(200003)21:2%3c221::aid-job36%3e3.0.co;2-0 http://dx.doi.org/10.1002/(sici)1099-1379(200003)21:2%3c221::aid-job36%3e3.0.co;2-0 http://dx.doi.org/10.1037/0021-9010.92.4.1006 http://dx.doi.org/10.1037/0033-2909.131.6.803 http://dx.doi.org/10.1017/s1138741600006624 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 muller, a. r., pfarrer, m. d., & little, l .m. (2014) a theory of collective empathy in corporate philanthropy decisions. acadademy management review, 39, 1–21. http://dx.doi.org/10.5465/amr.2012.0031 nadler, a., & liviatan, i. (2006). intergroup reconciliation: effects of adversary's expressions of empathy, responsibility, and recipients' trust. personality and social psychology bulletin, 32, 459-470.parasuraman, a., zeithaml, v. a., & berry, l. l. (1988). servqual. journal of retailing, 64, 12-40. peterson, c. (2006). a primer in positive psychology. new york, ny: oxford university press. peterson, c., & seligman, m. e. p. (2004). characters, strenghts and virtues (csv). oxford: oxford university press. peterson, c., ruch, w., beermann, u., park, n. & seligman, m.e.p. (2007). strengths of character, orientations to happiness, and life satisfaction. journal of positive psychology, 2,149-156. http://dx.doi.org/10.1080/17439760701228938 podsakoff, p. m., mackenzie, s. m., lee, j., & podsakoff, n. p. (2003). common method variance in behavioral research: a critical review of the literature and recommended remedies. journal applied of psychology, 88, 879-903. http://dx.doi.org/10.1037/0021-9010.88.5.879 price, l. l., arnould, e. j., & tierney, p. (1995). going to extremes: managing service encounters and assessing provider performance. the journal of marketing, 8397. http://dx.doi.org/10.2307/1252075 raferty, a. e. & griffin. m. a. (2004). dimensions of transformational leadership: conceptual and empirical extensions. the leadership quarterly, 15, 329-354. http://dx.doi.org/10.1016/j.leaqua.2004.02.009 roberge m. (2013). a multi-level conceptualization of empathy to explain how diversity increases group performance. international journal business. management, 8, 122–135. http://dx.doi.org/10.5539/ijbm.v8n3p122 solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 62 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.5465/amr.2012.0031 http://dx.doi.org/10.1080/17439760701228938 http://dx.doi.org/10.1037/0021-9010.88.5.879 http://dx.doi.org/10.2307/1252075 http://dx.doi.org/10.1016/j.leaqua.2004.02.009 http://dx.doi.org/10.5539/ijbm.v8n3p122 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 salanova, m. (2008). organizaciones saludables y desarrollo de recursos humanos. estudios financieros, 303, 179-214. salanova, m., agut, s. & peiró, j. m. (2005). linking organizational resources and work engagement to employee performance and customer loyalty: the mediation of service climate. journal of applied psychology, 90, 1217-1227. http://dx.doi.org/10.1037/0021-9010.90.6.1217 salanova, m., cifre, e., martínez, i., y llorens, s. (2007). caso a caso en la prevención de riesgos psicosociales. metodología wont para una organización saludable. bilbao: lettera publicaciones. salanova, m., lorente, l., chambel, m. j., & martínez, i. m. (2011). linking transformational leadership to extra rol behavior: the mediating role of self-efficacy and work engagement. journal of advanced nursing, 67, 2256-2266. http://dx.doi.org/10.1111/j.1365-2648.2011.05652.x salanova, m., llorens, s., cifre, e., & martínez, i.m. (2012). we need a hero! towards a validation of the healthy & resilient organization (hero) model. group & organization management, 37, 785-822. http://dx.doi.org/10.1177/1059601112470405 salanova, m., martínez, i.m., & llorens, s. (2005). psicología organizacional positiva. in f.j. palací (ed..), psicología de la organización (pp. 349-376). madrid: pearson, prentice-hall. schaufeli, w. b. y bakker, a .b. (2004). job demands, job resources, and their relationship with burnout and engagement: a multi-sample study. journal of organizational behavior, 25, 293-315. http://dx.doi.org/10.1002/job.248 schaubroeck, j., lam, s., & cha, s. a. (2007). embracing transformational leadership: team values and the impact of leader behavior on team performance. journal of applied psychology, 92, 1020-1030. http://dx.doi.org/10.1037/00219010.92.4.1020 solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 63 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1037/0021-9010.90.6.1217 http://dx.doi.org/10.1111/j.1365-2648.2011.05652.x http://dx.doi.org/10.1177/1059601112470405 http://dx.doi.org/10.1002/job.248 http://dx.doi.org/10.1037/0021-9010.92.4.1020 http://dx.doi.org/10.1037/0021-9010.92.4.1020 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2014.3694 social and technological sciences eissn: 2341-2593 seligman, m. e. p. (1999). the president’s address. american psychologist, 54, 559562. seligman, m. e. p., steen, t. a., park, n., & peterson, c. (2005). positive psychology progress: empirical validation of interventions. american psychologist, 60, 410-421. http://dx.doi.org/10.1037/0003-066x.60.5.410 seligman,m. e. p., & csikszentmihalyi, m. (2000) positive psychology: an introduction. american psychologist, 55, 5-14. http://dx.doi.org/10.1037/0003066x.55.1.5 sobel, m. e. (1982). asymptotic confidence intervals for indirect effects in structural equation models. in s. leinhardt (eds.), sociological methodology (pp. 290-312). washington, dc: american sociological association. http://dx.doi.org/10.2307/270723 woolley, l., caza, a., & levy, l. (2011). authentic leadership and follower development: psychological capital, positive work climate, and gender. journal of leadership studies & organizational development, 18, 438-448. http://dx.doi.org/10.1177/1548051810382013 yukl, g. (1998). leadership in organizations (4th ed.). nj: prentice hall. zaccaro, s. j., rittman, a. l. & marks, m. a. (2001). team leadership. the leadership quarterly, 12 , 451-483. http://dx.doi.org/10.1016/s1048-9843(01)00093-5 solares et al. (2015) http://polipapers.upv.es/index.php/muse mult. j. edu. soc & tec. sci. vol. 2 nº 2 (2015): 38-64 | 64 http://polipapers.upv.es/index.php/muse http://dx.doi.org/10.1037/0003-066x.60.5.410 http://dx.doi.org/10.1037/0003-066x.55.1.5 http://dx.doi.org/10.1037/0003-066x.55.1.5 http://dx.doi.org/10.2307/270723 http://dx.doi.org/10.1177/1548051810382013 http://dx.doi.org/10.1016/s1048-9843(01)00093-5 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 79 examining pedagogical knowledge content on mitosis in a university context n. gonzález*1, a. rossi2 1facultad de ciencias veterinarias, universidad nacional de la plata. 60 y 118, 1900 la plata, provincia de buenos aires, argentina. 2facultad de ciencias médicas, universidad nacional de la plata. 60 y 120, 1900 la plata, provincia de buenos aires, argentina. * corresponding author: email: nvgonzal@hotmail.com. +54 221 4236663 received: 2016-05-23; accepted: 2016-08-15 abstract mitosis is a process of cell division occurring in eukaryotic organisms. students from many countries experience difficulties learning this science topic, and its teaching demands substantial effort. effective teachers develop a wide range of knowledge types to successfully transform science matter for students; this transformation of knowledge has been conceptualized as pedagogical content knowledge (pck). in this study the pck of two university teachers on mitosis was explored. as informed by the instruments employed (content representation and pedagogical (core), and professional experiences repertoires, analytical rubric (pap-er), and semi-structured interviews) both participants’ pck on mitosis can be characterized as incomplete, however not identical. pck evolves throughout the professional practice so, in a context mostly limited to a traditional teacher-centered transmission of knowledge such as the university, development of teachers’ pck emerges as a strategy to reorient the teaching of mitosis to modalities based on the construction of scaffoldings to facilitate students’ learning. keywords pedagogical content knowledge (pck), mitosis, university, content representation questionnaire (core), professional and pedagogical experience repertoire (pap-er) http://dx.doi.org/10.4995/muse.2016.4653 mailto:nvgonzal@hotmail.com multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 80 1. introduction mitosis is a process of nuclear division in eukaryotic cells that occurs when a single parent cell divides resulting in generally two identical daughter cells each containing the same number of chromosomes and genetic content as that of the original cell. during cell division, mitosis refers specifically to the separation of the duplicated genetic material carried in the nucleus. this biological process is located at the intersection of unifying topics of biology i.e. continuity, in connection to reproduction, and development of cells and organisms for it relates to growth, tissue repair and regeneration. learning mitosis is also fundamental to the understanding of transmission genetics and molecular biology (ayuso and banet; 2002; locke and mcdermid, 2005). students from many countries experience difficulties particularly when discriminating biological concepts such as chromosomes, chromatids, diploid cells, and struggle to make appropriate and meaningful connections between mitosis and genetic information (lewis, leach and wood-robinson 2000; dikmenli, 2010; chattopadhyay, 2012; çimer, 2012). different strategies have been proposed to teach this topic, i.e. interactive videos (baggott and wright, 1996), chromosomal modeling by using pool noodles (locke and mcdermid, 2005), and socks (chinnici, neth and sherman, 2006), computer-based activities (tsui and treagust, 2013), web-based curriculum units (williams, montgomery and manokore, 2012), and also role-playing (chinnici, yue and torres, 2004). in large university classes, lecturers apply to visualizations of this cell division, among many those supported on powerpoint® slide shows (gonzález et al., 2014c). more than three decades ago teachers rated mitosis as one of the most difficult topics to teach (finley, stewart and yarroch, 1982). as students experience difficulties when learning this topic, its teaching demands considerable effort for novice (yip, 1998) or even experienced teachers (knippels, waarlo and boersma, 2005; oztap, ozay and oztap, 2003; gonzález and rossi, 2015). http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 81 many studies conclude that teachers’ performance in the classroom is one of the most important factors in students’ academic achievement. thus, effective teachers develop and display a wide range of knowledge types to successfully transform science matter for students (kind, 2009). magnusson, krajcick and borko (1999) define this type of knowledge as pedagogical content knowledge (pck): “the transformation of several types of knowledge from other domains” (p. 96). our research team has been working in the assessment of teachers’ pck on meiosis, a eukaryotic type of cell division process that shares with mitosis similar difficulties as those mentioned above and challenges for both students and faculty. we have successfully employed the loughran, mulhall and berry (2004) content representation and pedagogical, and professional experiences repertoires (core and pap-ers, respectively) to characterize the pck of pre-service and in-service secondary teachers on meiosis (gonzález and rossi, 2014a). in the same line of work, we employed a rubric to document university cell biology teachers’ pck focusing on the powerpoint® presentation they used in their classes (gonzález et al., 2014c). few works examine teachers’ pck on mitosis and are mainly focused on secondary education. the objectives of this study were to document the pck of two cell biology university teachers on the subject of mitosis following the methodology proposed by loughran et al. (2004), and complementarily to assess their pck as displayed in a set of slides. the research questions guiding the investigation were: what components of pck on mitosis can be identified in a university context? what pck content can be identified by an analytical rubric assessing a powerpoint presentation on mitosis? http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 82 2. conceptual framework pedagogical content knowledge (pck) as proposed originally by shulman (1986) includes “the most powerful analogies, illustrations, examples, explanations, and demonstrations in a word, the ways of representing and formulating the subject that makes it comprehensible for others” (1986, p. 9). pck has been interpreted by scholars in many ways addressing different features of it. kind (2009) reviews a variety of models of pck including those that follow shulman‘s line of thought explicitly and others that draw on empirical research findings or on principles from psychology. the magnusson et al. (1999) model is within the former group. these researchers conceptualize pck as consisting of five components: (a) orientations toward science teaching, (b) knowledge and beliefs about the science curriculum, (c) knowledge and beliefs about students’ understanding of specific science topics, (d) knowledge and beliefs about of assessment in science, and (e) knowledge and beliefs about instructional strategies for teaching science. in 2012, international research teams gathered at a pck summit and produced the following consensus definition: pck is a “personal attribute of a teacher, considered both a knowledge base and an action. it is the knowledge of, reasoning behind, planning for, and enactment of teaching a particular topic in a particular way for a particular reason to particular students for enhanced student outcomes” (carlson and gess-newsome, 2013). as stated by participants at the pck summit, teacher professional knowledge bases are the backbone of the profession thus including assessment knowledge, pedagogical knowledge, content knowledge, knowledge of students, and curricular knowledge. more interestingly, as depicted in figure 1, in the classroom practice, pck interacts with the classroom context, and with two sets of amplifiers and filters. amplifiers increase the potential of an idea being accepted or an action being implemented whereas filters can extinguish good intentions or ideas. one set of amplifiers and filters comprises teachers’ beliefs and orientations; the other set relates to students’ beliefs, prior knowledge, and behaviors. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 83 figure 1. the teacher professional knowledge bases consist in a series of knowledges. topic specific professional knowledge is derived from a cross section of the teacher professional knowledge bases for a specific topic. there are two sets of amplifiers and filters, the first set falls between topic specific subject matter professional knowledge and classroom practice; the second set falls between the latter and students outcomes. classroom practice is the context in which pck exists. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 84 3. methodology 3.1 context of the study the study took place in the veterinary college, university of la plata, argentina during a 3 h weekly instruction period of a cell biology course. it is a mandatory course for first year students. mitosis is a topic included in the course curriculum that is focused mainly on cytological aspects and corresponds to a teaching sequence after dealing with basic genetics, i.e. dna replication, transcription, protein synthesis, and dna recombination. it is presented to the students as the final phase of the cell-cycle. mendelian genetics are covered in a separate second year course. 3.2 participants two teachers (sylvia and juliette, pseudonyms) participated voluntarily in the study. both graduated as veterinary medical doctors and earned their ph.d. where the study took place and have had very little non-systematic teaching training. they have taught the cell biology course as teaching assistants since 2006, and are thoughtful about their classroom performance and highly appreciated by their students and colleagues. sylvia’s and juliette’s classes had 30 and 21 students, respectively. 3.3 instruments pck researchers have developed an array of methodologies and techniques to gain knowledge into this construct, e.g. paper and pencil assessments such as open-ended and multiple choice questions, concept maps, drawings, interviews, video observations of real instruction, classroom observations, and very commonly, some kind of combination of the previously mentioned (baxter and lederman, 1999; borowski, carlson, fischer, henze, gess-newsome, kirschner and van driel, 2012). in this study four instruments were employed. two of them corresponded to the loughran et al. (2004) core and pap-er tools. the core consisted of eight questions (see results http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 85 section). the core questionnaire was presented to the participants and discussed to orientate the elaboration of responses after the class observation. later, it was sent by email, and answers received a week later by the same via. the pap-er was developed from observations of classes, one of the sources proposed by loughran et al. (2004). the researchers, as non-participant observers, took detailed field notes and produced shortly afterwards a written version of the material originally gathered at the class. the third instrument employed was the analytical rubric (gonzález et al., 2014c). it comprised a content dimension focused on pck whereas the other dimensions aimed to the design of the slides as instructional materials. briefly, the rubric covered the following dimensions and indicators:  content: core conceptualizations (identified by a previous inquiry with expert colleagues), logical sequencing, relations between core conceptualizations, transition between concepts, data and/or examples inclusion.  organization: introduction-body-conclusion format, transition between slides, and internal coherence.  information: quantity, quality, and pertinence of data.  graphic aspects: quality, quantity, relevance and creative use of photographs, graphics and tables.  textual aspects: grammar and spelling, terminology, quality and extent of texts and titles. four levels were established for the assessment: exemplary, proficient, acceptable, and unacceptable. each indicator was assigned 3, 2, 1 or 0 points, respectively. the powerpoint® slides were kindly provided by the teachers. the assessment was performed independently by the researchers, and latter discussed so that scorings were consensual. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 86 the fourth instrument included in this study was a semi-structured interview. sylvia and juliette were interviewed individually to review their answers to core, and specific aspects of their classes and presentation slides. then, the final analysis of the core responses, the writing of pap-er and the scoring by the rubric were performed. 4. results 4.1 core for clarity, sylvia’s and juliette’s responses are presented together with the questions. 1. what do you intend the students to learn about this topic? our two participants’ central ideas are presented in table 1. table 1. the two participant’s central ideas on mitosis. sylvia´s central ideas juliette´s central ideas • mitosis is a regulated process that occurs in unicellular and multicellular eukaryotic organisms. • mitosis occurs in somatic cells e.g. keratinocytes (epidermal cells), hepatocytes (liver cells). • every cell originates from another existing cell like it (virchow´s omnis cellula e cellula). • the chromatids of a replicated chromosome separate in the anaphase. • in dividing cells, mitosis is the culmination of their cell-cycle. • daughter cells in mitosis are genetically identical to the parent cell and the reasons for this. • in mitosis, the parent cell transfers its genetic material previously replicated to daughter cells. • the comparison of cell division by mitosis and cell division by meiosis. • daughter cells originated by mitosis maintain the diploid number of chromosomes of the parent cell. • the sequence of events in mitosis is continuous; it is divided into stages mainly for didactic reasons. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 87 2. why is it important for the students to know this? sylvia stated that mitosis is a basic knowledge related to several biological concepts such as haploid and diploid cells, and processes like cell division by meiosis, and diploid chromosomal number restoration at fertilization. she highlighted that mitosis also provides knowledge basis for the understanding of molecular and cellular foundations of many diseases in which cell proliferation is altered. juliette’s answer referred to the role of mitosis in animals i.e. in development, growth, reparation and regeneration processes; she also mentioned its role in the reproduction of unicellular organisms. 3. what else do you know about this idea (that you don't intend students to know yet)? sylvia referred to a large body of knowledge on mitosis regulation, e.g. cascades of protein phosphorylation, the groups and subtypes of cyclin proteins family, and the metaphaseanaphase checkpoint. she also linked failures of the cell-cycle machinery to cancer development. as her colleague, juliette listed a number of molecular aspects of mitosis, the majority of them mostly in relation to chromosome structure (cohesines, condensines). she considered these topics should be addressed in the lectures given by professors. 4. which difficulties/limitations are connected with teaching this topic? the main difficulties (items 1, 2, and 3), and limitations (item 4) pointed out by sylvia and juliette are summarized in table 2. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 88 table 2. condensed main difficulties and limitations referred by sylvia and juliette. item sylvia juliette 1. the complex terminology as a main obstacle, i.e. chromatin, chromosome and chromatid, a set of terms closely similar for students although accurate in the domain-specific vocabulary. x x 2. misunderstanding in the timing of dna replication, metaphase alignment and anaphasic migration of chromosomes. x x 3. deficiencies in the students’ understanding of the cell-cycle regulation at the molecular level. x 4. students generally are not able to develop an adequate insight of mitosis within the cell-cycle. x x 5. as cell division is considered a difficult topic, students’ involvement in learning is minimal. x 5. knowledge about students' thinking that influences your teaching of this topic. from her teaching experience, sylvia mentioned those common students´ misconceptions referred as difficulties and limitations in the fourth question. additionally, she pointed out that the identification of those misconceptions can be used as a base to plan lessons. juliette observed that students generally acknowledge the definition of mitosis –as the one presented in the introductionand she takes advantage of this fact as a stepping stone to start the lesson and help students to build and deepen their knowledge of the topic. 6. which other factors influence your teaching of this topic? sylvia believed teaching mitosis requires initially two major decisions about its approach: mitosis can be focused from different points of view such as the course context in the curriculum, and the time assigned to cover this topic. for juliette a conditioning factor is the absence of students’ previous knowledge of mitosis, i.e. the lack of the stepping stone she mentioned in her previous answer that restricts her students’ progress. she also highlighted that students’ attitudes of indifference or disinterest are a challenge to her classroom management effectiveness. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 89 7. teaching procedures (and particular reasons for using these to engage with this topic). sylvia answered by means of a list of activities that can be parted as beforeand inclass activities. before-class activities included the students’ reading from their textbooks and doing assigned homework to revise previous concepts needed for the new topic and practice new ones as a way to engage their participation in class. in-class activities comprised a set of tasks dealing with factual knowledge such as changes in the nuclear compartment, number of chromosomes and chromatids, identification of mitosis phases, cytoplasmic changes e.g. in the cytoskeleton, the cell-cycle regulation, and differences between mitosis of plants and animals. these learning tasks include written explanations, simple mathematical calculations, and description of images of dividing cells. juliette stated that she intends to engage students by pointing out the relevance of mitosis as a more detailed explanation of the processes mentioned in her second answer, linking it to veterinary medicine, e.g. liver regeneration in domestic animals after ingestion of a toxicant. 8. specific ways of ascertaining students' understanding or confusion around this topic. sylvia assesses diverse aspects of mitosis by oral questioning in class (e.g. for ploidy: if a cell has a 38 diploid number, which is the ploidy and number of chromosomes in the daughter cells?). she also encourages her students to analyze, compare and discuss the mitosis stages depicted in the slides used in class as a way to explicit misconceptions held by students. juliette mused on how she relies on oral questioning during the class; moreover she employs questions to provide some scaffolding when students summarize the contents covered in class. questions quoted by juliette were similar to sylvia’s. 4.2 pap-er for brevity, one pap-er was constructed as a condensed, narrative, and comparative report extracted from the class observation notes. for clarity, the teaching sequences were depicted in two graphic representations (fig. 2). http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 90 figure 2. sylvia´s (left) and juliette´s (right) teaching sequences developed at the session. differences found were highlighted in a blue font. the sessions were carried out following a lecture and solving activities format. subject matter was presented with a set of powerpoint® slides. a short introduction for the session included the explanation of the learning objectives and the contextualization of mitosis (and meiosis) within the conceptual frame of sexual and asexual reproductions. after lecturing on cell-cycle, teachers presented the students an activity to assess their understanding and keep up their attention. it consisted in some questions presented in a slide (fig. 3) on how to differentiate cells in some phases of the cell-cycle. students engaged for ten minutes in small collaborative groups to discuss and elaborate their answers. during the following minutes students presented verbally their responses; teachers acted as moderators and posed additional questions to promote further discussion and understanding. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 91 figure 3. first activity presented to the students in the session. the expected responses involve the occurrence of rna, dna and protein synthesis at specific phases of the cell-cycle, e.g. dna is synthesized only in the s phase so it can be employed to differentiate g1 phase of s phase. the next subtopic explained by both teachers was the cell-cycle regulation, its processes and checkpoints. the explanation relied on molecular and cellular aspects, viz. kinases and cyclins; those contents were developed in a more extended segment by sylvia. a second activity regarding the ploidy and homologous chromosomes concepts was then introduced. it was a short problem, consisting of two questions about a hypothetical cell (fig. 4). a designated student of each group presented the answers to the class. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 92 figure 4. second activity presented to the students in the session. expected answers are (a) the haploid number is 3, and (b) the set of chromosomes is haploid. the last lecture segment was about the cytological aspects of mitosis. sylvia revised the organization of the interphase nucleus and guided students to elaborate verbally the interpretation of its changes through mitosis as they described the photographs on the slides. she also established relationships between metaphase and anaphase with the corresponding cell-cycle checkpoint. as a summary, and also to reinforce the covered material on mitosis, a 2.47 minutes youtube video was presented (https://www.youtube.com/watch?v=cvlpmmvb_m4). juliette also described the stages of mitosis but hardly got responses of her students to her questions about the images. as the time session was running out, she chose not to use the video. the student participation varied along the session. in sylvia´s classroom, during the lecture segments of the session, most students showed engagement when dealing with low complexity subtopics: they listened, took notes, nodded their heads as recognition of their understanding, and asked questions to confirm their comprehension. as the cell-cycle regulation is a much complex subtopic; when it was introduced students seldom did anything but listen. thus, the pacing of the class slowed down for students clearly struggled http://dx.doi.org/10.4995/muse.2016.4653 https://www.youtube.com/watch?v=cvlpmmvb_m4 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 93 to recall the functions of proteins, a basic factual knowledge needed to incorporate new meanings into their prior knowledge i.e. to conceptualize cell-cycle regulation. the two activities within the collaborative groups were solved enthusiastically. in the final segment of the session students accepted the challenge of a third activity based on photographs of cells in different stages of mitosis. the video was gladly received; when asked by researchers after class, students commented it was a useful material to facilitate and enhance their learning. interestingly, researchers noticed differences in juliette’s classroom. her students held less positive attitudes throughout the class; most of them seemed bored during the lecture segments and appeared reluctant to answer questions posed by juliette. students’ interaction was poor or inexistent in some groups while solving the activities. two students even left the classroom in the second half of the session as if they had decided that it wasn’t worth staying. in summary, both teachers acted alike during the session, promoting their students’ understanding, carrying out a lecture and in-lecture activities format for their classes. briefly, the main subtopics lectured by sylvia and juliette were the characterization of cellcycle in animal cells, its regulation, and the cytological aspects of mitosis (fig. 2). 4.3 analysis of the teaching slides by the rubric 4.3.1 descriptive overview sylvia’s presentation consisted in 22 slides, including a first title slide. sylvia employed slides containing solely images, only text and a combination of images + text (18, 1, and 1 slide, respectively). her final slide was dedicated to a short video, downloaded from youtube, depicting the cell division by mitosis. juliette’s presentation included a first title slide within a total of 22 slides. she also employed slides including only images, only text and a combination of images + text (19, 1 and 1 slide, respectively). she chose not to use the video due to time constraints. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 94 4.3.2 assessment by the rubric as it concerned our objective of documenting the participants’ pck, the assessment was focused on the content dimension. detailed results are presented in the appendix. short descriptive outcomes related to the remaining dimensions were also elaborated and here presented. content dimension: all core contents (identified by a previous inquiry with expert colleagues) were addressed in both set of slides; subtopics were logically sequenced but each participant’s elaboration differed in qualitative aspects. connection and transition between concepts were achieved in both presentations by means of similar precise links. sylvia’s slides received a higher score on depth of content due to a more extended elaboration on cell-cycle regulation based on one of the learning goals of the class1. her score was also higher for the variety of examples she presented to the students. the scores for sylvia’s and juliette’s presentations were 17/18 points and 15/18 points for the content dimension, respectively (see appendix for detailed results). organization and information dimensions: sylvia’s and juliette’s slides alike received the exemplary level. graphic aspects dimension: both presentations included a similar number of relevant slides to depict the topic. however, juliette incorporated images from different sources – 1 list the four stages of interphase, and describe the major events that occur during each stage in preparation for cell division. list the checkpoints that regulate the progression of cells through the cell cycle and explain the mechanisms. list the phases of mitosis in a eukaryotic cell, and discuss the major events that happen during each phase. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 95 books and web pages– in the same slide; such diversity of representations was considered a potential cognitive overload that diminished their pedagogical value. she excluded the use of the video due to time constraints. sylvia showed the video and revealed a creative use of images for she developed a short time extra in-lecture activity to summarize the phases of mitosis. textual aspects dimension: the two instructional materials had an appropriate simple design used as a mean to greater clarity; they both were exemplary on all indicators. 5. discussion in this study the pck of two university teachers on mitosis was explored. the choice of this topic was based on its disciplinary centrality, and on that it is regularly taught in introductory biology courses. the tools employed were the loughran et al. (2004) core and pap-er, an analytical rubric, and semi-structured interviews. the analyses of the results concerning the cores and pap-er through the magnusson’s et al. (1999) pck components that framed our investigation led to the following appreciations. orientation toward science teaching. an orientation represents a general way of viewing of conceptualizing science teaching; it is described with respect to the goals of science teaching and the characteristics of the instruction (magnusson et al. 1999). in reference to the goals of teaching this topic both teachers combined academic rigor and didactic orientations. their classes were teacher-centered: they presented a body of factual knowledge by explanations and short discussions. sylvia implicitly drove students to develop thinking processes whereas juliette directed questions to students to mostly hold them accountable for knowing the scientific information on the topic. knowledge of science curriculum. this component of pck consists of two categories: teachers’ knowledge of the goals and objectives for students in the subject they are teaching and their knowledge of what students have learned previously and what they are expected http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 96 to learn later on (magnusson et al. 1999). sylvia and juliette are knowledgeable about the learning objectives for students in this course. they both started the session drawing the students’ attention to them and throughout the lesson made clear connections to topics addressed in previous meetings such as dna replication and cytoskeleton components. as veterinary medicine graduates, they were able to link mitosis to other courses, i.e. pathology (sylvia) and basic genetics (juliette). knowledge of students’ understanding of science. two categories of knowledge are included in this third component: requirements for learning specific science concepts, and areas of science that students find difficult (magnusson et al. 1999). teaching experience accounts for sylvia’s and juliette’s awareness of students’ difficulties on mitosis; they were able to list alternative conceptions, difficulties, and misunderstandings. although they acknowledged the abstract nature of mitosis, they were not able to explicitly mention requirements for learning this topic. knowledge of assessment in science. there are two categories of knowledge included in this conceptualization: knowledge of the dimensions of science learning to assess and knowledge of the methods of assessments (magnusson et al. 1999). sylvia considered that oral quizzes were suitable for formative assessment in class; she believes individual and written summative evaluation should be implemented after the session. juliette shares sylvia’s first idea. they both value the final course evaluation (multiple choice test on conceptual understanding) as the formal assessment but mentioned no alternatives to it. knowledge of instructional strategies. this component is comprised by two types of knowledge: knowledge of subject-specific strategies and knowledge of topic-specific strategies (representations and activities). sylvia’s emphasized on the students’ involvement, and as observed in the actual class, she strongly intended her students to participate during the lecture. she mentioned activities dealing with factual knowledge of mitosis; some of them could be valued as topic-specific strategies e.g. simple mathematical calculations dealing with the number of chromosomes and chromatids in parental and daughter cells. although varied, it is doubtful to conceive that these activities would http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 97 actually engage students. juliette stated that she has no knowledge of other teaching strategies than lecturing. during the session she intended to promote students’ engagement with the topic by presenting examples related to veterinary medical practice. rubrics are widely employed as effective assessment tools in science teaching (allen and tanner, 2006); in the present work the innovative instrument by gonzález et al. (2014) was applied to our participants’ powerpoint® presentations to assess their pck and thus complement their core and the pap-er. two indicators –depth of content and data and/or examples inclusion– provided evidence of components of sylvia’s and juliette’s pck related to the curriculum domain: knowledge of the learning goals of the session and knowledge of what students have learned previously and what they are expected to learn afterwards. besides the differences detailed in the descriptive aspects, the analysis rendered other remarkable feature. the teaching sequence was prescriptive and encapsulated in the powerpoint® slides provided by the course coordinator. however, as some modifications were available for teaching assistants to make; sylvia’s choice rested on the incorporation of slides concerning molecular aspects of cell-cycle regulation thus reinforcing the explanation of academic aspects of this troublesome scientific idea. moreover, sylvia included an additional slide about homologues chromosomes, another conflicting issue for students. on the other hand, juliette included in her presentation supplementary images concerning the stages of mitosis. sylvia’s emphasis on the molecular aspects of cell-cycle regulation and conceptualization of chromosomes and juliette’s focusing on the cytological aspects of mitosis and cytokinesis revealed their knowledge of the students’ understanding of specific science topics, specifically topics that students find difficult. thus, the dimensions related to the slides as instructional materials showed that these teacher assistants chose different ways to represent and cover the content of the session, each highlighting those aspects of mitosis they valued the most relevant for their students and in coincidence of their responses to co-re question 2. few categories of the magnusson et al. (1999) pck components were identified in sylvia’s and juliette’s co-re, pap-er, and rubric outcomes consisting mainly in the transformation http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 98 of disciplinary knowledge oriented to the presentation of content, and some factual conceptions about teaching procedures, curriculum and evaluation. at this point, to refine results from the assessments, it is interesting to introduce talanquer’s perspective (2004) on the pck; this author argues that teachers’ pck must be sufficient to:  identify the main ideas associated with a topic.  recognize the students’ probable conceptual difficulties and the impact on learning.  identify questions, problems or activities that require students to recognize and challenge their preconceptions.  select experiments, problems or projects that allow students to explore central ideas in the discipline.  build explanations, analogies or metaphors to facilitate understanding of abstract concepts.  design assessment activities that allow the application of learning in realistic and varied contexts. viewed in light of talanquer’s requirements (2004), we believe that both participants’ pck on mitosis can be characterized as incomplete, however not identical. the greatest concurrence was found in the identification of the main subtopics in the process of mitosis, and the difficulties in its teaching, focusing on some students’ shortcomings. sylvia’s knowledge and beliefs informed by core were aligned with her transmission-orientated teaching revealed in the pap-er. nonetheless, sylvia´s central ideas are more numerous and cover a wider range of subtopics, and throughout her class, she continually and systematically re-visited the session’s goals. as evidenced by the rubric assessment and pap-er, in a format lecture she made room for a creative use of slides in an attempt to engage her students with the topic. moreover, in previous study sylvia’s beliefs and ideas on the teaching of this topic revealed her reflection on action and reflection in action (gonzález et al., 2014b). on the other hand, juliette was aware of some critical aspects of teacher’s pck based, as sylvia, on her classroom experience and her subject matter http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 99 knowledge. she was at all times the driving force in the classroom, and although she explicitly connected mitosis and biological processes related to domestic animals, it failed to captivate her students’ attention. it resulted evident that although both teachers had been receptive towards the incorporation of in-lecture activities as proposed by the course coordinator; their limited knowledge about instructional and assessment strategies shaped their orientations towards science teaching and, mostly they taught as they had been taught. we believe that the differences between sylvia’s and juliette’s pck can be further explained by taking in consideration the filters and amplifiers included in the teacher professional knowledge bases model presented in figure 1 (carlson and gess-newsome, 2013). filters and amplifiers comprise teachers’ beliefs, orientations, prior knowledge and context; as a counterpart also students’ beliefs, prior knowledge and behaviors are included as potential filters and amplifiers. our participants share most of their formative disciplinary backgrounds and teaching experiences but, as noted in the pap-er, the climate of the classrooms differed: in sylvia’s classroom it was more relaxed and, on demand of her students, she devoted time to clarify erroneous or incomplete concepts whereas in juliette’s session, at certain moments, an air of tension seemed to prevail. the students’ choices to attend to instruction or ignore it, embrace student-centered teaching practices or resist them as possible courses of action became crystal-clear in each classroom and, for juliette, developed in a filter that diminished her good intentions and actions. numerous studies concerning effective teaching strategies for mitosis at diverse educative levels reflect its presence and relevance in science curriculums all over the world. however, being pck a content-specific construct, investigations addressing this topic at universities are practically non-existent. the study reported here explored two university biology teachers' pck on mitosis and is part of a bigger project; these results encourage us to carry on so knowledge gained from the research program would be built in novel contributions for high quality classroom practice. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 100 6. conclusions and implications • the core and pap-er presented are autonomous because they illustrate how the pck is evident by revealing specific aspects of the action in the classroom (juliette) and reflection on action and reflection in action (sylvia). • the differences found in the teachers’ pck are consistent with the characteristics of this construct: pck is specific to a topic, a teacher and a context.  the analytical rubric provided pre-established performance criteria to make intelligible some aspects of the pck components from the magnusson and coworkers’ theoretical framework adopted in this study. its effectiveness and value as an assessment tool relied in its complementary character to other highly recognized survey instruments as the reco and pap-er. we would like to close this article with a reflection on the assumption of the pck as a continuum –from weak to strong– as it entails a second notion: pck can be strengthened (gess-newsome, carlson, gardner and taylor, 2010). the interaction of the components of pck has been noted to be limited in research with novice teachers (gonzález and rossi, 2014a); a similar situation has been demonstrated in investigations with preservice teachers (friedrichsen et al. 2009). kind (2009) points that pck remains unnoticed by many science teachers; for instance, as many of our fellow university teachers, sylvia and juliette, the two experienced teachers in the present work had never heard the term. our characterization of the participating teachers’ pck as incomplete indicates the strong need to pay special attention to the construction of pck as a way of improving teacher professional knowledge (kind, 2009; loughran et al. 2012). taking in consideration that the pck of a teacher evolves throughout his professional practice (olander and olander, 2013) and, furthermore given the acknowledged role of reflection in the development of science teachers’ pck (popovic and antink, 2010), we highlight the relevance of including in-service training to enhance the professional development of university teachers. moreover, in a context mostly limited to a traditional teacher-centered transmission of http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 101 knowledge such as the university, and in line with several investigations that encourage the use of cores and pap-ers as an strategy to develop and support science teachers’ pck (bertram, 2014), we strongly believe that may prompt university teachers to reflect meaningfully on their practice. the pck development of teachers emerges as a strategy to re-orient the teaching of mitosis to modalities based on the construction of scaffoldings to facilitate students’ learning. acknowledgements we thank the teaching assistant who participated in this study. this work was supported by grants from the national university of la plata (project 11/v216) “la construcción del conocimiento pedagógico del contenido (cpc) y del conocimiento del contenido tecnológico-pedagógico (cctp) para el proceso de mitosis en profesores universitarios”. http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 102 5. references ayuso g.e. and banet e. (2002). alternativas a la enseñanza de la genética en educación secundaria. enseñanza de las ciencias, 20(1), 133-157. baggott l. and wright b. (1996). the use of interactive video in teaching about cell division. journal of biological education, 30(1), 57-88. http://dx.doi.org/10.1080/00219266.1996.9655478 baxter j.a. and lederman n. g. (1999). assessment and measurement of pedagogical content knowledge. in j. gess-newsome and n.g. lederman (ed.), examining pck (pp. 147-163). boston, ma: kluwer. bertram a. (2014). cores and pap-ers as a strategy for helping beginning primary teachers develop their pedagogical content knowledge. educación química, 25(3), 292303. http://dx.doi.org/10.1016/s0187-893x(14)70545-2 borowski a., carlson, j., fischer, h.a., henze, i., gess-newsome j., kirschner s. and van driel j. (2012). different models and methods to measure teachers' pedagogical content knowledge. in c. bruguière, a. tiberghien and p. clément (ed.), e-book proceedings of the esera 2011 conference: science learning and citizenship. (pp. 37-40). retrieved from http://www.esera.org/media/ebook/strand13/ebook-esera2011_borowski-13.pdf carlson j. and gess-newsome j. (2013). the pck summit consensus model and definition of pedagogical content knowledge. the symposium “reports from the pedagogical content knowledge (pck) summit”, esera conference 2013. chattopadhyay a. (2012). understanding of mitosis and meiosis in higher secondary students of northeast india and the implications for genetics education. education, 2(3), 41-47. http://dx.doi.org/10.5923/j.edu.20120203.04 chinnici j. p., neth s. z. and sherman, l. r. (2006). using “chromosomal socks” to demonstrate ploidy in mitosis & meiosis. the american biology teacher, 68 (2), 106109. http://dx.doi.org/10.1662/0002-7685(2006)068[0106:ucstdp]2.0.co;2 http://dx.doi.org/10.4995/muse.2016.4653 http://dx.doi.org/10.1080/00219266.1996.9655478 http://dx.doi.org/10.1016/s0187-893x(14)70545-2 http://www.esera.org/media/ebook/strand13/ebook-esera2011_borowski-13.pdf http://dx.doi.org/10.5923/j.edu.20120203.04 http://dx.doi.org/10.1662/0002-7685(2006)068%5b0106:ucstdp%5d2.0.co;2 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 103 chinnici j. p., yue j. w. and torres k. m. (2004). students as human chromosomes in role-playing mitosis & meiosis. the american biology teacher 66 (1), 35-39. çimer a. (2012). what makes biology learning difficult and effective: students’ views. educational research and reviews, 7(3), 61-71. dikmenli m. (2010). misconception of cell division held by student teacher in biology: a drawing analysis. scientific research and essay, 5, 235-247. din‐yan y. (1998). identification of misconceptions in novice biology teachers and remedial strategies for improving biology learning. international journal of science education 20(4), 461-477. http://dx.doi.org/10.1080/0950069980200406 finley f.n., stewart j. and yarroch, w.l. (1982). teachers’ perceptions of important and difficult science content. science education, 66, 531-538. http://dx.doi.org/10.1002/sce.3730660404 friedrichsen p.j., abell s.k., pareja e.m., brown p.l., lankford d. m. and volkmann m.j. (2009). does teaching experience matter? examining biology teachers' prior knowledge for teaching in an alternative certification program. journal of research in science teaching, 46(4), 357-383. http://dx.doi.org/10.1002/tea.20283 gess-newsome j., carlson j., gardner a. and taylor, j. (2010). impact of educative materials and professional development on teachers’ professional knowledge, practice, and student achievement. retrieved from http://bscs.org/primepapers gonzález n.v. and rossi a.m. (2014a). la enseñanza de la meiosis en el nivel secundario: el conocimiento didáctico del contenido de profesores expertos y principiantes. in a. garritz, s. dazza and m.g. lorenzo (eds.), conocimiento didáctico del contenido. una perspectiva iberoamericana, (403-453). saarbrücken: editorial académica española. gonzález n.v., diessler m.e and rossi a.m. (2014b). a case study exploring a university teacher’s pedagogical content knowledge on mitosis, in c. p. constantinou, n. papadouris & a. hadjigeorgiou (eds.), e-book proceedings of the esera 2013 conference: science http://dx.doi.org/10.4995/muse.2016.4653 http://dx.doi.org/10.1080/0950069980200406 http://dx.doi.org/10.1002/sce.3730660404 http://dx.doi.org/10.1002/tea.20283 http://bscs.org/primepapers multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 104 education research for evidence-based teaching and coherence in learning. part 14 (d. couso and l. louca co-ed.), (pp. 1-11) nicosia, cyprus: european science education research association. isbn: 978-9963-700-77-6 gonzález n.v., solari m.j and rossi a.m. (2014c). an analytical rubric for powerpoint® presentations: proposal of a complementary instrument to assess pedagogical content knowledge, in c. p. constantinou, n. papadouris & a. hadjigeorgiou (eds.), e-book proceedings of the esera 2013 conference: science education research for evidencebased teaching and coherence in learning. part 3 (a. tiberghien and e. kyza co-ed.), (pp. 176-183) nicosia, cyprus: european science education research association. isbn: 9789963-700-77-6 gonzález n.v. and rossi a.m. (2015). conocimiento pedagógico del contenido para la enseñanza del tema mitosis: un estudio de casos con docentes universitarios de argentina. revista didáctica de las ciencias experimentales y sociales, http://dx.doi.org/10.7203/dces.29.4991 knippels m.c., waarlo a.j. and boersma k.t. (2005). design criteria for learning and teaching genetics. journal of biological education, 39, 108-112. http://dx.doi.org/10.1080/00219266.2005.9655976 lewis j., leach, j. and wood-robinson, c. (2000). chromosomes: the missing link-young people understand of mitosis, meiosis, and fertilization. journal of biology education, 34(4), 189-199. http://dx.doi.org/10.1080/00219266.2000.9655717 locke, j. and mcdermid, h.e. (2005). using pool noodles to teach mitosis and meiosis. genetics, 170, 5-6. http://dx.doi.org/10.1534/genetics.104.032060 loughran j.j., mulhall p. and berry a. (2004). in search of pck in science: developing ways of articulating and documenting professional practice. journal of research in science teaching, 41(4), 370-391. http://dx.doi.org/10.1002/tea.20007 loughran j., berry a. and mulhall p. (2012). understanding and developing science teachers’ pedagogical content knowledge (vol. 12). springer science & business media. http://dx.doi.org/10.4995/muse.2016.4653 http://dx.doi.org/10.7203/dces.29.4991 http://dx.doi.org/10.1080/00219266.2005.9655976 http://dx.doi.org/10.1080/00219266.2000.9655717 http://dx.doi.org/10.1534/genetics.104.032060 http://dx.doi.org/10.1002/tea.20007 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 105 http://dx.doi.org/10.1007/978-94-6091-821-6 magnusson s.; krajcik j. and borko h. (1999). nature, sources, and development of pedagogical content knowledge for science teaching. in j. gess-newsome and n.g. lederman (ed.), examining pedagogical content knowledge (pp 95-132). dordrecht: kluwer academic publishers. olande, c. and olander m.h. (2013). professional development through the use of learning study: contributions to pedagogical content knowledge in biology. procedia social and behavioral sciences, 89, 205-212. http://dx.doi.org/10.1016/j.sbspro.2013.08.835 oztap h., ozay e. and oztap, f. (2003). teaching cell division to secondary school students: an investigation of difficulties experienced by turkish teachers. journal of biological education, 38, 13-16. http://dx.doi.org/10.1080/00219266.2003.9655890 popovic g. and antink a. (2010). exploring the relationship between subject matter knowledge and reflective practice among preservice science and mathematics teachers. retrieved from http://files.ecetera.si/ioste/419.pdf williams m., montgomery b.l. and manokore v. (2012). from phenotype to genotype: exploring middle school students' understanding of genetic inheritance in a web-based environment. the american biology teacher, 74(1), 35-40. http://dx.doi.org/10.1525/abt.2012.74.1.8 http://dx.doi.org/10.4995/muse.2016.4653 http://dx.doi.org/10.1007/978-94-6091-821-6 http://dx.doi.org/10.1016/j.sbspro.2013.08.835 http://dx.doi.org/10.1080/00219266.2003.9655890 http://files.ecetera.si/ioste/419.pdf http://dx.doi.org/10.1525/abt.2012.74.1.8 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.4653 social and technological sciences eissn: 2341-2593 gonzález and rossi (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 79-106 | 106 appendix. detailed results of the assessment on the powerpoint® presentations. sylvia’s presentation juliette’s presentation content presence of core content 3 points 3 points logical sequencing 3 points 3 points connection between concepts 3 points 3 points transition between concepts 3 points 3 points depth of content 3 points 2 points data and/or example inclusion 2 point 1 point organization introduction-body-conclusion format 3 3 transition between slides 3 3 internal coherence 3 3 information quantity 3 3 quality (academic sources) 3 3 pertinence of data 3 2 graphic aspects quality 3 1 quantity 3 2 relevance 3 3 creative use 3 2 textual aspects grammar 3 3 spelling 3 3 terminology 3 3 quality and extent of titles 3 3 quality and extent of texts 3 3 total score 45 points 40 points http://dx.doi.org/10.4995/muse.2016.4653 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 11 the impact of the great depression in ecuador cristian paúl naranjo navas∗, andrés david naranjo navas∗∗, alegría cumandá navas labanda∗∗∗ ∗ facultad de ciencias de la educación, humanas y tecnologías. universidad nacional de chimborazo, avda. eloy alfaro y 10 de agosto, riobamba, ecuador. email: cnaranjo@unach.edu.ec, paulnaranjo@icloud.com ∗∗ departamento de ciencias políticas. facultad latinoamericana de ciencias sociales sede ecuador, calle la pradera e7-174 y av. diego de almagro. email: adnaranjofl@flacso.edu.ec, andynaranjob@hotmail.com ∗∗∗ facultad de ingeniería. universidad nacional de chimborazo, avda. eloy alfaro y 10 de agosto, riobamba, ecuador. email: anavas@unach.edu.ec, cumita9@hotmail.com received: 12 october 2018; accepted: 03 september 2019 abstract this investigation started with an inquiry: did the great depression impact latin america similarly? does the case of ecuador represent a point of dissimilarity? ecuador does represent an atypical case in the region. the data presented in this study shows that the impact of the great depression in latin america was uneven: while the region declined sharply from 1929 to 1932, and since 1933 it registers signs of accelerated growth, ecuador remained stagnant throughout the decade studied. however, the development of some macroeconomic data are similar in the entire region because of the close commercial links with the united states of america. the evolution of foreign trade and public finances were similar: a profound reduction until 1932-1933, and then a quick recovery. finally, the great depression represents a break point in latin american: the growth of the region passes from depending on the external trade to a development that focused on the ingrowth. keywords: ecuador, great depression, political economy, gold standard https://doi.org/10.4995/muse.2019.7766 mailto:cnaranjo@unach.edu.ec mailto:paulnaranjo@icloud.com mailto:adnaranjofl@flacso.edu.ec mailto:andynaranjob@hotmail.com mailto:anavas@unach.edu.ec mailto:cumita9@hotmail.com multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 12 1. introduction this article presents a review of the impact of the great depression in ecuador, making use of new economic statistics, as well as placing the case of ecuador in the context of the latin american region. this article has two main objectives: first, to construct a comprehensive regional context of the impact of the international crisis through the use of historical macroeconomic variables so that the study of the ecuadorian case is not isolated; second, to present new economic statistics of the ecuadorian case that had not been used in previous studies, these statistics will help to understand the real situation of the ecuadorian economy from 1927 to 1937. in addition, two hypotheses are proposed to understand the impact of the great depression in ecuador. first, the composition of the ecuadorian population became a support cushion that did not allow the international crisis to provoke signs of a decline in the ecuadorian economy while the latin american region deteriorated deeply until 1932; however, this same population composition became an extra burden that immobilized ecuador's economic acceleration when recovery was a trend throughout the region. second, the evolution of economic indices related to foreign trade is similar throughout the region, probably because the main commercial partner was the united states of north america, where the great depression originated. 2. population the first population census was conducted in 1950, thereafter; population data are officially published by the government of ecuador. before 1950, population projections are based on rear projections made by several authors. the population data of ecuador have been estimated by linda rodríguez (1992), the central bank of ecuador (2012) and moxlad data base (2015)1. 1 the central banck of ecuador (2012) and moxlad (2017) explain that their estimates are overhead projections made based on information available in the latin american demographic https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 13 because of the existence of different sources, one must understand the numerical differences in the projections made by the different authors: for example, the estimates made by central bank of ecuador (2012) for 1927 and by rodríguez (1992) for 1926. table 1 shows that the population for 1927 is 2.2 million people (central bank of ecuador 2012), while in table 2, for 1926, a population of 2.9 million people is estimated (rodríguez 1992). this difference must be understood not as a demographic problem occurred in these years, but as projections made by different authors. for ecuador, during the period studied, it was estimated that the total population was between 1.5 and 2.6 million inhabitants. as can be seen in table 1, ecuador's population grew at a rate of 2.6% per year, half a percentage point below the average in south america. the region grew at a rate of 3.05%. the country with the highest population growth is argentina, with an annual average of 4.1%, while the country with the lowest population growth is bolivia, with an annual average of 1.6%. if one takes into account the population of 1926 (table 2) for the regions of the coast and the mountains, and the approximate length of 60.039 km 2 for the coast, and 59.810 km 2 for the mountains, the population density is 18.57 for the coast, and 30.3 for the mountains. these data show that the population density in the sierra was 1.6 times that of the coast. center. moxlad, on it platform, adds that "the lack of census data also inhibited the realization of estimates before 1950 in the cases of haiti and ecuador" (2017). for rodriguez (1992), data come from: hamerly (1973); paz y miño (1926); and directorate general of vital statistics and census (1934). the latter source does not explain the methodology and sources of the published data. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 14 table 1. population of ecuador and south america, 1927-1930 año population, ecuador growth rate, ecuador population, south america growth rate, south america 1927 2.257.357 100.0 70.246.000 100.0 1928 2.297.561 101.7 71.659.000 102.0 1929 2.338.481 103.5 73.141.000 104.1 1930 2.380.129 105.4 74.591.000 106.1 source: author´s creation. data from ecuador come from: central bank of ecuador, 2012: 212; for south america: moxlad, 2015. data for south america include: argentina, brazil, bolivia, chile, colombia, uruguay, venezuela, peru, paraguay and ecuador as can be seen in table 2, on the coast, where the main coastal port is located, guayaquil, rodríguez (1992) estimated a population of 445,000 people; in the region of the andes, where the capital of the republic is located, quito, with an estimated 1,445,000 people; and, in the amazon, 100,000 people. the indigenous population was the largest class in the country, although the ruling class had spanish ancestry. most of the population lived in rural areas, with less than a quarter living in cities. the most populated city was guayaquil, with about 96 thousand inhabitants; and, quito with an approximate population of 50 thousand inhabitants. in addition, approximately one third of the population in ecuador were indigenous; a quarter of the population had some european ancestry; a little less than half of the population, were mestizos, with predominantly indigenous ties; and the remainder, a small percentage of the population were black, of whom the majority lived in coastal cities (stevens, 1940). table 2. population densities of ecuador, 1926 provinces population porcentages coast 1.115.264 38.07 esmeraldas 54.593 1.86 manabí 347.847 11.87 los ríos 161.800 5.52 guayas 483.508 16.51 el oro 67.516 2.30 mountain range 1.814.050 61.93 https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 15 carchi 78.125 2.67 imbabura 161.223 5.50 pichincha 304.794 10.40 cotopaxi 193.017 6.59 tungurahua 227.181 7.76 chimborazo 288.713 9.86 bolívar 88.657 3.03 cañar 94.743 3.23 azuay 201.911 6.89 loja 175.686 6.00 east galapagos islands total 2´929.314 100 source: rodríguez 1992, appendix h. despite the fact that there were no population censuses in the 1920s and 1930s, the report of the general directorate of statistics of the civil registry and census (1934), which does not explain the type of methodology was used, nor the origin of the data, estimated that the highest birth rate corresponded to the coastal province of manabí, with a record of 56.37 per thousand inhabitants, while the highest mortality was in the province of imbabura, with 31.94 per thousand inhabitants. the diseases that caused the most deaths were bronchitis, whooping cough, diarrhea and enteritis, measles and malaria. the highest mortality was between the ages of 0 to 4 years, the group most vulnerable to the consequences of the diseases mentioned were infants. infant mortality accounted for 16.2 percent of total births. tabla 3. birth, mortality, and marriage rates, ecuador, 1933 provinces registered births natality per thousand inhabitants deaths in the year mortality per thousand inhabitants registered married couples nuptiality per thousand inhabitants carchi 2.667 41,13 1.428 22,02 445 6,86 imbabura 5.306 40,85 4.148 31,94 1.044 8,04 pichincha 10.720 40,93 8.003 30,56 1.720 6,57 león 6.129 34,66 4.141 23,42 1.129 6,38 https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 16 tungurahua 6.596 35,70 5.048 27,32 1.142 6,18 chimborazo 9.041 40,37 5.019 22,41 1.566 6,99 bolívar 3.538 42,88 1.437 17,41 643 7,79 cañar 3.476 32,28 1.593 14,99 615 5,71 azuay 7.392 33,19 3.759 16,88 1.384 6,21 loja 6.974 44,46 2.453 15,64 1.147 7,31 el oro 2.799 43,08 1.107 17,04 326 5,02 guayas 17.735 50,46 10.751 30,59 1.472 4,19 los ríos 5.430 51,94 3.112 28,77 173 1,65 manabí 13.251 56,37 5.466 23,25 1.322 5,62 esmeraldas 2.145 48,88 629 14,33 121 2,76 total 103.199 42,77 58.094 24,08 14.249 5,90 source: peñaherrera, 1934: 7. economically active population it is estimated that approximately 32% were part of the economically active population, that is, about 832,000 people. the first data on the economically active population for 1962 came from the population and housing censuses conducted in 1962. research by the national planning and economic coordination board (1979) reveals that 32% of the total population can be considered as part of the economically active population. as there is no official data for the 1930s, this figure has been taken to estimate the economic active population for the 1930s. for this reason, this data should be considered as a conjecture based on the first national censuses. by 1930, approximately 14% of the population lived in urban areas2 (bethell 1998, 31). if we take this percentage and extrapolate it to the economically active population, we obtain that around 116,480 people worked in the urban sector, and around 715,520 in the rural sector. de la torre (1993) estimated that, by 1936, 55% of the population of quito was marginally employed: day laborers (10.4%), self-employed workers (23.5%) and domestic workers (21, 1%). the second most important category of labor was public sector workers: 16.6% in 1936 and 21.1% in the early 1940s. 2 leslie bethell (1998) explains that this percentage applies to cities with over 20,000 inhabitants. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 17 table 4. occupational structure in quito, 1936 occupation number percentage public employees 5.893 16,6 private employees 3.025 8,5 industrial workers 1.651 4,6 workshop workers 3.555 10,0 day laborers 3.678 10,4 freelance workers 8.133 23,5 workshop owners 1.085 3,1 servants 7.464 21,1 unemployed 795 2,2 total 34.276 100,0 source: de la torre 1993, 67 3. economic policies during the great depression the global crisis particularly affected the countries of latin america given the close trade relationship it had with the united states of america. in the case of ecuador, the economic crisis affected the foreign trade sector in particular because the united states supplied 45% of total imports and 47% of total exports (feiker 1931, 1). the global crisis created commercial, financial, monetary and exchange shocks in ecuador, which forced the government to make controversial decisions, decisions that were considered as possible solutions to the onslaughts of the great depression. these anti-crisis policies were implemented in a decade of strong political instability, as reflected by the fact that in nine years from 1929 to 1938 twelve different people were in charge of the governments3. this 3 from 1929 to 1938, among presidents and managers of power, there are twelve people in charge of the executive branch (salvador lara, 2002). most overthrows were led by the ecuadorian army. people in charge of the executive branch were: isidro ayora cueva (april 17, 1929august 24, 1931); crnl. luis larrea alba (24 august 1931 october 15, 1931); alfredo baquerizo (october 15, 1931 august 20, 1932); neptalí bonifaz (august 20, 1932august 28, 1932); carlos larrea freire (august 28, 1932september 2, 1932); alberto guerrero martínez (02 september 1932december 5, 1932); juan de dios martínez (05 december, 1932october 17, 1933); abelardo montalvo (20 october 1933september 1, 1933); josé maría velasco ibarra (01 september 193420 august https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 18 instability affected government decisions, as there was no way to establish long-term public policies. the decade of the 1930s was a turbulent decade, which was reflected in the extraordinary number of presidents, civilian and military dictators, "as the economy deteriorated, more and more ecuadorians turned to the government for immediate solutions to economic problems and social aspects of the nation" (rodríguez 1992, 198). anti-crisis policies had a time of slow decision-making because just a few months before the world crisis began, the american economist edwin kemmerer, along with most of his collaborators, had left the country after advising the government of ecuador for the creation of the central bank, the only legal institute that had the power to issue currency, and for the establishment of the gold standard as a monetary system. on february 11th 1927, along with other economic norms, the kemmerer mission presents to the government of ecuador the draft of the organic law of the central bank of ecuador; on march 4th, president isidro ayora signs the law; and on august 10th the central bank began its operations. edwin kemmerer was considered an international authority on economic issues. from 1919 to 1930, the kemmerer mission advised the governments of guatemala (1919), colombia (1923), chile (1925), ecuador (1926), bolivia (1928), and peru (1930). the relationship between the government of ecuador tightened sharply, so that "the absolute dependence that has since been created with the expert for the application of measures" (almeida, 1994: 54) remained until 1932. after the years in which the kemmerer mission offered their advice, the ecuadorian government began to make relatively independent decisions. the ecuadorian government set aside the “kemmerian" policies and began to make decisions based on its own expertise, including the abolition of the gold standard, the increase in the money supply, seizure and disinvestment of currency. policies during the great depression were partly given by the kemmerer's advice, partly by the expertise of national politicians, partly by the public 1935);antonio pons campusano (august 21, 1935september 25, 1935); federico páez (september 25, 1935october 23, 1937); gral. alberto enriquez gallo (23 october 1937 december 2, 1938) https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 19 pressure that was tired of the government prostrating itself before the "monetary doctor" (drake, 1995), and partly by the political instability that led the country to have 12 people in charge of the executive in less than a decade. thus, anti-crisis policies went through two distinct stages from 19274 to 1938 that the first has been named as the stage of "influence of kemmerer mission, 1927-1932", and the second stage is known as the stage of "inconvertibility and instability, 1932-1938". 4. influence of the kemmerer mission, 1927-1932 5 between 1927, the year in which the kemmerer mission ends its work in ecuador, and february 1932, when the formal step for the abandonment of the gold standard was taken, it was the stage of the influence of the policies established by kemmerer, which involved restructuring of the national financial system with its main axis located in the founding of the central bank of ecuador. when the great depression was felt in ecuador the anti-crisis economic policies did not differ from the policies structured in 1927: anti-crisis policies were the same before and during the international crisis. the ecuadorian government had the solid concept that the tools left by the new financial system were sufficient to control the onslaught of the international crisis. the economic policies influenced by the kemmerer mission had a central idea, maintaining the fiscal balance. when the international crisis began, this idea did not change, maintaining fiscal balance remained the main concern. this balance was synonymous with maintaining the gold system as a system that could be self-regulated through the increase or decrease of money circulation that could be controlled through the printing of money, through interest rates and through purchase and sale of currency abroad. these tools were the axes of a policy that was centralized in the money supply. this was a 4 although the international crisis began in 1929, we have decided to choose the year 1927 so that it can provide a clearer context for the policies that were implemented in this year and were maintained during the first two years of the great depression. 5 1932 is chosen as the end of the influence of the economic policies of edwin kemmerer and as the beginning of a decade of instability and inconvertibility. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 20 policy that sought to control the imbalances caused in the trade balance by controlling the money supply. economic policies during kemmerer's influence period the advice and policies of the kemmerer mission had similar patterns in each country they visited. the common elements in all the countries visited by the kemmerer mission were: the creation of central banks, superintendence of banks, and general comptrollers; and, proposals of laws by which the gold standard was stablished the main as monetary policy. these structural changes occurred from 1919 to 1930, gozzi & tappatá (2010) present these changes chronologically: guatemala (1919) institutions founded: • central bank of guatemala. established laws: • monetary law of the republic of guatemala. colombia (1923) institutions founded: • republic of colombia bank. • comptroller general of the republic of colombia. established laws: • bank law of the republic. • banking establishments law. • timbre law (reorganized the operation of customs and established the collection of national revenues). • income tax law. • comptroller law (transform the former court of auditors on the comptroller general of the republic). https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 21 • act restraining force of the budget (limited the powers of parliament to order spending public and leave the initiative in the hands of the executive branch). • law on number of ministries. • negotiable titles deeds law. chile (1925) institutions founded: • chile central bank. • superintendency of banks. • general comptroller of the republic of chile. established laws: • chile central bank law. • general law of banks. • monetary law. • orgnic budget law. • comptroller general of the dominican republic law. ecuador (1926) institutions founded: • ecuador central bank. • superintendency of banks of ecuador. • comptroller general of the republic of ecuador. established laws: • organic law of the central bank of ecuador. • organic law of banks. • organic law of the mortgage bank (banco nacional de fomento). • internal taxes law. • coins law. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 22 • contracts act on agricultural garments. • customs law. bolivia (1928) institutions founded: • bolivia central bank. • superintendency of banks. • comptroller general of the republic of bolivia. established laws: • reorganization law of the national bolivian bank. • general law of banks. • monetary law peru (1930) institutions founded: • central reserve bank of peru. • superintendent of banking peru. established laws: • law of the central reserve bank of peru. • banking law of peru. • monetary law the first central banks of latin america were designed to fulfill three objectives: to maintain monetary stability; to preserve the stability of banks; and government funding on a limited basis (jácome 2015). these objectives would be achieved through a currency issue associated with the gold standard which would lead to a stable exchange rate and a low inflation. in the case of ecuador, the kemmerer mission carried out the economic policy stage from 1927 to 1932. the economist edwin kemmerer, along with his working group, had an undeniable participation in the founding of a new financial system in ecuador. his advice https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 23 did not fail to influence the decisions on monetary policies that ecuadorian politicians took even several years after his official visit. before the beginning of the international crisis, the gold standard was already established in ecuadorian territory. the money supply was related to gold reserves. the existing circulation had to be backed by 50% of gold reserves. the money supply was in the hands of private banks until the founding of the central bank, which opened its doors on august 10th 1927. the foundation of the central bank was the central axis of the kemmerer mission. immediately, the central bank organized policies around the money supply. with this purpose, printing money was the tool the government possessed for the regulation of the circulation of money. on the other hand, the central bank balanced the money market also by regulating the discount and interest rate, raising it when there was an excess of cash and lowering it when the money was low (morillo 1996). the use of this tool had a logical justification, by raising interest rates, the monetary circulation would be reduced, and as interest rates lowered, the monetary circulation would increase. in addition to these instruments in hand, the central bank had the law of coins6 in their favor, which provided the government with power, on the one hand to control the exchange rate, and on the other to buy and sell money abroad. the central bank of ecuador could, by using the coins act, make automatic adjustments: if there was an excess of circulation, the government could sell unlimited currency in new york and london (carbo 1927). these instruments began to be questioned when the crisis began because there was the general impression that it was a straitjacket that did not allow to create other types of policies. for this reason, the ecuadorian government contacted edwin kemmerer to ask his opinion on the subject. in 1930, kemmerer responded to the ecuadorian government's request for the validity of established policies, in which kemmerer asserted that the 6 the law of coins was published in the official gazette n. 289, on march 1, 1927. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 24 automatic readjustment of the money supply, managed by the central bank, ensured the proper circulation of the currency, "and should not be altered despite protests. he denied that ecuador was going through a deflationary process or that it had a monetary deficit. he urged his hosts to maintain the gold standard and high discount rates" (drake 1995, 269). kemmerer had denied the fact that there was a pronounced deflation that was caused by a reduction in the money supply. according to the same source (drake 1995), kemmerer encouraged ecuadorian politicians to continue payments on the foreign debt. these policies were maintained in spite of a diversity of proposals that began to appear. among the most important proposals, morillo (1996) refers to three of them: first, the central bank was encouraged to buy mortgage bonds; second, that the resources destined to the service of the external debt should be invested in diverse projects in ecuador; and, third, a policy of rediscounts. none of these proposals were accepted, the government was going to be facing the crisis with the financial instruments that came along with the gold standard. with kemmerer's response, the ecuadorian government felt the necessary justification for maintaining the gold standard despite the fact that reserves were declining and despite the fact that the deflationary process was undeniable. the financial policies of the ecuadorian government were maintained until 1932. it was hoped that the monetary policy instruments, to say the printing of money, the regulation of the type of discount and interest and the purchase and sale of currency, were sufficient to control the impact of the great depression. taking into account that the international crisis was felt in ecuador from the unevenness in the trade balance, much of the effort to build an anti-crisis policy focused around the attempt to control the trade balance. of all the instruments that the ecuadorian government had at hand, the most used was the interest rate: when the trade balance was negative, the reserves decreased, and for that reason the central bank, and by repetition the rest of private banks, put in the market a greater amount of short-term loans, but with high and progressive interests. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 25 with respect to interest rates, in 1927, when the central bank was founded, it opened a line of credit for the associated banks with an interest of 10%, while for the general public the line of interest was 11%. banks charged the public 12% and sometimes charged higher rates. in 1928, the interest rate fell to 8% for the associated banks and 9% to the public. the reduction in discount rates was a result of the reduction in exports of the main agricultural product, cocoa. by the early 1930s, the central bank had set an interest rate of 10% for the general public and 9% for the associated banks, which charged their clients a 12% interest. in 1930, the central bank decided to establish differentiated interest rates as follows: 7% for associated banks, up to 30 days; 8% for associated banks, up to 90 days, for industrial purposes; and 9% for the public, up to 180 days, with agrarian pledges (morillo 1996). these rates were maintained until the first months of 1932. in september of 1931, great britain decided to leave aside the gold standard, which caused serious repercussions in ecuador, since an accelerated gold output began, the central bank lost about one million sucres during the same year. following britain's decision to abandon the pattern, gold, and after seeing an accelerated fall in reserves from the central bank of ecuador, "most banks in guayaquil had opted to make their loans dollar-based" (morillo 1996, 74). this problem prompted the ecuadorian government to modify the required percentage of 50% in reserves, to maintain the gold standard, to 25% of reserves in deposits and 50% of reserves for currency convertibility. in this way, it can be clearly seen that the stage from 1927 to 1932 was characterized by a central idea: to maintain fiscal balance. this fiscal balance was synonymous with maintaining the gold system as a system that could be self-regulated through printing of money, interest rates, and the purchase and sale of currency. consequences of policies influenced by kemmerer, 1927-1932 the consequences of the kemmerer mission's influence are established around the goal of maintaining fiscal stability that overcomes the instability of the international market. from the visit of the kemmerer mission in october 1926, the gold standard was maintained as a https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 26 system of immovable conversion. the gold standard was designed to maintain the stability of the money supply and also maintain the value of the ecuadorian currency, the value of the sucre. the consequences of this period are divided into five contexts: monetary circulation; variation of the gold reserves in the central bank of ecuador; exchange rates; inflationary fluctuations; and, the evolution of the gross domestic product. with regard to monetary circulation the monetary circulation, as shown in table 5, during the period of influence of the kemmerer mission, declined since 1927, year in which a circulation of 79 million sucres was registered. in 1929, the year of the beginning of the international crisis, the monetary circulation was 67 million sucres. these data clearly show that the money supply declined steadily before the international crisis began, opening the door for the founding of the central bank and its monetary policy: printing of new currency; and the regulation on discount and interest rates; exchange rates; and overseas transactions. from 1927 to 1929, the reduction of the monetary circulation happened at the rate of -8.2% annual average; and, from 1929 to 1931, the reduction was at the rate of -17.9% per year. in 1932, the monetary circulation increased by 15 million sucres, registering 58 million sucres. the policies of financial rigidity caused a rapid fall of monetary circulation. in two years, from 1929 to 1931, the money supply was reduced by one third, which, in turn, produced a sharp devaluation. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 27 table 5. monetary circulation in sucres, 1927-1932 año total 1927 79.290.000 1928 74.333.000 1929 67.812.000 1930 54.212.000 1931 43.870.000 1932 58.029.000 source: central bank of ecuador 1940, 150-151. the figures of the total money supply recorded in december each year. figure 1. monetary circulation, millions of sucres, 1927-1932 source: table 5. with regard to gold reserves gold reserves shows a similar development to money supply developments: the gold standard needed reserved valued at 50% of total money circulation, then it is logical to see https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 28 that the evolution of money supply and gold reserves are directly related. from 1927 to 1929 the gold reserves decreased at a rate of 10.2% on average, while from 1929 to 1931, reserves reduced at a rate of 28.5% annually. reserves moves from 44 million sucres in 1927 to 15 million in 1931. the steady decline in gold stocks gave way to abandon the gold standard in february 19327. table 6 shows the gold reserves in relation to monetary circulation. the percentage varies from 1927, when it was located at 51%, thereafter the percentage of gold with respect to the monetary circulation gradually lowered. this steady decline happened until 1932, year when the ratio was of 22%. anti-crisis policies that remained until 1932 were not effective to keep gold reserves: reserves decreased inevitably in half in less than three years. table 6 shows clearly the variation of the gold reserves, and also shows the percentage of total circulation. the logical consequence of the decline in gold reserves was the abandonment of the gold standard in 1932. table 6. gold reserves in the central bank of ecuador, in sucres, 1927-1932 year total value gold reserves (sucres) golden percentage relative to the total 1927 84.912.879 44.085.490 51,9 1928 79.195.465 38.029.255 48,0 1929 73.146.997 35.063.646 47,9 1930 60.816.151 28.639.062 47,1 1931 47.860.775 15.492.532 32,4 1932 75.984.779 17.408.986 22,9 source: author´s creation. data are from: central bank of ecuador 1940, 150-151; drake 1984, 270. the total involves deposits in the central bank and sucres in circulation. 7 the abandonment of the gold standard became official through decree no. 32 of february 8, 1932. in november of the same year, congress noted that the suspension of the gold standard was temporarily until a stable economic balance. since november, the gold reserves of the central bank could not be less than 25% of monetary circulation (morillo, 1996). https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 29 figure 2. gold reserves in millions of sucres, 1927-1932 source: table 6. with regard to exchange rates the value of the gram of gold was directly related to the price of the sucre: the price of gold gram and the price of the sucre, did not fluctuate freely, but was established by the central bank of ecuador. when the international crisis erupts, and gold reserves decreased, the value of gram of gold and the price of the sucre could be maintained only until 1932. the value per gram of gold remained at 3.32 sucres, from 1927 until 1931, while in the same period, the dollar was valued at 5 sucres. the value of the gram of gold and exchange rate begins to change in 1932 as a result of the abandonment of the gold standard. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 30 while nations8 that left the gold standard early opened a space for the devaluation of the currency, ecuador kept the price in unalterable values. gold systems guaranteed a fixed exchange rate. from 1927 to 1932, the share price was maintained at 5 sucres per us dollar. this rigidity produced that export sectors started seeking for currency devaluation to restore competitiveness, while the state "was initially determined to not devalue the currency for fear of a social backlash" (cueva 1991, 66). this devaluation did not occur until 1932. table 7. exchange rate, 1927-1932 year gram of gold valued in sucres sucres to the dollars gold reserves in dollars 1927 3,32 5,00 8.817.098,00 1928 3,32 5,00 7.605.851,00 1929 3,32 5,04 6.957.072,62 1930 3,32 5,05 5.671.101,39 1931 3,32 5,06 3.061.765,22 1932 3,42 5,93 2.935.748,06 source: author´s creation. data come from: rodriguez, 1992: appendix k; drake, 1984: 270; central bank of ecuador, 1940: 150-151. as displayed in table 7, while the value of gram of gold and the exchange rate did not change until 1931, gold reserves decreased steadily. this scenario could not last in time. policies based on fiscal stability collapsed in 1932, it became impossible to maintain the exchange rate as gold reserves had fallen dramatically in a few years; it was clear that the decline in gold reserves would cause a sharp devaluation. 8 argentina, uruguay and brazil in 1929; venezuela in 1930; mexico, chile, bolivia in 1931; colombia, nicaragua, costa rica, peru and ecuador in 1932 (ossa, 1993). https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 31 with regard to inflation or deflation the global crisis directly affected the fluctuation of prices because gold reserves decreased and therefore the money supply also decreased, resulting, ultimately, in deflating prices. in 1929, inflation stood at 4.24 percent over the previous year. thereafter, there is a marked deflation. although the international crisis marks pronounced deflation until 1932, ecuador presents vertiginous fluctuations before the great depression: in 1928, -11.4%; in 1929, an inflation of 4.24%. from 1929, there was a prolonged deflation process until 1932, reaching the lowest peak in 1931, with deflation -24.55 percent. in 1932, a -10.96% deflation is recorded. this percentage could have been much higher, however, because of the abandonment of the gold standard in february of the same year, the money supply grew and deflation was not as high as in 1931. if the gold standard remained in 1932, deflation could have been at similar levels to 1931. in the case of latin america, taking into account the price indices of argentina, chile, peru and venezuela, deflationary trends appeared from 1928 to 1931, with an annual average of -8.21. the year of highest deflation was recorded in 1928, with a record of -18.92. the trend presented latin america is similar to the trend in ecuador; however, in the case of ecuador, deflation presents a more dramatic evolution. this phenomenon can be understood by considering that the gold standard was in effect in ecuador until 1932, while in most of latin america, the gold standard was abolished in 1931. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 32 tabla 8. inflation rate in ecuador and latin america, 1928-1932 year ecuador latin america 1928 -11,49 -18,92 1929 4,24 -3,06 1930 -2,62 -6,53 1931 -24,55 -4,33 1932 -10,96 3,00 source: author´s creation. data come from: morillo 1996, 640; mitchell 1999, 693. the inflation rate for latin america has been calculated based on the price indices from mitchell (1999, 693) for argentina, chile, peru and venezuela. figure 3. inflation in ecuador and latin america, 1928-1932 source: table 8 as displayed in figure 3, the latin american region recorded deflationary rated until 1932. the restriction of the money supply had a logical consequence, a deflationary -11,49 4,24 -2,62 -24,55 -10,96 -18,92 -3,06 -6,53 -4,33 3 -30 -25 -20 -15 -10 -5 0 5 10 1928 1929 1930 1931 1932 ecuador latin america https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 33 phenomenon. however, the case of ecuador is unique because the trend is more pronounced than in latin america. in the case of ecuador, this deflation caused a steady growth. with regard to gross domestic product in table 9 can be noted that the nominal gdp 1927-1932 presents high and low peaks: peak low in 1931, with a decrease of -26.34%, and the highest peak is recorded in 1929, an increase of 13, 52%. on the other hand, the change in real gdp, systematically reduced until 1932. each year recorded an average decline of 2%. the change in gdp at constant prices presents a clearer picture of the evolution of the ecuadorian economy as inflation picture does not influence measurement. data from 1927 to 1932 show that there was a progressive increase in gdp until 1929. thereafter, gdp grows at a rate of 0.65 points in 1931 and 1932. this variation shows that the anti-crisis policies produced a deflationary phenomenon that caused no negative growth of the real gdp until 1932. to strengthen our data, it is important to mention the research made by twomey (1983), who analyzed the evolution of the domestic production of six countries in latin america, namely argentina, brazil, chile, colombia, honduras and mexico, concluding that after 1929 "the decline in production was greater in chile, except in colombia, while honduras did not experience a typical cycle during the early part of the decade" (1983, 221). in honduras, the gross domestic product increased from 1929 to 1931 by about 4%, while colombia registered a stagnation in 1929 and 1930, and from 1931 its gdp grows steadily (1983, 222). the evolution of ecuador's gdp is also an outlier because no recorded percentages of decline was registered during the decade of the thirties, but rather, it has an average minimum growth. as it is visible in figure 4, the evolution of real gdp of ecuador and latin america has similar trends from 1930. however, latin america has a much more pronounced trend than that of ecuador: in 1931, latin america presents a variation of -14.4%, while ecuador https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 34 recorded a variation of 0.61%. these data confirm the hypothesis that the impact of the great depression in ecuador was minimum compared to the impact on the rest of the region. table 9. evolution of gross domestic product, ecuador and latin america, 1927-1932 years gdp current prices (thousands of sucres) gdp current prices (%) gdp constant prices april 78may 79= 100 (thousands of sucres) gdp constant prices (%) variation real gdp latin america 1927 440 8.406 1928 402 -8,71 8.810 4,81 7,02 1929 456 13,52 9.367 6,33 2,64 1930 457 0,18 9.726 3,82 5,42 1931 336 -26,34 9.785 0,61 -14,40 1932 305 -9,4 9.848 0,65 -5,48 source: author´s creation. data for ecuador come from: morillo, 1996: 687. data for 1970 are based in latin america and sourced base moxlad, 2015; it includes: argentina, brazil, chile, colombia, uruguay, venezuela and peru. figure 4. variation real gdp, ecuador and latin america 1928-1932 source: table 9. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 35 policies from the kemmerer mission were similar in the countries where he had influence, however, this influence was limited by the governments of each country, which eventually decided to leave the gold standard, some early, some late: argentina, uruguay and brazil in 1929; venezuela in 1930; mexico, chile, bolivia in 1931; colombia, nicaragua, costa rica, peru and ecuador in 1932 (ossa 1993). the decision to leave the gold standard opened up the possibility for governments in latin america to increase the money supply as an anti-cyclical policy, while countries that did not leave the gold standard quick, deflationary processes appears, which minimized the decline of the gdp. therefore, it cannot be said definitively which policy was the the most appropriate for the region. in the case of ecuador, keeping the policy around the gold standard until 1932 produced that real gdp did not display negative growth. 5. inconvertibility and instability, 1932-1938 the second period of the anti-crisis policy is known as the stage of "inconvertibility and instability," which stretches from 1932 to 1938, years in which the expertise of ecuadorian politicians determined monetary policies. the two periods of the anti-crisis policies, namely, the period of the kemmerian influence and the period of inconvertibility and instability, make clear differences in decision-making: the first stablished an extremely rigid system, where economic tools focused on monetary policy and instruments that were adaptable enough to cope with the crisis; the second period is characterized by an extremely unstable system, which moves from the abolition of the gold standard, the management of interest rates and increased credit for an aggressive public debt, the seizure of foreign exchange, export and import control: these policies intended to stable the trade balance, while trying to control inflation. the main measures taken during the period of inconvertibility and instability to face the international crisis were four: 1. the abandonment of the gold standard, that is to say, the bank reserve that was needed for the printing of currency was halved. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 36 2. the ecuadorian government forced, through a series of decrees and laws, the central bank of ecuador to make large loans that allowed to solve the fiscal gaps and allowed to increase the monetary circulation. 3. government control of exports and imports from the second half of the decade 4. control of the foreign exchange market from 1933. abolition of the gold standard the money supply was linked to gold reserves; the existing circulation had to be backed by 50% of gold reserves. this policy continued until 1932, when the gold standard was disable because the social pressure against the kemmerian policies measures was growing rapidly. the expansion of money supply and credit was required, "even the superintendent of banks thought that kemmerer did not understand the devastating contraction of the money supply and prices, and that the central bank should ignore its recommendation that further limited the currency and the credit" (drake 1995, 269). to understand why this system worked for three years after the international crisis erupted, it is to be considered the fact that the kemmerer mission had a consolidated international fame as it had structured the financial system of some countries in latin america as guatemala, colombia and chile. in all these countries, including ecuador, the gold standard had the approval of edwin kemmerer. when the international crisis began, many voices rose against the financial advices structured by edwin kemmerer during his visit to ecuador in 1926. an emergency law that stablished a reserve of 50% was approved in december 1931. until then, the gold reserves were to represent 50% of total deposits and currency in circulation. finally, in february 1932, the ecuadorian government changed the required percentage of 50% to 25%. this decision was influenced by the decision of britain to leave the monetary system based on gold reserves in 1931. britain's decision influenced latin america, which leave systematically their monetary systems based on gold reserves. in ecuador, the decision to leave the gold standard also came from public pressure seeking ways to increase https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 37 the money supply, with the purpose to increase the money supply that would control deflation. in 1937, the government raised the gold lace 30% of the monetary circulation. in addition, it was decided to calculate limits of the necessary money supply through a system based on population estimates computing "to compute this limit is taken into account enforceable demand obligations in national currency to the central bank and is considered the census statistics whose population at the time was estimated at two and a half million people. it was concluded that the figure of 40 sucres per entailed a volume of 100 million sucres of this kind of current, enough to meet the most reasonable payment needs" (morillo 1996, 112). these limits allowed inject the ecuadorian economy about 150 million sucres. public debt decree no. 33 of february 1932 ordered as a measure to stop deflation, monetary inconvertibility. in the same decree, the government ruled one of the first loans that would come in a few years: the government ordered the central bank to grant a loan of 12 million sucres, which would serve in part to create the agrarian fund, an institution in charge of granting loans for agriculture (stancey 1990, 55). the loan was delivered in twelve installments at the rate of one million per month. this loan was intended to increase the circulation of money focusing on some areas: five million were allocated for roads and infrastructure, five million for agricultural investment, one million for international debt, and one million for commercial and industrial investments, which was meant to seek help for the export industry (morales, león camacho & oleas montalvo 1997). from february 1932, anti-crisis policies focused on government intervention in the resources of the central bank, which had the intention of injecting more funds to the ecuadorian economy, but it also had the intention to cover recurring tax holes. ecuadorian government debt at the end of 1932 was of 40 million sucres. since then, government debt grew significantly due to the use of resources from the central bank. thus, this period was https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 38 inaugurated with the repeated government intervention in the central bank requiring high loans. eight loans from the central bank was given to the ecuadorian government from 1932 to 1937, with a total of 44.8 million. these loans were made according to the following schedule: february 8th 1932, 12 million sucres; on december 29th 1932, through an emergency law, 13 million went to the mortgage bank of ecuador, bank administered by the government9; on december 30th 1933, there was a new loan of 6.4 million sucres, at two per cent of interest; in february 1934, 2.5 million sucres were lent for fiscal debts; on may 13th 1937, 1.7 million was loaned; on june 24th 1937, 3 million sucres; and on august 21st 1937, 6.2 million. on august 19th 1937, the "government debt balance is consolidated at 268,000,000 with a 3 percent annual interest, and with a fixed payment, from 1938, of 750,000" (stancey 1990, 62). recurring debts to the central bank of ecuador made it possible to have a bigger money supply, however, on the other hand, these debts were just trying to momentarily cover fiscal gaps. seizure of foreign currency, import and export control ecuadorian government policies during this period were not focused solely on the loans for the central bank loans to cover its fiscal gap and to inject resources into the economy, but it was also related to the seizure of foreign currency. on the 30th april 1932 the government, seeing that the price of foreign currency climbed rapidly, it published the decree no. 90, by which the centralization of 80% of foreign currencies available in private banks and exporters was ordered. this 80% were immediately transferred to the central bank, which was the only institution responsible for buying and selling currencies. the seizure took turns with a value of 5.95 sucres for purchase and 6.00 sucres for sale (carbo 1978). that 9 by 1936 the mortgage bank of ecuador had canceled half the debt, the other half to the central bank was forced to accept shares in the bank as payment. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 39 is, the central bank took over the foreign exchange market, in addition to generate an effective devaluation of 20% since 1927. on november 22th 1933, the ecuadorian congress approved the partial disincentive law, with which the economy turned to the open currency market, but still the seizure of 25% of the total value of exports of certain products were ordered. this policy was trying to boost exports, especially agricultural exports (central bank of ecuador 1935). in august 1936, the centralization of all the exchange market was decreed. the government established that the central bank would be in charge of the exchange market, plus the control of all exports and imports. through decree 596, signed by federico páez, head of the executive function of the ecuadorian government, mandated that "from this date, only the central bank of ecuador can buy and sell freely foreign currencies" (wither 1936, art. 1). the 596 decree was intended to improve the trade balance by limiting imports. in june 1937, the central bank decided to "implement a prior deposit before granting official permits for the importation of goods (50% of the total value)" (morillo 1996, 107). on july 31st, it was decided to annul the seizure on the market of currencies, but control over international trade was maintained through the establishment of requirements and permits for imports and exports. consequences of policies at the stage of inconvertibility, 1932-1938 in march 1932, the ecuadorian government received a large loan from the central bank. the loan was 12 million sucres, delivered in twelve parts, one million per month (morales, león camacho & oleas montalvo 1997). this loan exemplifies the economic objective of this period: increase the money supply. these policies produced the increase in money supply and gold reserves; which in turned caused the sucre devaluation; the detonation of inflation; and an insignificant growth of gross domestic product. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 40 regarding the monetary circulation at the time of inconvertibility and instability, the money supply had a direct connection with the decision of the abolition of the gold standard. from 1932, the money supply rose steadily. in 1932, 58 million sucres, thereafter, it increased annually by 12 million sucres. the largest increases in circulating was recorded in 1934 and 1936, an increment of 24 and 26 million sucres respectively. in table 10, the figures reveal that money supply rose continuously from 1932. from 1932 to 1938 the circulation increased 100 million sucres, which means an increase of 150 percent of the value recorded in 1932. these figures reveal that the decision to leave the gold standard, plus the aggressive public debt, made possible the increase of the money supply. table 10. monetary circulation, ecuador, 1932-1938 año total 1932 58.029.000 1933 85.324.000 1934 109.886.000 1935 106.179.000 1936 132.827.000 1937 149.827.000 1938 154.229.000 source: central bank of ecuador, 1940: 150-151. the figures of the total money supply recorded in december each year. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 41 figure 5. monetary circulation, ecuador, in millions of sucres, 1932-1938 source: table 10 the increase in the money supply directly affected the annual inflation. after leaving the previous monetary system, money supply increased prices at high averages when it is compared with the rest of the region. the increase in the money supply was a direct result of aggressive public debt. with regard to gold reserves since 1932, gold reserves are constantly increasing. this year, the gold reserves amounted 17 million sucres. from 1932 to 1936, gold reserves increased by an annual average of 6 million sucres. in 1937 and 1938 reserves decline 3 million sucres, recording, for two years, a total of 39 million sucres. from 1932 to 1938, reserves increased by 22 million sucres. in 1927, gold reserves accounted 44 million sucres (table 6), and in 1937 accounted 39 million sucres (table 11). these figures show that, ten years later, the reserves decreased https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 42 by 5 million sucres: the reserves of 1937 and 1938 amounted less than the reserves in the decade of the twenties. the relationship between gold reserves and total capital (money supply more deposits in the central bank) fluctuated constantly. the lowest peak is in 1933, recording a ratio of 16%. the highest peak is 1935, with a ratio of 28%. table 11. gold reserves in the central bank, 1932-1938 year monetary circulation central bank deposits total value gold reserves (sucres) golden percentage of total 1932 58.029.000 19.984.779 78.013.779 17.408.986 22,32 1933 85.324.000 26.165.454 111.489.454 18.340.216 16,45 1934 109.886.000 27.647.751 137.533.751 27.156.303 19,75 1935 106.179.000 30.267.780 136.446.780 38.830.834 28,46 1936 132.827.000 34.363.210 167.190.210 42.526.686 25,44 1937 149.827.000 39.563.215 189.390.215 39.890.498 21,06 1938 154.229.000 34.465.799 188.694.799 39.798.376 21,09 source: author´s creation. data are from: central bank of ecuador, 1940: 150-151. deposits at the central bank consist of three sources: according to the central baco (central bank of ecuador, feb., 1940), total deposits come from the sum of: first, government entities and special accounts; second, deposits of private banks; third, other transactions. the total value is the sum of the sucres in circulation plus deposits at the central bank. the percentage of gold comes from the amount in circulation plus deposits. although gold reserves steadily increased, the percentage of gold, relative to the monetary circulation, with variations, does not exceed 28%. this upper limit implies that it would had been impossible to increase the money supply with the previous monetary system: 50% of the monetary offer had to be supported in gold reserves. regarding the exchange rate of the sucre with the dollar after the decision of the ecuadorian government to abandon the gold standard, the price of the sucre to the dollar ranged significantly in the first two years. in 1932 and 1933, the sucre was traded at about six per one us dollar. this represented an increase of 20% over the price recorded in 1931. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 43 in 1934 the biggest devaluation of the thirties happened. this year, the price of the sucre was of 10.80 per dollar. since then, the price of the sucre remained relatively stable. in 1938 the second most pronounced devaluation of the decade was recorded. this year, the exchange rate was of 14.13 sucres per us dollar, a devaluation of 27% over the previous year. during the decade of the thirties, it can be seen clearly that there were two major picks in the devaluation of the sucre. the first major devaluation was recorded in 1934 when it passes from 6 to 10.80 sucres, and the second was in 1938 when the sucre was devalued from 11.39 to 14.13. to understand the devaluations of the sucre, we need to take into account that the government decided to control the currency market in 1932. this measure was repealed the following year, letting the market react to partial freedom of demand and supply of foreign exchange, yet the government still controlled 25% of the transactions. given that the foreign exchange market was partially self-regulated in 1933, the depreciation of the sucre came the following year. thus, the first major devaluation of the decade was over 40%. reacting to the devaluation in 1934, as a measure to prevent another one, the government decided, for the second time, to control de transactions of currency and the total control of exports and imports. the centralization of foreign exchange market allowed the central bank to manage partially the devaluations of the following years (stancey 1990). table 12. exchange rate, 1932-1938 year gram value of gold in sucres sucre to the dollar gold reserves in dollars 1932 3,42 5,93 2.935.748,06 1933 3,44 6,00 3.056.702,67 1934 4,65 10,80 2.514.472,50 1935 10,05 10,55 3.680.647.77 1936 11,67 10,48 4.057.889.88 1937 11,78 11,39 3.502.238.63 1938 12,37 14,13 2.816.587.12 source: author´s creation. data come from: rodriguez 1992, 206; central bank of ecuador 1940, 150-151. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 44 with regard to inflation or deflation with regard to inflation or deflation in 1932 was the last time in the decade that deflation occurred. in this year, the deflation recorded was of -10.96, an alarming figure which forced the ecuadorian government to give up its economic policies. thereafter, inflation rates have fluctuated unstably, with some years in which inflations were very high, and other years whose inflation levels were reasonable. as displayed in table 13, in the years 1933, 1934 and 1936 were the years where inflation was very high, in the first years and inflation of 34 percent was recorded, while in 1936, an inflation percentage of 19.20 was recorded. from 1932 to 1938, the economy lived years of significant inflationary fluctuations. the inflationary rated were a direct consequence which arose from the anti-crisis policies taken since the abolition of the gold standard. table 13. price index and inflation, ecuador and latin america, 1932-1938 year price index in ecuador inflation in ecuador inflation in latin america 1932 2,12 -10,96 3,00 1933 2,84 34,03 4,86 1934 3,82 34,63 2,51 1935 4,13 9,11 -1,99 1936 4,92 19,20 4,86 1937 5,33 8,46 11,06 1938 5,33 0,21 -4,00 source: author´s creation. data come from: morillo 1996, 640; mitchell 1999, 693. latin america, on the other hand, presents fluctuating inflationary signs: years of deflation followed by years of inflation. thus, from 1932 to 1934, latin america recorded an average annual inflation of 3.45, the following year a deflation of -1.99, and in 1936 and 1937, the region recorded an inflation of 4 and 11 points. as seen in figure 6, the trend of inflation in ecuador, with respect to latin america, is similar, although the trend of inflation in ecuador is much more pronounced than that recorded in the region. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 45 figure 6. inflation, ecuador and latin america, 1932-1938 source: table 13 the inflationary trend in the region was the product of increasing the money supply, and this came as a result of the abandonment of the monetary system based on gold reserves. inflation in ecuador eliminated the economic growth registered at current prices; therefore, as we shall see in the next section, recovery of the ecuadorian economy was very slow, a stationary growth for the decade. relative to the growth of gross domestic product in table 14, it can be noted that the nominal gdp, 1932-1937, has an average growth of 18%. although it seems an annual extraordinary growth, this scenario changes when it is taken into account gdp at constant prices. real gdp, based on prices in april 1978-may, growth has an average of two points. the change in gdp at constant prices presents a clearer picture of the evolution of the economy: data from 1932 to 1937 show that there was a progressive increase in gdp until https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 46 1936, in 1937 there is a decrease in real gdp due to currency seizure policies. the change in real gdp from 1932 to 1937 shows that the anticrisis flexibility and increased money supply policies helped little to the recovery and growth of the ecuadorian economy. as seen in figure 7, the real gdp of latin america recovered faster than the one recorded for ecuador: from 1933 to 1937, while latin america average annual growth was of 5%, ecuador grew at 1.7%. this scenario shows that although ecuador did not record negative growth rates until 1932, his recovery was much slower than in latin america. table 14. evolution of gross domestic product, ecuador and latin america, 19321937 years gdp current prices (thousands of sucres) gdp variation current prices (%) gdp constant prices april 78may 79= 100 (thousands of sucres) gdp variation contant prices (%) gdp variation latin america 1932 305 -9,40 9.848 0,65 -5.49 1933 437 43,33 10.064 2,19 -4.45 1934 600 37,26 10.284 2,19 15.75 1935 649 8,30 10.548 2,56 5.75 1936 801 23,29 10.910 3,43 4.91 1937 851 6,27 10.708 -1,85 3.04 source: morillo 1996, 687 https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 47 figure 7. variation of real gdp, ecuador and latin america, 1932-1937 source: table 14 monetary policy was the main tool that latin america had to face the great depression. as prices of raw materials, international reserves and tax revenues fell, the anti-cyclical policies implemented induced "to suspend and subsequently abandon the gold standard, and enter capital controls and a ban on the export of gold" (jacome 2015, 13). economic policies in latin america, from 1932, represent a breach of the principles of free market and non-intervention. economic policies during the first years of the international crisis, following the pattern set by the free market: "stimulate demand and thereby boost production by a deflationary policy. with this policy a decline in prices by reducing the money supply and tighten credit, state expenditures are reduced and the budget balance is desired" (ponce, 2015: 13). after the collapse of the gold system, reactivation policies focused on a substantial increase of money lending from the central banking, in the case of colombia, private credit policy also became very active. with this purpose, by 1932, in colombia, three entities were created: agrarian fund, central bank mortgage, and colombian corporation credit (ocampo 1996). https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 48 in the case of ecuador, the economic policies from 1932 had important similarities with the policies of the latin american region: leave the gold standard with one main objective, to increase the money supply. for this, the central government played an important role through an aggressive borrowing from the central bank of ecuador. this stage brought heavy inflation rates, which kept real gdp growth at rates below the region. the great depression marked the beginning of the end of the easy growth based on exports of primary goods (veliz 1980, 157). the reconstruction of the economies of latin america, after the impact of the great depression are characterized by the tendency towards internal development, through protectionism and industrialization development strategies that would depend on the lead of government, leaving aside the principles of free market that contextualized the principles of growth in the region until 1929. 6. conclusions at the beginning of this research, we wondered whether ecuador suffered with the impact of the great depression or whether, on the contrary, it is an atypical case of the region. to answer these initial concerns, it is necessary to remember the composition of the population. if the majority of the population lived in rural areas, and a large part of these people lived in a system of exchange of products that they cultivated for family use, it is undeniable that their subsistence did not change before or during the international crisis. this population cushions sustained the economic growth (real gdp) of the nation without registering negative data. however, the same population composition, which served as a cushion to sustain the economic growth of the nation, became a heavy weight to carry that did not allow the nation to recover at levels similar to those recorded in the rest of the region. this recovery depended primarily on the external sector, taking into account that the main trading partner was the united states of america. ecuador does represent an atypical case in the region, however, it is not the only one, colombia and honduras also do not show signs of economic decrease. these data show https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 49 that the impact of the great depression on the latin american region was uneven: ecuador, colombia and honduras are countries that show a lower impact. on the other hand, the symptoms of the international crisis were similar throughout the region. the evolution of foreign trade data and public finances were similar: a sharp decline until 1932-1933, and then a rapid recovery. there is, however, a small dissimilar behavior; the region leaves the gold standard in different years, thus changing the rhythms of economic growth. the great depression represents a break in the latin american region in its way of conceiving development. during the 1920s the growth of the region focused on trade relations with the nordic countries, those countries where prosperity seemed inexhaustible. however, when the crisis erupts, the region begins to wander through changes that will lead to try to reduce dependence from the hegemonic countries. the great depression would represent the beginning of another period, focusing growth on the inward, and fostering industrialization as the most important tool of this new approach. this study on the impact of the great depression in ecuador shows several economic statistics, which have recreated the historical development of the nation for a decade, and has created a logical context through which economic policies can be understood and linked with the results. however, certain issues remain unclear due to lack of sources, thus, it has not been possible to recreate statistics for unemployment, poverty, inequality, etc. certainly, these topics will be grounds for future research of our own or other authors. references almeida, r. (1994). kemmerer en el ecuador. serie tesis historia. quito, ecuador: facultad latinoamericana de ciencias sociales sede ecuador. https://doi.org/10.4995/muse.2019.7766 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 50 banco central de ecuador. (1937). decretos. boletín mensual , año xi (122), 1-20. banco central de ecuador. (enero de 1940). consumo interno. boletín mensual, año xiii, n.150-151. banco central del ecuador. (1939). notas informativas. boletín mensual , año xiii (145), 1-30. bethell, l. (1998). latin america: economy and society since 1930. cambridge: cambridge university press. https://doi.org/10.1017/cbo9780511626074 bethell, l. (1991). historia de américa latina. américa latina: economía y sociedad, c. 1870-1930. barcelona, españa: editorial crítica. https://doi.org/10.1017/chol9780521266529 brian news service. (1932). edwin w. kemmerer. washington, 1942: library of congress prints and photographs division. carbo, l. (agosto, 1927). boletín del banco central. quito: banco central del ecuador. carbo, l. (1953). historia monetaria y cambiaria del ecuador: desde época colonial. quito, ecuador: banco central del ecuador. carbo, l. (1978). historia monetaria y cambiaria del ecuador. quito, ecuador: banco central del ecuador. cueva, a. (1991). la crisis de 1929-32: un análisis. en r. thorp, la crisis en el ecuador. los años treinta y ochenta. quito, ecuador: corporación editora nacional. instituto de estudios avanzados, 61-77. de la torre, c. (1993). la seducción valasquista. (f. l. ecuador, ed.) quito, ecuador: ediciones libri mundi. dirección general de estadística, registro civil y censo. (1934). informe realizado por peñaherrera, a. para la dirección general de estadística, registro civil y censo. quito: ministerio de gobierno y estadística. dirección general de estadística, registro civil y censo. (1933). boletín general de estadística, año iii, num.6. quito: imprenta nacional. https://doi.org/10.4995/muse.2019.7766 https://doi.org/10.1017/cbo9780511626074 https://doi.org/10.1017/chol9780521266529 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 51 dirección nacional de estadística. (1944). ecuador en cifras 1938-1942. quito, ecuador: imprenta del ministerio de hacienda. drake, p. (1984). la misión kemmerer en el ecuador: revolución y regionalismo. revista del banco central del ecuador , vii (19), 211-280. drake, p. (1995). kemmerer en los andes: las misión kemmerer, 1923-1933. quito, ecuador: banco central del ecuador. feiker, f. (1931). economic and financial conditions in ecuador. u.s. department of commerce, bureau of foreign and domestic commerce. washington: united states government printing office. gozzi, e., & tappatá, r. (2010). fit & proper consulting. recuperado el 10 de 2015, de primera iniciativa de reforma financiera profunda en américa latina. la misión kemmerer: http://www.fitproper.com/documentos/propios/mision_kemmerer.pdf hamerly, m. (1973). historia social y económica de la antigua provincia de guayaquil 1763-1842. guayaquil, ecuador. jácome, l. (marzo de 2015). central banking in latin america: from the gold standard to the golden years. imf working paper/15/60. international monetary fund. https://doi.org/10.5089/9781484303184.001 junta nacional de planificación y coordinación económica. (1979). desarrollo y educación en el ecuador 1960-1978. quito, ecuador. mitchell, b. (1993). international historical statistics the americas 1750-1988 (2 ed.). new york: stockton press. morales, j. f., león camacho, p., & oleas montalvo, j. (1997). setenta años del banco central del ecuador pasado, presente y proyección al futuro. quito: banco central del ecuador. moreno, m. (1992). economías y sociedades de plantaciones en el caribe español, 18601930. en l. bethell, américa latina: economía y sociedad, 1870-1930 . barcelona: editorial crítica, 163-199. https://doi.org/10.4995/muse.2019.7766 https://doi.org/10.5089/9781484303184.001 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 52 morillo batlle, j. (1996). economía monetaria del ecuador. quito, ecuador: imprenta mariscal. moxlad. base de datos de historia económica de américa latina montevideo-oxford. (p. l. bertola, productor) recuperado el 01 de 10 de 2017, de moxlad: http://moxlad-staging.herokuapp.com/home/es# ossa, f. (1993). la creación del banco central de chile en 1925: antecedentes previos y los primeros años de operación. revista estudios de economía , 20 (1), 69-101. paz y miño, j. (11 de julio de 2011). revolución juliana. el telégrafo . paz y miño, j. (2004). lección de los estancos. taller de historia económica , v (vi), 13. paz y miño, j. (2013). la revolución juliana en ecuador (1925-1931). políticas económicas. quito, ecuador: academia nacional de historia. paz y miño, l. (marzo de 1926). población. el telégrafo. pazmiño, b., & ignacio, c. (1929). empleados del banco central del ecuador, quito. (archivo histórico del guayas, ed.) quito: ministerio de cultura y patrimonio. peñaherrera, a. (1934). informe de la dirección general de estadística, registro civil y censo al sr. ministro del ramo. quito: ministerio de gobierno y estadística. ponce, j. s. (1988). crisis económica del ecuador análisis comparativo de los períodos históricos. quito: banco central del ecuador. ponce alberca, j. (2015). la evolución económica (1918-1939): la gran depresión. lecturas de clase . sevilla: universiad de sevilla. registro oficial n. 289. (01 de marzo de 1927). quito, ecuador. rodríguez, l. a. (1992). las finanzas públicas en el ecuador (1830-1940). quito: ediciones banco central de ecuador. stancey, e. l. (1990). evolución de la política del banco central del ecuador 1927-1987. quito: ediciones del banco central del ecuador. stevens, r. b. (1940). commercial policies and trade relations of individual latin american countries. en r. b. stevens, the foreign trade of latin america (vols. https://doi.org/10.4995/muse.2019.7766 http://moxlad-staging.herokuapp.com/home/es multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.7766 social and technological sciences issn: 2341-2593 naranjo-navas et al. (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 2 (2019): 11-53 | 53 section 6.ecuador, 2-40). washington d.c., the united states: united states tariff commission. twomey, m. (1983). the 1930s depression in latin america: a macro analysis. (u. o. michigan, ed.) explorations in economic history (20), 221-247. https://doi.org/10.1016/0014-4983(83)90031-1 véliz, c. (1980). in the centralist tradition of latin america. princeton university press. https://doi.org/10.1515/9781400857302 villavicencio, manuel (1858). geografía de la república del ecuador. nueva york: robert craighead. wither n, f. (1936). decretos supremos no. 596, 629 y 873 sobre control de importación, exportación y cambios. decretos, ministerio de hacienda, quito. https://doi.org/10.4995/muse.2019.7766 https://doi.org/10.1016/0014-4983(83)90031-1 https://doi.org/10.1515/9781400857302 6. conclusions references multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 52 applications of ordered weighted averaging (owa) operators in environmental problems llopis-albert, carlosa; palacios-marques, danielb a departamento de ingeniería mecánica y de materiales, universitat politècnica de valència, camí de vera s/n, spain, 46022, email: cllopisa@upvnet.upv.es b departamento de organización de empresas, universitat politècnica de valència, camí de vera s/n, spain, 46022, email: dapamar@doe.upv.es received: 2016-01-04; accepted: 2016-07-22 abstract this paper presents an application of a prioritized weighted aggregation operator based on ordered weighted averaging (owa) to deal with stakeholders' constructive participation in water resources projects. they have different degree of acceptance or preference regarding the measures and policies to be carried out, which lead to different environmental and socio-economic outcomes, and hence, to different levels of stakeholders’ satisfaction. the methodology establishes a prioritization relationship upon the stakeholders, which preferences are aggregated by means of weights depending on the satisfaction of the higher priority policy maker. the methodology establishes a prioritization relationship upon the stakeholders, which preferences are aggregated by means of weights depending on the satisfaction of the higher priority policy maker. the methodology has been successfully applied to a public participation project (ppp) in watershed management, thus obtaining efficient environmental measures in conflict resolution problems under actors’ preference uncertainties. keywords: owa operators; stakeholders; decision-making; water resources management. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 53 1. introduction public participation projects in watershed management has expanded considerably during the last decades, largely in response to the environmental compulsory legislation in many countries (e.g., peña-haro et al., 2010; 2011; molina et al., 2012; llopis-albert et al., 2014). for instance, the eu wfd enacts the achievement of a good quantitative, qualitative and/or ecological of all water bodies by a series of measures ranked under the conditions of ecological efficiency, full cost recovery and public participation (ec, 2000). in order to achieve the environmental and socio-economic objectives the heterogeneous stakeholders’ interests must be balanced. public participation eases the enforceability of measures and policies, because they provide transparency, confidence in institutional actors, consensus among stakeholders, and legitimacy of the river basin management plan (e.g., llopis-albert et al., 2015). there are different levels of stakeholders’ participation on account of their degrees of influence in the decisionmaking process, and how they are affected by the measures undertaken. the levels range from information supply (stakeholders are only informed), consultation (only the actors’ opinions are considered), to active involvement (actors develop alternatives, identify solutions and take responsibilities). in this work, stakeholders are divided into three main sets: government (national, regional and local governments, and river basin authorities); experts and opinion formers (advisors and academics, mass-media); and the users of water resources (i.e., water user associations for agricultural, industrial and urban use). there are several factors that may lead to conflict of interest among actors, which range from the environmental objectives pursued, different rates of socioeconomic development for the region, level of engagement and means of participation, and alternative measures to be undertaken. in accordance with the stakeholders’ importance and influence their priority is defined. the government has been defined with the largest weights, the users have the second, while the experts present the smallest weights. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 54 in this paper, the stakeholders’ satisfaction in a ppp are analyzed through the ordered weighted averaging aggregation (owa) operator, although other approaches such as the fuzzy set/qualitative comparative analysis (fsqca) or structural equation modelling could also be applied (xu et al., 2014; berbegal-mirabent and llopis-albert, 2016; llopis-albert and palacios-marqués, 2016). owa operators (yager, 1988; yager et al., 2011) and prioritized multi-criteria decision-making problems have been widely used in the literature (e.g., yager, 2004; amin and sadeghi, 2010; yan et al., 2011). 2. methodology in conflict decision-making processes some actors are considered as prior to others. this is performed by taking under consideration their importance and influence and how they bear the measures to be undertaken. the aim is to obtain through prioritized aggregation operators (yager, 2008; 2009, wang et al., 2014) a prioritization among stakeholders, and the overall scores of each alternative. we assume that stakeholders 𝐷 = {𝐷1, 𝐷2, ⋯ , 𝐷𝑛 } can be divided into 𝑞 categories 𝐻1, 𝐻2, … , 𝐻𝑛; where 𝐻𝑖 = {𝐷𝑖1, 𝐷𝑖2, ⋯ , 𝐷𝑖𝑛𝑖 }, 𝐷𝑖𝑗 is the stakeholder in category 𝐻𝑖, 𝐷 = ⋃ 𝐻𝑖 𝑞 𝑖=1 and ∑ 𝑛𝑖 = 𝑛 𝑞 𝑖=1 . there is also a prioritization among the categories, which entail that 𝐻1 > 𝐻2 > ⋯ > 𝐻𝑞 . the set of alternatives are defined as 𝑋 = {𝑥1, 𝑥2, ⋯ , 𝑥𝑚} and for each alternative 𝑥 in 𝑋 and for each stakeholder, the value 𝐷𝑖𝑗 (𝑥) ∈ [0,1] expresses the stakeholders’ satisfaction or preference. the alternatives are ranked, which present a strict priority order when each priority level is assigned to only one stakeholder (i.e., 𝑛𝑘 = 1 for 𝑘 = 1,2, … , 𝑞) while otherwise a weakly ordered prioritization takes place. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 55 according to yager (2008) the value of 𝐷(𝑥) can be determined for alternative 𝑥 as: 𝐷(𝑥) = 𝐹 (𝐷𝑖𝑗 (𝑥)) = ∑ (∑ 𝜔𝑖𝑗 𝐷𝑖𝑗 (𝑥) 𝑛𝑗 𝑗=1 ) 𝑞 𝑖=1 (1) the priority relationship is defined by means of weights 𝜔𝑖𝑗. they are function of 𝑥 and are associated with a particular ordered position, contrary to the weighted means. in addition, the values of the variables are formerly ordered in a decreasing way. the satisfaction degree of stakeholders for each priority level can be expressed as: 𝑆𝑎𝑡𝑖 = 𝑂𝑊𝐴𝜔𝑖 (𝐷𝑖1(𝑥), 𝐷𝑖2(𝑥), ⋯ , 𝐷𝑖𝑛𝑖 (𝑥)) = ∑ 𝜔𝑖𝑘 𝑏𝑖𝑘 (𝑥) 𝑛𝑗 𝑘=1 (2) where 𝜔𝑖 is the owa weighting vector associated with each priority category 𝐻𝑖 and 𝑏𝑖𝑘 (𝑥) is the 𝑘 𝑡ℎ largest of 𝐷𝑖𝑗 (𝑥). the weights satisfy that 𝜔𝑖𝑘 ∈ [0,1] and ∑ 𝜔𝑖𝑘 = 1 𝑛𝑗 𝑘=1 . the weights can be calculated using a wide range of techniques (see, e.g., yager, 1988; o'hagan, 1988; xu, 2005). the priority induced importance weights (𝑇𝑖) of each priority level 𝐻𝑖 are obtained as follows: 𝑇𝑖 = ∏ 𝑆𝑎𝑡𝑘−1 𝑖 𝑘=1 (3) in which: 𝑆𝑎𝑡0 = 1; 𝑓𝑜𝑟 𝐻1, 𝑇1 = 1; 𝑓𝑜𝑟 𝐻2, 𝑇2 = 𝑇1𝑆𝑎𝑡1; 𝑓𝑜𝑟 𝐻3, 𝑇3 = 𝑇1𝑇2𝑆𝑎𝑡2 the aggregated value for each alternative is calculated through the prioritized owa (powa) operator: 𝐷(𝑥) = ∑ 𝑇𝑖 𝑆𝑎𝑡𝑖 𝑞 𝑖=1 (4) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 56 and the overall score of each alternative is determined by means of: 𝐷(𝑥) = ∑ 𝑢𝑖 𝑆𝑎𝑡𝑖 𝑞 𝑖=1 (5) where: 𝑢𝑗 = 𝑇𝑖 ∑ 𝑇𝑗 𝑞 𝑗=1 (6) 3. application to a case study this methodology is applied to the jumilla-villena aquifer (se spain), which have a great influence on the agricultural and economic development of the region. this has caused important environmental impacts, such as aquifer over-exploitation, drying out of springs and wetlands, and high concentrations of pollutants (e,g,, nitrates from the fertilizers applied to agriculture) and seawater intrusion. the aquifer has a surface of 338 km2 and is shared between the segura and júcar river basin authorities (chs, 2016; chj, 2017). groundwater reserves are estimated at 1400 hm3, and accumulated water abstractions from the 1980s is about 1000 hm3. however, the large uncertainties in hydrological parameters hamper the calculations (e.g, llopis-albert et al., 2016; 2016a; 2015; pulido-velazquez et al., 2011; llopis-albert and capilla, 2010; 2010a). groundwater abstractions are around 40 hm3/year with a recharge of 7 hm3/year, i.e., the water system has a disequilibrium of about 33 hm3/year. an important decrease in the piezometric levels of around a 100 meters have been recorded in several wells. this has led to consider the quantitative status of the aquifer to be one of the most important issues in the segura river basin management plan for the period 2015-2021 (chs, 2016). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 57 however, these drawdowns have been considerably reduced because the measures undertaken. as a consequence of all these facts, the water system is at risk of not achieving the good quantitative and qualitative status as stated by the eu wfd, thus leading to severe conflicts among stakeholders. in accordance to the wfd a ppp was performed by the segura river basin authority for the hydrological planning cycle 2015-2021 (chs, 2016). as above explained, three different categories of stakeholders with heterogeneous interests are considered: government (h1): including the national government (d11), the regional and local governments (d12), and the segura and júcar water agencies (d13). experts and opinion formers (h2): including advisors and academics from different fields (hydraulics and environmental sciences, economics, law, ngo’s, etc.) (d23). users of water resources (h3): including water user associations (d31), associations from the tourism sector because its influence in the coastal areas of the se of spain (d32), and private companies /e.g., hydroelectric and energy) (d33). table 1 presents the stakeholders’ satisfaction preference with regard to each alternative. different weights are assigned to the stakeholders on account of their capacity to influence on the measures to be undertaken. in this sense, the categories are prioritized as follows h1 >h3 >h2. eventually, the different groups of stakeholders considered in this study are: 3 from different level governments (d11, d12, d13), 3 from experts and opinion formers multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 58 (d21, d22, d23), and 3 from water user communities (d31, d32, d33). each actor provides its satisfaction level or preference to each alternative, thus allowing to obtain the scores dij(x) with regard to each alternative ak (k=1,2,…,5). the segura hydrologic plan for the cycle 2015-2021 entails different alternatives, measures and policies to be carried out that should be discussed by the actors. they include, in order to achieve the environmental and socio-economic objectives, the decrease of over-exploitation, transfer of water resources from other watershed, use of desalination and sewage water plants, government control measurements and regulations, etc. (chs, 2016). we consider 5 different alternatives, which entail different levels of priority regarding the environmental and socio-economic objectives, and different measures and policies to achieve those objectives: -alternative 1 (a1): it gives priority to the achievement of the environmental and ecological objectives before the horizon of 2027. -alternative 2 (a2): it tries to balance the environmental and ecological objectives with the socio-economic objectives through the use of external water resources. -alternative 3 (a3): it is similar to alternative a2 but instead of relying in the use of external water resources it also makes use of desalination plants and sewage treatment plants. -alternative 4 (a4): it gives more priority to the socio-economic objectives in detriment of the environmental ones. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 59 -alternative 5 (a5): it is similar to alternative a1 but entails a high government control, river basin authorities and user’s communities for achieving the good status of water bodies. 4. results and discussion table 1 shows the scores dij(x) obtained by means of meetings, interviews with actors, surveys, expert judgment, mass-media information and reports published by the segura and júcar water agencies (chs, 2016). table 1. satisfaction degree of each stakeholder for each alternative d11 d12 d13 d21 d22 d23 d31 d32 d33 a1 0.9 0.8 0.9 0.9 0.7 0.9 0.6 0.5 0.9 a2 0.8 0.9 0.8 0.6 0.5 0.6 0.9 0.9 0.5 a3 0.6 0.6 0.6 0.6 0.5 0.5 0.8 0.9 0.4 a4 0.7 0.8 0.6 0.7 0.8 0.5 0.9 0.9 0.3 a5 0.7 0.6 0.7 0.8 0.5 0.8 0.6 0.5 0.9 the weights associated to the owa operator in eq. (2) are determined using linguistic quantifiers (q) (yager, 2008): 𝜔𝑗 = 𝑄 ( 𝑗 𝑛 ) − 𝑄 ( 𝑗−1 𝑛 ) , 𝑗 = 1,2, … , 𝑛 (7) where q is (zadeh, 1983): multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 60 𝑄(𝑟) = { 0, 𝑖𝑓 𝑟 < 𝑎 𝑟 − 𝑎 𝑏 − 𝑎 , 𝑖𝑓 𝑎 ≤ 𝑟 ≤ 𝑏 1, 𝑖𝑓 𝑟 > 𝑏 therefore, the weight vector is 𝜔𝑗 = (0.066,0.666,0.268) 𝑇 , with 𝑗 = 1,2,3. in this way, the stakeholders’ satisfaction for each priority level can be obtained: 𝑆𝑎𝑡1(𝑥1) = 𝑂𝑊𝐴𝜔1(𝐷11(𝑥1), 𝐷12(𝑥1), 𝐷13(𝑥1)) = 𝑂𝑊𝐴𝜔1(0.9,0.8,0.9) = 0.8737 𝑆𝑎𝑡2(𝑥1) = 𝑂𝑊𝐴𝜔2(𝐷21(𝑥1), 𝐷22(𝑥1), 𝐷23(𝑥1)) = 𝑂𝑊𝐴𝜔2(0.9,0.7,0.9) = 0.8469 𝑆𝑎𝑡3(𝑥1) = 𝑂𝑊𝐴𝜔3(𝐷31(𝑥1), 𝐷32(𝑥1), 𝐷33(𝑥1)) = 𝑂𝑊𝐴𝜔3(0.6,0.5,0.9) = 0.5935 𝑇1 = 1; 𝑇2 = 𝑆𝑎𝑡0 · 𝑆𝑎𝑡1 = 0.8737; 𝑇3 = 𝑇2 · 𝑆𝑎𝑡2 = 0.7400 by using eq. (4) the global prioritized aggregated value can be obtained: 𝐷(𝑥1) = ∑ 𝑇𝑖 𝑆𝑎𝑡𝑖 3 𝑖=1 = 1 · 0.8737 + 0.8737 · 0.8469 + 0.7400 · 0.5935 = 2.0529 by proceeding in the same way: 𝐷(𝑥2) = 1.6373; 𝐷(𝑥3) = 1.1177; 𝐷(𝑥4) = 1.4536; 𝐷(𝑥5) = 1.4465 this leads the alternatives to be ranked as follows: 𝐷(𝑥1) > 𝐷(𝑥2) > 𝐷(𝑥4) > 𝐷(𝑥5) > 𝐷(𝑥3) we can conclude that after applying a prioritized weighted aggregation operator based on ordered weighted averaging (owa) the best alternative is a1, followed by alternative a2. 5. conclusions a methodology based on prioritized owa operators has been successfully applied to a real case study in the field of water resources. results show that stakeholders are more concerned about the environmental and ecological objectives (since the aquifer is at risk of not fulfilling the requirements of the wfd) rather than the socio-economic ones. we multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 61 have shown that the methodology is useful when applied to conflict resolution problems under stakeholders’ preference uncertainties. references berbegal-mirabent, j., llopis-albert, c. (2016). applications of fuzzy logic for determining the driving forces in collaborative research contracts. journal of business research 69(4), 1446–1451. doi: 10.1016/j.jbusres.2015.10.123. -chs (2016). segura river basin authority. http://www.chsegura.es/ -chj (2016). júcar river basin authority. http://www.chj.es/ -ec (2000). directive 2000/60/ec of the european parliament and of the council of october 23 2000 establishing a framework for community action in the field of water policy. official journal of the european communities, l327/1el327/72. 22.12.2000. -llopis-albert, c., merigó, j.m., xu, y., (2016). a coupled stochastic inverse/sharp interface seawater intrusion approach for coastal aquifers under groundwater parameter uncertainty. journal of hydrology, 540, 774–783. doi: 10.1016/j.jhydrol.2016.06.065. llopis-albert, c., palacios-marqués, d., merigó, j.m., (2016a). decision-making under uncertainty in environmental projects using mathematical simulation modeling. environmental earth sciences, . doi: llopis-albert, c., palacios-marqués, d., (2016). applied mathematical problems in engineering. multidisciplinary journal for education, social and technological sciences 3(2), 1-14. doi: 10.4995/muse.2016.xxxx. -llopis-albert, c., palacios-marqués, d., soto-acosta, p. (2015). decision-making and stakeholders' constructive participation in environmental projects. journal of business research 68, 1641–1644. doi: 10.1016/j.jbusres.2015.02.010. -llopis-albert, c., pulido-velazquez, d. (2015). using modflow code to approach transient hydraulic head with a sharp-interface solution. hydrological processes, 29(8), 2052–2064. doi: 10.1002/hyp.10354. -llopis-albert, c., merigó, j.m, palacios-marqués, d. (2015). structure adaptation in stochastic inverse methods for integrating information. water resources management 29(1), 95-107. doi: 10.1007/s11269-014-0829-2. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 62 -llopis-albert, c., pulido-velazquez, d. (2014). discussion about the validity of sharpinterface models to deal with seawater intrusion in coastal aquifers. hydrological processes 28(10), 3642–3654. doi: 10.1002/hyp.9908. -llopis-albert, c., palacios-marqués, d., merigó, josé .m. (2014). a coupled stochastic inverse-management framework for dealing with nonpoint agriculture pollution under groundwater parameter uncertainty. journal of hydrology 511, 10–16. doi: 10.1016/j.jhydrol.2014.01.021. -llopis-albert, c. and capilla, j.e. (2010). stochastic simulation of non-gaussian 3d conductivity fields in a fractured medium with multiple statistical populations: a case study. journal of hydrologic engineering, 15(7), 554-566. doi: 10.1061/(asce)he.1943-5584.0000214. -llopis-albert, c. and capilla, j.e. (2010a). stochastic inverse modeling of hydraulic conductivity fields taking into account independent stochastic structures: a 3d case study. journal of hydrology, 391, 277–288. doi: : 10.1016/j.jhydrol.2010.07.028. -molina, j.l., pulido-velázquez, m., llopis-albert, c., peña-haro, s., (2012). stochastic hydro-economic model for groundwater quality management using bayesian networks. water science & technology, 67(3), 579-586. doi: 10.2166/wst.2012.598. -o'hagan, m. (1988). aggregating template rule antecedents in real-time expert systems with fuzzy set logic. in, proceedings of 22nd annual ieee asilomar conference on signals, systems and computers, ieee and maple press, pacific grove, 681-689. -peña-haro, s., llopis-albert, c., pulido-velazquez, m. (2010) fertilizer standards for controlling groundwater nitrate pollution from agriculture: el salobral-los llanos case study, spain. journal of hydrology 392(3): 174–187. doi: 10.1016/j.jhydrol.2010.08.006. -peña-haro, s., pulido-velazquez, m., llopis-albert, c. (2011). stochastic hydroeconomic modeling for optimal management of agricultural groundwater nitrate pollution under hydraulic conductivity uncertainty. environmental modelling & software 26 (8), 999-1008. doi: : 10.1016/j.envsoft.2011.02.010. -pulido-velazquez, d.; llopis-albert, c.; peña-haro, s.; pulido-velazquez, m. (2011). efficient conceptual model for simulating the effect of aquifer heterogeneity on natural groundwater discharge to rivers. advances in water resources 34(11), 1377-1389. doi: 10.1016/j.advwatres.2011.07.010. -verma, r., sharma, b. (2016). prioritized information fusion method for triangular fuzzy information and its application to multiple attribute decision making. international journal of uncertainty, fuzziness and knowledge-based systems, 24, 265-290. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.7001 social and technological sciences eissn: 2341-2593 llopis-albert and palacios-marqués (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 52-63 | 63 -wang, h.m., xu, y.j., merigó, j.m., (2014). prioritized aggregation for nonhomogeneous group decision making in water resource management. economic computation and economic cybernetics studies and research, 48(1), 247-258. -xu, z.s., (2005). an overview of methods for determining owa weights. international journal of intelligent systems, 20, 843-865. xu, y., llopis-albert, c., gonzález, j. (2014). an application of structural equation modeling for continuous improvement. computer science and information systems 11(2), 797–808. doi: 10.2298/csis130113043x -yager, r.r. (1988). on ordered weighted averaging aggregation operators in multicriteria decision making, ieee transactions on systems, man and cybernetics b 18 (1988) 183-190. -yager, r.r. (2004). modeling prioritized multicriteria decision making. ieee transactions on systems, man, and cybernetics -part b: cybernetics; 34, 2396-2404. -yager, r.r. (2008). prioritized aggregation operators. international journal of approximate reasoning, 48, 263-274. -yager, r.r. (2009). prioritized owa aggregation. fuzzy optimization and decision making, 8, 245-262. -yager, r.r., kacprzyk, j., beliakov, g. (2011). recent developments on the ordered weighted averaging operators: theory and practice, springer-verlag, berlin. -zadeh, l.a. (1983). a computational approach to fuzzy quantifiers in natural languages. computers & mathematics with applications, 9, 149-184. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 185 spanish real estate bubble: a classroom experiment fernando castello-sirvent fernando.castello@esic.edu esic, business and marketing school eintlab research avda. blasco ibáñez, 55, 46021 valencia, spain. received: 2016-01-14; accepted: 2016-06-22 abstract the european higher education area (ehea) requires development of teaching methodologies that encourage active participation by students in the autonomous learning process, taking the figure of the professor as the mediator. on the other hand, the economy is a fundamental aspect in today’s society insofar that it contributes to explaining the behaviour of individuals. in this sense, the experimental economy applied in lecture rooms is presented as an innovative educational procedure that is perfectly adapted to university teaching, since it facilitates acquisition and consolidation of curricular knowledge and favours the development of capabilities by students, whilst at the same time improving the quality of teaching and has a positive impact on the interest students have in the subject. this paper describes the application of an economic experiment as an active learning method to prove the mechanisms used to set prices. the experience took place at the esic business and marketing school valencia university centre, within the subject of “spanish economy” in official business management and administration degrees and in marketing and sales management degrees. the starting point for this experience was the work by radim boháček (2002), adding changes aimed at empirically explaining inflation, the real estate bubble and subsequent economic crisis in spain. the obtained results prove that this educational experience favours learning of the economic concepts students study, and stimulates proactive interest in the subject. from the obtained results a second experiment is proposed in order to consolidate concepts concerning price setting mechanisms in the presence of intervention by the public sector (taxes and subsidies) or establishing incentives that model collusive behaviour leading to situations where market offers are concentrated in oligopolies. keywords ehea, autonomous learning, spanish economy, experimental economy, pricing, real estate bubble http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 186 1. introduction the implementation of the european higher education area (ehea) has transformed how knowledge is transmitted. in fact, one of the main goals of the ehea is to improve the quality of university teaching by focusing its attention on the adoption of competencies among students (cano, 2008). the incorporation of the new university system is an attempt to establish a dynamic, active and flexible training strategy to ensure students are able to carry out autonomous learning (whitehead, 2008). in this way, it is possible to develop the skillsets demanded on the current changing, competitive and complex employment market (hunt, eagle and kitchen, 2004). for this reason, universities must drive innovative experiences in teaching-learning processes in order to combine consistently the generic, overarching and specific skills that facilitate professional guidance to enable graduates to enter the working arena successfully (roger et al., 2016). at this point, it is appropriate to highlight that the skill acquisition process is more successful if it takes place through teamwork instead of resorting to the traditional transmission of knowledge (exley and dennick, 2007). in this sense, the ehea encourages methodologies driving the development of competencies in which co-operation and teamwork constitute fundamental method for teaching and learning, as the learning so acquired is considered to be deeper and longer lasting (gil et al., 2006). the conventional methodology based on the transfer of knowledge must precisely be replaced by a project contemplating the acquisition of skills and aptitudes based on selflearning (martín et al., 2014). in other words, students must become the protagonists of their own learning. to do so, it is of vital importance to conduct practical sessions and debates in the classroom to enable students to develop an intervention at both personal and team level materialized in the gaining of the skills needed to approach their professional career successfully (de juan, gonzález, parra, kanther and sarabia, 2008). the present article explains the different stages carried out to draw up a project for educational innovation based on the development of an economic experiment within the classroom. the experiment is established on the basis of that proposed by boháček (2002) by introducing a series of different alterations enabling the process and the outcomes to approach a large part of the conclusions drawn from the theoretical corpus of the subject within which this innovation project is housed. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 187 thus, co-operative learning is seen in this experiment as a very interesting option to organize the teaching processes and the acquisition of skillsets. nonetheless, it is important to point out that it is not enough to bring a series of students together to ensure co-operative effort. the success of teamwork lies in understanding how people learn, process and use the information provided to them (estrada et al., 2014). the goal of this experiment involved facilitating autonomous learning by students in the context of the curriculum for the subject of “spanish economy” as part of its integration into the syllabus for the degrees in business management and sales & marketing at esic business and marketing school. these official qualifications are awarded through the miguel hernández university in elche. the innovation project attempted to make more accessible a subject framed within the domain of applied economics and habitually distant and inaccessible for second-year students. this economic experiment involved 36 students, of whom 8 were erasmus exchange students from a german university. starting from the design of the pedagogical experience, consideration was given to achieving higher levels of development among students in the subject’s general and specific skills, making it possible for the learning outcomes established by the university to arise in a comprehensive and more satisfactory manner for students. 2. description of the teaching innovation project the goal of the innovation project was to achieve the articulation of a practical learning method whereby students could verify, in addition, the validity of part of the economic theory studied in the subject. in particular, one important goal involved analysing price formation and, in the specific case of the spanish property market, assessing the role of regulators in the design of incentives. it was also fundamental for participants in the experiment to be able to make it tangibly apparent how the various supply-side restrictions (land regulations) and, on the demand side on the other hand, the effects caused by the monetary policies on low interest rates applied by the european central bank (ecb) that ultimately contributed to the generation of our real estate bubble. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 188 thus, the elements involved in the formation of relative prices, i.e. why one item should be worth more than another, are crucial for students to be able to understand not only the process of inflation in the spanish property sector but also, on a wider scale, any distortion in market equilibrium. starting from the paper published by radim boháček (2002) and the variants proposed by rey-biel (2015), an economic experiment was designed around the exchange of various legumes (beans and chickpeas) that was subjected to three major conditions: a) contingency: the amount of incentive received by each participant must depend at least on the decision taken by the subject involved. b) dominance: changes in the subject’s satisfaction with respect to the experiment had to be due, fundamentally, to changes in the amounts of incentive received. c) monotonicity: a higher incentive payment had always to be preferred by students to a smaller payment, and subjects must not become satiated. the sequential design of the structure of the experiment proposed is presented below. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 189 table 1. structure of the economic experiment phase 1 a) normative stage share-out of the initial provision of beans and chickpeas, and of the paper records to all participants explanation of the methodology (introduction) how to carry out exchanges, how to record the timeline annotations on the paper record provided, the agent making the exchange and the exchange rate b) random control stage (duration: 5’) c) exchange stage (duration: 15’) free exchange between any pair of agents with monitoring of the timeline and annotations proposed with respect to the methodology phase 2 a) normative stage monetary injection of more chickpeas to leave the total proportion on the market at 1/6 explanation of the methodology (winner’s rules) b) random control stage (duration: 5’) c) exchange stage (duration: 15’) free exchange between any pair of agents with monitoring of the timeline and annotations proposed with respect to the methodology phase 3 a) normative stage explanation of the methodology (variation) b) random control stage (duration: 5’) c) exchange stage (duration: 15’) free exchange between any pair of agents with monitoring of the timeline and annotations proposed with respect to the methodology source: own production http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 190 in phase 1a, it was established that each individual could take an initial provision of red beans and ordinary chickpeas pre-established in different sacks distributed at random and contained in a box. this initial provision presented, for the class as a whole (the market), an uneven proportion of 1/3, in which 3 chickpeas were presented for each bean. in this way, some economic agents (students) might find themselves with a proportion of 2/1 in their initial provision; others with a proportion of 5/1; and others with a proportion of 1/6. the total amounts of legumes contained in each bag were the same for all students, but not the proportion between beans and chickpeas. distribution in the classroom was at random. in order to resolve the contingency condition, three exchange stages (1c, 2c, 3c) were designed with the aim of allowing students to perform unlimited free exchanges with their classmates. the moment at which the market was declared exhausted was when it was no longer possible to engage in any further transaction as all participants were satisfied with the amounts exchanged and, in consequence, with the combination of beans and chickpeas retained. in order to resolve the dominance condition, three normative stages (1a, 2a, 3a) were designed for the purpose of providing a sequenced explanation of the methodology and the rules governing the experiment so as to furnish participants with tools and criteria to enable them to increase their own utility. in phases 1a and 2a, the information presented was deliberately incomplete in order to trigger a design that would constrain the behaviour of individuals. in order to resolve the monotonicity condition, a final incentive was determined for the ‘most competitive’ economic agent. this consisted in a 50-euro gift voucher from a wellknown department store. although all the participants knew about the final incentive, they were gradually informed through phases 1a, 2a and 3a about the specific rules that would enable them to be ‘more competitive’. similarly, and in view of the possibility of fraud being perpetrated by participants, three random control stages (1b, 2b, 3b) were designed. in each of the control stages, the lecturer inspected the data sheets and the balances (beans and chickpeas) of three students, chosen at random, and their most recent counterparties, i.e. the last three students with whom agreement had been reached for a transaction with the student being inspected. in this way, a total of 9 students were inspected at random (25% of the class) along with their http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 191 final counterparties for each immediately preceding exchange phase, so the possibility of a check affected practically 50% of the market. this inspection and control condition is similar the function performed by the public exchequer facing tax fraud in a real market. the goal in our case, as for the tax office, involved inhibiting opportunistic behaviour leading to the exploitation of the regulatory cracks in the game, thus preventing the generation of distortions and externalities. in the event of fraud being detected, the penalty meant that both the student under inspection and the counterparty would lose their stock of legumes and would cease to participate in the game, thus making it impossible for them to win the final prize. it should be indicated that any inaccuracy in the amount noted between a pair of economic agents or any mismatch in these amounts, or in their stock of legumes with respect to the balance noted at the moment of the inspection, would imply the expulsion of both participants from the game. it must be remembered that student a in the last exchange of a phase with student b should have noted down the same amounts as student b but in the reverse order, and both should coincide in their tally of the real balances of beans and chickpeas, for both a and b. we must stress that there were no mistakes in the tallies kept and we can therefore infer the high level of attention given by participants during the experiment, as any carelessness would have implied a severe penalty. the design of the experiment also mitigated the impact that might have been caused by the economic agents becoming sated through tiredness or boredom in view of the repetitive mechanics of the game and this was possible through a flexible, functional design with a total duration of less than 75 minutes. during phase 2a, the lecturer, acting as a central bank, flooded the market with chickpeas by raising the monetary supply of chickpeas to create, on aggregate and analysing the market as a whole, a disproportion of 1/6 (beans/chickpeas). the share-out was fair and proportional, with each student receiving the same number of chickpeas. due to the intrinsic characteristics of this design, a student who had inferred an equilibrium price of 1/3 prior to the monetary injection passed to an inferred price of 1/6. during phase 2a, the lecturer explained the winner would not be the participant who, as most of the students had guessed, had the largest number of beans and chickpeas (added together), but rather that the outcome would be decided by multiplying the number of beans http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 192 by the number of chickpeas. thus, participants were able to infer that they should seek exchanges enabling them to achieve a certain equilibrium between beans and chickpeas because, if a player managed to accumulate a lot of beans but few chickpeas, for example 6x2 = 12, and another achieved a degree of balance between beans and chickpeas, for example 4x4 = 16, then the second one would beat the first. during phase 3a, the lecturer introduced a further modification. by then all the participants were aware of the disproportion that existed between the different legumes; they were not able to specify it in numerical terms but they were able to gauge it. at the end of phase 3c, the pair of participants involved in a last exchange that had been recorded as the final exchange representing the highest relative prices would obtain from the central bank an additional 50 beans and 50 chickpeas each, which would be added to their stock at that moment and the game would then come to an end and the final tally would be drawn up (multiplication of beans and chickpeas of all the participants) to determine the winner. 3. methodology a stopwatch was projected in the classroom to give an accurate reference of the time in minutes and seconds, programmed in three blocks of 15 minutes each, to provide the timeline during the exchanges in phases 1c, 2c and 3c for inclusion on the control sheets distributed together with the initial provision of legumes. each participant had to record the following information on his or her data sheet for each exchange: a) the phase in which it took place (1c, 2c, 3c) b) the exact moment it took place (minute and second) c) the player with whom the exchange was made1 d) the swap ratio (number of beans and the number of chickpeas exchanged2) 1 the participants were assigned numbers from 1 to 36 and noted the specific code assigned to their respective counterparties in their different trades. 2 the swap ratios admitted any proportional fraction in the form 2/5; 6/7; 9/4; representing 2 beans for 5 chickpeas, 6 beans for 7 chickpeas or 9 beans for 4 chickpeas. in this way, the exchange rate for a bean can be calculated by dividing the denominator by the http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 193 e) the balance at the end of the trade (how many beans and how many chickpeas) bearing in mind the market’s dimensions (36 participants trading with each other, all standing up) meant that it was difficult for players, and costly in comparative terms, to carry out a complete, overarching co-ordination of all the members on the market because, if they had tried to, they would have missed the chance to make more and better exchanges. at the end of phases 1c, 2c and 3c, the lecturer asked the participants to shout out the ratios of the last trades they had made on the market. in other words, the exchange rates for supply and demand in the last operations between pairs of students. this gave rise to ratios varying between 1/1 for the exchange of one bean for one chickpea and 1/8 for the exchange of one bean for 8 chickpeas. as these exchange rates were shared aloud, participants could become aware of the inflationary process that was being created on the market, even though this would not have been apparent to them until that moment due to the bias caused by the asymmetry in the information. 4. outcomes at the end of phase 1c, it was possible to see that inflation exceeded real scarcity on the market. while the proportion in the initial provision was 1/3, by the end of phase 1c, the closing prices between supply and demand were 1/3, 1/6 and 1/8. this indicates the existence of volatility due to the fact that, in a non-organized market, information asymmetry meant that not all the participants were aware of the last price traded and, in consequence, they were ignorant of the ‘best contractual position’ that would allow them to buy chickpeas or sell beans, and vice versa. numerator. in the preceding examples, the price of a bean would respectively be 2.5, 1.17 and 0.44 chickpeas. analogously, the price of a chickpea can be calculated by dividing the numerator by the denominator in the swap ratio indicated above. using the same examples, the price of a chickpea would respectively be: 0.4, 0.86 and 2.5 beans. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 194 at the end of phase 2c and, in particular, phase 2d, it was possible verify that the exchange ratios were getting higher and higher. after phase 2c, the monetary injection of chickpeas justified the increase in prices. however, in the course of phase 3c, alongside the price increases, there was a continuous process for the creation of alliances among the participants to the point where true oligopolies were established in terms similar to those expressed by garcía-gallego (2008). thus, by forcing dumping practices, as many as four hidden markets3 were created and enabled their members to accumulate a large number of chickpeas by effecting major alterations on the market price and achieving different concentration manoeuvres that ended up allowing the highest final exchange ratio to be set at 1/96 and the second highest to be recorded at 1/68. in this sense, after the corresponding inspection and control verifications had been carried out, the player who had registered the highest exchange ratio and the corresponding counterparty each received an additional provision of 50 beans and 50 chickpeas. as the winner acknowledged the collusive concentration strategy applied, it was finally announced that the prize would be shared with the other members in the team (oligopoly). from an analytical perspective, in order to calibrate the results of this innovation project, a questionnaire was distributed for quantitative metrics. the last question in this survey was descriptive and qualitative through a semi-structured block to allow students to deal with the following items: a) strategy followed in phase 1c and justification b) strategy followed in phase 2c and justification c) strategy followed in phase 2d and justification d) general comments and proposals for improvement 3 we are aware of the number of oligopolistic markets created following the mathematical and statistical analyses associated with the data contained on the data sheets from each of the participants. following this process, we studied the real-time formation of the supply and demand curves, which have given rise to very interesting conclusions. due to their nature and complexity, however, an in-depth analysis would exceed the scope of the present article. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 195 the general and specific skills4 in the curriculum for “spanish economy” in the degree for business management are described below. these are the ones covered in the educational innovation project presented here. table 2. specific competencies for the “spanish economy” subject degree in business management (esic valencia – umh) type competency specific ability to analyse the general problems within the sphere of micro-economy and the macroeconomy specific ability to relate knowledge of fundamental economic theories and institutions. specific ability to use and interpret the company’s data and information to draft specialized reports and to take decisions specific ability to identify and evaluate the impact of economic events on companies general ability to use the necessary tools to observe adequately the systems under study general critical and analytical ability in the corresponding to area of speciality. general ability to evaluate, optimize and contrast criteria tor taking decisions general ability to work in multi-disciplinary and multicultural teams. general ability to consolidate, extend and integrate fundamental knowledge in the field of the social and legal sciences general availability of efficient self-learning methodologies and skills for the adaptation and updating of new knowledge and scientific advances, as well as the trends in needs in order to adopt an aptitude for innovation and creativity in the exercise of the profession source: own production 4 according to the teaching guidelines established by the miguel hernández university in elche (umh) http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 196 the general and specific skills5 in the curriculum for “spanish economy” in the sales management and marketing degree are described below. these are the ones covered in the educational innovation project presented here. table 3. specific skills for the “spanish economy” subject degree in sales management and marketing (esic valencia – umh) type competency specific understand the marketing environment in which the business organizations are operating and not for profit organizations. specific know how to explain general problems in the sphere of economics. general identify, recognize and understand the basic problems in the area of social and legal sciences general use tools to analyse adequately the phenomena under study general have critical and analytical aptitudes source: own production the assessment questionnaire for the innovation project incorporated social and demographic variables and all responses were anonymous. the technical details of the research is given below, together with the profile of the students surveyed and their descriptive statistics. 5 according to the teaching guidelines established by the miguel hernández university in elche (umh) http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 197 table 4. technical details of the survey universe university students from esic valencia degree in business management degree in sales management and marketing subject: spanish economy (2nd year) geographical scope valencia, spain sample design personal survey sample size 36 valid questionnaires fieldwork november 5th, 2015 statistical techniques descriptive analysis statistical software spss version 21.0 source: own production table 5. profile of the students surveyed characteristics percentage weighting gender male 36.11% female 63.89% age mean 20.278 standard deviation 1.560 minimum 19 maximum 26 source: own production http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 198 table 6. descriptive statistics minimu m maximum mean st. dev. 1. i was interested in the economics experiment proposed by the lecturer 2 5 4.472 0.774 2. my interest in educational innovation projects has increased after the economics experiment project 2 5 4.167 0.845 3. i thought the structure of the economics experiment was correct 3 5 4.306 0.710 4. the quality of the practical experience acquired through the economics experiment is good 3 5 4.194 0.710 5. the economics experiment could have lasted longer 2 5 3.806 1.009 6. the lecturer responsible has correctly integrated us into the dynamics of the economics experiment 2 5 4.306 0.710 7. i found the economics experiment intellectually stimulating 2 5 4.278 0.849 8. the methodology of the economics experiment has enabled me to learn elements of the content that i consider valuable and that i would not otherwise have managed to understand 2 5 3.914 0.887 9. my interest in the material studied in the subject has increased as a consequence of the economics experiment 2 5 3.694 0.822 10. i have understood the economic theory studied in the syllabus thanks to the economics experiment 2 5 3.528 0.878 11. overall, i have improved my collaboration with other colleagues and with the lecturer through the economics experiment 2 5 3.944 0.860 12. overall, i have improved my professional competencies in the course of the economics experiment 2 5 3.722 1.003 13. i would recommend an economics experiment like this to my colleagues 3 5 4.194 0.856 http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 199 14. globally speaking, i am glad i participated in this economics experiment 3 5 4.306 0.749 15. i would like more economics experiments like this in other subjects 2 5 4.417 0.806 16. i would attend another economics experiment organized at the university 2 5 4.222 0.989 5. discussion the overall assessment of the innovation project by the students was positive as we can confirm from the quantitative results validated by the participants. the theoretical corpus contributing to explain price formation predicts that, in the presence of scarcity, prices tend to increase. students were able to become aware of the fundamental principles on which economic theory is formulated and the way in which an inflationary process was formed, stage by stage. in addition, with regard to the concept of transaction costs associated with information asymmetry that underlie a market (tadelis and williamson, 2012), the students were able to confirm that the perception of chickpea scarcity was uneven among the participants. not all the economic agents (students) had the same information regarding the scarcity of beans. only the lecturer was aware of the initial provision of legumes. insofar as the market was not an organized one, the information flowed in the exchanges through small constellations of economic agents, thus giving a biased vision of the scarcity and implying high prices with a lot of volatility. furthermore, the students confirmed that the individual action of an economic agent determines the action of the other agents, thus configuring the market. in phase 2a, when the lecturer acted as the central bank and injected liquidity into the market in the form of extra chickpeas, students were able to confirm that the price of beans increased in the exchange ratios with respect to the currency (chickpeas) that flooded the market. the low interest rates established by the ecb, together with the different regional laws on land and their collateral effects, lie at the base of the real estate bubble that spain suffered following the collapse of lehman brothers. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 200 in terms of the analysis of the outcomes from the questionnaire posed to the students following this innovation project, the results suggest a very positive assessment of this kind of experiment in the classroom. training actions of this type are highly appreciated by students as they enable them, through practice (tacit knowledge), to approach a reflexive iteration of the curriculum’s theoretical corpus (express knowledge) and finally crystallize in a deeper level of understanding of the mechanisms giving rise to this learning process based on experience (nonaka and takeuchi, 1995). the tabulation of the qualitative outcomes presented in the last section of the questionnaire which allowed participants to express themselves freely and openly showed great consistency among a large number of students regarding their assessment of the concertation and collusion strategy throughout phase 3c. with all the cautions inherent to a qualitative methodology (gutiérrez brito, j., 2009), we must divide the evaluations given by the participants themselves with respect to their respective oligopolistic strategies into two main schools of thought. on the one hand, students who deemed their behaviour reprehensible, even though they had engaged in it; and, on the other, students who openly evaluated it as honest and defensible. with respect to the motivations of the first group, as the virtually unanimous justification of their behaviour, they alleged they were taking advantage of a void in the system, in the design of the game, and the non-existence rule prohibiting such collusive behaviour. the second group, on the other hand, argued in favour of the exploitation of market synergies, and the competitive advantage represented by the size and volume of transactions acting around a central operator or group of economic agents. while the first alleged a vulnerability in the game design, the second group found a competitive opportunity. curiously, all the erasmus students were among the second group. qualitative observations of this type led us to propose the future design of new economic experiments inspired by the experiment explained in this paper. in this case, experiments offering greater relevance to the public sector, with the introduction of subsidies and taxes, with the aim of profiling efficient mechanisms on highly regulated markets with barriers to access on the supply side, such as the electricity market (staropoli and jullien, 2006). http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 201 similarly, in the assessment questionnaires used in the classroom for the result of the teaching innovation activity, we propose the future quantitative tabulation of arguments in favour and against the oligopolistic behaviour discovered by the participants in the course of the experiment, and an increase in sample so as to allow the performance of discriminating quantitative analyses in the light of social and demographic parameters such as gender, original qualification or nationality of the student. references bohacek, r. (2002). a market-clearing classroom experiment. southern economic journal, 69(1), 189. doi: http://doi.org/10.2307/1061564 cano, mª e. (2008). la evaluación por competencias en la educación superior. profesorado. revista de curriculum y formación de profesorado, 2008, vol. 12, nº 3, p. 116. de juan, m. d., gonzález, e., parra, j. f., kanther, a. y sarabia, f. j. (2008) antecedentes del aprendizaje autorregulado del estudiantes universitario de marketing actas del xx encuentro de profesores universitario de marketing, 1719 de septiembre de 2008, gran canarias (españa). estrada ,m., monferrer, d., segarra, m. y moliner, m. á. (2014). el trabajo cooperativo en base al modelo del cerebro total: una experiencia docente. redmarka uimauniversidad de a coruña – ciecid, nº. 12, pp. 87-112. exley, k. y dennick, r. (2007). enseñanza en pequeños grupos en educación superior: tutorías, seminarios y otros agrupamientos. narcea ediciones. garcía-gallego, a.; georgantzís, n. y kujal, p. (2008): «experimental insights into the efficiency of alternative water management institutions», en dinar, a.; albiac, j. y sánchez-soriano j. (eds.), game theory and policymaking in natural resources and the environment, ed., 209-235, routledge, taylor & francis group, london/new york. gil, c.; alías, a. y montoya, m.d.g. (2006). cómo mezclar diferentes metodologías docentes para motivar e impulsar a un mayor número de alumnos, vi jornadas de aprendizaje cooperativo, barcelona. http://dx.doi.org/10.4995/muse.2016.6377 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6377 social and technological sciences eissn: 2341-2593 castelló-sirvent (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 185-202 | 202 gutiérrez brito, j. (2009). introducción a la lógica del análisis del discurso. introducción a las técnicas de investigación, 245-260. hunt, l., eagle, l., y tchen, p. (2004). balancing marketing education y information technology: matching needs or needing a better match? journal of marketing education, 26, 75-88. http://dx.doi.org/10.1177/0273475303262350 martín, a.; león, c. y garcía, a. (2014). innovación docente para la integración de autoformación y autoevaluación en la plataforma web. pixel-bit: revista de medios y educación, nº. 44, p. 201-214. http://dx.doi.org/10.12795/pixelbit.2014.i44.14 nonaka, i., & takeuchi, h. (1995). the knowledge-creating company: how japanese companies create the dynamics of innovation. oxford university press. rey-biel, p. (2015). ¿funcionan los mercados? dos experimentos que muestran sus ventajas...y limitaciones, post i nadaesgratis.es http://nadaesgratis.es/pedro-rey-biel/experimentos-en-clase accessed 13 october 2015. roger-monzó, v.; guijarro-garcía, m.; martí-sánchez, m. (2015). digital signage: an experience of innovation in higher education. multidisciplinary journal for education, social and technological sciences, v. 2, n. 2, p. 1-13. doi: http://dx.doi.org/10.4995/muse.2015.3769 staropoli, c. y jullien, c. (2006): «using laboratory experiments to design efficient market institutions: the case of wholesale electricity markets». annals of public and cooperative econo mics, vol. 77, nº 4, pp. 555-577 http://dx.doi.org/10.1111/j.14678292.2006.00319.x tadelis, s., & williamson, o. e. (2012). transaction cost economics. available at ssrn 2020176 http://dx.doi.org/10.2139/ssrn.2020176 whitehead, d. p. (2008). thoughts on education y innovation, childhood education, 85(2), 106-118. http://dx.doi.org/10.4995/muse.2016.6377 http://dx.doi.org/10.1177/0273475303262350 http://dx.doi.org/10.12795/pixelbit.2014.i44.14 http://nadaesgratis.es/pedro-rey-biel/experimentos-en-clase http://dx.doi.org/10.4995/muse.2015.3769 http://dx.doi.org/10.1111/j.1467-8292.2006.00319.x http://dx.doi.org/10.1111/j.1467-8292.2006.00319.x http://dx.doi.org/10.2139/ssrn.2020176 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 107 district heating and cogeneration in the eu-28: current situation, potential and proposed energy strategy for its generalisation enrique rosales-asensio1 {enrique.rosales.uned@gmail.com} david borge-diez1,2 {david.borge@unileon.es} 1department of electric, electronic and control engineering, uned juan del rosal, 12 – ciudad universitaria. 28040 madrid spain phone +34-913-987-788 fax +34-913-986-028 2dpto. de ing. eléctrica y de sistemas y automática, universidad de león campus de vegazana s/n. escuela de ingenierías. 24071 león tf: (+34) 987 29 19 93; fax: (+34) 987 29 17 90 received: 2016-05-12; accepted: 2016-08-11 abstract yearly, eu-28 conventional thermal generating plants reject a greater amount of energy than what ultimately is utilised by residential and commercial loads for heating and hot water. if this waste heat were to be used through district heating networks, given a previous energy valorisation, there would be a noticeable decrease in imported fossil fuels for heating. as a consequence, benefits in the form of an energy efficiency increase, an energy security improvement, and a minimisation of emitted greenhouse gases would occur. given that it is not expected for heat demand to decrease significantly in the medium term, district heating networks show the greatest potential for the development of cogeneration. however, to make this happen, some barriers that are far from being technological but are mostly institutional and financial need to be removed. the purpose of this review is to provide information on the potential of using waste heat from conventional thermal power plants (subsequently converted into cogeneration plants) in district heating networks located in the eu-28. for this, a preliminary assessment is conducted in order to show an estimate of the cost of adopting an energy strategy in which district heating networks are a major player of the energy mix. from this assessment, it is possible to see that even though the energy strategy proposed in this paper, which is based on a dramatic increase in the joint use of district http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 108 heating networks and cogeneration, is capital-intensive and would require an annual investment of roughly 300 billion euros, its adoption would result in a reduction of yearly fuel expenses in the order of 100 billion euros and a shortening of about 15% of the total final energy consumption, which makes it of paramount interest as an enabler of the legal basis of the “secure, clean and efficient energy” future enacted by the eu-28 horizon 2020. keywords: district heating networks, cogeneration, energy potential, eu-28 nomenclature and units ahdd annual heat demand per dwelling dcns district cooling networks dhns district heating networks hhw heating and hot water hpr heat to power ratio las local authorities lf load factor http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 109 1. introduction from the simplified energy balances performed annually by eurostat (2013 is the last year for which data is available), conventional thermal power plants located in the eu-28 waste more energy in the form of residual heat than what finally residential and commercial buildings use for heating [1,2]. as is clear from (fig. 1), if this residual heat could be used previous energy valorisation from cogeneration in district heating networks, a noticeable decrease in the amount of imported fossil fuels and greenhouse gases emitted into the atmosphere would occur (in the theoretical background section, additional information on the relation between electricity and heat in a cogeneration plant is included), which would result in economic, environmental, and energy security benefits. figure 1. energy balance for the eu member states for the year 2013 [2] from (fig. 1), it is impossible to determine how much heat can be used. many power plants have to be necessarily located far from the thermal loads of cities (e.g., coal power plants are usually located close to a port to stock). however, in a future scenario, it could be more beneficial, from an economic perspective, to place those plants in the vicinity of the cities. as they are environmentally friendly they can be located closer to them. the old plants should be dismantled when their useful life ends. this supposes an additional expense of transporting coal or lignite in order to be able to use waste heat from power plants [3]. 0 20 40 60 80 100 total primary energy supply total heat consumption (tertiary sector) total heat consumption (residential sector) e j energy balance for the eu-28 member states for the year 2013 losses in the energy transformation sector losses in end use combustible renewables and waste solar/wind/geothermal/other hydro nuclear natural gas petroleum products coal and coal products transportation electricity heat http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 110 in (fig. 2), it can be seen as another way of representing the energy balance (which again shows the importance of waste heat from power plants) [4]. figure 2. power consumption for households located in the european union depending on the energy end-use. source: adapted from [5] as seen in (fig. 3), about 80% of the energy used in homes has as ultimate goal to provide heating and hot water; this means that, according to (fig. 2), there would be potentially about 15 000 pj/year (4100 twh) to be satisfied through the waste heat from power plants (note that this amount is less than the residual heat emitted by power plants located in the eu-28, 19 608 pj/year; fig. 2). finally, (fig. 4) shows the energy share for each district heating resource —it is possible to see that, for the eu, most of the heat comes from waste heat from power plants. 0% 10% 20% 30% 40% 50% 60% 70% 80% heating water heating cooking electricity (lighting and white goods) s h a re year energy consumption in dwellings by end uses 1990 2012 http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 111 (a) (b) figure 3. energy share (a) and generation for each district heating source (b) in the eu28 [10] 0% 20% 40% 60% 80% 100% 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 s h a re heat sources used in eu district heating networks fossil fuels, direct use renewables, direct use (biomass and waste heat) recycled heat, renewable cogeneration (biomass and waste heat) recycled heat, fossil cogeneration and industries 0 500 1000 1500 2000 2500 3000 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 p j/ y e a r energy sources in eu district heating networks fossil fuels (direct use) renewables (biofuels, geothermal, solar, etc.) + waste (direct use) waste heat recovered / recycled from biomass and waste (indirect use) waste heat recovered/recycled http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 112 despite these data, it should be noted that, due to the lack of energy matching throughout the year, as well as to the problem of the reject heat temperature (cogeneration can conduct an energy valorisation of this low-temperature heat in an economical way), not all of these thermal loads can be satisfied by the waste heat from thermal power plants. this makes necessary to estimate their real potential including assumptions as close as possible to the reality and to conduct an economic and environmental assessment (reduction of greenhouse gas emissions) that includes a generalised spread of district heating networks using as an energy source waste heat from conventional thermal power plants in the eu-28, which will be conducted in this review. the objective of this paper is to propose an energy strategy which enables, through the generalisation of the joint use of district heating networks and cogeneration in the eu-28, the fulfillment of the legal basis of the h2020 “secure, clean, and efficient energy” goals in a feasible, reliable, economically advantageous, and sustainable way. the method proposed here uses a systematic and easy approach (neither strong mathematical skills nor non-widely used software is required) and manages to propose an energy strategy for such a big region as eu-28 based mostly on available public data (with the exception of the plant data obtained through the gestis database by using the arcgis software). this is of great importance as other methods are network-oriented (district heating network); lack from massive surface applicability [6], being inadequate for such a big area as europe; are limited to industrial sites [7]; or present assessments which need a district heating network taxonomy [8], information not available in the vast majority of the cases. as a consequence, they are not useful for the purposes of this paper. to our knowledge, there is not a single scientific paper which specifically addresses the potential of the joint use of thermal power plant reject heat into district heating networks across the european union. thus, we offer new insights into this matter. in the first section, the first approach to the energy balance of the eu-28 member states is described, and the potential of using the waste heat from conventional thermal power plants is discussed. meanwhile, the second section will present the methodology used to develop the review conducted here. in the third section, studies evaluating the potential of using jointly district heating networks and cogeneration for different regions will be presented. in this section, information regarding prices of district heating networks, transport, distribution, and thermal storage, as well as a presentation of the case of copenhagen as an example of a city that has succeeded in combining electricity, district heating networks, natural gas, and waste management properly is presented. in the fourth section, different scenarios on the evolution of district heating networks and the estimated economically feasible conventional thermal power plant conversion potential will be presented. the fifth section is reserved for the discussion, and the sixth section presents the conclusion, in which the implications (restrictions and benefits) that http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 113 the generalisation of the joint use of cogeneration and district heating networks in the eu28 will be shown. 2. material and methods to give documentary strength to this review, it was necessary in the first instance to consult more than 100 scientific publications and/or gray literature in the field of cogeneration and district heating networks. while not absolute, it is a representative sample of the state of the art of the subject under study. subsequently, and to determine the degree of matching between the above publications and the perception of experts from various members of the eu-28, a total of nine experts from seven countries of the european union (see section "acknowledgements") were consulted through questionnaires, email correspondence and/or telephone conversations. once the scientific/gray literature and the experts were consulted, the third step taken was to use various databases to estimate the potential for the joint utilisation of district heating networks and cogeneration. among these databases, the most notable were eurostat, gestis database based on platts, and the european pollutant release and transfer register (european environment agency). the latter was essential for selecting those conventional ones potentially convertible in an economical way. the fourth step taken was to introduce some restrictions on the type of plant and the distance from it to the thermal load (nearest town). to estimate the cost related to the heat demand, member states of the eu-28 were firstly divided into zones, with the countries of central and southern europe assigned a cost on energy demand different from the one used only for the countries of southern europe. this discrimination is based on the fact that, obviously, the costs of providing heat to the thermal loads decrease as demand increases. finally, and in order to know the cost of providing heat through district heating networks for different typical climatic zones of the eu-28, [11] was consulted, which constitutes a supporting feature to the review presented here. to the heat demand for each consumption located in northern and central europe, a value of 2000 gwh/year was assigned, having been established the distance from the heat load at 75 km. meanwhile, for those loads located in the southern european countries, a type value of 750 gwh/year of heat and 400 gwh/year for cooling was established, being the distance in this case also 75 km. note that these conditions are much more restrictive than those described in [11]; we have proceeded in this way because such cases, although based on real plants, have better characteristics for the conversion to cogeneration plants than those that would be considered as “typical”. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 114 due to the fact that dhns and dcns lack a symmetrical end-use purpose (dhns are usually aimed at areas where there are residential buildings within a city, while dcns tend to be focused only on the central areas of the cities) and its proximity-to-load requirement, it is not possible to distribute cold appropriately from an economic perspective. district cooling is based on a difference of temperatures much smaller than district heating; see in theoretical background section for additional information on the distribution of heat and cold in modern heating/cooling systems) [12]. it would be impractical to transmit the cold at a distance of 75 km, so for the countries of southern europe, it has been supposed the use of absorption heat units that use the heat from district heating networks for cooling, not being, therefore, necessary to use pipes for transporting cold at a large scale. some restrictions have been imposed, e.g. only those thermal power plants with a rated power of over 300 mw located within 75 km of urban centers of more than 100 000 inhabitants are considered as valid for conversion, and some data such as the following was found out:  total installed capacity of thermal power plants using fossil fuels  total installed capacity of cogeneration  chp generation percentage  chp maximum installable capacity from thermal plants from this data, a feasible chp capacity that would comply with the aforementioned restrictions is obtained and presented in the results section. from these feasible convertible thermal power plants and from data obtained from the literature, usable heat, the investment needed, and fuel savings are obtained and shown in the results section. a summary of every stage of study and its relationship with the results section would be as follows: 1) scientific publicatons and/or gray literature review. 2) experts in the field of cogeneration and district heating networks consultation. 3) databases (eurostat, gestis, and the european pollutant release and transfer register) consultation.  the obtainment of conventional thermal combustion capacity, chp electrical capacity, share of chp electricity production, electricity generation from generation plants, heat generation from chp plants, final total energy consumption, total district heating network sales, and percentage of citizens served by district heating, is immediate from eurostat databases.  the methodology to assemble the data obtained from gestis and eprtr was as follows: a. plant data: gestis database based on platts http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 115 i. this data was accessed through the gestis database using the arcgis software. b. plant coordinates: eper/ e-prtr database i. it can be accessed through the european pollutant release and transfer register website. then following steps were undertaken “search the register”> “facility level”> “country”> “eu27”> “region”> “all regions”> “year”> “2013”> “industrial activity”> “energy sectors” > “thermal power stations and other combustion industries” > “no sub-activities” c. city coordinates: google earth i. no description needed 4) calculation of maximum conversion of chp by using total conventional thermal combustion capacity and chp electrical capacity; heat to power ratio by using heat generation from chp plants and electricity generation from chp plants; and maximum annual additional heat output by using the maximum conversion of chp calculated, the heat to power ratio, and supposing a load factor of 0.6. 5) restrictions on the type of plant (only conventional thermal plants with a rated power of over 300 mw are to be considered), distance (thermal plants shall be located at a distance below to 75 km of urban centers), population of urban centers (higher to 100 000 inhabitants) and heat demand in northern and central europe (2000 gwh/year) and heat and cooling demand in southern europe (respectively 750 gwh/year and 400 gwh/year) are applied. 6) calculation of dhns feasible potential by using the maximum annual additional heat and the fact that, after the restrictions imposed in 5), about 50% of the installed capacity of all conventional thermal power plants comply with those restrictions; calculation of conversion and district heating costs by classifying countries into regions (norhtern and central europe and southern europe) and taking into account heating (and cooling) costs in those regions. 7) once dhns' feasible potential is achieved and yearly savings in fuel are known, the cost of natural gas for domestic consumption is determined, and the usual fuel consumption per unit of heat supplied for boilers and cogeneration plants (for both cases separately) are assumed. through the cost difference between the two technologies, fuel savings are obtained. 8) finally, the results from previous steps (1 to 7) are analysed, and an energy strategy for the generalisation of district heating networks and cogeneration is proposed, achieving, consequently, the objectives of the research. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 116 3. theoretical background to know the true potential of any technology, in this case, the joint use of cogeneration and district heating networks, it is necessary to perform an assessment of its economic feasibility. in the existing literature, it is possible to find a relatively high volume of gray literature and scientific publications that carry out feasibility studies relating to the implementation of district heating networks. one of the first was conducted in 1979 by the ministry of energy of the united kingdom [13], where it was indicated that the joint use of cogeneration and district heating networks was competitive already at that time from an economic perspective in comparison to the fuels used to satisfy the heating demand (the largest infrastructural costs were offset by lower operating costs). this study recommended the development of a program that would implement district heating networks associated with cogeneration plants. although this publication saw the light more than three decades ago, the results remain equally valid because, firstly, the technical installation of the district heating networks have improved and the costs have decreased [14] and, secondly, comparatively fuel-related costs have increased more than the cost of the necessary infrastructure [15]. a computerised methodology to estimate the potential of the joint use of cogeneration and district heating for various discount rates was used [16]. as shown in (table 1), it is only in the uk with a peak demand of 60 gw [17] that there is a potential of 33 gw for this technology for a discount rate of 3.5%. due to similar weather conditions, this study is representative for much of northern and central europe. as is clear from (table 1) and (fig. 4), the choice of the discount rate is crucial to the feasibility of the project. table 1. potential joint use of cogeneration and district heating networks in the uk for various discount rates [16] ch/chp potential units discount rate 3.5% 6% 9% total net ch/chp potential for uk mwe 33 125 21 517 75 total electricity produced – 6897 4204 46 total heat sold gwh 189 472 123 119 518 primary energy saving gwh 230 358 149 686 630 in [18,19], the possibility of using cogeneration and district heating networks is studied again, reaching similar conclusions. as is clear from these publications, the joint use of these technologies provokes one of the greatest savings, in terms of co2 emissions, at one of the lowest costs per not emitted tonne [18,19] (fig. 4 and fig. 5). http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 117 figure 4. the cost of co2 reduction for the typical uk dwelling [18] -100 0 100 200 300 400 500 r e d u ct io n c o st s in c o 2 e m is si o n s co m p a re d t o t h e r e fe re n ce te ch n o lo g y ( g a s co n d e n si n g b o il e rs ) (€ / t o n c o 2 ) type of technology co2 reduction cost for a typical dwelling net present value per tonne of carbon dioxide saved using a discount rate of 3.5% net present value per tonne of carbon dioxide saved using a discount rate of 7% http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 118 figure 5. comparison of different energy solutions with low emissions of carbon dioxide [19] at a european level, [20] confirms the possibility of stopping to emit 400 mtco2 per year, which would mean a reduction of 9% compared to current emissions, if the penetration of district heating and cooling were to be doubled. to give documentary strength to this review, information regarding prices of district heating networks, transport, distribution, and thermal storage, as well as a presentation of the case of copenhagen as an example of a city that has succeeded in combining electricity, district heating networks, natural gas, and waste management properly is presented in this theoretical background section as follows. cogeneration power plant-district heating cogeneration districtheating (gas) district heating (biomass) cogeneration-district heating networks (biomass) solar thermal aerothermal heat pump geothermal pumps individual biomass boiler -500 0 500 1000 1500 2000 2500 3000 3500 0 50 100 150 c o 2 sa vi n g s (k g /y e a r) cost (€/mwh) comparison of low carbon heating options district heating reference individual energy sources with low or zero co2 emissions lineal (reference) http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 119 3.1 sankey diagram with the eu energy balance for 2012 figure 6. sankey diagram with the eu energy balance for 2012. source: adapted from [4] http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 120 3.2 historical of district heating prices, market share in (fig. 7), the development of district heating prices for different european countries is presented from 2008 to 2012, while (fig. 8) does the same with the percentage of users who have access to district heating networks for different countries that belong to the oecd, being of a 12.4% in 2012 for the member countries of the european union. finally, the contribution of cogeneration to the production of electricity for some oecd countries is presented in (fig. 9). figure 7. price of heat supplied by district heating networks placed in europe for the period 2008–2012 [21] 0 5 10 15 20 25 € /g j year district heating prices in europe, 2008 2012 2008 2009 2010 2011 2012 http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 121 figure 8. percentage of citizens who have access to district heating networks for different countries of the oecd (2012) [21] figure 9. contribution of chp in electricity production for different countries of the oecd (year 2012) [21] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% s h a re citizens with access to district heating networks (year 2012) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% a u st ri a b u lg a ri a c ro a ti a c ze ch r e p u b li c d e n m a rk e st o n ia f in la n d f ra n ce g e rm a n y ic e la n d it a ly s o u th k o re a la tv ia li th u a n ia t h e n e th e rl a n d s n o rw a y p o la n d r o m a n ia s lo v a k ia e sl o v e n ia s w e d e n s w it ze rl a n d u n it e d k in g d o m u s a s h a re share of cogeneration in electricity production (year 2012) http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 122 3.3 large scale heat transport technology heat transmission expenses through large-diameter pipes (even with distances being considerable) is low [22-25]. as an example, [22] shows that the cost of transmitting 2 gw of heat through large-diameter pipes at a distance of about 140 km would be about 0.25 €/kwh, this being due to the fact that the ability to transport heat through the pipe is proportional to the square of the diameter, while the cost of the pipe increases proportionally with the diameter [22]. therefore, the larger the piping and the demands of users, the greater the heat loss and the cost per unit of energy delivered (table 2). table 2. comparison of the heat transport cost depending on the diameter and length of the pipe [23] dn flow capacity price pr. km trench price pr. km pr. capacity price pr. km pr. annual sale heat loss pr. km mm m/s mw m€/km €/km/mw €/km/mwh/year %/km 100 200 300 400 500 600 700 800 900 1000 1.0 1.5 2.0 2.3 2.6 2.9 3.2 3.5 3.5 3.5 2 13 38 69 125 203 301 431 551 681 0.6 1.1 1.7 2.2 2.8 3.4 3.9 4.3 4.9 5.7 274 000 78 000 45 000 31 300 22 400 16 700 12 600 10 000 9000 8200 68.0 20.0 11.0 7.0 6.0 3.5 3.5 2.5 2.5 2.5 2.96 1.08 0.50 0.28 0.15 0.11 0.09 0.07 0.06 0.05 supply temperature (pre-insulated pipes) return temperature (designed maximum operating temperature = 130 ºc) pressure loss (variable flow pumps) 120 ºc 60 ºc 10 mm/m an example of large-scale heat transport technology can be cited the case of aarhus or that of prague. although the distance from the cogeneration plant to the center of the city of aarhus is only 20 km [26], the total length of the heat transport network rises to 130 km [27], representing the total length of all pipelines that are continuously connected. in (fig. 10), it can be observed that this large-scale heat transport network is part of a system consisting of generating plants (studstrupvaerket), closed vessels in which water or other fluid is heated for consumption peaks, and apparatuses for burning waste material (aca). http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 123 figure 10. heat transport system of aarhus [28] as for prague, heat transport piping from the cogeneration plant of melnik have a diameter of 1.2 m and a system length of 63.5 km [29]. as can be seen in (fig. 11), to cover much of the distance between melnik and prague, pipes are laid above the surface. figure 11. pipes for transporting large-scale heat (2xdn 1200) between melnik and prague [30] kedelanlaeg gjellerup forbraendingsanlaeg aca kedelanlaeg jens juuls vej halmvarmevaerk solbjerg kedelanlaeg arhusvaerket dong energy studstrupvaerket heat exchangers system pumping plant http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 124 3.4 heat distribution in modern district heating systems in modern systems, the heat distribution is almost always conducted through the use of preinsulated pipes that distribute hot water [31] (hwp are usually buried directly in the ground, so their installation is cheap, requiring no maintenance) [32]. furthermore, it is noteworthy that to obtain a higher efficiency regarding electricity, heat temperature extraction should be as small as possible, being as consequences that the heat is cheaper in fuel costs and the carbon dioxide emissions are lower [33]. these are the main reasons why there is a trend towards the use of water at a lower temperature—although, in a limited number of systems, such a small temperature as 45 ºc–55 ºc may be working properly [34], a temperature between 70 ºc and 80 ºc is a more “reasonable” one [35–37]. in some circumstances, and very especially in locations with flat terrain where district heating systems have a low-temperature design (below 90 ºc) [38,39], a direct connection from the dwelling to the principal district heating network can be performed [37]. this approach has the advantage of the ability to use lower temperatures and provide the highest efficiency possible, thus giving lower losses per unit of heat used. with a direct connection, it is possible to eliminate the cost of any heat exchanger [37]. a valve that reduces the pressure for each consumer to prevent from overpressure to end-user facilities is used [37]. direct connections, therefore, have many advantages because it is not necessary to allocate any investment for heat exchangers and it allows a more efficient operation of cogeneration regarding energy generation [37]. 3.5 thermal storage in accumulators for district heating systems currently, a very limited percentage of the eu-28 produced electricity is stored (predominantly in reversible hydroelectric power stations) [40,41], so any significant increase (such as by the use of wind energy for heating buildings) would be extremely costly [42,43] and would involve a considerable increase in the capacity of transmission and distribution [44]. the electricity storage through isentropic heat pumps is currently the only long-term electricity technology storage that can be used in the electrical network in an economical way, having a cost of about 35 €/kwh [45]. meanwhile, the thermal storage is considerably cheaper as the cost varies in a band range of 0.1 and 10 €/kwh, in this case not needing any upgrading of the electricity grid [46]. thermal storage allows the cogeneration plant to generate electricity whenever and to increase system flexibility due to the fact that thermal storage heat output change is almost immediate and that, unlike backpressure cogeneration plants that have to generate heat even when electricity price is high, condensing-extraction cogeneration plants have increased flexibility thanks to the possibility of generating electricity exclusively [47,48]. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 125 in (fig. 12) and (fig. 13) are respectively shown the heat storage directly and indirectly connected to the district heating network. figure 12. thermal storage directly connected to a district heating system [49] figure 13. accumulator with hydraulic separation from the district heating system [49] return water forward water cogeneration plant heat accumulator heat consumers network pumps heat accumulator heat consumers expansion vessel cogeneration plant http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 126 3.6 fuel used invariability and efficiency with the exception of those buildings with a large number of computer equipment and due to climatological reasons, domestic air conditioning, in some parts of europe, is still being considered a luxury (meanwhile heating is seen as essential). district cooling systems are similar to district heating networks in the sense that a fluid is distributed and conducted through tubings [50]. however, there are some major distinctions. for example, there are differences between the residential and commercial sectors, which for the latter, are dominated by district cooling networks [22]. this situation can be given the following factors:  air conditioning systems are profusely used in the services sector, whereas they have a limited penetration in dwellings [22].  the size of the district cooling network is a point to be considered so that it is viable from an economic perspective [51].  normally, there is a refrigeration density need requirement in order to be profitable to use as a district cooling system, being compulsory that the users are located close to each other [52].  another factor is the limited ratio energy/liquid quantity in comparison with dhns [53] because of the lower temperature differences. for instance, in district cooling systems, flow temperature is around +5 ºc, and back temperature is +15 ºc (temperature difference of about 10 ºc). meanwhile, in a district heating system, flow temperature would be around 90 ºc, and back temperature is around 40 ºc (therefore, there is a temperature difference of about 50 ºc) [12,53]. this is the reason why pipes used in district cooling networks are normally higher than those used in district heating networks and thus have a higher cost (fig. 14). http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 127 figure 14. comparison between the pipe diameters taking into account the differences between flow and back temperatures [53] most cooling applications make use of compressors driven by motors [54–57], where their coefficient of performance depends heavily on the temperature difference [58] (the better the coefficient of performance, the lower the temperature of the heat sink is) [59], as well as the type of compressor, the rating power of the cooling unit, and whether water or air is used to carry out this task [56]. for its part, the absorption chillers or the heat pumps operate under a different principle [60,61]. here, instead of using the mechanical power from a motor, they make use of the heat to drive the cycle [62-65]. it should be noted that the coefficient of performance for this technology lags behind that of compressors driven by electricity [56,66] and varies for commercially available lowtemperature district cooling systems from 0.65 to 1.2 [53] —that is, a unit of heat at low temperature is capable of providing between 0.65 and 1.2 units of low-temperature refrigeration depending primarily on the number of times that the heat entering the chiller is internally used [53]. traditionally, and due to the elevated temperature's utilisation (those that occur with lowpressure steam) [67], absorption chillers lower the coefficient of performance and the refrigerating capacity greatly as the inlet temperature of the heating medium decreases (fig. 15). this has the important effect that for a given cooling capacity, the price of the device can become excessive (fig. 15) —nevertheless, these devices are often in district cooling systems [68]. 0 10 20 30 40 50 60 0 100 200 300 400 500 600 t e m p e ra tu re d if fe re n ce ( °c ) pipe diameter (mm) comparison between diameters (v < 2.5 m/s) 5 mw 20 mw http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 128 figure 15. cooling capacity of absorption chillers in relation to water temperature [69] in (fig. 16), it is shown that for 1000 hours of operation at full load, cooling costs by absorption (excluding the costs of constructing the networks) are significantly higher than the costs incurred with the technology of refrigeration through compression. 32 ºc31 ºc29.5 ºc27 ºc 0 10 20 30 40 50 60 65 70 75 80 85 90 95 100 c o o li n g c a p a c it y ( k w ) heat medium inlet temperature (ºc) cooling water inlet temperature http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 129 figure 16. comparison of refrigeration costs through compression and refrigeration through absorption (excluding construction costs of the networks) for power ranging from 300 kw to 500 kw and 1000 hours of operation at full load [70] as it is clear from (fig. 17) and as far as installation costs are concerned, compression refrigeration is significantly cheaper than refrigeration through absorption. however, as the installed power increases, the differences decrease considerably. 91 66 161 95 0 50 100 150 200 250 300 absorption refrigeration unit (centralised) compression unit (centralised) absorption refrigeration unit (decentralised) compression unit (decentralised) € /m w h compression refrigeration and absorption refrigeration cost comparison http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 130 figure 17. comparison of the investment cost for different refrigeration machines [70] 3.7 fuel used invariability and efficiency normally, when using cogeneration power plants and sacrificing a unit of electricity, it is possible to obtain about eight times this energy as heat (in contrast, for example, when using a heat pump, it would only be possible to get that amount multiplied by three) [71– 74]. when a plant starts operating as a cogeneration plant and its electricity production falls, another condensation plant which is not operating at maximum output will be responsible for increasing the power in order to compensate for this change, thus resulting in a modification of fuel consumption [75]. by definition, marginal power plants are the ones that dictate the price in every hour to all technologies [76] and, in this case, combustible used to generate electricity will remain constant [77] and at a constant efficiency [78]. 3.8 cogeneration, district heating networks and smart cities: the case of copenhagen the european commission's “smart cities and communities” initiative has already expressed in the “strategic energy technology plan” (set-plan) an interest in positioning a number of cities at the forefront of combating and minimising co2 emissions [79,80]. 0 200 400 600 800 1000 1200 1400 0 100 200 300 400 500 600 0 20000 40000 60000 80000 100000 120000 s p e ci fi c in ve st m e n t (€ /k w ) cold capacity (kw) in ve st m e n t (€ ) investment cost comparison investment absorption (linear regression) investment absorption (curve fitting) investment compression specific investment compression investment compression (linear regression) specific investment compression (curve fitting) investment absorption specific investment absorption http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 131 district heating and cooling networks will be smarter as they make the electric grid smarter [81,82]. consequently, it will be especially important that the heating and cooling have heat reservoirs [83,84] so that the electric grid reacts quickly to price signals and can stabilise itself when there are fluctuating renewable energy sources. in denmark, heating networks are the backbone of the energy system, providing enough flexibility to integrate renewable energies (especially wind) into the electric grid in a safe way [85]. one of the objectives of the eu-28 regarding energy is precisely to make these smart cities adopt the danish energy strategy [86,87]. renewable energy can generate electricity when prices in the electricity market are high, and even though it continues to produce electricity at night when demand is low, the stored heat may cover peak consumption that occurs during the morning. with respect to the loads of heating/cooling, district heating networks and intelligent cooling contribute to smoothing heating and cooling peak demand, thereby reducing peak production [88]. undoubtedly, copenhagen is the world region that more has managed to integrate electricity, district heating networks, natural gas, and waste management. among its highlights may be mentioned the following:  98% of the population of copenhagen is, in those areas where this infrastructure is present, connected to district heating networks [89].  las have developed heating plans for the entire region, dividing the areas by optimal technologies, taking into account economic criteria established at a national level by the energy authority [90–92].  municipalities have created companies with a municipal participation (ctr and veks) to develop the heat transport system, while companies entirely belonging to the municipality are in charge of distributing this heat.  for its part, the waste management company (vestforbrænding) is in charge, apart from the recycling, of treating the waste produced by 20 municipalities [93] to an optimum level to introduce cogeneration in large incineration plants to ensure that all the material that cannot be recycled is used as energy. in denmark, the providing of the service of district heating depends on companies that are themselves local monopolies, and that typically have the necessary infrastructure for the generation, supply and sale of heat to the end user [94]. in contrast to the companies supplying gas and electricity, for these local monopolies, any benefit from the sale of heat will not be permitted to be obtained [94]. this means that only the necessary costs of construction, operation, and maintenance of district heating networks can finally be included in the fees [95]. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 132 to get a visual idea of how the system of generation and heat transport of the city of copenhagen is, in (fig. 18) a map depicting the generation and transmission system operated by veks, ctr, and vestforbraendings is shown. figure 18. map from veks showing the heating system in copenhagen (only local heating systems in the area of veks are presented) [96] 4 results given its economic and environmental benefits, the joint use of cogeneration and district heating networks should experience significant growth in coming years (fig. 19) [97]. for district heating networks to achieve a penetration of over 90% in a big city, several decades may elapse, in the case of copenhagen, it took more than 40 years [98], depending largely on the commitment of the authorities [99]. theoretical background section includes additional information concerning the case of copenhagen. note that the actions to take ishoj albertslund gladsaxe vf kara/noveren hcv heat pipeline district heating networks-steam area district heating area operated by vf district heating area operated by ctr project for the conversion from natural gas to dhns (veks) district heating area operated by veks transportation heat system under construction incineration plant cogeneration plant smv rlv amf amv kkv avv v hoje-taastrup greve solrod koge roskilde ballerup tarnby http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 133 for the widespread introduction of district heating networks from the market entry stage (going through the market penetration phase) to the stage of market maturity will depend, in any case, on the casuistry of each location; therefore, an adaptation of the measures to be taken depending on specific circumstances will be necessary. figure 19. different scenarios for the heat supplied by district heating networks and cogeneration plants located in the eu until 2050 [97] to know the potential costs and energy savings, it has been supposed as a target that 50% of all domestic and commercial heating loads from the eu-28 can be satisfied from waste heat from conventional power stations and district heating networks. these results are compared country by country assuming power plants are converted into cogeneration plants. 0 200 400 600 800 1000 1200 1400 1980 1990 2000 2010 2020 2030 2040 2050 2060 e n e rg y ( t w h /y e a r) past and forecasted heat supplied by district heating networks in the eu reference 1b current policies 2 energy efficiency 3 diversified 4 high renewable energy penetration 5 delay in the implementation of carbon capture and storage 6 low nuclear energy scenario iea eurostat eurostat including industrial sector http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 134 the estimate has been made as follows: 1. table 3 gives the total installed capacity of thermal power plants using fossil fuels by member state in column a. 2. column b shows the total installed capacity of cogeneration per member state (table 3). 3. column c presents the combined heat and power (electrical) generation percentage from each member state (table 3). 4. column d (table 3) shows the installable capacity in the eu-28, assuming that 100% of conventional thermal power plants that use fossil fuels are converted into cogeneration plants (although not all the heat is used because the heat generated is higher than the actual demand). table 3. installed capacity per member state to achieve full conversion of conventional thermal power plants in cogeneration plants. source: adapted from [100,101] a b c d total conventional thermal combustion capacity [100] (gw) chp electrical capacity [101] (gw) share of chp in electricity production [101] maximum conversion of chp (gw) 2013 2013 2013 vision at – austria 6.3 4.4 14.4% 1.9 be – belgium 7.6 2.3 15.2% 5.3 bg – bulgaria 4.6 1.2 8.5% 3.4 cy – cyprus 1.6 0.0 1.4% 1.6 cz – czech republic 10.5 4.6 13.7% 5.9 de – germany 82.0 27.0 12.4% 55.0 dk – denmark 7.6 5.7 50.6% 1.9 ee – estonia 2.6 0.5 9.3% 2.1 es – spain 49.8 4.0 8.5% 45.8 fi – finland 8.4 6.3 34.5% 2.1 fr – france 23.0 5.5 2.4% 17.5 gr – greece 10.7 0.6 3.4% 10.1 hu – hungary 5.8 1.6 13.4% 4.2 http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 135 hr – croatia 1.7 0.7 12.6% 1.0 ie – ireland 6.0 0.3 7.8% 5.7 it – italy 70.6 7.6 12.7% 63.0 lt – lithuania 2.8 1.2 35.0% 1.6 lu – luxembourg 0.5 0.5 14.7% 0.0 lv – latvia 1.2 1.0 38.3% 0.2 mt – malta 0.6 0.0 0.0% 0.6 nl – netherlands 21.4 9.2 34.5% 12.2 pl – poland 28.2 8.3 15.9% 19.9 pt – portugal 6.4 1.4 13.8% 5.0 ro – romania 9.0 2.2 11.2% 6.8 se – sweden 6.6 4.5 10.2% 2.1 si – slovenia 1.2 0.3 7.1% 0.9 sk – slovakia 2.8 4.4 77.0% 0.0 uk – united kingdom 59.0 6.2 5.5% 52.8 eu–28 438.5 111.5 328.6 column e (table 4) represents the additional energy obtained from converting all conventional power plants placed in the eu-28 into cogeneration plants, as shown in column d (table 3). it is supposed that once the conventional thermal power plant has been converted, lf is 0.6. hpr calculations were carried out, taking into account columns a and b (table 4), which correspond to the statistics available from eurostat for the main power producers and not for the one of the auto-producers, and assuming that the heat of a chp plant will feed district heating networks. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 136 table 4. calculation of the annual heat obtained from the conversion of conventional thermal power plants located in the eu-28. source: adapted from [101] a b c d e electricity generation from chp plants [101] (twh) heat generation from chp plants [101] (twh) heat to power ratio maximum conversion of chp (gw) maximum annual additional heat output (twh) 2013 2013 2013 vision vision at – austria 9.9 30.8 3.1 1.9 31.0 be – belgium 12.7 7.6 0.6 5.3 16.7 bg – bulgaria 3.7 11.2 3.0 3.4 53.6 cy – cyprus 0.1 0.0 / 1.6 8.4 cz – czech republic 12.0 33.6 2.8 5.9 86.8 de – germany 78.7 181.7 2.3 55.0 664.9 dk – denmark 17.6 28.7 1.6 1.9 16.0 ee – estonia 1.2 3.5 2.9 2.1 32.0 es – spain 24.1 48.6 2.0 45.8 481.4 fi – finland 24.3 69.8 2.9 2.1 32.0 fr – france 14.0 41.8 3.0 17.5 275.9 gr – greece 2.0 2.9 1.5 10.1 79.6 hu – hungary 3.9 7.5 1.9 4.2 41.9 hr – croatia 1.7 3.7 2.2 1.0 11.6 ie – ireland 2.0 3.4 1.7 5.7 51.0 it – italy 36.7 59.1 1.6 63.0 529.8 lt – lithuania 1.7 4.3 2.5 1.6 21.0 lu – luxembourg 0.4 0.9 2.3 0.0 0.0 lv – latvia 2.4 3.1 1.3 0.2 1.4 mt – malta 0.0 0.0 / 0.6 3.2 nl – netherlands 34.8 60.5 1.4 12.2 89.8 http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 137 pl – poland 26.1 71.5 2.7 19.9 282.4 pt – portugal 7.2 19.0 2.6 5.0 68.3 ro – romania 6.6 16.1 2.4 6.8 85.8 se – sweden 15.6 45.9 2.9 2.1 32.0 si – slovenia 1.2 3.0 2.5 0.9 11.8 sk – slovakia 22.2 7.7 0.3 0.0 0.0 uk – united kingdom 19.7 39.6 2.0 52.8 555.0 eu–28 390.1 789.4 / 328.6 3563.3 we have proceeded in this way due to the fact that: 1. the vast majority of non-cogeneration thermal power plants belong to large power producers and not to auto-producers 2. and that the uncertainty of connecting the waste heat from a power plant to a district heating network is much lower than the one that would be obtained if industrial waste heat were to be used, where other factors, such as the possibility that a given industry remains at least 60 years working in a given location, should be evaluated. because district heating networks always will have availability restrictions, this paper has centered on hhw requirements for tertiary and residential sectors. an analysis of the technical potential of the district heating networks is shown in (table 5). as indicated earlier in this section, it has been assumed that district heating networks can supply roughly 50% of all heating requirements, domestic and commercial ones. in (table 5), it is possible to see that it has been supposed that the potential feasible for district heating networks (column e) is 50% of the maximum theoretical heat, a scenario in which all domestic and commercial loads have access and are satisfied through district heating networks (column c). http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 138 table 5. analysis of the technical potential of district heating networks. source: adapted from [2,11,102,103] a b c d e f g final total energy consumption [2] (pj) total district heating networks sales [102] (pj) percentage of citizens served by district heating [102,103] maximum annual additional heat (pj) dhns feasible potential (pj) region conversion cost + dhns cost [11] (annual cost of investment in milliards of €) 2013 2013 2013 vision vision vision at – austria 1170.2 80.7 24% 111.6 55.8 n–c 2.4 be – belgium 1457.1 21.0 3% 60.1 30.1 n–c 1.4 bg – bulgaria 367.2 18.0 18% 193.0 96.5 n–c 4.2 cy – cyprus 67.6 0.0 0% 30.2 15.1 s 1.0 cz – czech republic 1000.0 89.4 38% 312.5 156.3 n–c 6.8 de – germany 9096.0 254.8 12% 2393.6 1196.8 n–c 52.6 dk – denmark 594.3 105.5 63% 57.6 28.8 n–c 1.4 ee – estonia 120.2 23.0 62% 115.2 57.6 n–c 2.6 es – spain 3397.0 0.0 0% 1733.0 866.5 s 52 fi – finland 1030.6 114.2 50% 115.2 57.6 n–c 2.6 fr – france 6366.3 96.1 7% 993.2 496.6 n–c 20.2 gr – greece 642.2 1.0 0% 286.6 143.3 s 8.6 hu – hungary 628.8 31.0 15% 150.8 75.4 n–c 3.4 hr – croatia 243.4 9.7 10% 41.8 20.9 n–c 1.0 ie – ireland 449.6 0.1 0% 183.6 91.8 n–c 4.0 it – italy 4969.6 33.1 6% 1907.3 953.7 s 57.2 lt – lithuania 198.3 27.1 57% 75.6 37.8 n–c 1.6 lu – luxembourg 172.9 2.0 6% 0.0 0.0 n–c 0.0 lv – latvia 161.4 21.5 65% 5.0 2.5 n–c 0.2 mt – malta 21.0 0.0 0% 11.5 5.8 s 0.4 http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 139 nl – netherlands 2141.8 26.1 4% 323.3 161.7 n–c 7.2 pl – poland 2655.0 248.7 53% 1016.7 508.4 n–c 22.4 pt – portugal 663.5 9.0 2% 245.9 123.0 s 7.4 ro – romania 911.0 54.0 23% 308.9 155.0 n–c 6.8 se – sweden 1322.8 176.0 52% 115.2 57.6 n–c 2.6 si – slovenia 200.9 7.7 15% 42.5 21.3 n–c 1.0 sk – slovakia 454.9 82.7 35% 0.0 0.0 n–c 0.0 uk – united kingdom 5712.2 41.5 2% 1998.0 999.0 n–c 44.0 eu–28 46 214.5 1573.9 11.8% 12827.9 6414.9 315.0 finally, the costs related to the conversion of conventional thermal power plants into cogeneration plants and the investment required for the expansion of the district heating networks are estimated in column g (table 5). expansion costs include costs relating to both heat transport as those incurred in the district heating network itself. it should be noted that the cost of network expansion will vary depending on the region under analysis. locations where ahdd is high incur smaller expenses (for district heating) than those for regions with a low heat demand, where a lower number of buildings will need to be connected to the district heating network. for those projects located in northern and central europe, denoted respectively under the “n-c” acronym (see column f of table 5), it has been assumed that they incur an annual heat demand cost of 45 €/gj [8]. the costs of providing heat and cold jointly to those projects located in southern european countries, denoted by the acronym “s” (column f of table 5), were assumed to be 60 €/gj [8]. note, however, that carrying out an appropriate assessment of the factual potentiality of associating current thermal generating plants to dhns (district heating networks) requires a detailed analysis of each particular case, in which various techno-economic parameters are taken into account –see [11]. that is an extra feature to this paper, as this analysis is beyond the scope of this paper. as can be seen in column c (table 5), about 12% of the citizens of the eu-28 have access to district heating networks, with significant differences between countries. it is possible to find cases where the contribution of district heating networks to heat demand is virtually zero, as in the case of spain, and others where the weight of district heating networks is high (nearly 2 out of 3 danish have access to this technology). a historical of the heat price provided by the district heating networks, the contribution of cogeneration to produce http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 140 electricity, and the percentage of citizens who have access to district heating networks is presented in (fig. 6), (fig. 7), and (fig. 8); theoretical background section. it is possible to see that the length of the pipes for transporting heat from the thermal power station located at wilhelmshaven (757 mw) to the city of oldenburg is of more than 60 km and that this conversion would be feasible from an economic point of view. so it can be assumed conservatively that about 50% of all the electric power allocable to conventional thermal power plants is located within a reasonable distance to convert these plants in cogeneration plants [11]. this percentage has been obtained – from the database (e-ptr) [104] – after imposing as constraints that only will be valid for conversion to cogeneration plants those conventional thermal power plants with a rated output of over 300 mw placed at a distance of less than 75 km from towns of more than 100 000 inhabitants, which results in about 23% (this represents a 56% of the installed capacity of this technology) of all conventional thermal power plants not yet converted to cogeneration plants placed in the eu-28 (in 2013) to comply with these restrictions (see table 6). this is the reason why in this paper it has been set as a goal that 50% of the thermal load (domestic and commercial) of the eu-28 is satisfied through the joint use of cogeneration plants and district heating networks. table 6. conventional thermal power plants (not converted to cogeneration power plants) with a rated output of over 300 mw placed at a distance of less than 75 km from towns of more than 100 000 inhabitants. source: database of e-ptr [104] number of conventional thermal plants (unconverted) near urban centers. (distance <75 km and p> 300 mw) total conventional thermal power plants (unconverted) near urban centers. (gwe) total number of conventional thermal power plants 393 246.5 1690 according to (table 5), if all potentially convertible thermal power plants, those with a rated power of over 300 mw located within 75 km of urban centers of more than 100 000 inhabitants, were converted into cogeneration plants and they would have access to district heating networks, about 6400 pj of heat could be used, with a low carbon footprint, if an investment of some 315 billion euros will be made. after having consulted the database [104], it was found that in most countries, the limiting factor is, rather than lack of heat loads, the availability of thermal plants. however, in some locations, it was possible to observe the opposite, as groups of thermal power plants located relatively remote or at uneconomical distances to transport heat happened. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 141 to get a clearer idea of the economic benefits of investing in district heating networks, let us assume that fuel consumption per unit of heat supplied is 1.1 in the case of boilers [105], being meanwhile 0.125 in the case of cogeneration plants [105]. assuming that the cost of natural gas for domestic consumers was about 15 €/gj for the second half of 2014 [106], yearly savings of 93.6 billion euros would be achieved only in fuel for an annual investment 315 billion euros. this estimate is only intended to give context about the treated magnitudes; in any case, it needs further evaluation case by case to debug this rough approximation; see [11]. it should be noted that, obviously, this amount is in itself an upper limit, which is subject to considerable uncertainty and that some of the conventional thermal plants that here have been supposed to be convertible eventually will not. however, if a greater commitment at the european level in promoting the joint use of district heating networks and cogeneration would be obtained, in the long term, it would be possible to approach the upper limit as new cogeneration power plants were built close to the thermal loads. certainly, the barriers that the joint use of cogeneration and district heating faces are not technological but institutional and financial; see [107] to know these barriers in detail. 5 discussion as in most of the district heating networks of the member countries of the european union that belonged to the former ussr and in the early stages as compared to the rest of the eu, it was common practice to use auxiliary boilers that only generated heat without there being at first a willingness to convert them into cogeneration plants [108-111]. commonly, they used cheap or subsidised hydrocarbons [112] that had difficulties in burning [113]. with the passage of time, a number of auxiliary boilers from some cities were connected to other thermal generating plants that could use their reject heat. in prague, what really triggered the connection of the poříčí ii power plant with the associated desulfurised emissions to the existing district heating network, even taking into account that the pipes carrying steam and hot water should have a length of about 40 km each one, was the realisation of the fact that there was a need to desulfurise emissions from plants that burned dirty fuels [114]. it might be thought that the joint use of cogeneration and district heating networks would only be profitable in the northern european states due to their cooler climate and greater thermal requirements, rather than be consequently applicable to the rest of the eu-28. however, the fact that homes in these countries have better insulation has as a consequence that the specific thermal demands are lower (fig. 20). moreover, it is unlikely that the thermal loads in buildings will decrease so that the joint use of cogeneration and district heating networks becomes non-viable [115]. on the other hand, usual reductions of heat losses in buildings are not as high as expected in the first instance (the users to see an http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 142 improvement in the efficiency of an energy service that leads to a reduction of the actual cost of the service tend to consume more, causing the known rebound effect) [116]. this effect, although difficult to quantify (but real), might result in a decrease in thermal load reductions of between 10% and 30% [116]. the vision of the not or, at least, modest changes in heat demand is based on forecasts conducted by each of the member states. as an example, it is presented the case of ireland (table 7), where it is possible to see that for the period 2008–2020, it is expected to increase the thermal energy consumption by only a 0.3% [117], very much in line with other countries eu-28. figure 20. useful energy consumption for heating in equivalent kilograms of oil per m2 and heating days with respect to the percentage of centralised heating for some eu member states [118] 1 1,5 2 2,5 3 3,5 4 4,5 40% 50% 60% 70% 80% 90% 100% u se fu l co n su m p ti o n p e r m 2 ( k o e /m 2 /d d ) % central heating useful energy consumption for heating spain bulgaria slovakia portugal norway netherlands slovenia czech republic latvia sweden italy france denmark irelandpoland austria finland uk germany http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 143 table 7. thermal energy consumption by sector for ireland (2008-2020) [117] total final demand (ktoe) growth (%) average annual growth (%) sectoral shares (%) e n d -u s e 2008 2012 2020 08-20 0820 0812 12-20 2008 2010 2020 industrial 139 148 150 0.6 1.5 0.2 2.5 2.8 2.6 residential 44 43 41 -0.5 -0.5 -0.5 0.8 0.8 0.7 services 15 14 14 -0.5 -1.0 -.02 0.3 0.3 0.2 total renewable energies 198 205 205 3.7 0.3 0.9 0.0 total consumption 5502 5209 5718 4.0 0.3 -1.3 1.2 heating from renewable energies 3.6 3.9 3.6 furthermore, there are several reasons why it is easier to increase generation from renewable sources (particularly wind) than from cogeneration. in the eu-28, wind energy (along with hydroelectric power) is the dominant energy source from renewable origins [119,120], being necessary in this case only that the promoter locates a location and would obtain the various needed permits. because there are usually many locations where to deploy a project, there are high chances that the same is carried out. by contrast, cogeneration is undoubtedly much more restricted because it inevitably has to be used in cities and industries —obviously, the difficulties associated with implementing a district heating network to an entire city are of a different order than those found in obtaining permission for implementing a wind farm. the cogeneration plants are also subject to more restrictions than those plants that produce heat and electricity separately. this is because although the electricity is cheap and easy to transport over long distances (in theoretical background section it is possible to find additional information concerning large-scale heat transportation), heat needs to be generated locally and can only be transported to much shorter distances. consequently, a heat-generating unit will always be connected to the end user; meanwhile, electricity will feed to the electric grid in general. in other words, the cogeneration plant will have to compete in the heat market with, for example, heat boilers or heat pumps, and in the electricity market with more flexible power plants [121]. despite these restrictions, it is to be emphasised that a higher chp and dhns generalisation (and their associated thermal storage; see theorecital background sectio) is of vital importance in reducing overall energy consumption in the eu-28, which obviously http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 144 will impact co2 emissions. this view has been the one that has enabled denmark, along with increased energy requirements of buildings, to prevent an increase in its energy consumption over the last 30 years despite having experienced an increase of 70% of gdp during this period [122]. literature findings are certainly inconclusive in what district heating networks and cogeneration potential are respected at a pan-european level. in fact, even though some studies have been undertaken for specific countries or regions, such as sweden [123], latvia [124], the baltic states [125], or usa [126], scientific literature has not given the same attention to the potential and the economic and environmental benefits resulting from the implementation of an energy strategy based on a massive increase of the joint use of district heating networks and cogeneration in the eu-28. as a result, and due to the aforementioned lack of attention, a comparison between this paper and other scientific publications is, at least, problematic (in particular, and for southern europe countries, attention given has been virtually zero). however, it is possible to indicate that one of the main strengths of this paper is that, by using a simple methodology, its approach and perspective provide both novel results for the scientific literature and new insights to the european union as a whole about the true potential of this underexploited technology. paradoxically, this strength of the simple methodology is at the same time a shortcoming due to inaccuracies in the district heating and cogeneration potential assessment, and further research on this issue would be interesting for refinement purposes. finally, and as another research extension, it would be interesting to evaluate the applicability of other reject heat from other types of thermal power plants such as biomass and nuclear ones for district heating purposes. 6 conclusions about half of the installed electric capacity in the eu-28, from conventional thermal power plants, is located at an appropriate distance to be able to convert these plants into cogeneration plants and to use their wasted heat in district heating networks. taking into account the maximum penetration that is economically feasible, from an annualised investment in infrastructures of 315 billion euros, the costs, assignable to fuel, would be reduced by 95 billion euros per year and would save about 6400 pj of primary energy. this represents about 15% of the total final energy consumption in the eu-28 in 2013 (46 214.5 pj). all these savings are achieved from a by-product, such as the waste heat from conventional power plants, currently not utilised in order to meet any thermal load due to their low temperature (about 30 °c). if the energy vision, proposed in this paper, were implemented, it would quadruple (from 12% to 50%) the access of the citizens of the eu28 to the technology of district heating networks. this estimate has important political implications as a generalisation of the joint use of cogeneration plants to raise the http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 145 aforementioned temperature, and district heating networks would entail some economic, environmental, and energy security benefits that are necessary for the fulfillment of the legal basis of the h2020 societal challenges “secure, clean, and efficient energy” in a sustainable way. it has been consequently justified that, definitely, the barriers that the proposed new vision face are not of a technological nor of an economic nature but are institutional and financial. it is worth to remark the forecast that, if a policy, committed to the development of district heating networks and cogeneration, is obtained, it would imply that over the next decades more and more power plants will be built in proximity to thermal loads and decommission those located remotely. finally, and as a possible research extension, it would be interesting to consider reject heat from other thermal power plants, such as biomass and nuclear ones. even though the latter ones should be considered cautiously due to their typical locations (normally far from heat loads), rejected heat from nuclear power plants might represent a significant increase in the joint use of chp and dhns, as currently they represent about 30% of the electricity provided in the eu-28. acknowledgements we are grateful to the experts contacted who gave us their support, in particular and especially to poul alberg østergaard (aalborg universitet), capezzali massimiliano (école polytechnique fédérale de lausanne), kevin sartor (université de liège), erwin cornelis (vito nv), chiara wolter (ambiente italia), alexandra tudoroiu (cogen europe), olu ogunbadejo (department for environment, food & rural affairs), aleksandrs zajacs (riga technical university), and carsten magass (forschungszentrum jülich gmbh). without their participation, this article would not have been possible. 7 references [1] ecoheatcool and euroheat & power. the european heat market: final report. brussels: euroheat & power; 2006. [2] eurostat. simplified energy balances – annual data. luxembourg: eurostat. accessed at <http://ec. europa.eu/eurostat/en/web/products-datasets/-/nrg_100a> on october 3, 2015. [3] the connecticut academy of science and engineering. a study of the feasibility of utilizing waste heat from central electric power generating stations and potential applications. hartford: the connecticut academy of science and engineering; 2009. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 146 [4] iea. world balance. paris: iea. accessed at <http://www.iea.org/sankey/> on september 29, 2015. [5] european environment agency. progress on energy efficiency in europe; 2015. accessed at <http://www.eea.europa.eu/data-and-maps/indicators/progress-on-energyefficiency-in-europe-2/assessment> on october 7, 2015. [6] bordin c, gordini a, vigo d. an optimization approach for district heating strategic network design. european journal of operational research 2016 (in press). http://dx.doi.org/10.1016/j.ejor.2015.12.049 [7] cooper sjg, hammond gp, norman jb. potential for use of heat rejected from industry in district heating networks, gb perspective. journal of the energy institute 2016;89(1):5769. http://dx.doi.org/10.1016/j.joei.2015.01.010 [8] köfinger m, basciotti d, schmidt rr, meissner e, doczekal c, giovannin a. low temperature district heating in austria: energetic, ecologic and economic comparison of four case studies. energy 2016 (in press). [9] international energy agency. energy technology perspectives 2015: mobilising innovation to accelerate climate action. paris: organisation for economic and co-operation and development; 2015. [10] euroheat & power. district heating and cooling–statistics. brussels: euroheat & power; 2015. [11] colmenar-santos a, rosales-asensio e, borge-diez d, collado-fernández e. evaluation of the cost of using power plant reject heat in low-temperature district heating and cooling networks. applied energy 2016; 15:8. http://dx.doi.org/10.1016/j.apenergy.2015.10.161 [12] boysen h. hydraulic balance in a district cooling system. nordborg: danfoss district energy; 2003. [13] combined heat and power group. combined heat and electrical power generation in the united kingdom: report to the secretary of state for energy. london: h.m. stationery off.; 1979. [14] euroheat & power. district heating in buildings. brussels: euroheat & power; 2011. [15] international energy agency. cogeneration and district energy. paris: international energy agency; 2009. [16] department for environment food and rural affairs. analysis of the uk potential for combined heat and power. london: queen’s printer and controller of hmso; 2007. http://dx.doi.org/10.4995/muse.2016.6339 http://dx.doi.org/10.1016/j.ejor.2015.12.049 http://dx.doi.org/10.1016/j.joei.2015.01.010 http://dx.doi.org/10.1016/j.apenergy.2015.10.161 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 147 [17] osborne m. making electricity transmission smarter. cambridge: cambridge university energy network; 2013. [18] kelly sj, pollit mg. an assessment of the present and future opportunities for combined heat and power with district heating (chp-dh) in the united kingdom. energy policy 2012;38(11):6936-6945. http://dx.doi.org/10.1016/j.enpol.2010.07.010 [19] woods p, turton a. smart heat grids – the potential for district heating to contribute to electricity demand management to facilitate renewable and nuclear electricity generation. abingdon: the solar energy society conference c92; 2010. [20] werner s. benefits with more district heating and cooling in europe. rome: 20th world energy conference; 2007. [21] euroheat & power. district heating and cooling country by country survey 2015. brussels: euroheat & power; 2015. [22] orchard w. cost of transmitting 2 gw of heat 140 km: also longest heat main in europe 60 km. london: orchard partners london ltd.; 2011. accessed at <https://groups.google.com/forum/#!msg/energy-discussion-group/hoqhtmohic8/0bro uaij3v8j> on march 11, 2014. [23] riddle a. district heating & the future smart cities approach. copenhagen: ramboll group a/s; 2013. [24] pirouti m. modelling and analysis of a district heating network. cardiff: university of cardiff; 2013. 137 p. [25] kitsios k, chrysis i. assessment of national potential for cogeneration in cyprus. nicosia: republic of cyprus ministry of commerce, industry and tourism energy service; 2009. [26] dyrelund a. the future of the energy supply: smart energy cities. euroheat & power 2012 (english edition);9(1):12-15. accessed at <https://www.stateofgreen.com/ cmspages/getazurefile.aspx?path=~%5ccache%5cstateofgreen%5c33%5c3305a15939c0-4c5f-ae58-70883774a7d4.pdf&hash=3e2d92789d181deb3076da90e24ce0ab e6548c7 3131c70d1d8e172d7661ebec9> on march 11, 2014. [27] worm j. delrapport 3 miscellaneous investigations: co2 reductions in low-energy buildings and communities by implementation of low-temperature district systems – demonstration cases in energyflexhouse and boligforeningen ringgården, journalnr. 63011-0152. copenhagen: danish energy agency; 2011. accessed at <http://fjvu.dk/si tes/default/files/delra pport_3_miscellaneous_investigations.pdf> on march 11, 2014. [28] dong energy. the studstrupværket chp plant. skødstrup: dong energy a/s; 2007. http://dx.doi.org/10.4995/muse.2016.6339 http://dx.doi.org/10.1016/j.enpol.2010.07.010 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 148 [29] wagner i. local success stories: prague district heating system. brussels: ecoheat4.eu; 2011. accessed at <http://ecoheat4.eu/en/country-by-country-db/czech-republic/localsuccess-stories/> on march 11, 2014. [30] piskac m. prague district heating system: application for 1st global district energy climate awards. prague: pražská teplárenská a.s; 2009. accessed at <http://www.co penhagenenergysummit.org/applications/prague%20czech%20rep-dis trict%20energ y%20climate%20award.pd f> on march 11, 2014. [31] harvey ldd. energy and the new reality 1: energy efficiency and the demand for energy services. 1st ed. oxon: routledge; 2010. [32] wade dw. district heating conversion from steam to hot water at the savannah regional hospital. marietta: rda engineering; 1995. [33] de souza gfm. thermal power plant performance analysis. 1st ed. london: springerverlag london limited; 2012. http://dx.doi.org/10.1007/978-1-4471-2309-5 [34] burzynski r, crane m, yao r, becerra vm. low temperature district heating network serving experimental zero carbon homes in slough, uk. copenhagen: dhc13, the 13th international symposium on district heating and cooling; 2012. [35] ramboll energy. lavenergi fjernvarmekoncepter. copenhagen: ramboll; 2010. [36] urecon. pre-insulated municipal service pipe system. quebec: urecon ltd.; 2009. accessed at <http://www.urecon.com/applications/municipal_pipe.html> on march 17, 2014. [37] eliseev k. district heating systems in finland and russia. mikkeli: mikkeli university of applied sciences; 2011. [38] hepbasli a. low exergy (lowex) heating and cooling systems for sustainable buildings and societies. renewable and sustainable energy reviews 2012;16:74-104. http://dx.doi.org/10.1016/j.rser.2011.07.138 [39] allegrini j, orehounig k, mavromatidis g, ruesch f. a review of modelling approaches and tools for the simulation of district-scale energy systems. renewable and sustainable energy reviews 2015;52:1391-1404. http://dx.doi.org/10.1016/j.rser.2015.07.123 [40] pomper de. electric storage: technologies and regulation. silver spring: national regulatory research institute; 2011. [41] pauli ga. the blue economy: 10 years, 100 innovations, 100 million jobs. 1st ed. taos: paradigm publications; 2010. http://dx.doi.org/10.4995/muse.2016.6339 http://dx.doi.org/10.1007/978-1-4471-2309-5 http://dx.doi.org/10.1016/j.rser.2011.07.138 http://dx.doi.org/10.1016/j.rser.2015.07.123 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 149 [42] danish energy agency. technology data for energy plants: generation of electricity and district heating, energy storage and energy carrier generation and conversion. fredericia: energinet; 2012. [43] taylor p, bolton r, stone d, zhang xp, martin c, upham p. pathways for energy storage in the uk. york: the centre for low carbon futures; 2012. [44] dirección general de energía. the future role and challenges of energy storage. brussels: european commission; 2013. [45] macnaghten j. utility scale pumped heat electricity storage. fareham: isentropic ltd; 2009. [46] iea-etsap. thermal energy storage: technology brief. bonn: international renewable energy agency; 2013. [47] schröder d. introducing additional heat storage to the hässelby chp plant: a case study on economic and ecological benefits achievable with heat storage in a deregulated electricity market. stockholm: kth school of industrial engineering and management; 2011. [48] international energy agency. combined heat and power: evaluating the benefits of greater global investment. paris: international energy agency; 2008. [49] overbye p. thermal storage in district heating systems. copenhagen: ramboll; 2012. accessed at <http://www.districtenergy.org/assets/pdfs/03annualconference/mo ndaya/a4.1overbyeramboll-a4thermalstoresindh-slidesonly.pdf> on february 27, 2014. [50] poeuf p, senejean b, ladaurade c. district cooling system: the most efficient system for urban applications. paris: climespace – gdf suez; 2010. [51] thornton r. idea report: the district energy industry. westborough: international district energy association; 2005. [52] capital cooling. eu district cooling market and trends. stockholm: capital cooling; 2013. [53] skagestad b, mildenstein p. district heating and cooling connection handbook. paris: interntational energy agency. iea district heating and cooling; 2001. [54] wulfinghoff dr. energy efficiency manual. 1st ed. montgomery county: energy institute press; 1999. [55] kavanaugh s. how air-conditioners and heat pumps work. northport: energy information services; 2011. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 150 [56] medved s. energy production: generators of cold chillers. maastrich: cauberghuygen consulting engineers bv; 2013. [57] jamadar vm, patil am. investigation of performance analysis of compressor cooling in vapour compression system. international journal of advanced engineering research and studies 2012; 2(1):29-31. [58] sellars b, barbour a. technical report: geothermal heat pump study. auckland: beca carter hollings & ferner ltd; 2009. [59] tovey k. env-2d02 energy conservation: section 11. norwich: university of east anglia; 2006. [60] baechler mc. energy renovations hvac: a guide for contractors to share with homeowners. richland: pacific nothwest national laboratory; 2011. [61] zimny j, michalak p, szczotka k. polish heat pump market between 2000 and 2013: european background, current state and development prospects. renewable and sustainable energy reviews 2015;48:791-812. http://dx.doi.org/10.1016/j.rser.2015.04.005 [62] jolly a. energy efficiency of buildings and renewable energy. hurup: nordic folkecenter for renewable energy; 2006. [63] u.s. department of energy. use low-grade waste steam to power absorption chillers. washington: industrial technologies program energy efficiency; 2006. [64] srikhirin p, aphornratana s, chungpaibulpatana s. a review of absorption refrigeration technologies. amsterdam: renewable and sustainable energy reviews 2001;5(4):343-372. http://dx.doi.org/10.1016/s1364-0321(01)00003-x [65] energy management association of new zealand. energy audit standard for process heat systems: a standard for the auditing of the energy efficiency of direct and indirect heating systems. wellington: energy management association of new zealand; 2012. [66] brown rby. an experimental study of heat driven absorption cooling systems. salford: university of salford; 1990. [67] bhatia a. overview of vapor absorption cooling systems. stony point: continuing education and development; 2011. [68] labus j. modelling of small capacity absorption chillers driven by solar thermal energy or waste heat. tarragona: universitat rovira i virgili; 2011. [69] atecyr. equipos de absorción. madrid: asociación española de climatización y refrigeración; 2008. http://dx.doi.org/10.4995/muse.2016.6339 http://dx.doi.org/10.1016/j.rser.2015.04.005 http://dx.doi.org/10.1016/s1364-0321(01)00003-x multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 151 [70] hübner v, kallmann k, piller s, thamling n, hesse d, wetzel a. meeting cooling demands in summer by applying heat from cogeneration. berlin: berliner energieagentur gmbh; 2008. accessed at <http://www.chaleurfraicheur.org/doc/publis hable_report_final.pdf > on march 8, 2014. [71] basel ii, ahmed wh. thermoelectric power generation using waste-heat energy as an alternative green technology. recent patents on electrical engineering 2009;2:27-39. http://dx.doi.org/10.2174/1874476110902010027 [72] icf international. technology characterization: steam turbines. washington dc: environmental protection agency; 2008. [73] elsaket g. simulating the integrated solar combined cycle for power plants application in libya. cranfield: cranfield university; 2007. [74] lowe r. combined heat and power considered as a virtual steam cycle heat pump. energy policy 2011;39:5528-5534. http://dx.doi.org/10.1016/j.enpol.2011.05.007 [75] fahlen e, ahlgren eo. accounting for external costs in a study of a swedish districtheating system: an assessment of environmental policies. energy policy 2010;28:49094920. http://dx.doi.org/10.1016/j.enpol.2010.03.049 [76] cortes generales. diario de sesiones del congreso de los diputados, 8 de marzo de 2006. núm. 507. madrid: congreso de los diputados; 2006. [77] unep. thermal energy equipment. accessed at <www.retscreen.net/fichier.php/ 1000> on december 17, 2013. [78] uche-marcuello fj. thermoeconomic analysis and simulation of a combined power and desalination plant. zaragoza: universidad de zaragoza; 2000. [79] climate and energy fund. smart cities and initiative. vienna: climate and energy fund; 2013. [80] european technology platform on renewable heating and cooling. strategic research and innovation agenda for renewable heating & cooling. luxembourg: european union; 2013. [81] gadd h. to measure is to know! lund: lund university; 2012. [82] dhc+. district heating & cooling: strategic research agenda. brussels: euroheat & power; 2012. [83] møller-jørgensen j. r&d storage activities in denmark and eu. fredericia: energinet; 2013. http://dx.doi.org/10.4995/muse.2016.6339 http://dx.doi.org/10.2174/1874476110902010027 http://dx.doi.org/10.1016/j.enpol.2011.05.007 http://dx.doi.org/10.1016/j.enpol.2010.03.049 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 152 [84] wu w, wang b, shi w, li x. an overview of ammonia-based absorption chillers and heat pumps. renewable and sustainable energy reviews 2014;31:681-707. [85] kristensen s. the backbone of the modern energy system. frederiksberg: danish board of district heating; 2010. accessed at <http://dbdh.dk/the-backbone-of-themodern-energy-system/> on january 17, 2014. [86] dyrelund a. smart energy cities: seminar in dg17 28.09.2011. accessed at <https://stateofgreen.com/files/download/325> on december 17, 2013. [87] desire-project. guidelines to promote chp concepts with heat accumulators and the perspective of chp plants and other technologies that use thermal energy storage and their implementation in the european union. aalborg: aalborg university; 2007. [88] schmidt r, fevrier n, dumas p. key to innovation integrated solution: smart thermal grids. brussels: smart cities stakeholder platform; 2013. [89] elsman p. conpenhagen district heating system. copenhagen: first global district energy climate awards; 2009. [90] elleriis j. heat planning in the greater copenhagen area. copenhagen: heat plan copenhagen; 2011. [91] dyrelund a. european energy efficiency legislation adopts 35 years of danish experience. copenhagen: ramboll; 2011. [92] ministry of environment and energy. denmark’s second national communication on climate change. copenhagen: danish environmental protection agency; 1997. [93] dyrelund a, kozùowski w. dh/chp in eu smart energy cities. wrocław: ramboll; 2012. [94] danish ministry of energy, utilities and climate. electricity, district heating and natural gas supply. copenhagen: danish ministry of energy, utilities and climate; 2012. accessed at <http://www.efkm.dk/en/climate-energy-and-building-policy/denmark /energy-supply-and-efficiency/electricity-district-heating> on september 30, 2015. [95] dbdh. forsyning helsingør: portfolio items. accessed at <http://dbdh.dk/portfolio items/forsyning-helsingor/> on september 30, 2015. [96] veks. annual report 2012. albertslund: veks; 2013. [97] aalborg universitet. heat roadmap europe 2050. aalborg: aalborg universitet; 2013. [98] dhc+ technology platform. district heating & cooling: a vision towards 2020– 2030–2050. brussels: dhc+ technology platform; 2009. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 153 [99] thornton r. copenhagen’s district heating system: recycling waste heat reduces carbon emissions and delivers energy security. westborough: international district energy association; 2009. [100] eurostat. infrastructure – electricity – annual data. luxembourg: eurostat. accessed at <http://ec.europa.eu/eurostat/en/web/products-datasets/-/nrg_113a> on october 2, 2015. [101] eurostat. combined heat and power (chp) data 2005-2013. luxembourg: eurostat. accessed at <http://ec.europa.eu/eurostat/web/energy/data> on october 2, 2015. [102] eurostat. eurostat statistics explained: consumption of energy. luxembourg: eurostat. accessed at <http://ec.europa.eu/eurostat/statistics-explained/index.php/con sumption_of_e nergy> on september 29, 2015. [103] leka. new district heating countries. vilnius: lithuanian energy consultants association; 2012. [104] e-ptr. the european pollutant release and transfer register. copenhagen: european environment agency european environment agency. accessed at <http://prt r.ec.europa.eu/facilitylevels.aspx> on october 3, 2015. [105] riddle a. district energy & smart energy grids experience from denmark. london: rambøll energy; 2013. [106] eurostat. gas prices for domestic consumers – bi-annual data. luxembourg: eurostat. accessed at <http://ec.europa.eu/eurostat/en/web/products-datasets/-/nrg_ pc_202> on october 3, 2015. [107] colmenar-santos a, rosales-asensio e, borge-diez d, mur-pérez f. cogeneration and district heating networks: measures to remove institutional and financial barriers that restrict their joint use in the eu-28. energy 2015;85:403-414. [108] martinot e. investments to improve the energy efficiency of existing residential buildings in countries of the former soviet union. washington: world bank; 1997. [109] danish board of district heating. district heating history. frederiksberg: danish board of district heating. accessed at <http://dbdh.dk/district-heating-history/> on february 19, 2014. [110] gaigalis v, markevicius a, katinas v, skema r, tumosa a. analysis of energy transition possibilities after the decommission of a nuclear power plant in ignalina region in lithuania. renewable and sustainable energy reviews 2013;24:45-56. [111] streimikiene d, ciegis r, grundey d. promotion of energy efficiency in lithuania. renewable and sustainable energy reviews 2008;12:772-789. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 154 [112] biomass cogeneration network. current situation on chp and biomass chp in the national energy sector. pikermi: biomass cogeneration network; 2001. [113] alternative petroleum technologies. fuel oil emulsions. reno: alternative petroleum technologies; 2013. accessed at <http://www.altpetrol.com/en/2d-pd-foe.ht ml> on february 19, 2014. [114] cez group. the porici power stations. prague: cez group; 2014. accessed at <http://www.cez.cz/en/power-plants-and-environment/coal-fired-power plants/cr/porici.html > on february 19, 2014. [115] wan kkw, li dhw, liu d, lam jc. future trends of building heating and cooling loads and energy consumption in different climates. building and environment 2011;46(1):223-234. [116] sorrell s. the rebound effect: an assessment of the evidence for economy-wide energy savings from improved energy efficiency. london: uk energy research centre; 2007. [117] walker n, scheer j, clancy m, ó gallachoir b. energy forecasts for ireland to 2020. dublin: sustainable energy ireland; 2010. [118] enerdata. quantitative evaluation of explanatory factors of the lower energy efficiency performance of france for space heating compared to european benchmarks. angers: agence de l’environnement et de la maîtrise de l’energie; 2011. [119] hinrichs-rahlwes r. sustainable energy policies for europe: towards 100% renewable energy. 1st ed. boca raton: crc press; 2013. [120] eurostat. electricity generated from renewable energy sources, eu-28, 2003–13 yb15. luxembourg: eurostat. accessed at <http://ec.europa.eu/eurostat/statistics-expl ained/index.php?title=file:electricity_generated_from_renewable_energy_sources,eu28,_2003%e2%80%9 313_yb15.png&oldid=238269> on october 4, 2015. [121] pantaleo a, candelise c, bauen a, shah n. esco business models for biomass heating and chp: profitability of esco operations in italy and key factors assessment. renewable and sustainable energy reviews 2014;30:237-253. [122] traberg rl. district energy: the danish experience. london: royal danish embassy; 2008. [123] di lucia l, ericsson k. low-carbon district heating in sweden – examining a successful energy transition. energy research & social science 2014;4:10–20. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2016.6339 social and technological sciences eissn: 2341-2593 rosales-asensio and borge-diez (2016) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 3 nº 2 (2016): 107-155 | 155 [124] ziemele j, gravelsins a, blumberga a, vigants g, blumberga d. system dynamics model analysis of pathway to 4th generation district heating in latvia. energy 2015 (in press). [125] ziemele j, pakere l, blumberga d. the future competitiveness of the non-emissions trading scheme district heating systems in the baltic states. applied energy 2016;162:1579-1585. [126] christian gils h, cofala j, wagner f, schöpp w. gis-based assessment of the district heating potential in the usa. energy 2013;58:318-329. http://dx.doi.org/10.4995/muse.2016.6339 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 88 do continuous assessment results affect final exam outcomes? evidence from a microeconomics course juan carlos reboredo dpto. de fundamentos da análise económica. universidade de santiago de compostela, avda. xoán xxiii, s/n, 15782 santiago de compostela, spain. * corresponding author: email: juancarlos.reboredo@usc.es; phone: + 34 881811675 received: 2016-01-09; accepted: 2016-07-21 abstract continuous assessment aims to enhance student learning and understanding of a subject and so achieve better educational outcomes. we investigated how continuous assessment grades affected final exam grades. using a dataset for six academic post-bologna process years (2009-2015) for a first-year undergraduate microeconomics course offered at a spanish public university, we examined conditional dependence between continuous assessment and final exam grades. our results would indicate a limited contribution of continuous assessment results to final exam results: the probability of the final exam performance improving on the continuous assessment grade was lower than the probability of the opposite occurring. a consistent exception, however, was students who obtained an a grade for continuous assessment. our results would cast some doubt on the beneficial effects of continuous assessment advocated by the bologna process. keywords continuous assessment; final exam performance; conditional dependence multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 89 1. introduction implementation of the european higher education area (bologna process) led to the introduction of continuous assessment as a way to enhance and measure student learning. traditional lectures and final examinations have therefore been replaced by expositive and practical classes combined with student self-learning activities that are continuously assessed. a student’s final grade is comprised of marks for continuous assessment and for a final global exam. grades therefore depend not only on the ability of the lecturer but also on the student’s own motivation, dedication and abilities. examining the impact of continuous assessment on final student performance has several potentially important implications: for the efficient allocation of scarce resources in higher education institutions, for student time management aimed at maximizing academic performance and for the bologna process aim of enhancing student learning and understanding. despite the fact that continuous assessment is a core principle underlying the bologna process, however, evidence on the impact of continuous assessment on final exam grades for different subjects is still lacking in most eu countries. for a microeconomics undergraduate course, we investigate the relationship between continuous assessment and final exam grades by examining the conditional dependence between these two kinds of grades. we considered four different grades (a, b, c and d) for both kinds of assessment and computed the probabilities of each final grade conditional on specific continuous assessment grades. we formulate three hypotheses regarding the positive, null or negative contribution of continuous assessment grades to final exam grades and tested the hypotheses using a likelihood ratio test for conditional dependence. we applied our modelling procedure to a sample of students enrolled in the same first-year undergraduate microeconomics course over six years (2009-2015). our empirical evidence would indicate that continuous assessment grades made a limited contribution to final exam grades: the conditional probability of achieving a better grade in the final exam compared to continuous assessment was significantly lower than the conditional probability of getting multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 90 a lower grade. however, for students who obtained a grade a in continuous assessment, the conditional probability of obtaining an a in the final exam was no lower than the conditional probability of getting a lower grade. overall, however, final exam grades were generally lower than the grades that would be expected on the basis of continuous assessment outcomes. our results for the analysed microeconomics courses would suggest that continuous assessment aimed at fostering student learning and understanding may have neutral or even detrimental effects on student learning outcomes. poorer performance in final exams is likely due to students not fully assimilating a subject, which, in turn, may be due to rational management of scarce study time aimed at maximizing the probability of a pass on the basis of continuous assessment and dedicating more time to subjects for which they may obtain better marks. our results have implications for the efficiency of resource allocation to more personalized learning aimed at improving overall educational outcomes (e.g., the greater cost of reduced-size classes), which are highly dependent on student motivation and choices. the remainder of this article is organized as follows: section 2 reviews relevant literature; section 3 describes the data; section 4 outlines our methodological approach; section 5 presents and discusses the results; and, finally, section 6 concludes. 2. brief literature review previous literature on student performance that examines the relationship between lecture attendance and exam performance and between study time and student grades reports mixed evidence for the impact of attendance and study time on academic performance. a seminal study by romer (1993), based on attendance records for an intermediate-level macroeconomics course, reported attendance to have a positive and significant impact on academic performance. this conclusion, which has been corroborated by other empirical studies (see, e.g., durden and ellis, 1995; devadoss and foltz, 1996; chan et al., 1997; multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 91 rodgers, 2002; kirby and mcelroy, 2003; dolton et al., 2003), would suggest that mandatory attendance policies enhance student performance. however, other studies have found otherwise, namely, that incentives to attendance have no impact on academic performance (chen and lin, 2015; rodgers, 2002). yet other studies have found that the positive link between attendance and performance may be explained by endogeneity problems. krohn and o’connor (2005), for a study of macroeconomics courses, found no relationship between attendance and grades when using instrumental variables to account for endogeneity. similarly, martin and walker (2006), using fixed effect estimators to account for endogeneity, found that the positive and significant effect of class attendance disappeared when using panel data estimations instead of ordinary least squares (ols). more recently, andrietti (2014) also reported similar findings for a panel-data study. another strand of the literature has examined the impact of study time on student grades. several studies have reported no significant impact of study time on grades (schuman et al., 1985; hill, 1991; rau and durand, 2000) and plant et al (2005) found that the amount of study by college students was a poor predictor of academic performance. in contrast, in a study that considered causal effects, stinebrickner (2008) found that study time mattered, reporting that each additional study hour increased grades by 0.36 points. similarly, bonesrønning and opstad (2012) found that increased study effort improved test grades. in contrast with the existing literature, we do not consider attendance or study time but, instead, outcomes for both these variables as reflected in continuous assessment grades. continuous assessment provides useful information to both teachers and students, as teachers can identify the main areas of difficulty and students can pinpoint lack of sufficient knowledge, allowing an opportunity for both to rectify. continuous assessment can therefore act as an early-warning system that alerts both teachers and students to the need to refocus their respective efforts in the light of results. on that basis, if the final exam covers all the topics reflected in the continuous assessment tests, its outcome will be no worse than the outcome for the continuous assessment. this is the spirit underlying multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 92 continuous assessment of student learning as introduced by the bologna process. we test for this hypothesis by considering the conditional dependence of final exam grades on continuous assessment results. 3. data we collected data from assessments for students taking a first-year microeconomics course taught in the spring semester of a business administration undergraduate degree offered at a spanish public university (universidade de santiago de compostela). course content included analyses of consumption and production decisions and competitive market functioning. the course was delivered in a two-hour theoretical lecture plus a 90-minute reduced-size, interactive practical session per week. our database of 32 hours of theoretical classes and 17 hours of interactive classes contained data for six academic years (20092015) subsequent to launch of the bologna process. students were evaluated using scored tests and exercises, weighted in a similar way and with the same level of difficulty as the final exam (which students had to take at the end of the course) consisting of theoretical and practical content. the course was taught each year by the same teacher using the same syllabus and the same assessment procedures. grading was as follows: a, 90% or more; b, 70%-89%; c, 50%-69%; and d, 49% or less. table 1 reports descriptive statistics for our data referring to 589 students. the sample reflected academic performance data unevenly distributed over the sampling years. the percentages for students obtaining each grade point to a variation in final exam grade distribution with respect to continuous assessment grade distribution. thus, largely the same proportions of students obtained an a in both kinds of assessments (except for 20092010); the share of students who obtained a b-grade for continuous assessment and who maintained this grade in the final exam, however, was greatly reduced; and the share of c and d-grade students receiving the same grades in the final exam increased. in other words, multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 93 the probability distribution for continuous assessment grades moved left: the probability of an a remained similar, the probability of a b dropped and the probability of a c or d rose. this descriptive evidence would indicate that continuous assessment yields confusing evidence of future student performance: the hard-working and high-performing students (a grade) performed consistently, obtaining the same top grades in both assessments; students with reasonably good continuous assessment results (b grade) would seem to have slackened off their efforts and, consequently, obtained poorer final exam grades; and students with poor continuous assessment results (c and d grades) would seem to have invested lower effort and, consequently, do not improved their final exam grades. the descriptive statistics in table 1 also show that student performance worsened with class size: the larger the class, the greater the leftward movement of the final exam grade distribution with respect to the continuous assessment grade distribution (see, for instance, results for the 46 students of 2014-2015 compared to results for the 194 students of 20102011). table 1. descriptive statistics. continuous assessment grade vs final exam grade. continuous assessment grade (% students) final exam grade (% students) year # students a b c d a b c d 2009-2010 60 13.5 47.5 25.4 13.6 6.8 16.9 52.6 23.7 2010-2011 194 0.5 17.1 46.6 35.8 2.6 11.4 30.6 55.4 2011-2012 80 3.7 7.5 30.1 58.7 10 22.5 28.7 38.8 2012-2013 139 2.9 4.4 47.8 44.9 0.7 3.6 52.9 42.8 2013-2014 70 5.7 34.3 22.9 37.1 5.7 12.9 14.3 67.1 2014-2015 46 13.0 39.2 13.0 34.8 10.9 13.0 34.8 41.3 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 94 4. modelling conditional dependence we modelled continuous assessment and final exam grades to test for conditional dependence between grades as follows. consider indicators for the grades that student i can obtain for the final exam (fe) and for continuous assessment (ca), fe i i and ca i i , respectively, as taking the values j=a, b, c or d. conditional dependence of the final exam grade on the continuous assessment grade for each student is given by  fe cai ipr i h | i j  , for h,j=a, b, c, d, and conditional dependence implies that    fe ca fei i ipr i h | i j pr i h    . conditional dependence between grades can thus be represented by the conditional dependence matrix p: aa ab ac ad ba bb bc bd ca cb cc cd da db dc dd p p p p p p p p p p p p p p p p p               , (1) where each element, given by  fe cajh i ip pr i h | i j   for h,j=a, b, c, d, reports the probability of a grade h in the final exam provided the continuous assessment grade was j. conditional probability estimates are obtained by maximizing the maximum likelihood function under conditional dependence, given by:   jh n jh j,h p p  , (2) where jhn is the number of students with continuous assessment grade ca i i j followed by final exam grade fe i i h . estimated parameters arising from eq. (2) are simply the ratio of the counts for the corresponding cells: jh jh ja jb jc jd n p̂ n n n n     , (3) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 95 in light of the independence tests proposed by christoffersen (1998) and reboredo (2014) for indication sequences, hypotheses regarding the impact of the continuous assessment grade on final exam performance can be formulated as specific restrictions on the conditional probabilities inside matrix p in eq. (1). these restrictions can be tested using standard likelihood ratios tests. we consider three hypotheses. first, we consider the hypothesis that final exam grade is independent of the continuous assessment grade (h1), formulated as:  hypothesis 1: 0 jh hh :p p the conditional and unconditional probability of a specific grade h in the final exam are thus similar. note that this hypothesis states that the conditional probabilities in the hcolumn of the matrix p in eq. (1) are equal. taking the likelihood function under the null hypothesis 1, we can obtain the likelihood ratio test as: jhh nn h jh h j 2log p p           lr . (4) a second hypothesis consists of testing whether the final exam grade is better than the continuous assessment grade. in this case, the conditional probability of getting a better final exam grade is greater than the conditional probability of obtaining the same or a lower continuous assessment grade. this hypothesis (h2) can be formulated as:  hypothesis 2: 0 jh jzh :p p , where h is a better grade than j and z is a poorer grade than h. using the likelihood function we can estimate the likelihood ratio for h2 as: jh jh n n jh jh h z h 2log p p           lr . (5) multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 96 finally, the third hypothesis is that continuous assessment and final exam grades are the same. thus, the conditional probability of obtaining the same grade in the continuous assessment and final exam is greater than the conditional probability of obtaining a different grade. accordingly, each conditional probability in the diagonal of matrix p will be greater than the conditional probabilities in the same row. this hypothesis (h3) can be formulated as:  hypothesis 3: 0 hh hzh :p p , h z  using the likelihood function under null h3, we obtain the likelihood ratio as: hh hz n n hz h z 2log p p          lr . (6) 5. results table 2 reports estimates for the conditional probability matrix p in eq. (1) considering the full sample of student grades. the empirical results show that conditional dependence between continuous assessment and final exam performance varied according to grade (a, b, c or d). they also reveal substantial differences in conditional transition probabilities for the different grades. the implications for the effectiveness of continuous assessment in terms of final exam performance merit investigation, as, broadly speaking (and excluding the a grades), the conditional probabilities of getting a better final exam grade than continuous assessment grade were notably lower than the conditional probabilities of gaining a similar or lower grade. table 2. conditional probabilities. 0.39 0.27 0.30 0.04 0.09 0.30 0.49 0.12 p̂ 0.04 0.06 0.45 0.45 0.01 0.05 0.22 0.72               multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 97 we next tested the three hypotheses formulated in the previous section. first, we considered the hypothesis that final exam grade was independent of the continuous assessment grade (h1). the results (table 3) indicate that the null of independence was rejected for different grades, indicating that final exam performance depended on continuous assessment outcomes. the key question, answered by h2 and h3, is what shape does the conditional dependence take and what is its contribution to final exam outcomes. table 3. test for hypothesis 1. 0 ja a h :p p 0 jb bh :p p 0 jc ch :p p 0 jd dh :p p rejection rejection rejection rejection h2 tested whether students improved in their continuous assessment grade in their final exam (table 4). our evidence indicates that a-grade students were likely to achieve the same grade in the final exam, whereas b-, cand d-grade students were unable to achieve a better grade. thus, the results of testing h2 would point to continuous assessment being of dubious value, as only the hardest-working students with excellent continuous assessment results performed equally as well in the final exam. table 4. test for hypothesis 2. 0 aa az h :p p z b,c,d   0 ba bz h :p p z b,c,d   0 cj cz h :p p j a,b;z b,c,d    0 dj dd h :p p j a,b,c   non-rejection rejection rejection rejection h3 tested whether final exam grades reflected continuous assessment grades (table 5). consistent with the evidence reported for h2, the h3 results indicate that a-grade students were more likely to achieve a in the final exam than b or a lower grade. however, gradeb students were more likely to drop a grade to c, so we were not able to reject the multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 98 hypothesis that the conditional probability of obtaining the same grade was greater that the conditional probability of obtaining a lower grade. as for grade-d students, the probability of getting a better final exam grade was significantly lower than the conditional probability of maintaining a d grade. table 5. test for hypothesis 3. 0 aa az h :p p z b,c,d   0 bb bz h :p p z a,c,d   0 cc cz h :p p z a,b,d   0 dd dz h :p p z a,b,c   non-rejection rejection non-rejection non-rejection we checked the robustness of our results by examining these across different years. our evidence for the whole sample held for each year, with the exception of the 2012-2013 academic year where there was a lower probability of gaining an a grade in the final exam conditional on an a grade in the continuous assessment and that the probability of obtaining a b grade conditional on a b grade in the continuous assessment was lower than any other probabilities. we also tested for the possible impact of teacher competence by performing a similar analysis — for the same syllabus, years and assessment procedure — for two other teachers of different groups, obtaining results similar to those reported here. overall, our results regarding conditional dependence of student final exam grades on continuous assessment grades would indicate that: (a) for the top students, continuous assessment results quite faithfully reflect final exam results; and (b) for more mediocre students, continuous assessment results are indicative of even more mediocre final exam grades. we interpret these results as reflecting how students manage study time: they do not maximize knowledge of the course as a whole but rationally managing their time so as to maximize the probability of a pass while dedicating more time to subjects in which they might perform better. this evidence would cast some doubt on the usefulness of continuous multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 99 assessment as advocated in the bologna process as a means to improve learning and educational outcomes. 6. conclusions continuous assessment is an important pillar of the bologna process aimed at creating a common european higher education area. it is intended to improve student learning and understanding and, ultimately, academic performance. we examined whether continuous assessment results contributed to final exam outcomes by studying conditional dependence between the two kinds of grades for a first-year microeconomics undergraduate course offered at a spanish public university. our empirical evidence for courses taught between 2009 and 2015 indicate that continuous assessment grades were a poor indicator of final exam grades, given that the conditional probability of a better grade in the final exam than in the continuous assessment was lower than the corresponding probability of getting the same or lower grade, except for students who attained an excellent continuous assessment grade (a). final exam grades were generally poorer than would be expected if continuous assessment was fulfilling its goal of enhancing and encouraging student learning. for the microeconomic courses analysed in our study, we conclude that continuous assessment grades did not yield useful information on final outcomes. our findings may be explained by the time-management choices of students, who may rationally manage their scarce time in such a way that, once they have maximized the probability of a pass (informed by the continuous assessment grade), they dedicate more time to other subjects for which they could probably get a better grade. our results have implications for the efficiency of resources earmarked for more personalized education modes aimed at improving educational outcomes. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 100 references andrietti, a. (2014). does lecture attendance affect academic performance? panel data evidence for introductory macroeconomics. international review of economics education 15, 1–16. bonesrønning, h., opstad, l. (2012). how much is students’ college performance affected by quantity of study? international review of economics education 11(2): 46–63. chan, k.c., shum, c., wright, d.j. (1997). class attendance and student performance in principles of finance. financial practice and education 7 (2) 58–65. chen, j., lin, t-f(2015). effect of peer attendance on college students’ learning outcomes in a microeconomics course. the journal of economic education 46(4), 350359. christoffersen, p. (1998). evaluating interval forecast. international economic review 39 (4), 841–862. devadoss, s., foltz, j. (1996). evaluation of factors influencing student class attendance and performance. american journal of agricultural economics 78 (3) 499–507. durden, g.c., ellis, l.v. (1995). the effects of attendance on student learning in principles of economics. american economic review: papers and proceedings 85 (2) 343–346. hill, l. (1991). effort and reward in college: a replication of some puzzling findings, in james w. neuliep, ed., replication research in the social sciences. newbury park, ca: sage, 139–56. krohn, j.b., o’connor, c.m. (2005). student effort and performance over the semester. the journal of economic education 36 (1) 3–28. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2017.6548 social and technological sciences eissn: 2341-2593 reboredo (2017) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 88-101 | 101 martin, p., walker, i. (2006). student achievement and university classes: effects of attendance, size, peers, and teachers. discussion paper 2490. iza. plant, e.a., ericsson, a.k., hill, l., asberg, k. (2005). why study time does not predict grade point average across college students: implications of deliberate practice for academic performance. contemporary educational psychology 30, p96116. rau, w., durand, a. (2000). the academic ethic and college grades: does hard work help students to ‘make the grade’? sociology of education 73: 19–38. reboredo, j.c. (2014). can gold hedge and preserve value when the us dollar depreciates? economic modelling 39, 168–173. rodgers, j.r. (2002). encouraging tutorial attendance at university did not improve performance. australian economic papers 41 (3) 255–266. romer, d. (1993). do students go to class? should they?. journal of economic perspectives 7 (3) 167–174. schuman, h., walsh, e., olson, c., etheridge, b. (1985). effort and reward: the assumption that college grades are affected by the quantity of study. social forces 63, 945–66. stinebrickner, r, stinebrickner, t.r. (2005). how much does studying matter? federal reserve bank of cleveland proceedings: 5559. multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 44 present and future of the e-learning in economics schools and faculties t. casasús estellés, a. ivars escortell, m.i. lópez rodríguez dpto. de matemáticas para la economía y la empresa, universitat de valència, avda. dels tarongers s/n, 46022, valència, spain. e-mail: casasus@uv.es; phone:963828373 dpto. de economía aplicada, universitat de valència, avda. dels tarongers s/n, 46022,valència, spain. e-mail: antonia.ivars@uv.es: phone: 963828617 dpto. de economía aplicada, universitat de valència, avda. dels tarongers s/n, 46022, valència, spain. e-mail: maria.i.lopez@uv.es; phone: 963828617 received: 2018-01-03; accepted: 2018-03-24 abstract the goal of this paper is to justify/motivate the existence of moocs of quantitative subjects, particularly of mathematics and statistics, in the degrees taught in faculties of economics and business, complementary to the standard university courses. the changes in curricula carried out in spain in the last decade have boosted access to these faculties’ degrees for all kind of students. however, some of these students lack skills and abilities enough to successfully tackle the first university mathematics and statistics courses, as these are currently designed. empirical studies support the use of multimedia material as a very effective supporting tool for successful learning. thus, after assessing the student’s academic performance in quantitative subjects before and after the introduction of the bologna process, and obtaining worrying results, the implementation of the use icts on a regular basis is proposed in order to optimize the teaching-learning process. the data used in the study correspond to those obtained for students of the degree of tourism and the degree in international business, two degrees with very different characteristics and that, therefore, reflect a great diversity of the students of the faculty of economics of the university of valencia. keywords moocs, tics, movies, clickers, quantitative subjects https://doi.org/10.4995/muse.2018.9777 mailto:casasus@uv.es mailto:antonia.ivars@uv.es mailto:maria.i.lopez@uv.es multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 45 1. introduction the work presented here is due to a situation that, for a few years now, has been confirmed and accepted as a fact in the spanish university: the scarce level of knowledge of a quantitative nature of the newly admitted students, who access to the area of social sciences . students are faced with serious difficulties to be able to follow fluently the matters related to mathematics and statistics, taught in the first years of faculty. in particular, in the case we will deal with, students who access the degrees offered by the faculty of economics of the university of valencia, present a pre-university level that is not enough to successfully tackle subjects with mathematical content and / or statistical. this sometimes leads to a sense of failure, both for teachers and for students. the problem is accentuated if one observes the high dropout rates in the first courses. this perception is not given as an isolated case. in (cabrera, 2015) a compilation table of factors and variables related to abandonment is described. in this table prior knowledge appears as one of the educational factors related to abandonment. this problem has prompted certain university teachers to look for icts and new pedagogical tools, as a help to alleviate the detected situation as much as possible. the study of this work focuses on a particular case, the faculty of economics of the universitat de valència. however, concern for the results of students in quantitative matters is a recurring theme in the meetings of university teachers. see, for example, the papers presented at the annual meetings of the spanish association of university teachers of mathematics (asepuma) for the economy and business, www.asepuma.org. in these meetings, a section is dedicated specially to teaching proposals that motivate and facilitate the work of the student. in this sense, for example, (masero, 2016) presents a didactic proposal based on icts and active methodologies focused on students. in the case of the authors, the experience of years in the teaching of quantitative subjects has allowed us to have a general vision about the evolution of university studies since the https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 46 nineties. from this perspective, the objective that we set out in this paper is to justify our vision of where, and for which reason, e-learning and, consequently, moocs, will be introduced inevitably and necessary in regulated studies of economics and business. because of this, we will rely fundamentally on the analysis of the use of icts evolution in the degrees of the faculties of economics, from the last years of the bachelor's degrees to the present. the structure of this work is the following: in sections 2 and 3, we analyze which factors are significant in the academic performance of our students in two temporary spaces, before the implementation of the degrees and after their implementation. the results obtained, in favor of the use of icts, lead us, in section 4, to analyze whether the situation is still present, when the incorporation of the e-learning methodology has become widespread finally, in section 5 we present the conclusions obtained. 2. pre-bolonia precedents doing a bit of history, and focusing on the particular case of the faculty of economics of the universitat de valència, the background of the current situation can be said to have started in the 2003-2004 academic year. in this course, an educational innovation project (eip) was launched that involved the two centers with the highest volume of enrollment, and consequently of teaching activity, of the universitat de valència, the faculties of economy and law. the eip we refer to, is the implementation of the double degree of business administration law (ba-law) (dasí et al., 2007, lópez, diez and ivars, 2010), a project that began to take shape in march 2003 in response to the call made public by the universitat de valència, a call that aimed to develop experiences of educational innovation in the context of the european convergence. in the 2003/04 academic year, a first pilot group was launched in which, during the seven promotions that took the degree course, in addition to implement the ects credit as a unit of https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 47 measurement, all the tools that would characterize the bologna plan were introduced, as if a laboratory (a teaching one in this case) will be: evaluation of skills, skills, joint schedules, coordination, incorporation of icts in the teaching-learning process, etc. the observation and analysis of the results that were obtained gave way to numerous studies with different objectives: to predict what might be the strengths / weaknesses of the bologna plan; detect the factors that, being related to icts, significantly influenced academic performance, etc. one of the results obtained from a survey made to the first year students of the 5th class (ivars et al., 2009) concluded that the option of baccalaureate course (opcbachi) had an influence on the average grade obtained, this being higher in students coming from the baccalaureate of biomedical studies, followed by those who had taken the options of technology, social and ultimately those who came from a bachelor of humanities. table 1 contains some of the parameters obtained: table 1. descriptive average grade according to options of baccalaureate minimum maximum average typical deviation pearson v.c. median biomedical 4,67 9,6 7,89 1,31 0,17 8 technological 3,88 9,33 7,64 1,74 0,23 7,97 humanities 2,67 9,67 6,91 1,66 0,24 7,17 social 3,17 9,83 7,51 1,35 0,18 7,77 own elaboration the results are shown in figure 1: https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 48 figure1. graphical representation of the average mark according to options of baccalaureate. own elaboration. there were also clear differences between the grades according to the level of studies of the parents, obtaining better results those students whose parents were highly educated. as in the case of the previous factor (opcbachi), table 2 and figure 2 contain the results of a descriptive analysis and the graphic representation of the average grade according to the level of studies of the parents (estumadrepadre21). table 2. descriptive average grade according to the level of studies of the parents. minimum maximum average typical deviation pearson c.v. median parents level=2 3,17 9,6 7,802 1,36 0,18 8 1 we consider the characteristic estumadrepadre as the level of studies of the parents (obtained considering the sum of the level of studies of the mother and the father, each one of them at 5 levels ordered from lowest to highest). and according to operative criteria, the cases in which the characteristic "estumadrepadre" took the value 8, 9 and 10 were grouped, giving rise to the characteristic estumadrepadre2. . https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 49 parents level=3 6 9,37 8,061 0,96 0,12 7,9 parents level=4 4,17 9,17 7,356 1,47 0,20 7,57 parents level=5 4,67 8,17 6,091 1,16 0,19 5,93 parents level=6 2,67 9,33 6,761 1,75 0,26 6,92 parents level=7 4,87 9,83 7,024 1,68 0,24 7 parents level≥8 7,13 9,67 8,267 0,81 0,18 8,17 own elaboration figure 2. graphical representation of the average mark according to estumadrepadre2. own elaboration the careful observation of these results led us to the application of the anova technique (sig = 0.001) that confirmed the differences between the different cases. however, among the items considered in this study, the pre-university level of quantitative nature was not taken into account, since it was understood that this level was the appropriate to tackle successfully the subjects that depended on the said content. moreover, the agents responsible for the entrance exams to the university (pau) communicated this fact year after year to the university services. in this sense, the https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 50 conclusions of casasús and crespo (2004) show the "useless/invalid indicative value of the results obtained in the tests of selectivity", being a great difference between the levels that could be expected by the grades obtained in the pau and the ones that were really reached in the mathematics course of the first year of the degree2. in the introduction of the new methodologies that began in that period, activities that required cooperative work and self-learning gained strength. see, for example, (esteban et al, 2009) as an example of activities that would be a starting point for new methodologies used later in the bologna era. 3. post-bolonia precedents it should be remembered that the bologna declaration, carried out in 1999, was the beginning of the process of building the european higher education area (ehea). in addition, in the case of spanish universities, the implementation of this process was covered by royal executive order 1393/2007, of october 29th, which laid the foundations for the structures of official university education, granting autonomy to the universities in the development of study plans. in the case of the universitat de valència, on july 27th, 2010, in agreement of the government council of the universitat de valència 158/2010, the general guidelines that allowed the implementation of the dual degree programs (pdt) were approved. one of the pdt that began its journey in this university (2010-2011 academic year) was the double degree ade-law, being this the logical successor of the previous double adelaw. 2 see annex a with the result of a questionnaire in the first day at a lecture of mathematics i (academic year 2003/2004) in the faculty of economics, for a student with a qualification of 10 in the mathematics exam in the pau. https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 51 in addition, the start-up of the bologna process was accompanied by a notable increase in icts (agudo, hernández-linares, rico and sánchez, 2014) and tools based on so-called simulation games (bezanilla et al., 2014; méndez, garcía-pernía y cortés, 2014, urquidi martín and calabor prieto, 2014). once the degrees were implemented, an increase in the deficit of the pre-university level was detected, as well as a decrease in interest on the part of the students in subjects with a mathematical load. a comparative study of the academic results between the bachelor's degrees and the new degrees (lópez et al., 2016) tried to corroborate / discard this "sensation". to do this, the data corresponding to the academic performance of the adelaw pie students and the ade-derecho pdt were selected. the selection of these degrees was considered adequate given that both programs had similar characteristics 1. obtaining two degrees: bachelor / bachelor in business administration and bachelor / graduate in law. 2. integration into the curriculum of the contents of each degree. 3. equality in the requirements required of students to continue within the program. 4. use of ects credits as a unit of measurement. thus, the possible differences detected at the performance level, most likely due to other factors not considered, among them the pre-university level of quantitative character of the students who accessed the degrees offered by the faculty of economics. therefore, the analysis carried out focused on a subject that required such a requirement. in the analysis carried out, almost all the factors constituting the inputs in the teaching-learning process remained constant: subject matter, methodology used, didactic material, timetable, evaluation system, use of pedagogical tool (group activity), ..., concluding the existence of two factors that marked significant differences in student performance (sig = 0.0093 for factor tt (type of degree) and sig. = 0.0097 for ag (assistance to group https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 52 activity)), in favor of the students of the double degree and of those who attended the group activity: 1. "type of degree", at two levels: double degree (dl) and double degree (dg). 2. "attendance to the group activity", at two levels: "the student did attend", and "the student did not participate in the said activity". these results can be corroborated from the observation of least significant difference (lsd) for both factors figure 3. lsd intervals for tt and ag factors (response variable: examination mark "). own elaboration. as it can be verified, the non-overlap of the aforementioned intervals makes it possible to reject the hypothesis of equality of average scores for both inputs, the mark being higher for the students who belonged to the double degree and for those who had attended the group activity. 4. current situation in order to study if the situation described in the previous section is generalizable to the rest of the degrees, a basic quantitative test has been carried out for the students of other groups of the same faculty. this test, conducted on the first day of class, is a continuation of those that we have been doing since 2008. this has allowed us to have a good database, from which we have extracted some of the conclusions previously stated. https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 53 initially, the information was collected in written format, and was carried out since 2014 using the electronic voting system (evs). tables 3 and 4 summarize some of the results obtained and that were carried out in the degrees of tourism and international business (gib), among others. the structure of the information presented is as follows: 1. in the first column the statement of the question is collected. 2. in the second column the possible answers (only one is correct). 3. the rest of the columns present the percentage of correct answers, according to the promotions (three from tourism, table 3, and one from gib, table 4) table 3. survey conducted with evs to students of the tourism degree. question answer options % success (2014/15) % success (2015/16) % success (2016/17) 1.the characteristic a= “number of times you get face when flipping a coin 1200 times” is type: a) attribute b) ordinal c) discrete variable d) continuous variable 29,63% 53,57% 86,67% 2.the output of 1, 4 and 10 is: a) 15 b) 9 c) 10 27,78% 15,09% 21,43% 3.the typical deviation of 1, 2, 3, 4 and 5 is a) 5 b) 1,41 c) 1 44,44% 25,49% https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 54 question answer options % success (2014/15) % success (2015/16) % success (2016/17) 4.knowing that variable x takes the values: x1 =2, x2=4, x3=6, x4=8, x5=10 then, the value of ∑ 𝑥𝑖 23 𝑖=1 is: a) 12 b) 144 c) 56 32,08% 66,04% 45,65% 5.given the functions: a) y = 8x 5x + 3 b) y = x4 + 8x c)y = x + 1/x the corresponding to the line equation is a) the a) b) the a) and c) c) all 22,00% 10,71% 29,41% own elaboration table 4. survey conducted with evs to students of gib. answer answer options % success (2016/17) 1. –the variance of 1, 4 y 10 is: a) 9 b) 3’74 c) 14 46,67% 2.when analyzing the dispersion of the following two groups of values: group a: {2, 4, 6, 8, 10} and group b: {4, 8, 12, 16, 20} it is possible to conclude: a) both groups have the same variance. b) variance of group b is twice the variance of group a. c) variance of group b is 4 times the variance of group a. 7’55% https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 55 answer answer options % success (2016/17) 3.it is known that the average salaries (in hundreds of €) per month in two companies x and y are x ̅̅ ̅̅ = 7 and y̅̅̅ = 11 with a dispersion of sx 2 = 9 and sy = 3. if a worker of company x has a monthly salary of € 700 and another, of company y, of € 1100. which of the two workers has a relatively lower monthly salary? a) worker of company x b) worker of company y c) both workers have, relatively, the same monthly salary. 17’24% own elaboration from this information we can deduce the low percentage of correct answers, even in the case of questions with a mathematical load corresponding to lower levels than the baccalaureate one. for example, in the fifth question asked to the students of tourism, the difficulty was in identifying the general form of the equation of a line. none of the three promotions considered exceeded 30% of success. in the case of gib students, the situation becomes worse if one takes into account that the results presented correspond to a second test carried out in the fifth week of the course, after having emphasized basic descriptive concepts. note that the number of correct answers does not reach 50% in the question referring to a measure of elementary dispersion (the variance), nor to 20% in a simple question of categorization (several questions of this type had been previously resolved). the reason for the choice of these degrees lies, basically, in that they are two degrees with very different characteristics, and therefore they collected a great diversity of students, making a fairly representative sample of the students of the faculty of economics. some differences of interest between both degrees are: 1. the entry mark is very different, about 7 out of 14 in tourism students and 12, out of 14, in the case of gib students, which, in principle, should have an impact on https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 56 their level of pre-university knowledge, in general, and consequently in the quantitative ones, in particular. 2. election of the degree for completely different reasons, many of those enrolled in tourism have chosen it as their last option, while the gib is usually their first choice since they have enough access mark to choose what they want. obviously this affects their motivation when it comes to continuing studies. 3. different number of credits in the syllabus, 6 credits for the students of tourism for the subject of descriptive statistics without any mathematics credit, and 4.5 for the students of gib for the subject of descriptive statistics, probability and introduction to the inference and 6 mathematics credits. figures 4 and 5 show the results of the three tourism classes for each question (figure 4) and the distribution of the percentage of responses to the three possible options of the second question formulated to the gib students (figure 5). it is noteworthy, in the case of figure 4, that only three percentages of the fourteen represented exceed 50%. figure 5 shows that the students answer the incorrect answers in a high percentage (options a and b) and only 7.55% the right answer. figure 4. success percentage per question in tourism classes. own elaboration 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% 100,00% 2014-2015 2015-2016 2016-2017 https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 57 figure 5. answer percentage, question 2, gib course. own elaboration. the generalized verification of results such as those described above, together with the possibilities currently offered by new technologies and the help of the servei de formació permanent i innovació educativa led us to the development of new instruments that complemented and supported the regulated courses we teach at the university. in particular, the movies produced are hosted on the university's website (mmedia.uv.es) and we can state that they have been very well received (we consider proof of this, in addition to the favorable comments of the students, the increase in their attention in class, the decrease in absenteeism and the number of visits the movies had); these data are shown in table 53. 3 the values contained in table 5 are biased, because after entering the video links on the website of the educational innovation project to which the authors belong (http://pages.uv.es/piclickers/cat/menuh_muvies.wiki ) all the visits were not reorded in the mmedia.uv.es counter https://doi.org/10.4995/muse.2018.9777 http://pages.uv.es/piclickers/cat/menuh_muvies.wiki multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 58 table 5. list of movie titles and number of registered visits. tittle visit number a) methodology for obtaining confidence intervals 151 b) confidence interval for the average of a normal population with unknown variance and small sample 1332 c) determination of n to estimate the average of a normal population with known variance. 191 d) confidence intervalfor the average of a normal with known variance 225 e) confidence interval for the difference of averages 111 f) use of pearson’s chi-two tables. 132 g) use of table t of student 355 h) use of tables f of snedecor 509 i) confidence interval for the ratio of variances of two normal populations 383 j) use of normal distribution tables 231 k) clickers project 704 l) evolution of the inter-professional minimum wage 1992-2014 270 m) obtaining information from the ine regarding the cpi 304 n) application on the link of two series of indexes with a different base period 250 own elaboration being aware that these recordings, made at a first moment, only solved partially the deficiencies detected in the subject matter of the subject of the degree, but not the preuniversity gap detected, and coinciding with the first call for mooc courses for https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 59 teachers of the university of valència, we decided to participate by presenting a proposal that would allow us to develop a mooc whose content would cover the deficiencies detected, a proposal that was approved. in the mooc that we are finishing right now we have taken into account some of the deficiencies detected in the elaboration of previously recorded movies, somewhat long in some cases. in this sense, (guo, et al., 2014) perform a study on how the design of the videos affects the ability of the video to attract the student, one of its results being that the short videos are more effective. thus, the mooc developed is in line with the cited authors, since the videos meet the following requirements: 1. they are of short duration (an average of about 5-8 minutes). 2. the image of the teacher and the slide are inserted. 3. the explanation developed by the teachers is shown on a tablet, as a blackboard. specifically, the mooc developed consists of nine modules of about 6 lessons each, together with the complementary material that has been considered necessary (links, bibliography, self-evaluating issues / problems, evaluation test ...). the lessons are of mathematical-statistical content at the pre-university basic level, and with them the student is asked to complete the knowledge that will be necessary to continue with satisfaction his/her university studies. students can take their own pace, as well as skip some lessons if they already know them, and address in particular the one they need. from our point of view, it would be like having a book and consulting a chapter when necessary, although the content as a whole has coherence and can be fully studied as an introductory course to the university. as can be inferred from the above, our objective at this time is not to convert our classes into inverted classes or flipped classroom, but we agree with (aguado, 2017) that the "use of a mooc as a complement to regulated training" can be an opportunity for students, with advantages such as: 1. the possibility of accessing the available materials at any time. https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 60 2. the realization of tests that allows them to check the assimilation of the subject. 3. the access to the material individually and at their own pace. it is even more important in this case, in which the mooc developed is, for our economics and business degrees, a necessary tool, almost essential, for the correct followup of the subjects of quantitative content (as we have seen in the previous sections). the process of preparing this and any other mooc related to the regulated degrees requires the participation of both the corresponding universities and the agents involved, mainly teachers. aguado (2017) states that the effort involved in planning, organizing and carrying out a mooc requires a significant involvement that not all teachers are willing to take on, especially in these times when teaching is much less valued than researching . this statement is in line with what years ago concluded (kindelán, 2009), who after affirming the need for teaching and research tasks should go hand in hand in the daily work of university teachers, recognized that the current teaching model is giving priority to research to the detriment of teaching. 5. conclusions after the comparative study carried out between undergraduate and graduate degrees, it has been detected the fact that undergraduate students in economics graduates presented a higher performance than graduates in subjects of a quantitative nature. it also detects a downward trend in the pre-bologna era, sharpened later in the post-bologna era. in addition, this situation is prolonged / aggravated in the following courses. the study reveals a low level of quantitative knowledge in almost all the degrees offered by the faculty of economics. as an example, the results of three tourism promotions and one of international business have been exposed, providing conclusions that are, at least, worrying. https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 61 the deep concern that produces the evidence obtained, together with the widespread use of new technologies in society, and in particular in young people, has led us to the use of icts that complement the usual teaching in the university. in particular, the use of a tool such as moocs gives us a great opportunity to help students with difficulties in following certain subjects (in our case, quantitative subjects), or to follow a complete course of introduction to the quantitative subjects as a training for the university, if for some reason they had not completed it. icts in general, as well as moocs in particular, seem to postulate, if we follow the development of this work, as the precursors of future models of mass education, online and open, which are being considered as a very useful tool in education, both regulated or not. aguado (2017) argues that the use of a mooc as a complement to formal training can be an opportunity for students and our response to the question posed by (garcía aretio, 2015) about moocs such as tsunami, revolution or old fashion would be that have come to stay, in the current format or in future formats. 6. references aguado, j. (2017). ¿pueden los mooc favorecer el aprendizaje disminuyendo las tasas de abandono universitario? revista iberoamericana de educación a distancia, 20(1), 125143. agudo, j., hernández linares, r., rico, m., & sánchez, h. (2014). seguimiento y autoevaluación en el aula universitaria con una tablet pc. revista complutense de educación, 25(2), 245-270. bezanilla, m., arranz, s., rayón, a., rubio, i., menchaca, i., guenaga, m., & aguilar, e. (2014). propuesta de evaluación en competencias genéricas mediante un juego serio. new aprroaches in educational research, 3(1), 44-54. doi:10.7821/naer.3.3. https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 62 cabrera, l. (2015). efectos del proceso de bolonia en la reducción del abandono de estudios universitarios: datos para la reflexión y propuestas de mejora. revista fuenntes, 16, 39-62. doi:http://dx.doi.org/10.12795/revistafuentes.2015.i16.02 casasús, t., & crespo, e. (2004). titulación ade-derecho: primer análisis de una experiencia. actas de las xii jornadas de asepuma. murcia. http://asepuma.org/xii/comunica/casasus_crespo.pdf consultado el de 1 junio de 2017. dasí, a., garcía, j., huguet, a., juan, r., montagud, m., & gollnert, g. (2007). innovación educativa en la universidad: ade-derecho. (u. d. valència, ed.) valencia: puv. esteban, j., bachero, j., ivars, a., & lópez, m. (2009). descripción de una actividad grupal en el pie de ade-derecho de la universitat de valència. http://www.uv.es/motiva2/ponencias%20motiva2009/docs/62.pdf. consultado el 7 de junio de 2017. garcía aretio, l. (2015). mooc: ¿tsunami, revolución o moda? revista iberoamericada de educación a distancia, 18(1), 9-21. garcía aretio, l. (2017). los mooc están muy vivos. respuestas a algunas preguntas. revista iberoamericana de educación a distancia, 20(1), 9-27. guo, p., kim, j., & rubin, r. (2014). how video production affects student engagement: an empirical study of mooc videos. proceedings of the first acm conference on learning at scale. doi:10.1145/2556325.2566239 ivars, a., lópez, m., & ruiz, f. (2009). ade-derecho en la universitat de valència: estudio de una experiencia piloto. delta publicaciones universitarias. kindelán, m. (2009). una perspectiva sobre el binomio enseñanza-investigación en la universidad del s. xxi. revista complutense de educación , 27-45. https://doi.org/10.4995/muse.2018.9777 http://asepuma.org/xii/comunica/casasus_crespo.pdf http://www.uv.es/motiva2/ponencias%20motiva2009/docs/62.pdf multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 63 https://revistas.ucm.es/index.php/rced/article/viewfile/41190/39391. consultado el 8 de junio de 2017. lópez, i., díez, r., & ivars, a. (2010). la docencia de la estadística en el pie de adederecho: el volument de trabajo del alumno y el profesor. @tic.revista d'innovació educativa, 5, 4-17. lópez, m., palací, d., & palací, j. (2016). disminución del rendimiento académico con el plan bolonia respecto al plan anterior en españa. revista complutense de educación, 633-651. http://dx.doi.org/10.5209/rev_rced.2016.v27.n2.46915. consultado el 25 de mayo de 2017. masero, i. (2016). una propuesta didáctica basada en la tic y las metodologías activas centradas en el alumno para el desarrollo de competencias. anales de asepuma, 24. granada. méndez, l., garcía-pernía, m., & cortés, s. (2014). comunidades virtuales en el instituto. descubriendo la gramática interna del videojuego. new approaches in educational research, 3(1), 2-11. doi:10.7821/naer.3.3. 2-10 urquidi martín, a., & calabor prieto, m. (2014). aprendizaje a través de juegos de simulación: un estudio de los factores que determinan su eficacia pedagógica. edutec. revista electrónica de tecnología educativa, 47. http://www.edutec.es/revista/index.php/edutec-e/article/view/75/pdf_6. consultado el 25 de mayo de 2017. https://doi.org/10.4995/muse.2018.9777 https://revistas.ucm.es/index.php/rced/article/viewfile/41190/39391 http://dx.doi.org/10.5209/rev_rced.2016.v27.n2.46915 http://www.edutec.es/revista/index.php/edutec-e/article/view/75/pdf_6 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9777 social and technological sciences issn: 2341-2593 casasús et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 44-64 | 64 annex a https://doi.org/10.4995/muse.2018.9777 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 47 educational activities for acquiring skills in the eees adapted degrees tolosa, santiago1*, hidalgo, antonio1, ojalvo, evaristo a.1, guiberteau, agustina2 university of extremadura, department of physical chemistry1 university of extremadura, department of analytical chemistry2 * corresponding author: email: santi@unex.es; phone: + 34 924289775 received: 2016-07-02; accepted: 2017-09-13 abstract the main objective of this paper is to describe the performance of a serial of training activities focused on the acquisition of skills by different groups of undergraduate students related to science as a branch of knowledge. specifically, we used those activities where students receive knowledge, such as theoretical classes (t), computational (pc) and laboratory (pl) sessions, exercises (p), ects tutorials and bachelor thesis (tfg). among the different activities, it must also be highlighted the undergraduate dissertation. the activities were conducted in different courses of the chemistry degree of the university of extremadura, in order to obtain information from the different competences that appear in the verification report of this title. finally, the students filled out a survey giving their opinion about the skills acquired in each of the activities. the results of the experience is that students achieve most of the skills considered in the title. likewise, the results from the activities considered show that not all skills are achieved and other activities should be considered to cover those deficiencies. the activity that most contributes to the acquisition of transversal competences are laboratory practices, while for specific skills, the activities developed with large group of students (t and p) are the most significant. tfg is the activity that best covers most of competences, especially basic and general. keywords teaching methodologies; innovation in education, applied sciences. education competences, formative activities. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 48 1. introduction adaptation to european higher education area (eees) has led to the spanish universities the need for more comprehensive training that includes, in addition to the specific competences of each degree, basic, general and transversal competences. thus, the memories of the new curricula verified by the agencia nacional de evaluación de la calidad y acreditación (aneca) incorporate a detailed compendium of the competences of the professional profile of each qualification, understanding that are key to enhancing employability. the current design of the curriculum requires that learning are oriented to the acquisition of skills by students, teachers must assess the degree of achieving them. therefore, in this new context, the challenge is to design a program of training activities and planned experiences specifically to support the student to achieve particular learning outcomes and, in turn, develop a framework to assess the different skills the student has to acquire. regarding the design of training activities to achieve the acquisition of the competences of a degree, it requires the use of different teaching methods and evaluation criteria adapted to each type of activity. the importance of this study is even greater if we consider the insufficient previous evidence linking the full acquisition of skills, the training activities, and methodologies to achieve with the evaluation of learning results. consequently, the fundamental objective of this work is to check to what extent the training activities employed have achieved the acquisition of skills, and to what extent an application of training activities can improve the acquisition of skills. 2. competences/skills degree in chemistry at the university of extremadura (boe 05/01/2010 and 08/03/2010) stated in his verification report the competences given in table 1, which can be grouped into four main groups: basic (cb), general (cg), transversal (ct), and specific (ec) skills. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 49 only 12 of the 35 specific skills listed in the degree report have been considered, particularly those related with the disciplines considered in the conducted surveys. table 1. uex chemistry degree competences. basic competences cb1 demonstrate knowledge and understanding cb2 knowing how to apply their knowledge to their work cb3 having the ability to gather and interpret data cb4 communicate information and solutions cb5 have developed learning skills general competences cg1 students engage in learning tasks cg2 students must develop an interest in learning cg3 students acquire knowledge and practical skills cg4 students develop skills/abilities transversal competences ct1 ability to use the method of induction, analysis and synthesis, organization and planning, work in an international context, both oral and written expression, critical thinking, problem solving, decision making and teamwork ct2 ability to communicate knowledge and conclusions ct3 ability to learn new techniques and knowledge ct4 development of personal learning abilities and to acquire skills in interpersonal relationships ct5 show sensitivity to environmental issues ct6 recognition of diversity and multiculturalism ct7 commitment to respect human rights ct8 motivation for quality multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 50 ct9 knowledge of a foreign language (preferably english) ct10 use of ict ct11 manage technical or professional projects specific competences ce1 (*) gain knowledge of chemical terminology, nomenclature, conventions and units ce2 interpret the atomic structure and the quantum chemistry principles ce3 relate the variation of the characteristic properties of chemical elements according to the periodic table ce4 (*) identify the characteristics of the different states of matter and the theories used to describe them ce5 (*) distinguish the main types of chemical reaction. principles of thermodynamics, kinetics and electrochemistry ce6 identify chemical elements and their compounds, understanding the synthesis, structure and properties of the compounds ce7 analyze the radiation-matter interaction. understanding the principles of spectroscopy ce8 differentiate principles and procedures for the determination, identification and characterization of chemical elements and compounds ce9 assimilating the relationship between macroscopic properties and properties of individual atoms and molecules ce10 distinguish and apply the methods of structural determination ce11 identify and develop unit operations of chemical engineering ce12 perform mathematical processing of data from chemical processes and quality management in laboratories ce13 recognize the structure and reactivity of the main classes of biomolecules and the chemistry of the main biological processes ce14 differentiate materials and deduce their properties ce15 (*) demonstrate knowledge and understanding of essential facts, concepts, principles and theories related to the areas of chemistry ce16 (*) solving qualitative and quantitative problems ce17 recognition and analysis of new problems and plan strategies for their solution ce18 (*) ability to function safely in a chemical laboratory ce19 (*) evaluation, interpretation and synthesis of data and chemical information. collection, processing and treatment of chemical data using computational techniques ce20 execution of laboratory procedures involved in synthesis and analytical works, involving organic and inorganic systems multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 51 ce21 (*) interpretation of data derived from observations and measurements in the laboratory ce22 capacity to carry out projects in the chemical industry ce23 knowledge of a foreign language (preferably english) ce24 (*) using the most appropriate ict in each situation ce25 (*) recognition and measurement of chemical processes in daily life ce26 (*) understanding of qualitative and quantitative aspects of chemical problems ce27 (*) understanding the relationship between chemistry with other disciplines ce28 demonstrate have reached an adequate understanding of the different physical phenomena ce29 have current knowledge in some aspects of physics ce30 ability to identify the essential elements of a complex physical situation ce31 having a good knowledge and mastery of mathematical and numerical methods most commonly used in physics ce32 problem solving in physics ce33 knowledge and understanding of mathematics ce34 solve problems and basic exercises of mathematics ce35 read and understand mathematical textbooks (marked with asterisks those competences studied in the surveys) 3. work methodology the data used in this study were collected from surveys filled by students from different courses of the degree in chemistry, covering various subjects with different training activities: theoretical (t) and problem (p) classes, laboratory (pl) and computer (pc) practices, tutorials (ects), and bachelor thesis (tfg). specifically, the surveyed subjects were: aggregation states of the matter and chemical kinetics in the second year of the degree (including activities of t, p, pc, and ects types); classical methods of quantitative analysis also in the second year course (with activities such pl); chemistry iii in the first year (only with pl type activities), and the bachelor thesis, considering only in this case those students who submitted their thesis during the course 2014/2015. we pay special attention to the latter group of students as many of the skills listed in the chemistry verification report are collected in this activity. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 52 almost a total of 100 surveys were collected. different surveys were completed according with the type of learning activity. thus, surveys about theoretical and problem classes were conducted during the aggregation states subject; ects surveys were performed in chemical kinetics tutorial activities. the information related with computer practices were obtained in the simulations of the liquid state activity, on the other hand, laboratory survey was done during the electrochemistry and chemical analysis laboratory hours. finally, bachelor thesis information was collected from surveys filled by students of different areas of chemistry. we will consider in the analysis that a competence is "achieved" when the percentage from the survey exceeds 50% and it is "adequately achieved" when it does a percentage above 75%. in the same way a percentage less than 25% can be considered that the competence were "not achieved". 4. result analysis 4.1 basic competences figure 1 shows the percentage of basic skills acquired from different training activities. it can be seen that in general, these skills were well acquired in the learning activities carried out in a large group of students (namely, t and p activities) and in practical laboratory sessions (pl), but they were not achieved in the computer sessions and in ects tutorials. it can be seen that cb1 competence, related to knowledge and understanding, reached the maximum rate possible of acquisition with theoretical lessons. by other hand, cb2 competence (knowing how to apply knowledge to work) was achieved in the greatest rate with laboratory activities, and cb3 one (having ability to gather and interpret data) with in the elaboration of the bachelor thesis. the cb5 competence (have developed learning skills) appears as the most successful on all training activities, the sum of the percentages is the highest and has the narrowest range between 45-80%. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 53 it is also worthy to note that cb3 (have the ability to gather and interpret data) and cb4 competences (communicate information and solutions) are acquired during the performance of the bachelor thesis rather than with the traditional activities of lectures and problems. figure 1. percentage of basic skills acquired in each activity finally, we will mention that there is always some activity that makes that all the basic competences reach the qualification of "adequately achieved", as seen in figure 1, there is at least one activity for each competence that exceed the dotted line. 4.2 general competences 0 10 20 30 40 50 60 70 80 90 100 cb1 cb2 cb3 cb4 cb5 t p pc ects pl tfg multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 54 the acquisition of general skills in chemistry degree has been analyzed in a similar way that the basic ones, showing the results in figure 2. it can be appreciated for the cg1 (engage in learning tasks), cg2 (interest in learning), and cg3 (acquisition knowledge and practical skills), that the activities with a large group students (t and p) are responsible for reaching a percentage that qualifies as "adequately achieved". figure 2. percentage of general skills acquired in each activity the cg4 competence (development of skills/abilities) is the largest overall percentage; in fact, it is the only one that reaches more than 50% in all activities (beyond the dashed line of the graph), and the only one in which ects tutorials acquires a significant value (greater than 80%). 0 10 20 30 40 50 60 70 80 90 100 cg1 cg2 cg3 cg4 t p pc ects pl tfg multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 55 in general, students consider that general skills are acquired very well with the theoretical lectures and problems sessions, and with laboratory practices, although to a lesser extent in this case. 4.3. transversal competences figure 3 shows the results obtained for the transversal competences of the chemical degree. the activity that most contributes to the acquisition of these skills are laboratory practices, particularly in the ct1 (using the method of induction, analysis and synthesis, organization and planning, work in an international context, speaking and writing properly, critical thinking, problem solving, decision making and teamwork) and ct8 ones (ability to learn new skills and knowledge), whose percentages exceed the value of 75%, considering them as "adequately achieved". note that there are specific activities for the acquisition of certain transversal competences. namely, computer activities (pc) contribute to the highest percentage (80%) in the ct10 skill (use of ict). in the same way, ects tutorials contribute to the ct2 competence (communication knowledge and conclusions). the highest percentage is reached in ct3 competence (learn new techniques and knowledge) with theoretical classes reaching a value of almost 90%. however there are skills that barely reach a 25% of acquisition with all training activities, such as ct6 (recognition of diversity and multiculturalism), or as in the case of the ct7 (commitment to respect human rights) where most activities do not allow to obtain this competence, which will require other activities for their achievement. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 56 figure 3. percentage of transversal skills acquired in each activity 4.4 specific competences in figure 4, the results of specific skills related to the subjects of aggregation states of matter, kinetics, chemical analysis and general chemistry are displayed. theoretical activities contribute, in the opinion of students, with a 100% in the achievement of ce1 competence (gain knowledge of chemical terminology, nomenclature, conventions and units). this kind of activities also contribute over 90% in the ce4 (characteristics of the different states of matter and the theories used to describe them) and ce5 skills (principles of thermodynamics, kinetics and electrochemistry). this same observation could apply for these three competences using sessions of seminars/problem though with lower percentages. 0 10 20 30 40 50 60 70 80 90 100 ct1 ct2 ct3 ct4 ct5 ct6 ct7 ct8 ct9 ct10 ct11 t p pc ects pl tfg multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 57 figure 4. percentage of specific skills acquired in each activity highlight the good response provided by laboratory practices in much of the skills, the results of the computational practices in ce19 (collection, processing and treatment of chemical data using computational techniques) and ce24 skills (use of ict), and the bachelor thesis in ce1 (knowledge of chemical terminology, nomenclature, conventions and units) and ce18 ones (ability to function safely in a chemical laboratory). in general, we can say that with theoretical sessions 75% of specific skills are acquired, a 66% is achieved with problem activities, a 66% with laboratories, a 16% with computing sessions 16%, and bachelor thesis about a 41%. it can be said again that ects tutorials contribute very little to the acquisition of such skills. 0 10 20 30 40 50 60 70 80 90 100 ce1 ce4 ce5 ce15 ce16 ce18 ce19 ce21 ce24 ce25 ce26 ce27 t p pc ects pl tfg multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 58 4.5 bachelor thesis activity the bachelor thesis is a degree final work (tfg) and covers very well the cb3 (gather and interpret data), cb4 (transmit information and solutions), cg3 (possess knowledge and practical skills), ce1 (gain knowledge of chemical terminology, nomenclature, conventions and units) and ec19 competences (evaluate, interpret, synthesize, obtain, process and treat chemical data). it shows a surprisingly low percentage of acquisition of the cb1 skill related to the acquisition of knowledge, and of the ce5 competence on solving qualitative and quantitative problems. on the contrary, the results of ct6 and ct7 competences (related to environmental issues, diversity and multiculturalism) can be expected because the work cannot have cover this subject. figure 5. percentage of competences acquired in tfg 0 10 20 30 40 50 60 70 80 90 100 c b 1 c b 2 c b 3 c b 4 c b 5 c g 1 c g 2 c g 3 c g 4 c t 1 c t 2 c t 3 c t 4 c t 5 c t 6 c t 7 c t 8 c t 9 c t 1 0 c t 1 1 c e 1 c e 4 c e 5 c e 1 5 c e 1 6 c e 1 8 c e 1 9 c e 2 1 c e 2 4 c e 2 5 c e 2 6 c e 2 7 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 59 tfg is able to achieve most of the competences, 16 of the 32 competences analyzed exceeds the minimum value of 50% to acquire the skill in this activity. moreover, according to the data collected, there is a competition (ct11) related to the management of technical/professional projects that is not acquired in tfg, perhaps because it is not a topic offered in the degree in chemistry. 5. conclusions in general, we can say that the activities undertaken by students throughout the degree in chemistry allow obtaining, in greater or lesser extent, the skills set out in the chemistry verification report. 1. there is always some activity, which makes the basic competences to reach the qualification "adequately achieved". 2. the general skills are acquired very well with the theory and problem sessions, and with laboratory practices, although to a lesser extent. 3. the activity that most contributes to the acquisition of transversal competences are laboratory practices, while for specific skills, the activities developed with large group of students (t and p) are the most significant. 4. tfg is the activity that best covers most of competences, especially basic and general. 5. the least appropriate activity for the acquisition of skills are the ects tutorials. 6. finally, ct6 (recognition of diversity and multiculturalism), ct7 (respect for human rights), and ec11 skills (technical project management/professional) are poorly acquired with the activities performed. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.6474 social and technological sciences issn: 2341-2593 tolosa et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 47-60 | 60 6. acknowledgment this work was sponsored by the consejería de infraestructuras y desarrollo tecnológico de la junta de extremadura (project gr15003). 7. references and web links 1. mª dolores montagud mascarell, juan luis gandía cabedo, https://doi.org/10.15581/004.28.79-116 2. https://www.educacion.gob.es/ruct/solicitud/competencias?actual=menu.solicitud. competencias.generales&tipo=g&cod=25009252013070401 3. https://www.educacion.gob.es/ruct/estudio.action?codigociclo=sc&codigo estudio=2500925 4. http://www.unex.es/conoce-la-uex/centros/ciencias/titulaciones/info/ competencias?id=125 5. http://www.unex.es/organizacion/gobierno/vicerrectorados/vicecal/archivos/ ficheros/informacion-titulos/ciencias/plan0125/memoriaplan.pdf 6. http://srv.aneca.es/listadotitulos/node/1160235578 https://doi.org/10.15581/004.28.79-116 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 1 the significance of private tutoring in improving english language literacy: a structural equation modelling approach douglas andabati candia*, james mukoki2, claire ashaba3, peter jegrace jehopio2, brenda kyasiimire5 department of planning and applied statistics, makerere university, kampala, uganda douglascandia@gmail.com, jammy.mukoki@gmail.com, ashaba7587@gmail.com, jehopio@yahoo.com, kyasiimirebrenda@gmail.com brenda kyasiimire department of management sciences, makerere university business school, kampala, uganda kyasiimirebrenda@gmail.com * corresponding author: email: douglascandia@gmail.com; phone: +256 788306848 received: 2018-01-24; accepted: 2018-04-28 abstract low quality of primary education demonstrated by low academic achievement has persisted as a challenge to uganda’s education sector. although the country has progressed in improving other education indicators, this hasn’t translated into better schooling outcomes. therefore, this study sought to determine the significance of private tutoring on english literacy levels. the study utilized secondary data from the uwezo uganda national learning assessment 2014 survey and generalized structural equation modelling to determine significant predictors. the variables that had a direct effect included private tutoring, child’s age and gender, pre-school attendance, school type, household size, household head gender and education level. the variables that had an indirect effect through private tutoring were region of residence, school type and household head education level. the study observed a need for the government through the ministry of education to come up with a regulatory framework to manage and control the practice of conducting private. keywords private tutoring, uganda, literacy, english, sem multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 2 1. introduction ensuring inclusive and equitable quality education and promotion of life-long learning opportunities for all is the fourth goal of the united nations sustainable development goals (sdgs) with the first target concerning the ensuring of all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes (osborn, cutter & ullah, 2015). towards the implementation of the sdgs, the uganda vision 2040 identifies human capital development as one of the key fundamentals that need to be strengthened to accelerate the country’s transformation and harnessing of the demographic dividend. the availability of appropriate and adequate human capital facilitates increase in production, productivity and technological growth thus making it one of the key endogenous drivers of economic growth. education as a human right is the heart of sustainable development and it is important to the development of individuals and economies, as it helps to pave the way to a successful and productive future and therefore is a key element of human capital development because it is viewed as the primary means of developing knowledge and skill (npa, 2015). in uganda, from 1986, the education sector has witnessed comprehensive policy reforms including the adoption of universal primary education (upe) in 1997 which made an immediate impact on primary school enrollment level from 2.8 million in 1996 to 8,485,005 (npa, 2015). nevertheless, low quality of primary education remains the main challenge demonstrated by low learning achievement (school outcomes) evidenced by only slightly above average performance in literacy for primary six and primary three pupils, standing at 52.6% and 60.2% respectively (uneb, 2015). the variations in literacy are mostly evident across school locations with rural schools (49.5%) performing below average with regards to literacy compared to urban schools (67.7%) at primary three level. in order to address the poor learning outcomes in pupils, a number of parents have resorted to private tutoring. private tutoring consists of a series of activities, supplementary to multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 3 mainstream schooling, whose aim is to enhance a student’s academic performance in exchange for monetary payment (bray, 2005) and can be provided on a one-to-one basis or through engaging pupils in small groups. in sub-saharan africa, tutoring has become more prominent, largely as a means to generate extra income for teachers with sacmeq ii data showing that 82% of grade 6 pupils in uganda received extra tutoring and that over half of these paid for their tuition (unesco, 2014). tutoring has not only been used as a strategy to improve academic performance but also to address issues of student retention, progression and completion (braine & parnell, 2011; guerra-martín, 2015). several approaches for tutoring have been theorized including the classification by jones and brown (2011)who distinguished three models to tutoring, that is, the traditional model, the reciprocal model and the emergent model. the traditional model is based on a hierarchical relationship between a tutor and a pupil in which the first transmits knowledge, information, or support to the latter. this kind of relationship could have several benefits for the protégé such as psychosocial or career-outcome (crisp & cruz, 2009; jones & brown, 2011; guerra-martín, lima-serrano & lima-rodríguez, 2017). the reciprocal model emphasizes the collaborative nature of the tutoring relationship. in this case both tutor and protégé could benefit, and an emotional connection could be created (guerramartín et al., 2017). finally, although traditional and reciprocal models are the most commonly mentioned in literature, new conceptualizations have emerged such as reverse model, peer tutoring, or the complex adaptive systems that include the role of the university institution or the socio-cultural environment (jones & brown, 2011). although the majority of quantitative studies regarding tutoring have previously been based on non-experimental methods, quantitative researchers have begun to move beyond descriptive investigations and towards an understanding of the causal relationship between tutoring and student success (crisp & cruz, 2009; guerra-martín et al., 2017). the current study comes to fill the gap especially on the effectiveness of private tutoring on primary pupils in uganda with a special focus on their english language literacy levels. this will supplement on current studies which have mostly been done in multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 4 developed countries (guerra-martín et al., 2017) but mostly focusing on numeracy or mathematics (guill & bos, 2014; unal, ozkan, milton, das & das, 2013) and a few in developing countries (cole, 2016). furthermore, most studies on student or pupil learning outcomes have assumed direct associations between the predictors of student learning outcomes. this study took into consideration the possibility of having some predictors influencing student learning outcomes indirectly through other predictors: a case in point being private tutoring which has been found to have an effect on student academic performance but has also been found to be affected by some factors which influence academic performance as well. there is need to study these interrelationships so that policy decisions regarding education quality improvement address all possible links and avenues through which the plausible predictors affect student academic performance with specific focus on private tutoring. 1.1 objective of the study the purpose of the study was to identify the significance of private tutoring on english language literacy levels of primary school pupils in uganda. furthermore, the study sought to determine whether these effects are both direct and indirect and if so to what extent they do influence english language literacy levels. 2. material and methods 2.1 data source the study was based on the uwezo uganda national learning assessment 2014 survey. children aged 6-16 were assessed in the household setting and other data were obtained through related surveys of the households, their local communities and selected local schools where majority of the children in the community were enrolled (uwezo, 2016). household characteristics were recorded by interviewing the heads or representatives of the households where assessments were conducted. basic indicators on the local communities multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 5 or enumeration areas (eas) were obtained through interviews with local council leaders. school indicators were obtained from school heads and through direct observation. overall, 3,347 eas, 3,347 schools and 51,835 households were visited (uwezo, 2016). a two stage cluster sampling design was adopted in the assessment within the 112 districts, with households as the elements and eas as the clusters. in the first stage, 30 eas were selected per district using the probability proportional to size (pps) methodology. thus, eas with larger numbers of households had a greater chance of being selected. the second and ultimate stage was the simple random selection of 20 households from each of the selected 30 eas in each district. this provides a sample of households that is self-weighting up to the district level. within the selected households, however, all available children in the age range 6-16 were assessed and relevant information both on the children and on their households were obtained. the study focused on pupils attending grade one to grade four. 2.2 dependent variable the outcome variable measuring english language literacy level of a pupil was categorized as follows. the criteria used for testing and categorizing children’s reading levels in the survey was as follows; if the child couldn’t recognize even 4 out of 5 common letters from the letters list mark, he/she was considered to be very weak; if she/he could correctly recognize at least 4 out of 5 letters with ease and read 4 out of 5 letters but couldn’t read words, the child was considered to be weak; if the child could correctly read at least 4 out of the 5 words with ease but struggled to read an easy paragraph, he/she was marked as average; if the child could read any paragraph like he/she is reading a sentence rather than a string of words, they would be categorized as good; if the child could read a story fluently, with ease and speed, they were marked as very good (uwezo, 2016). a pupil’s private tutoring status was considered an endogenous variable influencing their english literacy performance whereas it was also influenced by the plausible exogenous variables in the study. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 6 2.3 exogenous variables the proposed plausible independent variables for this study include; region of residence, age of the child, gender of the child, disability status of a child, years a child attended pre-school, school type, gender of household head, education level of household head, household size and whether a child receives extra lessons. 2.4 data analysis the data was analyzed using stata 14.2 in three stages. firstly, we carried out a descriptive summary of the variables in the study. secondly, a bivariate analysis was conducted using pearson’s chi-square test to check for association between a child’s literacy level, private tutoring status and the plausible exogenous variables. thirdly, generalized structural equation modelling (sem) was used at multivariate level with binomial logit and ordinal logit link functions for the two endogenous variables, that is, private tutoring and english language literacy level respectively. variables that were significantly associated (p ≤ 0.05) with the two endogenous variables at bivariate level were considered for further analysis. 3. results 3.1 description of respondents table 1 provides a description of the pupils with regards to their individual, school and household characteristics. based on table 1 which follows, majority of the pupils’ literacy levels were weak (32.55%) followed by the very weak (28.39%), average (20.10%), very good (12.66%) and lastly good (6.29%). majority of the pupils never received private tutoring (86.00%), were male (50.38%) and were not disabled (92.29%). the highest proportion of pupils never attended pre-school (32.57%), were aged 8-9 years (28.05%) and resided in the eastern region (29.18%) of uganda. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 7 table 1. description of respondents variable frequency percent english language literacy very weak 4663 28.39 weak 5346 32.55 average 3301 20.10 good 1033 6.29 very good 2079 12.66 private tutoring no 14496 86.00 yes 2359 14.00 region eastern 4919 29.18 northern 4805 28.51 central 4523 26.83 western 2608 15.47 child's age 6-7 years 4195 24.89 8-9 years 4728 28.05 10-11years 3884 23.04 12-13 years 2658 15.77 14-16 years 1390 8.25 child's gender male 8491 50.38 female 8364 49.62 disability no 15361 92.29 yes 1284 7.71 pre-school never attended 4112 32.57 one year 2279 18.05 two years 2674 21.18 three years 3559 28.19 school type public 11543 69.59 private 5045 30.41 household head gender male 6464 41.03 female 9290 58.97 household head education none 3125 19.64 primary 9390 59.03 secondary 2729 17.15 higher 664 4.17 household size 1-5 members 4603 27.31 6-7 members 5370 31.86 8-9 members 3753 22.27 10 plus members 3129 18.56 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 8 private tutoring and plausible independent variables from table 2, child’s gender, disability and household head gender were the only independent variables that had no significant association (p ≥ 0.05) with private tutoring. the highest proportion of pupils who received private tutoring; resided in the central region of uganda (20.21%), were aged 6-7 years (14.64%), females (14.41%), went to private schools (27.69%) and resided in households with 1-5 members (15.51%) and with heads having at least higher education (22.29%). table 2: association between private tutoring and plausible independent variable variables private tutoring no yes region central 79.79 20.21 eastern 88.72 11.28 northern 91.53 8.47 western 81.48 18.52 = 341.3019 p = 0.000 child's age 6-7 years 85.36 14.64 8-9 years 85.41 14.59 10-11 years 85.99 14.01 12-13 years 86.79 13.21 14-16 years 88.49 11.51 = 11.3465 p = 0.023 child's gender male 86.41 13.59 female 85.59 14.41 = 2.3315 p = 0.127 disability no 86.15 13.85 yes 84.27 15.73 = 3.4999 p = 0.061 school type public 91.91 8.09 private 72.31 27.69 = 1.1e+03 p = 0.000 household head gender male 86.15 13.85 female 85.67 14.33 = 0.7278 p = 0.394 household head education none 89.70 10.30 primary 87.40 12.60 secondary 78.56 21.44 higher 77.71 22.29 = 213.0798 p = 0.000 household size 1-5 members 84.49 15.51 6-7 members 86.61 13.39 8-9 members 86.94 13.06 10 plus members 86.07 13.93 = 13.1905 p = 0.004 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 9 english language literacy and plausible independent variables from table 3, apart from disability status, the rest of the plausible independent variables had a significant association (p ≤ 0.05) with a pupil’s english language literacy performance. the highest proportion of pupils that had very good literacy levels were; privately tutored (22.35%), residents of the central region (21.23%), aged 14-16 years (40.42%), females (13.73%), attended four years of pre-school (40.15%), attended a private school (20.07%), from female headed households (13.58%), from households with 1-5 members (14.27%) and headed by adults with at least higher education (23.05%). multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 10 table 3. association between english language literacy and plausible independent variables variables english language literacy very weak weak average good very good private tutoring no 30.24 33.22 19.45 6.01 11.08 yes 17.10 28.47 24.09 7.99 22.35 = 366.4367 p = 0.000 region central 17.78 27.29 25.18 8.51 21.23 eastern 33.66 36.5 17.35 4.37 8.11 northern 37.13 35.86 15.71 4.67 6.62 western 20.93 28.21 24.47 8.99 17.39 = 1.3e+03 p = 0.000 child's age 6-7 years 46.94 32.88 13.37 2.65 4.15 8-9 years 31.47 35.22 19.40 5.13 8.78 10-11 years 20.39 34.33 23.34 7.77 14.17 12-13 years 15.35 30.83 27.04 10.34 16.44 14-16 years 9.21 20.66 20.36 9.36 40.42 = 2.5e+03 p = 0.000 child's gender male 28.98 33.23 20.08 6.11 11.61 female 27.81 31.87 20.12 6.48 13.73 = 19.7880 p = 0.001 disability no 28.17 32.61 20.19 6.36 12.68 yes 30.5 32.74 18.70 5.70 12.36 = 4.3736 p = 0.358 pre-school never attended 36.89 34.38 16.48 5.15 7.09 one year 27.79 34.01 21.53 5.09 11.58 two years 20.74 33.13 22.7 7.48 15.95 three years 11.53 29.35 27.34 9.28 22.50 four years 13.87 18.25 22.63 5.11 40.15 = 1.1e+03 p = 0.000 school type public 32.78 34.01 18.10 5.61 9.49 private 17.5 29.45 25.03 7.96 20.07 = 717.8426 p = 0.000 household head gender male 30.4 32.47 19.64 6.00 11.48 female 26.83 32.51 20.60 6.48 13.58 = 32.6208 p = 0.000 household head education none 32.54 33.65 19.11 5.85 8.85 primary 31.02 33.39 19.09 5.90 10.60 secondary 18.11 29.05 24.16 7.69 20.99 higher 17.13 28.66 23.36 7.79 23.05 = 498.7572 p = 0.000 household size 1-5 members 25.69 32.01 21.16 6.87 14.27 6-7 members 29.84 31.80 19.54 5.72 13.10 8-9 members 28.54 34.14 19.55 5.98 11.79 10 plus members 29.73 32.76 20.16 6.80 10.56 = 55.0461 p = 0.000 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 11 direct and indirect determinants of english language literacy results on the interrelationship between english language literacy performance, private tutoring and the exogenous variables are presented in table 4. the direct determinants of a pupil’s english language literacy level were region of residence, disability status, school type and household head education. the direct determinants of a pupil’s english language literacy level were private tutoring, child’s age, child’s gender, pre-school attendance, school type, household head gender, household head education and household size. table 4. relationship between english language literacy, private tutoring and exogenous variables variables english language literacy private tutoring or (95% ci) or (95% ci) region central 1.00 1.00 eastern 0.52 (0.47-0.57)** 0.67 (0.59-0.75)** northern 0.51 (0.45-0.56)** 0.68 (0.59-0.79)** western 0.98 (0.88-1.08) 1.25 (1.09-1.42)** private tutoring no 1.00 yes 1.30 (1.17-1.43)** child's age 6-7 years 1.00 1.00 8-9 years 2.31 (2.10-2.54)** 1.08 (0.95-1.22) 10-11 years 4.41 (3.97-4.89)** 1.16 (1.01-1.32)** 12-13 years 7.09 (6.29-7.99)** 1.27 (1.09-1.48)** 14-16 years 22.24 (18.97-26.09)** 0.97 (0.79-1.19) gender male 1.00 female 1.17 (1.09-1.26)** pre-school never attended 1.00 one year 1.30 (1.17-1.46)** two years 1.58 (1.43-1.76)** 3 plus years 2.46 (2.22-2.73)** school type public 1.00 1.00 private 1.72 (1.58-1.86)** 3.78 (3.41-4.18)** household head gender male 1.00 female 1.16 (1.08-1.25)** household head education none 1.00 1.00 primary 1.13 (1.03-1.24)** 1.21 (1.05-1.38)** secondary 2.33 (2.08-2.62)** 1.93 (1.65-2.25)** higher 2.62 (2.17-3.17)** 2.07 (1.65-2.60)** household size 1-5 members 1.00 1.00 6-7 members 0.89 (0.81-0.97)** 0.90 (0.80-1.02) 8-9 members 0.87 (0.78-0.96)** 0.93 (0.81-1.06) 10 plus members 0.82 (0.74-0.92)** 1.02 (0.88-1.117) or-odds ratio, ** p≤ 0.05, (-) variable not considered for particular outcome multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 12 table 4 above provides a summary of the results from the structural equation model to identify both the direct and indirect (mediated through private tutoring) determinants of a pupil’s english language literacy performance. the variables that had a direct significant effect (p ≤ 0.05) on the english language literacy performance of a pupil included private tutoring, child’s age and gender, pre-school attendance, school type, household size, household head gender and education level. concerning private tutoring, for pupils who received private tutoring, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 1.30 times higher than for pupils who never received private tutoring, other variables held constant. regarding child’s age, for pupils aged 8-9 years, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 2.31 times higher than for pupils aged 6-7 years, other variables held constant. the odds increased further for pupils aged 10-11 years (or=4.41), 12-13 years (or=7.09) and 14-16 years (or=22.24). as for gender of a child, for females, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 1.17 times higher than for males, other variables held constant. concerning pre-school attendance, for pupils who attended for one year, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 1.30 times higher than for pupils who never attended other variables held constant. the odds increased even more for pupils who attended for; two years (or=1.58) and three plus years (or=2.46). pertaining to school type, for pupils going to private schools, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 1.72 times higher than for pupils going to public schools, other variables held constant. relating to household head gender, for pupils from female headed households, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 1.16 times higher than for pupils from male headed households, other variables held constant. as regards household head education level, for pupils residing in households with heads having at most primary education, the odds of being very good versus the combined good, average, weak and multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 13 very weak literacy levels were 1.13 times higher than for pupils residing in households with heads having no education, other variables held constant. the odds increased for secondary (or=2.33) and higher (or=2.62) education levels as well. finally, regarding household size, for pupils residing in households with 6-7 members, the odds of being very good versus the combined good, average, weak and very weak literacy levels were 0.89 times lower than for pupils residing in households with 1-5 members, other variables held constant. the odds reduced further for households with 8-9 members (or=0.87) and 10 plus members (or=0.82) the variables that had an indirect effect (mediated through private tutoring) on english language literacy levels of primary pupils through private tutoring were region of residence, school type and household head education level. for pupils residing the eastern region, the odds of receiving private tutoring were 0.67 times lower compared to pupils residing in the central region, other factors constant; subsequently resulting into higher odds of english language literacy levels for pupils from the central region. this wasn’t any different for the northern region (or=0.68). on the contrary, for the western region, the odds of receiving private tutoring were 1.25 times higher compared to pupils residing in the central region, other factors constant. as for school type, for pupils who went to private schools, the odds of receiving private tutoring were 3.78 times higher compared to pupils who went to public schools, other factors constant subsequently resulting into higher english language literacy level for private students. concerning the education level of the household head, for pupils residing in households where the head has primary level education, the odds of receiving private tutoring were 1.13 times higher compared to pupils residing in households where the head had no education, other factors constant. the trend was similar for secondary (or=1.93) and higher (2.07) education levels. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 14 figure 1. results of the structural model adopted for the study 4. discussion the study sought to ascertain the effect of private tutoring on english language literacy levels of primary pupils in uganda. focus was placed on both the direct and indirect (mediating through private tutoring) determinants of english language level literacy. findings from the study indicate the existence of a relationship between private tutoring and english language literacy level, in agreement with cooper (2010) and allen (2015). this is can be attributed to the fact that privately tutored pupils are offered maximum attention hence the tutor has opportunity to identify areas of weakness in a pupils’ learning, try out various teaching methods as well as provide clinical guidance which may help a pupil gain more understanding of subject matter, motivation and positive attitude towards the subject which subsequently translates to better grades. the indirect effect of school type and region of residence can be attributed to the imbalance in access to quality education across the regions of uganda with the central and western regions 3.region multinomial logit english ordinal logit 4.region multinomial logit 2.region multinomial logit tuition bernoulli logit -2.5 1.age_grp ordinal logit 2.age_grp ordinal logit 3.age_grp ordinal logit 4.age_grp ordinal logit 2.gender ordinal logit 1.disability bernoulli logit 1.pre_sch ordinal logit 2.pre_sch ordinal logit 3.pre_sch ordinal logit 4.pre_sch ordinal logit 2.schooltype bernoulli logit 2.hh_gender bernoulli logit 1.hh_edu bernoulli logit 2.hh_edu bernoulli logit 3.hh_edu bernoulli logit 1.hhsize ordinal logit 2.hhsize ordinal logit 3.hhsize ordinal logit -.59 -.035 -.68 .32 .82 1.7 2.6 3.7 .16 .28 .39 .77 .56 .47 .12 .13 .86 .99-.1 -.16 -.21 -.45 .36 -.34 .074.19 .37 .5 .064 .21 1.2 .17 .63 .79 multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 15 having best access with regards to facilities and skilled personnel. these are mostly private schools attended by pupils from families that can afford to educate pupils in them as well as pay for private tutoring classes for pupils who are not performing well. furthermore, most public primary schools are under the universal primary education programme which specifically targeted children from relatively poor households and could not afford higher school fees and hence are not in position to afford private tutoring subsequently resulting into poor academic performance. this is further aggravated by the high levels of pupil-to-teacher ratio and teacher absenteeism, minimum content knowledge by teachers in public schools (the world bank, 2013) and poor facilities that are unconducive for learning. the education level of the household head had both a direct and indirect effect on english language literacy of a pupil since the more educated a parent is, the more likely he or she is likely to invest in his or her child’s education through identifying good schools as well as paying for private tutoring in case a child isn’t performing well. this can be attributed to the appreciation of education as both a right a child is entitled to and also to enable a child earn a livelihood in the future through gainful employment. the indirect effect of disability status can be attributed to the fact children with disabilities tend to have learning challenges especially when taught collectively with fellow pupils during normal class hours hence the likelihood of them receiving private tutoring due to the need for special attention to enable them keep up with their normal counterparts although the subsequent effect on their literacy levels is not significant. 5. conclusion the purpose of this study was to determine the significance of private tutoring on the english language literacy levels of primary pupils in uganda. directly, english language literacy levels were highest among pupils who; received private tutoring, were aged 14-16 years, were females, had attended three years of pre-school, attended private schools, resided in households with 1-5 members, headed by females and with heads who multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 16 had attained higher education. though private tutoring comes with cost implications to parents and increases study pressure on students, it can’t be done away with especially in this era where; school enrollment has greatly increased highly compared to available teacher numbers reducing one to one classroom interaction between pupils and teachers, numerous distractions especially social media and other electronic media take up a lot of student’s time, deteriorating reading culture among others. once regulated and managed well, private tutoring can help increase student time invested in their studies, improve their interest in studies and understanding as well as discipline them through counselling and motivated them. based on the study findings, there is need for the government through the ministry of education to come up with a regulatory framework to manage and control the practice of conducting private tutoring as opposed to the current crackdown and ban on private tutoring yet it would be beneficial especially for children with learning difficulties who through such initiatives can catch up with fellow pupils with regards to understanding taught content. this will eventually translate into better performance for pupils and enable them compete favorably with fellow pupils with regards to enrolling for higher education levels. this will furthermore reduce on student idle time which would have been wasted engaging in unproductive activities. 6. acknowledgment we would like to acknowledge twaweza east africa for providing the data used in the study. 7. references allen b. (2015). after-school tutoring increases academic performance. journal of teaching, learning and scholarship, 2(3). braine m.e., parnell j. (2011). exploring student’s perceptions and experience of personal tutors. nurse education today, 31, 904-910. doi: 10.1016/j.nedt.2011.01.005. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 17 bray m. (2005). private supplementary tutoring: comparative perspectives on patterns and implications. paper presented at the oxford international conference on education and development, oxford-uk. https://doi.org/10.1080/03057920601024974 cole r. (2016). estimating the impact of private tutoring on academic performance: primary students in sri lanka. education economics, 25(2), 142-157. https://doi.org/10.1080/09645292.2016.1196163 cooper e. (2010). tutoring center effectiveness: the effect of drop-in tutoring. journal of college reading and learning, 40(2), 21-34. https://files.eric.ed.gov/fulltext/ej887303.pdf crisp g, cruz i. (2009). mentoring college students: a critical review of the literature between 1990 and 2007. research in higher education, 50, 525-545. doi: 10.1007/s11162-009-9130-2 das g.c., das r. (2013). an empirical view on private tutoring in school mathematics of kamrup district. international journal of scientific and research publications, 3(5). fowler j, norrie p. (2009). development of an attrition risk prediction tool. the british journal of nursing, 18(19), 1194-1200. guerra-martín m.d. (2015). características de las tutorías realizadas por el profesorado de los estudios de enfermería de la universidad de sevilla. sevilla: punto rojo libros. guerra-martín m., lima-serrano m., and lima-rodríguez j.s. (2017). effectiveness of tutoring to improve academic performance in nursing students at the university of seville. journal of new approaches in educational research, 6(2), 93-102. multidisciplinary journal for education, http://dx.doi.org/10.4995/muse.2018.9872 social and technological sciences eissn: 2341-2593 andabati et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 2 (2018): 1-18 | 18 guill k., and bos w. (2014). effectiveness of private tutoring in mathematics with regard to subjective and objective indicators of academic achievement evidence from a german secondary school sample. journal for educational research online, 6(1), 34-67. jones, r., and brown, d. (2011). the mentoring relationship as a complex adaptive system: finding a model for our experience. mentoring & tutoring: partnership in learning, 19(4), 401-418. doi:10.1080/13611267.2011.622077 npa. (2015). pre-primary and primary education in uganda: access, cost, quality and relevance. kampala-uganda: national planning authority. osborn d., cutter a., and ullah f. (2015). universal sustainable development goals: understanding the transformational challenge for developed countries. londonunited kingdom: stakeholder forum. the world bank. (2013). education and health services in uganda. data for results and accountability. washington, d. c-usa: the world bank. unal h., ozkan m., milton s., price k. and curva f. (2010). the effect of private tutoring on performance in mathematics in turkey: a comparison across occupational types. procedia social and behavioral sciences, 2, 5512–5517. uneb. (2015). achievement of primary school pupils in uganda in numeracy and literacy in english. kampala-uganda: uganda national examinations board. unesco. (2014). efa global monitoring report 2013/14. paris-france: united nations education, scientific, and cultural organization. uwezo. (2016). are our children learning? uwezo uganda 6th learning assessment report. kampala-uganda: twaweza east africa. << /ascii85encodepages false /allowtransparency false /autopositionepsfiles true /autorotatepages /none /binding /left /calgrayprofile (dot gain 20%) /calrgbprofile (srgb iec61966-2.1) /calcmykprofile (u.s. web coated \050swop\051 v2) /srgbprofile (srgb iec61966-2.1) /cannotembedfontpolicy /error /compatibilitylevel 1.4 /compressobjects /tags /compresspages true /convertimagestoindexed true /passthroughjpegimages true /createjobticket false /defaultrenderingintent /default /detectblends true /detectcurves 0.0000 /colorconversionstrategy /cmyk /dothumbnails false /embedallfonts true /embedopentype false /parseiccprofilesincomments true /embedjoboptions true /dscreportinglevel 0 /emitdscwarnings false /endpage -1 /imagememory 1048576 /lockdistillerparams false /maxsubsetpct 100 /optimize true /opm 1 /parsedsccomments true /parsedsccommentsfordocinfo true /preservecopypage true /preservedicmykvalues true /preserveepsinfo true /preserveflatness true /preservehalftoneinfo false /preserveopicomments true /preserveoverprintsettings true /startpage 1 /subsetfonts true /transferfunctioninfo /apply /ucrandbginfo /preserve /useprologue false /colorsettingsfile () /alwaysembed [ true ] /neverembed [ true ] /antialiascolorimages false /cropcolorimages true /colorimageminresolution 300 /colorimageminresolutionpolicy /ok /downsamplecolorimages true /colorimagedownsampletype /bicubic /colorimageresolution 300 /colorimagedepth -1 /colorimagemindownsampledepth 1 /colorimagedownsamplethreshold 1.50000 /encodecolorimages true /colorimagefilter /dctencode /autofiltercolorimages true /colorimageautofilterstrategy /jpeg /coloracsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /colorimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000coloracsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000colorimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasgrayimages false /cropgrayimages true /grayimageminresolution 300 /grayimageminresolutionpolicy /ok /downsamplegrayimages true /grayimagedownsampletype /bicubic /grayimageresolution 300 /grayimagedepth -1 /grayimagemindownsampledepth 2 /grayimagedownsamplethreshold 1.50000 /encodegrayimages true /grayimagefilter /dctencode /autofiltergrayimages true /grayimageautofilterstrategy /jpeg /grayacsimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /grayimagedict << /qfactor 0.15 /hsamples [1 1 1 1] /vsamples [1 1 1 1] >> /jpeg2000grayacsimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /jpeg2000grayimagedict << /tilewidth 256 /tileheight 256 /quality 30 >> /antialiasmonoimages false /cropmonoimages true /monoimageminresolution 1200 /monoimageminresolutionpolicy /ok /downsamplemonoimages true /monoimagedownsampletype /bicubic /monoimageresolution 1200 /monoimagedepth -1 /monoimagedownsamplethreshold 1.50000 /encodemonoimages true /monoimagefilter /ccittfaxencode /monoimagedict << /k -1 >> /allowpsxobjects false /checkcompliance [ /none ] /pdfx1acheck false /pdfx3check false /pdfxcompliantpdfonly false /pdfxnotrimboxerror true /pdfxtrimboxtomediaboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxsetbleedboxtomediabox true /pdfxbleedboxtotrimboxoffset [ 0.00000 0.00000 0.00000 0.00000 ] /pdfxoutputintentprofile () /pdfxoutputconditionidentifier () /pdfxoutputcondition () /pdfxregistryname () /pdfxtrapped /false /createjdffile false /description << /ara <feff06270633062a062e062f0645002006470630064700200627064406250639062f0627062f0627062a002006440625064606340627062100200648062b062706260642002000410064006f00620065002000500044004600200645062a064806270641064206290020064406440637062806270639062900200641064a00200627064406450637062706280639002006300627062a0020062f0631062c0627062a002006270644062c0648062f0629002006270644063906270644064a0629061b0020064a06450643064600200641062a062d00200648062b0627062606420020005000440046002006270644064506460634062306290020062806270633062a062e062f062706450020004100630072006f0062006100740020064800410064006f006200650020005200650061006400650072002006250635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e0635062f0627063100200035002e0030002006480627064406250635062f062706310627062a0020062706440623062d062f062b002e> /bgr <feff04180437043f043e043b043704320430043904420435002004420435043704380020043d0430044104420440043e0439043a0438002c00200437043000200434043000200441044a0437043404300432043004420435002000410064006f00620065002000500044004600200434043e043a0443043c0435043d04420438002c0020043c0430043a04410438043c0430043b043d043e0020043f044004380433043e04340435043d04380020043704300020043204380441043e043a043e043a0430044704350441044204320435043d0020043f04350447043004420020043704300020043f044004350434043f0435044704300442043d04300020043f043e04340433043e0442043e0432043a0430002e002000200421044a04370434043004340435043d043804420435002000500044004600200434043e043a0443043c0435043d044204380020043c043e0433043004420020043404300020044104350020043e0442043204300440044f0442002004410020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200441043b0435043404320430044904380020043204350440044104380438002e> /chs <feff4f7f75288fd94e9b8bbe5b9a521b5efa7684002000410064006f006200650020005000440046002065876863900275284e8e9ad88d2891cf76845370524d53705237300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /cht <feff4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef69069752865bc9ad854c18cea76845370524d5370523786557406300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /cze <feff005400610074006f0020006e006100730074006100760065006e00ed00200070006f0075017e0069006a007400650020006b0020007600790074007600e101590065006e00ed00200064006f006b0075006d0065006e0074016f002000410064006f006200650020005000440046002c0020006b00740065007200e90020007300650020006e0065006a006c00e90070006500200068006f006400ed002000700072006f0020006b00760061006c00690074006e00ed0020007400690073006b00200061002000700072006500700072006500730073002e002000200056007900740076006f01590065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f007400650076015900ed007400200076002000700072006f006700720061006d0065006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076011b006a016100ed00630068002e> /dan <feff004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000620065006400730074002000650067006e006500720020007300690067002000740069006c002000700072006500700072006500730073002d007500640073006b007200690076006e0069006e00670020006100660020006800f8006a0020006b00760061006c0069007400650074002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /deu <feff00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f006300680077006500720074006900670065002000500072006500700072006500730073002d0044007200750063006b0065002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /esp <feff005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f00730020005000440046002000640065002000410064006f0062006500200061006400650063007500610064006f00730020007000610072006100200069006d0070007200650073006900f3006e0020007000720065002d0065006400690074006f007200690061006c00200064006500200061006c00740061002000630061006c0069006400610064002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /eti <feff004b00610073007500740061006700650020006e0065006900640020007300e4007400740065006900640020006b00760061006c006900740065006500740073006500200074007200fc006b006900650065006c007300650020007000720069006e00740069006d0069007300650020006a0061006f006b007300200073006f00620069006c0069006b0065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740069006400650020006c006f006f006d006900730065006b0073002e00200020004c006f006f0064007500640020005000440046002d0064006f006b0075006d0065006e00740065002000730061006100740065002000610076006100640061002000700072006f006700720061006d006d006900640065006700610020004100630072006f0062006100740020006e0069006e0067002000410064006f00620065002000520065006100640065007200200035002e00300020006a00610020007500750065006d006100740065002000760065007200730069006f006f006e00690064006500670061002e000d000a> /fra <feff005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f0075007200200075006e00650020007100750061006c0069007400e90020006400270069006d007000720065007300730069006f006e00200070007200e9007000720065007300730065002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /gre <feff03a703c103b703c303b903bc03bf03c003bf03b903ae03c303c403b5002003b103c503c403ad03c2002003c403b903c2002003c103c503b803bc03af03c303b503b903c2002003b303b903b1002003bd03b1002003b403b703bc03b903bf03c503c103b303ae03c303b503c403b5002003ad03b303b303c103b103c603b1002000410064006f006200650020005000440046002003c003bf03c5002003b503af03bd03b103b9002003ba03b103c42019002003b503be03bf03c703ae03bd002003ba03b103c403ac03bb03bb03b703bb03b1002003b303b903b1002003c003c103bf002d03b503ba03c403c503c003c903c403b903ba03ad03c2002003b503c103b303b103c303af03b503c2002003c503c803b703bb03ae03c2002003c003bf03b903cc03c403b703c403b103c2002e0020002003a403b10020005000440046002003ad03b303b303c103b103c603b1002003c003bf03c5002003ad03c703b503c403b5002003b403b703bc03b903bf03c503c103b303ae03c303b503b9002003bc03c003bf03c103bf03cd03bd002003bd03b1002003b103bd03bf03b903c703c403bf03cd03bd002003bc03b5002003c403bf0020004100630072006f006200610074002c002003c403bf002000410064006f00620065002000520065006100640065007200200035002e0030002003ba03b103b9002003bc03b503c403b103b303b503bd03ad03c303c403b503c103b503c2002003b503ba03b403cc03c303b503b903c2002e> /heb <feff05d405e905ea05de05e905d5002005d105d405d205d305e805d505ea002005d005dc05d4002005db05d305d9002005dc05d905e605d505e8002005de05e105de05db05d9002000410064006f006200650020005000440046002005d405de05d505ea05d005de05d905dd002005dc05d405d305e405e105ea002005e705d305dd002d05d305e405d505e1002005d005d905db05d505ea05d905ea002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e05d005de05d905dd002005dc002d005000440046002f0058002d0033002c002005e205d905d905e005d5002005d105de05d305e805d905da002005dc05de05e905ea05de05e9002005e905dc0020004100630072006f006200610074002e002005de05e105de05db05d90020005000440046002005e905e005d505e605e805d5002005e005d905ea05e005d905dd002005dc05e405ea05d905d705d4002005d105d005de05e605e205d505ea0020004100630072006f006200610074002005d5002d00410064006f00620065002000520065006100640065007200200035002e0030002005d505d205e805e105d005d505ea002005de05ea05e705d305de05d505ea002005d905d505ea05e8002e> /hrv (za stvaranje adobe pdf dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. stvoreni pdf dokumenti mogu se otvoriti acrobat i adobe reader 5.0 i kasnijim verzijama.) /hun <feff004b0069007600e1006c00f30020006d0069006e0151007300e9006701710020006e0079006f006d00640061006900200065006c0151006b00e90073007a00ed007401510020006e0079006f006d00740061007400e100730068006f007a0020006c006500670069006e006b00e1006200620020006d0065006700660065006c0065006c0151002000410064006f00620065002000500044004600200064006f006b0075006d0065006e00740075006d006f006b0061007400200065007a0065006b006b0065006c0020006100200062006500e1006c006c00ed007400e10073006f006b006b0061006c0020006b00e90073007a00ed0074006800650074002e0020002000410020006c00e90074007200650068006f007a006f00740074002000500044004600200064006f006b0075006d0065006e00740075006d006f006b00200061007a0020004100630072006f006200610074002000e9007300200061007a002000410064006f00620065002000520065006100640065007200200035002e0030002c0020007600610067007900200061007a002000610074007400f3006c0020006b00e9007301510062006200690020007600650072007a006900f3006b006b0061006c0020006e00790069007400680061007400f3006b0020006d00650067002e> /ita <feff005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f00620065002000500044004600200070006900f900200061006400610074007400690020006100200075006e00610020007000720065007300740061006d0070006100200064006900200061006c007400610020007100750061006c0069007400e0002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /jpn <feff9ad854c18cea306a30d730ea30d730ec30b951fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e305930023053306e8a2d5b9a306b306f30d530a930f330c8306e57cb30818fbc307f304c5fc59808306730593002> /kor <feffc7740020c124c815c7440020c0acc6a9d558c5ec0020ace0d488c9c80020c2dcd5d80020c778c1c4c5d00020ac00c7a50020c801d569d55c002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /lth <feff004e006100750064006f006b0069007400650020016100690075006f007300200070006100720061006d006500740072007500730020006e006f0072011700640061006d00690020006b0075007200740069002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b00750072006900650020006c0061006200690061007500730069006100690020007000720069007400610069006b007900740069002000610075006b01610074006f00730020006b006f006b007900620117007300200070006100720065006e006700740069006e00690061006d00200073007000610075007300640069006e0069006d00750069002e0020002000530075006b0075007200740069002000500044004600200064006f006b0075006d0065006e007400610069002000670061006c006900200062016b007400690020006100740069006400610072006f006d00690020004100630072006f006200610074002000690072002000410064006f00620065002000520065006100640065007200200035002e0030002000610072002000760117006c00650073006e0117006d00690073002000760065007200730069006a006f006d00690073002e> /lvi <feff0049007a006d0061006e0074006f006a00690065007400200161006f00730020006900650073007400610074012b006a0075006d00750073002c0020006c0061006900200076006500690064006f00740075002000410064006f00620065002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006100730020006900720020012b00700061016100690020007000690065006d01130072006f00740069002000610075006700730074006100730020006b00760061006c0069007401010074006500730020007000690072006d007300690065007300700069006501610061006e006100730020006400720075006b00610069002e00200049007a0076006500690064006f006a006900650074002000500044004600200064006f006b0075006d0065006e007400750073002c0020006b006f002000760061007200200061007400760113007200740020006100720020004100630072006f00620061007400200075006e002000410064006f00620065002000520065006100640065007200200035002e0030002c0020006b0101002000610072012b00200074006f0020006a00610075006e0101006b0101006d002000760065007200730069006a0101006d002e> /nld (gebruik deze instellingen om adobe pdf-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. de gemaakte pdf-documenten kunnen worden geopend met acrobat en adobe reader 5.0 en hoger.) /nor <feff004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d00200065007200200062006500730074002000650067006e0065007400200066006f00720020006600f80072007400720079006b006b0073007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c0069007400650074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /pol <feff0055007300740061007700690065006e0069006100200064006f002000740077006f0072007a0065006e0069006100200064006f006b0075006d0065006e007400f300770020005000440046002000700072007a0065007a006e00610063007a006f006e00790063006800200064006f002000770079006400720075006b00f30077002000770020007700790073006f006b00690065006a0020006a0061006b006f015b00630069002e002000200044006f006b0075006d0065006e0074007900200050004400460020006d006f017c006e00610020006f007400770069006500720061010700200077002000700072006f006700720061006d006900650020004100630072006f00620061007400200069002000410064006f00620065002000520065006100640065007200200035002e0030002000690020006e006f00770073007a0079006d002e> /ptb <feff005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020006d00610069007300200061006400650071007500610064006f00730020007000610072006100200070007200e9002d0069006d0070007200650073007300f50065007300200064006500200061006c007400610020007100750061006c00690064006100640065002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /rum <feff005500740069006c0069007a00610163006900200061006300650073007400650020007300650074010300720069002000700065006e007400720075002000610020006300720065006100200064006f00630075006d0065006e00740065002000410064006f006200650020005000440046002000610064006500630076006100740065002000700065006e0074007200750020007400690070010300720069007200650061002000700072006500700072006500730073002000640065002000630061006c006900740061007400650020007300750070006500720069006f006100720103002e002000200044006f00630075006d0065006e00740065006c00650020005000440046002000630072006500610074006500200070006f00740020006600690020006400650073006300680069007300650020006300750020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e00300020015f00690020007600650072007300690075006e0069006c006500200075006c0074006500720069006f006100720065002e> /rus <feff04180441043f043e043b044c04370443043904420435002004340430043d043d044b04350020043d0430044104420440043e0439043a043800200434043b044f00200441043e043704340430043d0438044f00200434043e043a0443043c0435043d0442043e0432002000410064006f006200650020005000440046002c0020043c0430043a04410438043c0430043b044c043d043e0020043f043e04340445043e0434044f04490438044500200434043b044f00200432044b0441043e043a043e043a0430044704350441044204320435043d043d043e0433043e00200434043e043f0435044704300442043d043e0433043e00200432044b0432043e04340430002e002000200421043e043704340430043d043d044b04350020005000440046002d0434043e043a0443043c0435043d0442044b0020043c043e0436043d043e0020043e0442043a0440044b043204300442044c002004410020043f043e043c043e0449044c044e0020004100630072006f00620061007400200438002000410064006f00620065002000520065006100640065007200200035002e00300020043800200431043e043b043504350020043f043e04370434043d043804450020043204350440044104380439002e> /sky <feff0054006900650074006f0020006e006100730074006100760065006e0069006100200070006f0075017e0069007400650020006e00610020007600790074007600e100720061006e0069006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b0074006f007200e90020007300610020006e0061006a006c0065007001610069006500200068006f0064006900610020006e00610020006b00760061006c00690074006e00fa00200074006c0061010d00200061002000700072006500700072006500730073002e00200056007900740076006f00720065006e00e900200064006f006b0075006d0065006e007400790020005000440046002000620075006400650020006d006f017e006e00e90020006f00740076006f00720069016500200076002000700072006f006700720061006d006f006300680020004100630072006f00620061007400200061002000410064006f00620065002000520065006100640065007200200035002e0030002000610020006e006f0076016100ed00630068002e> /slv <feff005400650020006e006100730074006100760069007400760065002000750070006f0072006100620069007400650020007a00610020007500730074007600610072006a0061006e006a006500200064006f006b0075006d0065006e0074006f0076002000410064006f006200650020005000440046002c0020006b006900200073006f0020006e0061006a007000720069006d00650072006e0065006a016100690020007a00610020006b0061006b006f0076006f00730074006e006f0020007400690073006b0061006e006a00650020007300200070007200690070007200610076006f0020006e00610020007400690073006b002e00200020005500730074007600610072006a0065006e006500200064006f006b0075006d0065006e0074006500200050004400460020006a00650020006d006f0067006f010d00650020006f0064007000720065007400690020007a0020004100630072006f00620061007400200069006e002000410064006f00620065002000520065006100640065007200200035002e003000200069006e0020006e006f00760065006a01610069006d002e> /suo <feff004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f00740020006c00e400680069006e006e00e4002000760061006100740069007600610061006e0020007000610069006e006100740075006b00730065006e002000760061006c006d0069007300740065006c00750074007900f6006800f6006e00200073006f00700069007600690061002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a0061002e0020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /sve <feff0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400200073006f006d002000e400720020006c00e4006d0070006c0069006700610020006600f60072002000700072006500700072006500730073002d007500740073006b00720069006600740020006d006500640020006800f600670020006b00760061006c0069007400650074002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /tur <feff005900fc006b00730065006b0020006b0061006c006900740065006c0069002000f6006e002000790061007a006401310072006d00610020006200610073006b013100730131006e006100200065006e0020006900790069002000750079006100620069006c006500630065006b002000410064006f006200650020005000440046002000620065006c00670065006c0065007200690020006f006c0075015f007400750072006d0061006b0020006900e70069006e00200062007500200061007900610072006c0061007201310020006b0075006c006c0061006e0131006e002e00200020004f006c0075015f0074007500720075006c0061006e0020005000440046002000620065006c00670065006c0065007200690020004100630072006f006200610074002000760065002000410064006f00620065002000520065006100640065007200200035002e003000200076006500200073006f006e0072006100730131006e00640061006b00690020007300fc007200fc006d006c00650072006c00650020006100e70131006c006100620069006c00690072002e> /ukr <feff04120438043a043e0440043804410442043e043204430439044204350020044604560020043f043004400430043c043504420440043800200434043b044f0020044104420432043e04400435043d043d044f00200434043e043a0443043c0435043d044204560432002000410064006f006200650020005000440046002c0020044f043a04560020043d04300439043a04400430044904350020043f045604340445043e0434044f0442044c00200434043b044f0020043204380441043e043a043e044f043a04560441043d043e0433043e0020043f0435044004350434043404400443043a043e0432043e0433043e0020043404400443043a0443002e00200020042104420432043e04400435043d045600200434043e043a0443043c0435043d0442043800200050004400460020043c043e0436043d04300020043204560434043a0440043804420438002004430020004100630072006f006200610074002004420430002000410064006f00620065002000520065006100640065007200200035002e0030002004300431043e0020043f04560437043d04560448043e04570020043204350440044104560457002e> /enu (use these settings to create adobe pdf documents best suited for high-quality prepress printing. created pdf documents can be opened with acrobat and adobe reader 5.0 and later.) >> /namespace [ (adobe) (common) (1.0) ] /othernamespaces [ << /asreaderspreads false /cropimagestoframes true /errorcontrol /warnandcontinue /flattenerignorespreadoverrides false /includeguidesgrids false /includenonprinting false /includeslug false /namespace [ (adobe) (indesign) (4.0) ] /omitplacedbitmaps false /omitplacedeps false /omitplacedpdf false /simulateoverprint /legacy >> << /addbleedmarks false /addcolorbars false /addcropmarks false /addpageinfo false /addregmarks false /convertcolors /converttocmyk /destinationprofilename () /destinationprofileselector /documentcmyk /downsample16bitimages true /flattenerpreset << /presetselector /mediumresolution >> /formelements false /generatestructure false /includebookmarks false /includehyperlinks false /includeinteractive false /includelayers false /includeprofiles false /multimediahandling /useobjectsettings /namespace [ (adobe) (creativesuite) (2.0) ] /pdfxoutputintentprofileselector /documentcmyk /preserveediting true /untaggedcmykhandling /leaveuntagged /untaggedrgbhandling /usedocumentprofile /usedocumentbleed false >> ] >> setdistillerparams << /hwresolution [2400 2400] /pagesize [612.000 792.000] >> setpagedevice multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 1 to study and compare perception of health care professionals regarding the role of pharmacist in health care system in pakistan tashfeen akhtar, mehak maqsood, m adnan bashir, umer hayat gul, amina rashid department of pharmacy, the university of lahore, islamabad city campus, 24-west, jinnah avenue, blue area, islamabad, pakistan. tashfeenakhtar87@gmail.com, mehakmaqsood001@gmail.com, adnanbashir28@yahoo.com, umerraja149@gmail.com, fastidious_11@yahoo.com muhammad shahzad aslam* school of bioprocess engineering, universiti malaysia perlis, kompleks pusat pengajian, jejawi 3, 02600 arau, perlis, malaysia. aslammuhammadshahzad@gmail.com saima nisar school of computing, college of art and science, university utara malaysia, sintok, kedah, 06010, malaysia. saimaanisar@gmail.com received: 2016-05-01; accepted: 2017-09-12 abstract the healthcare team is mainly a triad of physicians, pharmacist & nurses. objective: the purpose of this paper is to help healthcare professionals understand more clearly the role of pharmacists within a health care team, especially interprofessional communication, pharmacists' responsibilities, and availability issues. a total of 200 samples were selected from 4 hospitals which include 100 samples of doctors and 100 of the nurses. each sample is basically a questionnaire comprising of 23 questions. a total of two hundred questionnaires were distributed and one hundred and seventy-six questionnaires were returned resulting in the response rate of 88%. pharmacists are being one of the major healthcare professional groups in the world after physicians and nurses are playing a very significant role in health care system. this understanding is a requirement for better communication and collaboration among the professions and for accomplishing the combined goal of better health care system. keywords pharmacist; healthcare; quantitative analysis; pakistan multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 2 1 introduction a health care system is a group of people, resources, organizations or institutions that provide the basic health care services to the targeted population. according to who health report, goals of health care systems' are good health for the public, receptiveness to the expectations of the people, and fair means of financial support (wh report, 2000). a good health system delivers class services to all people, when and where they need them. the effectiveness of any country’s health sector depends upon the budget owed to it. according to the who ranking of healthcare systems of its 191 members, france toped the chart with italy at second number. pakistan is placed at 122nd position (world health report geneva, 2000). britain's national health service (nhs) was recently judged the “world’s best health-care system” by the washington-based commonwealth fund in its latest ranking of 11 rich countries’ health provision. in pakistan, the setup of health sector consists of basic health units (bhu), rural health centers (rhcs) and the district hospitals. the vast network of health care facilities consists of 5,349 basic health units, 4755 dispensaries, 562 rural health centers (rhcs), 945 hospitals, 903 maternal child health centers (mchs) and 290 tb centers (existing infrastructure of healthcare in pakistan, 2009) but functionally it is being abandoned due to the lack of resources (health sector of pakistan, 2014). according to the world health organization (who) health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity (who, 1964). in terms of modern health care delivery, studies have shown that multidisciplinary knowledge is one of the goals for achieving vital population health (anderson, 2002). throughout the year's different healthcare professionals have been doing their duty in their field of expertise yet there has been a greater interest in increasing the collaboration between health care providers. collaborative care is when several health providers work together with patients, their families, caregivers and multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 3 communities to provide high-quality care. it involves engaging any health provider whose expertise can help improve the patient’s health. it has the potential to improve patient care, enhance patient safety and reduce workload issues that cause burnout among healthcare professionals (makowsky, 2009). health providers can also benefit through healthier work environments and increased job satisfaction. health care cannot function without medicines and pharmacists play important roles in achieving desired health and economic outcomes by assuring the proper use of medicines. in many parts of the world, pharmacists have played an important role in the provision of pharmaceutical care services. moreover, pharmacists can not only make a great contribution to the provision of the primary health care in developed countries but also in developing countries. in addition to that, pharmacists have provided clinical proficiency about selection, handling, preparation, procurement, and utilization of medications in patients (higby, 1996) the pharmacy profession is committed to ensuring the safe and effective use of medication and acts as a bridge between health sciences with the basic sciences (hadzović, 1996). pharmacist’s professional roles and responsibilities have progressed from a focus on medication compounding and dispensing to extended pharmaceutical care services (worley et al., 2007). generally, the physician diagnoses and prescribes while the pharmacist compounds and dispenses medicines. however, the pharmacist has gained recognition as an important professional in the multidisciplinary provision of health care. the paradigm shift for pharmacy practice took place in 1990 when hepler and strand introduced the concept of “pharmaceutical care” as the responsible provision of drug therapy for the purpose of achieving definite outcomes that improve the patients’ quality of life by preventing and resolving drug-related problems (hepler and strand, 1990). the concept of pharmaceutical care can only be reached if doctors, pharmacists, and nurses agree on each other’s role because different opinions by pharmacists and doctors regarding the pharmacist’s role could decrease the level of their cooperation (muijrers et multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 4 al., 2003). a better interaction between physicians and pharmacists has led to more effective, safer and less costly drug therapies. several studies on pharmacist–physician collaborations have proven that direct patient care i.e. interaction with the patient and observing his condition, is still wholly in the hands of the physician and that pharmacist participation in managing drug therapy is dependent on the physician’s attitude (morely et al.,1983; griffin et al.,1986). it is therefore important, that physicians understand and appreciate how pharmacists can contribute directly to the overall provision of patient care. physicians’ approval of pharmacists’ services also depends on the value physicians attached to the service and the physicians’ perception of the pharmacist’s competency (thomas, 1991). pharmacists being the third largest healthcare professional group in the world after physicians and nurses are playing a very significant role in health care system in developed countries (babar, 2006). however; their place is not yet acknowledged in developing countries like pakistan. there are several reasons for the little recognition given to the pharmacy profession in pakistan, and this includes less number of pharmacists in the public health sector (khan, 2009). it has been estimated that around 8102 pharmacists are in pakistan with 2,836 working in the public sector, 5,023 in private settings and 243 in non-governmental nonprofit organizations (anonymous, 2007). of the total, approximately 55 % are involved in the production of pharmaceuticals, 15 % in federal and provincial drug control and hospital pharmacy establishments, another 15 % in the sales and marketing of pharmaceuticals, 10 % in community pharmacy, and 5 % in teaching and research (azhar et al., 2011). furthermore, there is no separation of the role of prescribing from that of dispensing in the pakistani setting (azhar and hassali, 2009). continuous development in medication use by society and the expansion of the pharmacist’s role in direct patient care continue to generate demand for pharmacist services (department of health and human services, 2009). at the same time, increased competences for pharmacists’ medication providing roles have been achieved through the multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 5 use of advanced technology (e.g. bar code scanning, e-prescribing). pharmd graduates have the knowledge, skills, and capability to optimize therapeutic outcomes and improve the medication use system. as a result, pharmacists have gained recognition as “medication therapy experts” (owen and burns, 2010). insufficient communication between pharmacists, physicians, and nurses are one of the reasons that affect the patient care. one barrier that has been noted is pharmacists' lack of confidence in their ability to persuade physicians to accept their recommendations (mitchell, 1990). another possible cause is an ineffective or needless communication initiated by pharmacists, which makes physicians less willing to listen to pharmacists during future interactions. in addition, physicians do not see pharmacists as readily available drug information specialists because of location and telephone delays (anonymous, 1983). the increased inter-professional communication seen in certain situations has produced drug therapy that is safer, more effective, and less costly. furthermore, multiple studies report that physicians want pharmacists to communicate with them. in one study, physicians recommended that pharmacists notify them first, even before the patient, when possible problems with drug dose, allergy, or interaction arise (glaser, 1988). in another study of physician opinion, the physicians believed that pharmacists were competent and knowledgeable and thus should be doing more than just dispensing medication (hirsch et al, 1990). several studies on nurse-pharmacist collaboration concluded that many differences regarding medications can be resolved without causing any harm (fletcher et al., 2012). according to another study shared learning between pharmacist and nurse resulted in improving their professional working relationship in order to provide good quality care to patients (horsburgh et al., 2001). the purpose of this paper is to help healthcare professionals (doctors, nurses pharmacists) understand more clearly the role of pharmacists within a health care team, especially inter-professional communication, pharmacists' responsibilities, and availability issues. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 6 furthermore, our study explores pharmacists, physicians, and nurses’ experiences around working as a team and comparison of associated perceptions and expectations. 2 method and material the study was carried out from october through november 2014 in islamabad. it is fundamentally a cross-sectional and comparative study. the study is related to the healthcare system based survey comprises of hospitals of islamabad/ rawalpindi which includes poly clinic hospital, holy family hospitals, pims and benazir bhutto hospital rawalpindi and others. the questionnaire was hand delivered to a random sample. a total of 200 samples were selected from these 4 hospitals which include 100 samples of doctors and 100 of the nurses. each sample is basically a questionnaire comprising of 23 questions. the study consists of 4 sections; (a) demographic information, (b) indicate the level of comfort with pharmacist carrying out their duties, (c) indicate expectations about pharmacist working in different sectors and (d) indicate experience with a pharmacist. section b of the questionnaire includes a set of statements about pharmacist role in which the respondents were asked to indicate their level of agreement opting from 3 choices with 1 = uncomfortable, 2 = moderately comfortable and 3 = comfortable. six questions were asked each from section c and d and options were asked using a 5-point likert scale, where 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree. the data were computed and analyzed using statistical package for social sciences (spss, version 20) and descriptive analysis was conducted. the results of each item in the questionnaire were reported as percentage and frequencies. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 7 3 result a total of two hundred questionnaires were distributed and one hundred and seventy-six questionnaires were returned resulting in the response rate of 88%. details of the physicians’ personal information are given in table 1. there were no associations between physician variables such as age, gender, nationality, year of graduation and their perception of pharmacists. table 1 demographic data analysis profession doctor nurse n % n % 25-30 years 50 29.2% 47 27.5% 31-35 years 15 8.8% 14 8.2% 36-40 years 18 10.5% 15 8.8% > 40 years 5 2.9% 7 4.1% male 46 27.5% 36 21.6% female 39 23.4% 46 27.5% ali hospital 2 1.1% 0 0.0% ali medical center 4 2.3% 10 5.7% benazir bhutto hospital 11 6.2% 4 2.3% bilal hospital 1 0.6% 0 0.0% bph 2 1.1% 0 0.0% cda hospital 1 0.6% 0 0.0% cmh rawalpindi 11 6.2% 18 10.2% dhq hospital 2 1.1% 1 0.6% fauji foundation hospital 2 1.1% 5 2.8% holy family hospital 21 11.9% 6 3.4% kulsoom international hospital 4 2.3% 7 4.0% mahroof international hospital 2 1.1% 2 1.1% nirm 1 0.6% 0 0.0% multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 8 ogdcl 1 0.6% 0 0.0% pims 7 4.0% 4 2.3% polyclinic hospital 3 1.7% 10 5.7% quaid-e-azam international hospital 2 1.1% 2 1.1% shifa international hospital 11 6.2% 19 10.8% <1 year 12 7.4% 7 4.3% 1-10 years 63 38.7% 63 38.7% 11-20 years 8 4.9% 8 4.9% >20 years 2 1.2% 0 0.0% the results of the level of comfort of health care professionals in dealing with pharmacists carrying out specific duties are shown in ¡error! no se encuentra el origen de la referencia.. table 2 level of comfort of health care providers in dealing with pharmacist pharmacist’s role uncomfortabl e moderately comfortable comfortable n % n % n % providing patient education 30 17.3% 95 54.9% 48 27.7% suggesting use of non-prescription medications 50 29.4% 67 39.4% 53 31.2% suggesting use of certain prescription medications to patients 34 20.1% 80 47.3% 55 32.5% suggesting use of certain prescription medications to physician and nurses 21 12.3% 74 43.3% 76 44.4% treating to minor illnesses 42 24.6% 72 42.1% 57 33.3% identifying and preventing prescription errors 28 16.1% 73 42.0% 73 42.0% designing and monitoring pharmacotherapeutic regimens 29 16.8% 92 53.2% 52 30.1% monitoring outcomes of pharmaco24 14.0% 86 50.0% 62 36.0% multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 9 therapeutic regimens and plans figure 1 level of comfort with providing patient education according to the results, 17% doctors were uncomfortable regarding pharmacists role in providing patient education. 59% showed a moderate level of comfort while 22% felt comfortable with ±5% confidence interval. nurses also showed the same results with 17% as uncomfortable, 50% moderately comfortable while rest of the 32% was comfortable with ±5% confidence interval as shown in figure 1. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 10 figure 2 level of comfort with suggesting use of non-prescription medications regarding suggesting the use of non-prescriptions medications, 30% doctors were uncomfortable, 45% were moderately comfortable and 23% were comfortable with ±5% confidence interval. in the case of nurses, 28% were uncomfortable and 32% and 38% were moderately comfortable and comfortable respectively with ±5% confidence interval as shown in figure 2. figure 3 level of comfort with pharmacists' suggestion of prescription medications multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 11 19% doctors were uncomfortable with pharmacists suggesting prescription medicines to patients, 53% were moderately comfortable and 27% were comfortable with ±5% confidence interval. in nurses the ratio was 21%, 41% and 37% respectively with ±5% confidence interval as shown in figure 3. figure 4 level of comfort with treating minor illnesses suggesting prescriptions medicines to other healthcare professionals was perceived in case of doctors as 16%, 44% and 39% and in case of nurses 8%, 42% and 49% as uncomfortable, moderately comfortable and comfortable respectively with ±5% confidence interval. the pharmacist can treat minor illnesses and in this respect 39% doctors and 44% nurses showed a moderate level of comfort. 29% doctors and 21% nurses were uncomfortable with this notion while 32% doctors and 34% nurses were comfortable with ±5% confidence interval (figure 4) pharmacists’ role as monitoring the outcomes for pharmacotherapeutic plans is acknowledged with doctors showing 47% and nurses showing 52% moderate comfortability with ±5% confidence interval. 36 % doctors and 35% nurses were comfortable giving us nearly equal results. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 12 expectations of healthcare professionals about pharmacists working in different sectors are summarized in ¡error! no se encuentra el origen de la referencia.. table 3 expectations about pharmacists working in different sectors expectation s strongly disagree disagree neutral agree strongly agree n % n % n % n % n % a 1 6 9.2 % 2 6 14.9 % 4 0 23.0 % 7 1 40.8 % 2 1 12.1 % b 2 1.1 % 1 9 10.9 % 2 1 12.1 % 9 5 54.6 % 3 7 21.3 % c 7 4.1 % 2 5 14.5 % 3 5 20.3 % 7 2 41.9 % 3 3 19.2 % d 6 3.5 % 1 6 9.4% 2 8 16.5 % 7 9 46.5 % 4 1 24.1 % e 6 3.4 % 1 8 10.3 % 3 5 20.1 % 7 4 42.5 % 4 1 23.6 % f 1 4 8.1 % 3 4 19.8 % 4 1 23.8 % 5 5 32.0 % 2 8 16.3 % a = to take personal responsibility for resolving any drug-related problems they discover involving patients. b = to be knowledgeable drug therapy experts c = to assist in designing drug therapy treatment plans for patients d = to educate patients about the safe and appropriate use of their medication e = to monitor patients’ response to drug therapy and give report if a patient encounter any drug-related problem f = to assist patients in selecting appropriate non-prescription medications multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 13 figure 5 expectations regarding pharmacists' responsibility for resolving drug related problems 42% doctors and 39% nurses expect pharmacists to take personal responsibility regarding any drug related problem that occur involving the patient with ±5% confidence interval as shown in figure 5. figure 6 expectations about pharmacists as drug therapy expert multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 14 according to this study 61% nurses and 48% doctors believe pharmacists to be knowledgeable drug therapy experts. 12% doctors and 11% nurses gave a neutral opinion in this regard with ±5% confidence interval as elaborated in figure 6. with respect to pharmacist’s role in assisting in designing drug therapy 6% doctors disagree strongly, 18% disagree, 20% are neutral, 39% agree and 14% strongly agree. 1% of nurses disagree strongly, 10% disagree, 20% are neutral, 44% agree and 23% strongly agree. the results are with ±5% confidence interval. figure 7 expectations regarding the monitoring of patients' response to drug therapy pharmacists are to provide patient education regarding safe medication. 3% doctors disagree strongly, 15% disagree, 17% are neutral, 44% agree and 18% strongly agree. 3% nurses disagree strongly, 3% disagree, 15% are neutral, 48% agree and 24% strongly agree. the results are with ±5% confidence interval. pharmacists’ role as monitoring the outcomes for pharmacotherapeutic plans and reporting any problem that the patient acquires during the course of therapy is known as 3% doctors disagree strongly, 11% disagree, 22% are neutral, 41% agree and 20% strongly agree. 3% nurses disagree strongly, 9% disagree, 11% gave neutral opinion, 43% multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 15 agree and 26% strongly agree. the results are with ±5% confidence interval as shown in figure 7. different opinions on pharmacist assistance in selecting the medication are; 7% doctors disagree strongly, 21% disagree, 31% are neutral, 27% agree and 12% strongly agree. 9% nurses disagree strongly, 18% disagree, 16% are neutral, 36% agree and 19% strongly agree. the results are with ±5% confidence interval. table 4 experience of health care professionals with pharmacists experienc e strongly disagree disagree neutral agree strongly agree n % n % n % n % n % a 8 4.7 % 2 3 13.4 % 3 0 17.4 % 8 8 51.2 % 2 3 13.4 % b 1 0.6 % 2 7 15.6 % 2 9 16.8 % 9 0 52.0 % 2 6 15.0 % c 1 5 8.9 % 3 5 20.7 % 4 1 24.3 % 4 8 28.4 % 3 0 17.8 % d 9 5.4 % 2 0 12.0 % 3 2 19.2 % 7 3 43.7 % 3 3 19.8 % e 1 0 6.0 % 2 3 13.7 % 3 3 19.6 % 7 0 41.7 % 3 2 19.0 % f 7 4.2 % 1 3 7.7% 2 5 14.9 % 8 0 47.6 % 4 3 25.6 % a = in my experience, pharmacists are a reliable source of general drug information (i.e., specific facts about drugs which can be found in standard references) b = in my experience, pharmacists are a reliable source of clinical drug information (i.e., information regarding the clinical use of drugs in specific situations) c = pharmacists routinely counsel patients regarding the safe and appropriate use of their medications multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 16 d = pharmacists routinely inform if they discover clinical problems with prescriptions’ e = pharmacists routinely inform about more cost-effective alternatives to the drugs prescribed f = i am willing to in-corporate the pharmacotherapy for the patient with consultation of the pharmacist figure 8 pharmacist: a reliable source of drug information in case of pharmacists as general drug information source 6% doctors disagree strongly, 15% disagree, 19% are neutral, 47% agree and 10% strongly agree. 2% of nurses disagree strongly, 11% disagree, 15% are neutral, 54% agree and 16% strongly agree. the results are with ±5% confidence interval. 47 % doctors and 56% nurses agree that the pharmacists are reliable source of information regarding clinical scenarios while 19% doctors and 11% nurses disagree with this. the results are with ±5% confidence interval as described in figure 8. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 17 figure 9 counseling the patient 23% doctors agree that pharmacists routinely counsel patients with respect to safe and appropriate use of medications. almost same percent of doctors i.e. 25% disagree with this point thus giving us uncertain results in this respect. on the other hand 32% of nurses agree with pharmacists’ counseling as shown in figure 9. the results are with ±5% confidence interval. pharmacists routinely inform about any clinical problem regarding prescription. 4% doctors disagree strongly, 19% disagree, 19% are neutral, 38% agree and 18% strongly agree. 5% of nurses disagree strongly, 4% disagree, 19% are neutral, 48% agree and 21% strongly agree. the results are with ±5% confidence interval. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 18 figure 10 pharmacists' role in cost effective treatment 42% nurses followed by 40% doctors believe that pharmacists routinely inform about cost effective treatment by giving an alternative to the prescribed medications. 15% doctors disagree and 14% strongly agree. 21% of nurses gave a neutral opinion with 23% agreeing strongly. the results are with ±5% confidence interval as shown in figure 10. a successful healthcare system is possible only in the case if the healthcare professionals work in corporation with each other. according to this survey somewhat positive results have been achieved with 51% doctors and 43% of nurses willing to work in coordination with the pharmacist for the better pharmacotherapy for the patient.6% of the doctors and 9% nurses disagree while 18% doctors and 32% nurses strongly agree. the results are with ±5% confidence interval. 4 discussion health care system of every country complies with the policies of the government of that particular country. if we talk about the health care system of pakistan is basically categorized into several health care units dhqh (district headquarter hospital) working on district level there at least one hospital on district level it may also be increased to 2 or multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 19 3. then there is one hospital on tehsil level covering all the areas of that particular tehsil called as thqh (tehsil headquarter hospital). every tehsil is governing several day care centers and small dispensaries on union council level. health care system of pakistan also constitutes several private hospitals giving people health care services these hospitals may be working under some trust established for the welfare of people or may be established by the people for the sake of earning. some of the professionals also established their own setup in the form of the clinic and providing people health care. health care system of pakistan is built of three basic pillars physician, nurses, pharmacist. physician working in different hospitals have a well-defined role and therefore they feel the least difficulty after their degree. same is the case with the nursing field they also have a well-established role in the health care system of the pakistan. but pharmacist is the only pillar of health care profession whose role is still not defined in any field either he is working as a clinical pharmacist, hospital pharmacist or the community pharmacist. if we talk about the policies of our government for the promotion of pharmacy profession we hardly find any work done in this perspective. although pharmacy professional were the first time graduated late in the sixties. the pharmacy profession is progressing at snail’s pace which must be boosted a little in order to publish this profession in our society. although some of the private organizations have hired pharmacist they (pharmacist) have lemmatized themselves to the pharmacy and are not available on the bedside for the continuous monitoring of the patient medication. there are hardly some hospitals in which pharmacist are available on the bedside for the evaluation of patient medication. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 20 so we conducted this work to re-evaluate of the opinion of other health care professionals regarding the positioning of the pharmacist in health care setups because still other health care professionals still consider pharmacist as the dispenser. the demographic data includes age and gender of the selected health care professional (nurse and doctor), working place (name of hospital) and practical experience (in years). the assistance or the guidance provided by the pharmacist to a patient in a hospital in collaboration with nurses and doctors is to educate the patient regarding the proper use of the medicine in order to evoke his/her role in the public. 17% doctors were uncomfortable regarding pharmacists role in providing patient education. 59% showed a moderate level of comfort while 22% felt comfortable. nurses also showed the same results with 17% as uncomfortable, 50% moderately comfortable while rest of the 32% was comfortable. this was because of the lack of the direct contact of a pharmacist with the patient. otc drugs are very often used by the people in our society so; this need can also be fulfilled if the availability of pharmacist is ensured to the expense of the people. awareness of otc drugs must be launched by the pharmacist. proper use of the medication by the patient is also very necessary to get him better in the shortest enlisted time. example proper distances while the use of inhaler by asthmatic is very necessary and should be taught. again in some of the hospital pharmacist were available to educate the people and in some, they were absent so that is why we obtained 31.2% are comfortable only and many were in the opinion of moderate comfortable the prescription is needed to be evaluated before the medicine is dispensed to the patient. so that the chances of error be minimized. expert in medicine can sort errors in a better way than any other. cumulatively 42% of the professionals do admit about the expertise of the pharmacist. the only little number was not agreeing because of the lack of awareness. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 21 drug regimen should be designed by considering pharmacoeconomics and pharmacovigilance perspective of the patient and should save the record of the given medicine to facilitate for the next time when patient visits. deficit regarding this point was a lack of data sorting devices and again the lack of pharmacist in the setup. doctors still don’t want to give this authority to pharmacist therefore on 30.1 comply with the level comfortable regarding designing of the drug regime. the pharmacy profession is all about expertise in the field of medicine than these duties are obligatory for the pharmacist. resolving any drug-related problems they discover involving patient if the patient complains about any drug-related problem to the physician he should tell him to consult a pharmacist. the physician will only send him to the pharmacist if he recognizes his designation of the pharmacist which is still lacking in our health care system. 41% as whole agree with resolving of drug related issues by the pharmacist. if we talk about the private setups in pakistan pharmacist are playing their role clinical setting and are serving as the source of knowledge regarding the drug therapy prescribed to the patient. they are also sharing the responsibility for the designing of the drug regime selected by the doctor. now after the selection of the drug regime proper use of the drug should also be ensured by the pharmacist working. all health care professionals do admit this designation of the pharmacist where he is available. in government hospitals presence of pharmacist should be ensured. the maximum level of disagrees regarding the roles was from the government hospitals. pharmacovigilance and pharmacokinetics are the two foremost duties of the pharmacist regarding the continuation of the medicine until relieved and selection of the appropriate brands which suits to the economic conditions of the patient. because many brands are available in the market multinational and the local ones, if the generic is prescribed this issue of the selection of the brand according to the patient economic condition, would be solved. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 22 so that is why drug regime should be designed in collaboration with a pharmacist. many health care professionals comply with us regarding the expertise of pharmacist in medicine many of them filled agree and strongly agree on the box that was aware of the responsibilities of a pharmacist and did spend their time with him. many of them were neutral and disagreeing with us specifically in the government institutes. when we visited hospitals many of the doctors refused to fill the questionnaire and we came across one more experience a nurse serving in the government hospital wrote a statement on the questionnaire when she was filling ‟i am here but where is pharmacist”. the government of pakistan should work upon the basic and third pillar of the health care profession in order to improve the health of society. advanced countries have improved their health because they did specify the role of the pharmacist and designated his status in the health care system. ministry of health should pay special attention in this regard so that we may start moving on the road map of the countries like u.s having strong clinical setup. ministry should create vacancies in the government hospitals for the clinical and hospital pharmacist. pharmacy council should also revise the curriculum and should pay special attention to the institutes running degree program in pharmacy that either they are fulfilling the criteria of teaching in all aspects. 5 conclusion health care system of pakistan needs to be revalidated in comparison to the advanced countries to raise the status of health. this can only be done if every single member of the health care contributes its role physician, pharmacist or nurses at their end. pharmacist working in different health care systems should work at their best in order raise the level of their field in healthcare system of pakistan. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 23 6 references anderson, s. (2002). the state of the world's pharmacy: a portrait of the pharmacy profession. journal of interprofessional care, 16 (4), 391-404. https://doi.org/10.1080/1356182021000008337 anonymous (2007). report of health system review mission pakistan. azhar, s., hassali, m., & ibrahim, m. perceptions of hospital pharmacist’s role in pakistan’s healthcare system: a cross-sectional survey. tropical journal of pharmaceutical research , 10 (1), 2011. https://doi.org/10.4314/tjpr.v10i1.66535 anonymous, (1983). schering report v; pharmacists and physicians: attitudes and perceptions of two professions. kenilworth, nj: schering corporation. azhar, s., & hassali, m. (2009). evaluating the perception of doctors towards the role of pharmacist in pakistan's healthcare system. malaysia journal of pharmacy , 1 (7), 105. babar, z. (2006). pakistan national university of pharmaceutical sciences. am j pharm educ , 70 (123). department of health and human services, health resources and services administration, bureau of health professions,. (2008, december). the adequacy of pharmacist supply: 2004 to 2030 . fletcher, j., hogg, w., farrell, b., woodend, k., dahrouge, s., lemelin, j., & dalziel, w. (2012). effect of nurse practitioner and pharmacist counseling on inappropriate medication use in family practice. canadian family physician, 58(8), 862-868. higby, g. (1996). pharmaceutical care. in c. knowlton, & r. penna, from compounding to caring: an abridged history of american pharmacy (pp. 18-45). new york: chapman & hall. hadzović, s. (1996). pharmacy and the great contribution of arab-islamic science to its development. medicinski arhiv , 51 (1-2), 47-50. https://doi.org/10.1080/1356182021000008337 https://doi.org/10.4314/tjpr.v10i1.66535 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 24 health sector of pakistan : redefining roles and overcoming challenges . (2014, april). retrieved october 2014, from journal of pioneering medical sciences blog: http://blogs.jpmsonline.com/2014/04/29/health-sector-of-pakistan-redefining-roles-andovercoming-challenges/ hepler, c., & strand , l. (1990). opportunities and responsibilities in pharmaceutical care. journal of american pharmacists association , 47 (3), 533-543. hirsch, j., gagnon, j., & camp, r. (1990). value of pharmacy services: perceptions of consumers, physicians, and third party prescription plan administrators. am pharm , 2025. https://doi.org/10.1016/s0160-3450(15)31531-2 horsburgh, m., lamdin, r., & williamson, a. (2001). multi professional learning: the attitudes of medical, nursing and pharmacy students to shared learning. med educ , 35 (9), 876-883. https://doi.org/10.1046/j.1365-2923.2001.00959.x griffin, j. (1986). more physician–pharmacist cooperation needed. n engl j med , 315 (17), 1097. https://doi.org/10.1056/nejm198610233151719 glaser, m. (1988). what today's doctors think about pharmacists. drug top , 132, 3234,38,40. khan, a. (2009). re-defined role of pharmacist in public sector hospitals of pakistan. 15th international pharmacy conference and exhibition. lahore: ppa. mitchell, j. (1990). building cooperation with physicians: an interview with charles fortner. am pharm , 24-26. https://doi.org/10.1016/s0160-3450(15)31555-5 makowsky, m., schindel, t., rosenthal, m., campbell, k., tsuyuki, r., & madill, h. (2009). collaboration between pharmacists, physicians and nurse practitioners: a qualitative investigation of working relationships in the inpatient medical setting. journal of interprofessional care , 23 (2), 169-184. https://doi.org/10.1080/13561820802602552 https://doi.org/10.1016/s0160-3450(15)31531-2 https://doi.org/10.1046/j.1365-2923.2001.00959.x https://doi.org/10.1056/nejm198610233151719 https://doi.org/10.1016/s0160-3450(15)31555-5 https://doi.org/10.1080/13561820802602552 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7714 social and technological sciences issn: 2341-2593 akhtar et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 1-25 | 25 muijrers, p., ja, k., j, s., r, j., & rp, g. (2003). changing relationship:attitude and opinion of general practitioners and pharmacist regarding the role of the community pharmacist. pharm world sci, 25 (5), 235-241. https://doi.org/10.1023/a:1025872907411 morely, a., jepson, m., edwards, c., & stillman, p. (1983). what do doctors think of pharmacists treating minor ailments? pharm j. (231), 387-388. world health report 2000 – health systems: improving performance. geneva, who. (2000). retrieved october 2014, from world health organization: http://www.who.int/whr/2000/en/index.html who: preamble to the constitution of the world health. (1964). international health conference. new york. worley, m., schommer, j., brown, l., hadsall, r., ranelli, p., stratton, t., et al. (2007). pharmacists' and patients' roles in the pharmacist-patient relationship: are pharmacists and patients reading from the same relationship script? research in social and administrative pharmacy , 3 (1), 47-69. https://doi.org/10.1016/j.sapharm.2006.03.003 world health report (2000). geneva: world health organization. owen, j. a., & anne burns. (2010). medication use, related problems and medication therapy management,” in the pharmacist in public health: education, applications, and opportunities,”. american pharmacists association . thomas, p. (1991). professional relationships between general practitioners and pharmacists in health centers. br j gen pract , 41, 84-85. https://doi.org/10.1023/a:1025872907411 https://doi.org/10.1016/j.sapharm.2006.03.003 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 1 optimization approaches for robot trajectory planning carlos llopis-albert*, francisco rubio**, francisco valero** * departamento de ingeniería mecánica y de materiales. universitat politècnica de valència, camí de vera s/n, valencia, 46022, spain. email: cllopisa@upvnet.upv.es ** centro de investigación en ingeniería mecánica. universitat politècnica de valència, camí de vera s/n, valencia, 46022, spain. emails: frubio@mcm.upv.es; fvalero@mcm.upv.es received: 2017-11-03; accepted: 2018-02-13 abstract the development of optimal trajectory planning algorithms for autonomous robots is a key issue in order to efficiently perform the robot tasks. this problem is hampered by the complex environment regarding the kinematics and dynamics of robots with several arms and/or degrees of freedom (dof), the design of collision-free trajectories and the physical limitations of the robots. this paper presents a review about the existing robot motion planning techniques and discusses their pros and cons regarding completeness, optimality, efficiency, accuracy, smoothness, stability, safety and scalability. keywords algorithms, optimal trajectory, kinematic and dynamic constraints, minimum time, energy, obstacle avoidance. 1. introduction trajectory planning is moving a robot between two different configurations over time in order to perform a certain task while fulfilling robot’s constraints. a certain configuration entails a set of joint angles of the robot manipulator and the set of all possible joint angles is called the configuration space. the constraints encompass the physical https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 2 limitations of the robot. they include geometric constraints, which can be expressed in terms of the robot joint angles (i.e., bounds on the joint angles, avoidance of collision with the environment). they also cover kinematics and dynamics constraints that include higher-order time derivatives of the joint angles (i.e., bounds on the joint velocities, accelerations, torques, or motor current inputs). furthermore, the task should be performed between the successive configurations in an efficiently and accurately way while optimizing a certain objective, such as minimizing the path traveling distance or execution time, energy consumption (or actuator effort) and jerk or maximizing the smoothness ([1], [2], [3], [4], [5]). this article reviews the most significant methodologies in trajectory planning of mobile robots with kinematics and dynamics constraints and optimization objectives. on the one hand, inverse kinematics finds a continuous set of intermediate joint angles of the robot arms between the starting and goal joint angles that allow achieving the desired end-effector position and orientation while avoiding collisions. on the other hand, through a time parameterization the algorithms allow to meet the torque bounds and/or optimizing the execution time or the energy consumption. finally, a controller takes the inputs and adjusts its outputs by defining a sequential motion law so that the robot can carry out its task. the inputs cover the geometric path, the kinematic and dynamic constraints, while the output are the trajectory of the joints, expressed as a time sequence of position, velocity and accelerations. 2. trajectory planning algorithms 2.1. classic approaches path planning entails the generation of a geometric path without a time law, while the trajectory planning assigns a time law to the geometric path. two main categories of trajectory planning algorithms can be distinguished in accordance to the available https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 3 information, namely off-line and on-line. off-line robots compute the entire trajectory to the goal before motion begins (i.e., information about obstacles is known in advance), and may lead to globally optimal solutions when the environment is fully known. in this category different issues are analyzed, such as optimality (local and global), complete (a solution will be found if exists), and computational cost and efficiency (allow changes without recomputing or replanning everything). on-line robots generate the trajectory to the goal incrementally during motion, and lead to locally optimal solutions at best. in this case, the mobile robot obtains the information through sensors while it moves through the environment. in this category the issues raised are completeness (is the robot guaranteed to reach the goal if a solution exists), computational cost and efficiency at each step, and optimality (how far is a solution from the optimal and is it bounded by an upper limit). there exists a large variety of approaches to trajectory planning. the most important classical techniques are bug-like algorithms, the combinatorial methods, potential field methods and sampling-based methods. the bug-like algorithms are among the earliest and simplest sensor-based algorithms with reasonable results [6]. robot is assumed to be a point in the plane with perfect positioning and with the workspace bounded. they have a contact sensor, which detects the obstacle boundary if it touches it. they are straightforward to implement since entail a movement towards the goal, unless an obstacle is encountered. in that case, they circumnavigate the obstacle until motion toward the goal is again allowable. this is achieved by measuring the distance between any two points. combinatorial methods are geometric representation planners, based on the configuration space as the fundamental concept, which are used by most off-line robots. the geometric representations of the environment may consist of roadmaps or graphs that capture the topology of the free space, generated by different well-known methods such as a voronoi diagram ([7], [8]); a visibility graph ([9], [10], [11], [12]), a https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 4 tangent graph [13]; cell decomposition and grid method ([14], [15], [16], [17], [18], [19], [20], [21]); silhouette [22], and the subgoal network [23]. they differ in the way it represents the free space (non-collision space), but all are based on a connected network of path segments that can be traversed from start to goal. the main computational effort in these approaches is the representation of the free space, which includes the mapping of obstacles. once the roadmap is constructed, the search for the shortest trajectory is carried out by using standard graph search techniques such as dijkstra’s search [24] or a* [25]. these methods have the advantage of providing that the general motion planning problem is np-complete, but they have the disadvantages of being too slow to be used in practice, especially in high-dimensional problems, and to require an explicit representation of obstacles, which is very complicated to obtain in most practical problems. another approach is to overlay a uniform grid over the search space and represent the entire space by an undirected graph [2]. these methods assign high costs to edges that intersect obstacles, which allows to effectively separate between inaccessible nodes and nodes in the free space. the resolution of this method is complete, as all approaches based on a discrete representation of the search space, which implies that at low grid resolutions paths that pass through tight spaces between obstacles can be disregarded. instead an increase in the graph resolution would lead to high computational effort. as a disadvantage, the number of nodes for the uniform grid representation is much greater than for the roadmap-based algorithms. however, this approach is applicable to problems where obstacles are not clearly defined, such as for mobile robots. the potential field method constructs a potential field which is high near the obstacles and low at the goal configuration ([1], [26]). the robot is guided towards the goal configuration while avoiding the obstacles by letting its configuration evolve in that potential field. that is, the robot is attracted towards the goal configuration and repulsed from the obstacles. the gradient is a vector which points in the direction that locally https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 5 maximally increases the artificial potential field and the local variations of the robot reflect the structure of the free space. this method allows real-time control, but the possibility of getting trapped in a local minimum of the potential field prevents its use in highly cluttered environments. sampling-based methods probably are the most widely used methods for trajectory planning because they are efficient and robust algorithms. contrary to previous algorithms, the sampling-based planners accepts probabilistic completeness, i.e., the goal may not be reached in a finite time; accepts any solution, not necessarily the optimal; and neglects the explicit geometric representation of the free configuration space in terms of roadmaps or graphs. a roadmap is a graph whose vertices are configurations of free space and connects them by a path entirely contained in the free space. there are two ways of building the roadmap, i.e., by a deterministic or probabilistic approach. in the probabilistic roadmap planner (prm) instead of following a regular grid, samples are taken at random in free space. since there is no a priori grid structure, there are several methods for choosing the pairs of vertices to make the connection. this approach works very well for a wide variety of problems ([27], [28]), and it is based on the fact that checking if a single robot configuration is in the free space is less computationally expensive. prm creates a roadmap in the free space using a coarse sampling to obtain the nodes of the roadmap and a fine one to obtain the roadmap edges (i.e., the free paths between node configurations). then, planning queries can be answered by connecting the initial and goal configurations to the roadmap. a uniform random distribution ensures the probabilistic completeness of the planner [29]. there are other sampling-based planners depending on the node sampling scheme that may be more effective for single-query planning, such as the expansive-spaces tree planner (est) [30] and the rapidly-exploring random tree planner (rrt) [24]. there are also methods based on a combination of the previous methods, such as the sampling-based roadmap of trees (srt) method, which https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 6 constructs a roadmap using a prm and single-query trees. it has been observed that for very difficult path planning problems, single-query planners need to construct large trees in order to find a solution. in some cases, the computational cost of constructing a large tree may be high and it is worthy to use a multiple-query planning. sampling-based methods are able to deal with robots with many degrees of freedom and constraints. for instance, kinematic and dynamic constraints, energy and stability constraints, closed-loop kinematics, visibility and constraints, and reconfigurable robots. as a summary, the main disadvantages of classic approaches that make them inefficient in practice are that they entail a high computational cost to determine a feasible collision-free path in high dimensions; tend to get locked in local optimal solution; lead to non-deterministic polynomial time hard problems (np-hard) for trajectory planning of mobile robots with multiple obstacles [22]; and the solution is quite complicated when the environment is dynamic and complex [31]. these drawbacks prevent their use in complex environments. 2.2. heuristic approaches to solve the aforementioned drawbacks of classic approaches the heuristic and metaheuristic approaches have been developed. they encompass methods such as probabilistic roadmaps (pr); rapidly-exploring random trees (rrt); ant colony optimization (aco), that relies on the foraging behavior of ants for finding the shortest path to the food source ([32], [33], [34]); simulated annealing (sa), which is a heuristic random search approach that resembles the cooling process of molten metals through annealing ([35], [36]); neural network [37]; genetic algorithms (ga), which are based on the mechanics of natural genetics and selection ([38], [39], [40]); particle swarm optimization (pso), which are inspired by social behavior of bird flocking or fish schooling and are easier to implement than ga and with a fewer parameters to be adjusted ([41], [42], [43], [44], [45], [46]); stigmergy, which is a mechanism of indirect https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 7 coordination, through the environment, between agents or actions [47]; wavelet, which is based on wave-like oscillation theory [48]; fuzzy logic, which is a form of many-valued logic where the truth values of variables may be any real number between 0 and 1 [49]; and tabu search, which is a local-search method used for mathematical optimization [36]. heuristic algorithms do not ensure to find a solution, but when they do it is performed much faster than deterministic methods. 2.3. decision-making process techniques for trajectory planning there are several approaches for tackling the problem of predicting the trajectory of a moving object when its exact geometric description and information about its environment is not available. in such cases, the information about the environment derives from measurements provided by a set of imperfect noisy sensors. therefore, the trajectory planning is carried out under uncertainty, which needs to be modelled. this uncertainty has an effect on the predictability about the current and future states (in either discrete or continuous state spaces and continuous time) of the robot and its environment. those states are based on the initial conditions, sensors, and the memory of formerly applied actions. therefore, trajectory planning methods under uncertainty cover problems such as localization, map building, pursuit-evasion and manipulation [2]. some methods are able to account for the uncertainty and the decision-making process in a greater or lesser extent. for instance, the worst-case, expected-case or probabilistic models, game theory analyses (with players with conflicting goals) and more complex techniques such as sequential decision making (which is a sequence of basic decision-making problems), control theory and artificial intelligence. probabilistic estimation methods rely on probability density function (pdf) of the robot location over the space instead of a deterministic location, which allows dealing with uncertainties. the aim is to keep a position pdf over all possible robot poses. an efficient https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 8 example is the kalman filter, which provides a recursive method for estimating the state of a noisy dynamical systems [50]. this is carried out by means of bayesian inference and estimating a joint probability distribution over the variables for each timeframe. its output is a gaussian probability density function (pdf) of likely robot positions instead of a single position estimate, with the mean and covariance of the error covariance matrix a distribution. another approach for tracking mobile robots under dynamic environments is by means of the markov process or markov decision process (mdp), which also make use a probabilistic framework for dealing with decision making in situations where outcomes are partly random and partly under the control of a decision maker it has the advantage of generating an optimal path, but has the disadvantage of limiting the robot to choose from a finite set of action. this lead to a non-smooth path. however, fuzzy markov decision processes (fdmps) are able to generate smooth trajectories using a fuzzy inference system [51]. bayesian methods uses the same iterative prediction-update process than in the kalman filter, but they do not rely on its restrictive assumptions [52]. the pros are that they can use nonlinear models for both trajectory planning and sensing and an arbitrary distribution instead of a gaussian. however, this may lead to higher computational cost compared to kalman filters. 2.4. mathematical programming the methods based on mathematical programming deal with obstacle avoidance by means of a set of inequalities on the configuration parameters. then the motion planning is posed as a mathematical optimization problem that finds a curve between the start and goal configurations minimizing or maximizing a certain objective function, such as minimizing the path traveling distance or execution time, energy consumption (or actuator effort) and jerk or maximizing the smoothness (e.g., [53], [54], [55]). this leads to a https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 9 complex non-linear optimization problem with many inequality and differential constraints, which needs be solved by a numerical method. furthermore, multi-objective optimization problems have been developed in the literature through pareto optimal solutions ([56], [57]). 3. comparison of approaches the pros and cons of the different approaches are presented in table 1. they cover aspects such as completeness (if the path exists, the path and the trajectory are found), optimality (the plan obtained is optimal regarding some parameter, not trapped in a local minimum), efficiency (computational cost of the algorithm, i.e., if it can change world and queries without recomputing everything or replannig from scratch), accuracy (high precision path tracking and control even at high speed), smoothness (i.e., chattering avoidance), stability (dynamically-stable motion planning), safety (for the robot, its environment and humans), scalability (the problem scales well when increasing configuration space dimensions), and execution time (lower times are desirable). however, the optimization approaches for robot trajectory planning are in continuous developing (e.g., [59], [60], [61]). table 1. approaches comparison. approach pros cons potential fields real-time, good scalability not complete, not efficient world and queries updates, path not optimal (local minimum), potential field forces must be set cell decomposition complete, robust high computational cost, and high execution time visibility graph complete and yields minimum length paths, optimal high computational cost, and high execution time, bad dof scalability, not efficient world and queries updates voronoi diagram complete and generates roadmap with maximum possibly inefficient paths, time, bad dof scalability, path not optimal https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 10 distance, efficient world and queries updates heuristic approaches low execution time, parallel search not complete, possibility of providing smooth paths exact cell decomposition complete high computational cost, high execution time approximate cell decomposition robust and useful when only a coarse representation of workspace is available not complete bug (bug 1 and bug 2) complete, easy implementation, parameters easy to adjust long paths, high execution time a* complete, optimal grid not efficient, bad dof scalability, not efficient world and queries updates rapidly exploring random tree (rrt) complete, semi-efficient world and queries updates, good scalability path not optimal probabilistic roadmaps (prms) complete, semi-efficient queries updates, optimal graph, good scalability not efficient world updates 4. conclusions this paper provides a review about optimal trajectory planning algorithms for autonomous robots. they cover a wide range of aspects such as the kinematics and dynamics of robots, the achievement of collision-free trajectories and the consideration of the physical limitations of the robots. the different motion planning techniques are discussed, and their advantages and disadvantages presented. as a consequence of these pros and cons, diverse solutions can be used for the wide variety of robot’s applications. references [1] j.c. latombe, robot motion planning. kluwer academic publisher, boston, 1991. [2] s.m. lavalle, planning algorithms. published by cambridge university press, 842p., 2006. https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 11 [3] s.h. tang, w. khaksar, n.b. ismail, and m.k.a. ariffi, a review on robot motion planning approaches. pertanika journal of science & technology 20(1), (2012) 15–29. [4] p. raja, and s. pugazhenthi, optimal path planning of mobile robots: a review. international journal of physical sciences 7(9) (2012) 1314-1320. doi: 10.5897/ijps11.1745. [5] z. shiller, off-line and on-line trajectory planning. in carbone, g., and gomez-bravo, f. (eds.), motion and operation planning of robotic systems, mechanisms and machine science 29 (2015), springer, cham. doi: 10.1007/978-3-319-14705-5_2. [6] v. lumelsky and a. stepanov, path planning strategies for point mobile automaton moving amidst unknown obstacles of arbitrary shape. algorithmica 2 (1987) 403–430. [7] c.o. dunlaing, and c.k. yap, a retraction method for planning the motion of a disc. journal of algorithms 6 (1985) 104-111. [8] t. osama and r.j. schilling, motion planning in a plane using generalized voronoi diagram. ieee transactions on robotics and automation 5(2) (1989) 143-150. [9] t. lozano-perez, and m.a. wesley, an algorithm for planning collision-free paths among polyhedral obstacles. communications acm, 22(10) (1979) 560–570. [10] t. asano, l. guibas, j. hershberger, and h. imai, visibility-polygon search and euclidean shortest paths. paper presented at the foundations of computer science, 26th annual symposium on, 21-23 october. automation. etfa ‘06. ieee conference on. (1985). [11] l. li, t. ye, m. tan, and x. chen, present state and future development of mobile robot technology research. robot 24(5) (2002) 475-480. [12] r. siegwart, and i.r. nourbakhsh, introduction to autonomous mobile robot. massachusetts institute of technology press, cambridge, u.s.a (2004). [13] y.h. liu, and s. arimoto, path planning using a tangent graph for mobile robots among polygonal and curved obstacles. international journal of robotics research 11(4) (1992) 376–382. [14] t. lozano-perez, spatial planning: a configuration approach. ieee transactions on computers c-32(3) (1983) 108-120. https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 12 [15] r.a. brooks, and t. lozano-perez, a subdivision algorithm in configuration space for find path with rotation. proceedings of eighth international joint conference on artificial intelligence, karlsruhe, germany, pp. 799-806 (1983). [16] j.m. keil, and j.r. sack, minimum decomposition of polygonal objects. machine intelligence and pattern recognition 2 (1985) 197-216. [17] d.j. zhu, and j.c. latombe, new heuristic algorithms for efficient hierarchical path planning. technical report stan-cs-89-1279, computer science department, stanford university, usa (1989). [18] d.w. payton, j.k. rosenblatt, and d.m. keirsey, grid-based mapping for autonomous mobile robot. robotics and autonomous systems 11(1) (1993) 13-21. [19] m. likhachev, d. ferguson, g. gordon, a. stentz, and s. thrun, anytime dynamic a*: an anytime, replanning algorithm. proceedings of the international conference on automated planning and scheduling, pp. 262-271 (2005). [20] o. hachour, the proposed genetic fpga implementation for path planning of autonomous mobile robot. circuits, systems and signal processing 2(2) (2008) 151-167. [21] o. hachour, path planning of autonomous mobile robot. international journal of systems applications, engineering & development 4(2) (2008) 178-190. [22] j. canny, and j. reif, new lower bound techniques for robot motion planning problems. proceedings of ieee symposium on the foundations of computer science, los angeles, california, 49-60(1987). [23] b. faverjon, and p. tournassoud, a local approach for path planning of manipulators with a high number of degrees of freedom. in proceeding of ieee international conference on robotics and automation, 1152-1159 (1987). [24] j.j. kuffner, and s.m. lavalle, rrt-connect: an efficient approach to single-query path planning. in ieee international conference on robotics and automation, 995–1001 (2000). https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 13 [25] q.-c. pham, and s. caron, y. nakamura, kinodynamic planning in the configuration space via velocity interval propagation, in robotics: science and system (2013). [26] o. khatib, real time obstacle avoidance for manipulators and mobile robots. international journal of robotics research 5(1) (1986) 90-98. [27] l.e. kavraki, p. švestka, j.c. latombe, and m.h. overmars. probabilistic roadmaps for path planning in high-dimensional configuration spaces. ieee transactions on robotics and automation 12(4) (1996) 566–580. [28] m. overmars, and p. svestka, a probabilistic learning approach to motion planning. in k. goldberg, d. halperin, j. c. latombe, and r. wilson (eds.). algorithmic foundations of robotics (wafr), 19–37, a. k. peters, ltd (1995). [29] l.e. kavraki, j.c. latombe, r. motwani, and p. raghavan, randomized query processing in robot path planning. journal of computer and system sciences 57(1) (1998) 50–60. [30] d. hsu, randomized single-query motion planning in expansive spaces. phd thesis, department of computer science, stanford university (2000). [31] k. sugihara, and j. smith, genetic algorithms for adaptive motion planning of an autonomous mobile robot. proceedings of the ieee international symposium on computational intelligence in robotics and automation, monterey, california, pp. 138-143 (1997). [32] m.m. mohamad, n.k. taylor, and m.w. dunnigan, articulated robot motion planning using ant colony optimisation. paper presented at the intelligent systems, 2006 3rd international ieee conference on (2006). [33] t. guan-zheng, h.e. huan, and a. sloman. global optimal path planning for mobile robot based on improved dijkstra algorithm and ant system algorithm. journal of central south university of technology 13(1) (2006) 80-86. [34] m.a.p. garcia, o. montiel, o. castillo, r. sepulveda, and p. melin, path planning for autonomous mobile robot navigation with ant colony optimization and fuzzy cost evaluation. applied soft computing 9 (2009) 1102-1110. https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 14 [35] k. manousakis, t. mcauley, r. morera, and j. baras, using multi-objective domain optimization for routing in hierarchical networks. proceedings of the international conference on wireless networks, communication and mobile computing, 1460-1465 (2005). [36] e. masehian, and m.r. amin-naseri, composite models for mobile robot offline path planning, mobile robots: perception and navigation. in tech publisher, croatia (2007). [37] a. zhu, and s.x. yang, a neural network approach to dynamic task assignment of multirobots. ieee transaction on neural networks 17(5) (2006) 1278-1287. [38] j.h. holland, adaptation in natural and artificial systems. university of michigan press, ann arbor, usa (1975). [39] d.e. goldberg, genetic algorithms in search, optimization and machine learning. addisonwesley, delhi, india (2000). [40] z. qingfu, s. jianyong, x. gaoxi, and e. tsang, evolutionary algorithms refining a heuristic: a hybrid method for shared-path protections in wdm networks under srlg constraints. systems, man, and cybernetics, part b: cybernetics, ieee transactions on, 37(1) (2007) 51-61. [41] j. kennedy, and r.c. eberhart, particle swarm optimization. proceedings of the ieee international conference on neural networks, perth, australia, 1942-1948 (1995). [42] y.q. qin, d.b. sun, n. li, and y.g. cen, path planning for mobile robot using the particle swarm optimization with mutation operator. proceedings of the ieee international conference on machine learning and cybernetics, shanghai, 2473-2478 (2004). [43] m. saska, m. macas, l. preucil, and l. lhotska, robot path planning using particle swarm optimization of ferguson splines. emerging technologies and factory automation, 2006. etfa '06. ieee conference on. prague, czech republic, 20-22 september. doi: 10.1109/etfa.2006.355416 [44] j. kuffner, s. kagami, k. nishiwaki, m. inaba, and h. inoue, dynamically-stable motion planning for humanoid robots. autonomous robots 12(1) (2002) 105-118. https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 15 [45] a.z. nasrollahy, h.h.s. javadi, using particle swarm optimization for robot path planning in dynamic environments with moving obstacles and target. third european symposium on computer modeling and simulation, athens, greece, 60-65 (2009). [46] d.w. gong, j.h. zhang, and y. zhang, multi-objective particle swarm optimization for robot path planning in environment with danger sources. journal of computers 6(8) (2011) 1554-1561. [47] r.r. cazangi, f.j. von zuben, and m.f. figueiredo, evolutionary stigmergy in multipurpose navigation systems. evolutionary computation, 2006. cec 2006. ieee congress on. vancouver, bc, canada, 16-21 july. (2006). doi: 10.1109/cec.2006.1688332. [48] d.k. pai, and l.m. reissell, multiresolution rough terrain motion planning. ieee transaction on robotic and automation, 14(1) (1998) 1933. [49] q.r. zhang, and g.c. gu, path planning based on improved binary particle swarm optimization algorithm. proceedings of the ieee international conference on robotics, automation and mechatronics, chendu, china, 462-466 (2008). [50] t.l. lee, and c.j. wu, fuzzy motion planning of mobile robots in unknown environments. journal of intelligent and robotic systems 37(2) (2003) 177-191. doi:10.1023/a:1024145608826. [51] p.s. maybeck, the kalman filter: an introduction to concepts. in: cox i.j., wilfong g.t. (eds) autonomous robot vehicles. springer, new york, ny (1990). [52] m. fakoor, a. kosari, and m. jafarzadeh, humanoid robot path planning with fuzzy markov decision processes. journal of applied research and technology 14(5) (2016) 300-310. doi:10.1016/j.jart.2016.06.006. [53] h. choset, k.m. lynch, and s. hutchinson, principles of robot motion: theory, algorithms, and implementations. mit press. isbn: 9780262033275. (2005). [54] l. yang, j. qi, d. song, j. xiao, j. han, and y. xia, survey of robot 3d path planning algorithms. journal of control science and engineering (2016) pp. 22. doi: 10.1155/2016/7426913. https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.9867 social and technological sciences issn: 2341-2593 llopis-albert et al. (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 1-16 | 16 [55] f. rubio, c. llopis-albert, f. valero, and j.l. suñer, assembly line productivity assessment by comparing optimization-simulation algorithms of trajectory planning for industrial robots. mathematical problems in engineering (2015). doi: 10.1155/2015/931048. [56] f. rubio, c. llopis-albert, and f. valero, j.l. suñer, industrial robot efficient trajectory generation without collision through the evolution of the optimal trajectory. robotics and autonomous systems 86 (2016) 106–112. doi: 10.1016/j.robot.2016.09.008. [57] c. llopis-albert, f. rubio, and f. valero. improving productivity using a multi-objective optimization of robotic trajectory planning. journal of business research 68(7) (2015) 1429-1431. doi: 10.1016/j.jbusres.2015.01.027. [58] c. llopis-albert, and d. palacios-marqués, applied mathematical problems in engineering. multidisciplinary journal for education, social and technological sciences 3(2) (2016) 1-14. doi: 10.4995/muse.2016.6679. [59] w. xing, j. zhang, w. lu, and p. bao, an improved potential field based method for crowd simulation. international journal of software engineering and knowledge engineering 25(03) (2015) 427-451. doi: 10.1142/s021819401540015x. [60] j. sobecki, comparison of selected swarm intelligence algorithms in student courses recommendation application. international journal of software engineering and knowledge engineering 24(1) (2014) 91-109. doi: 10.1142/s0218194014500041. [61] a. saeed, s.h.a. hamid, and a.a. sani, cost and effectiveness of search-based techniques for model-based testing: an empirical analysis. international journal of software engineering and knowledge engineering 27(04) (2017) 601-622. doi: 10.1142/s021819401750022x. https://doi.org/10.4995/muse.2018.9867 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 61 efficiency in social enterprises ballester-miquel, josé carlos1*, perez-ruiz, pilar 2, hernandez-gadea, javier 3, de juan jordán, hugo4, guijarro garcía, maria5 1esic business & marketing school avda. de blasco ibáñez, 55, valencia (spain), josecarlos.ballester@esic.edu , +34 659279402 2 esic business & marketing school pilar.perez@esic.edu 3 esic business & marketing school javier.hernandez@esic.edu 4 esic business & marketing school hugo.dejuan@esic.edu 5 esic business & marketing school maria.guijarro@esic.edu received: 2016-07-06; accepted: 2017-09-13 abstract the aim of this article is to offer a review of the impact the different methodologies of analysis have on social enterprises, focusing the study on processes that establish a greater universality based on the degree of success achieved in their social objectives, the social reinvestment of their benefits and their democratic organisation, parameters that should favour the creation of a clear and simple method, as well as adaptable to change. the systematics will allow to establish systems for the measurement of the efficiency of social enterprises, in order to both organise objective procedures of comparison and offer support when applying for public aid derived from european and national funds set up for this purpose. the quantification of the social impact of the companies that constitute the social economy is vital to assess and follow up on their social mission. keywords efficiency, social enterprise, social impact mailto:josecarlos.ballester@esic.edu mailto:pilar.perez@esic.edu mailto:javier.hernandez@esic.edu mailto:hugo.dejuan@esic.edu mailto:maria.guijarro@esic.edu multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 62 1. introduction the 2020 strategy for a smart, sustainable and inclusive growth marked by the eu in 2010 aims to implement seven flagship initiatives and propose five measurable objectives based on employment, research and innovation, climate change and energy, education and the fight against poverty. this strategy has found that the last of the aforementioned objectives deviates from the initially established indicator, which was to achieve 20 million fewer poor by 2010. the control carried out by the commission in 2014 in its statement1 on the balance of the europe 2020 strategy using the data gathered in 2012 confirms the difficulty that europe faces in its objective of social integration. table 1. forecast of the 2020 strategy objectives analysed in 2012. objective 2010 2012 2020 (forecast) to increase the employment rate between the ages of 20-64 by up to 75% 68.5% 68.4% 72% to invest 3% of gdp in r&d 2.08% 2.06% 2,2% to reduce gases by 20% 18% 24% to increase renewable energies by 20% 7.5% 14.4% 21% to increase energy efficiency by 20% consumption should be cut by 6.3% in order to reach the 2020 target school dropout < 10% 12,7% 10% higher education–youth> 40 35.7% 40% 20 million fewer poor +10 m not reached source: data from the european commission com (2014) 130 final as seen in the table and considering the european average, the environmental objectives are going to be achieved in what gas reduction is concerned, with a decrease of 20%, which shows the greatest achievable margin. the situation is similar in educational policies, which means a better qualified future for europe, since the united kingdom and 1 com (2014) 130 final. balance of the europe 2020 strategy for a smart, sustainable and inclusive growth. 5.3.2014 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 63 ireland are the only european countries ranked eighth and tenth, respectively, in the world top ten2. the european average is close to 36%. the employment objective is reduced by approximately three points, employment being again one of the main targets of the eu. but the most dramatic consequence of the crisis is that, just two years after having established it in the 2020 strategy, we have to abandon the objective of reducing by 20 million the number of poor existing in 2010, because by 2012 we had 10 million more poor than two years before. the reading of these data indicates that europe is moving away from the welfare state and that social economy must be one of the main drives to correct this drift, both in employment values and in the actions of insertion of the most disadvantaged groups. the measuring of the risk of poverty and / or social exclusion uses a multidimensional3 concept, being more extensive than the use of the income value. it uses three sub-indicators: a poverty risk rate after social transfers, a severe material shortage of goods and households with very low labour intensity. poverty and social exclusion have worsened in spain as a consequence of the crisis with more intensity than in the eu, getting to 29.4% of the population, compared to 24.4% in the european union. this figure represents an increase of 1.3 million people compared to 2010. income inequality that is the ratio between the highest income (in the upper 20%, the upper quintile) and the lowest income (in the lower / last quintile) is set at 6.8 in 2014, compared to an average 5.2 in europe. this inequality compared to europe is translated inside spain with variations of more than 20 points between the north and the south of the country. although it is true that there is an increase of employment, with basically parttime and temporary contracts, this growth that does not occur in the most excluded groups and given the precariousness of the salaries, it is clear that this increase is not enough to overcome the situation of social exclusion to which these groups are subject. the main objective of the social enterprise as an operator of the social economy is to have a social impact instead of obtaining a benefit for its owners or shareholders. it produces goods and services for the market in an innovative and entrepreneurial way and 2 oecd ranking 2014. the countries with the highest rate of adult population with university studies 3 arope indicator (at risk of poverty and / or exclusion). multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 64 uses its benefits mainly to achieve social objectives, monzón, marcuello and nachar (2013). a relevant consideration is made by san josé and retolaza (2012) with reference to the influence of the stakeholders where they point out how antagonistic their interest may be sometimes and move a stakeholder into the category of a non-stakeholder. the ontological perspective of the stakeholders considers that each company generates a unique ecosystem of value creation around the social, economic and environmental parameters. comparing the three axes of business value mentioned before, the economic axis has a better and more extensive development of the metrics necessary to control the economic parameters of the companies. the economic results achieved in each company have international recognition, that is to say, a taxonomy has been established and a consensus has been achieved in the eu regarding the way in which results are calculated and what they represent. the use of the esa-20104 which replaces the sec-955 makes it possible to group non-financial corporations into the different standardised categories: a) non-financial companies; b) financial institutions; c) public administrations; d) households; e) non-profit institutions serving households. on the other hand, a group of satellite accounts is authorised, which includes, among others, social economy companies, which take into account parameters other than those strictly economic, such as environmental assets and pollution. on the other hand, the environmental parameter is based on the "environmental impact assessment"6 directive (eia directive) and the "strategic environmental assessment"7 directive (sea directive), making all environmental-orientated projects undergo the control of compliance with these directives. all projects and programmes cofinanced by cohesion policies must comply with the aforementioned directives in order to 4 regulation (ec) no 549/2013 of the european parliament and of the council of 21 may 2013 on the european system of national and regional accounts of the european union. 5 council regulation (ec) no 2223/96 of 25 june 1996 on the european system of national and regional accounts in the community 6 directive 2011/92 / eu of the european parliament and of the council of 13 december 2011 on the assessment of the effects of certain public and private projects on the environment. 7 directive 2001/42 / ec of the european parliament and of the council of 27 june 2001 on the assessment of the effects of certain plans and programs on the environment. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 65 obtain financial assistance, thus eia and sea directives becoming sustainable development procedures. the current challenge is to achieve, as in the case of the economic and environmental parameters, a measurement system that should bring concepts together and call for a union of criteria on social impact in europe. the fundamental criteria in the social impact metrics are the importance of the objectives over the favoured groups and the ultimate destination of the benefits. monzón, marcuello and nachar (2013) with the measurement of social impact the intention is to control the effects generated by specific activities of social enterprises, although their structure contributes to generate added value. the requirements that the measurement must meet are to support social enterprises´ activities, proportionality and not to hinder social innovation. it must strive for a balance between qualitative and quantitative data, considering the memory or narration of the qualitative data as an essential aspect and assuming the difficulty of transferring the data from a micro level to a macro level 8. traditionally, the main indicator of country development has been economic and now we must move towards a combination that should contain environmental, social and economic consequences. in the current crisis situation in europe, with a project of economic growth based on precarious employment according to the international labour organisation (ilo) 2015 report, where part-time and seasonal jobs increase, as well as unemployment in the groups with high risk of exclusion by more than 10 million compared to 2008, the implementation of social economy becomes a priority and, therefore, the measurement of the social results becomes necessary to be able to provide it with financing mechanisms. social economy, in this context, is considered as a structural way out of the crisis, monzón and chaves (2012, 91), where private initiative establishes a new relationship between individuals and society by showing a greater resistance to the negative economic environment. this is due to its characteristics of democratic functioning, non-sharing of benefits and reinvestment in social objectives, as shown by zevi et al. (2011) regarding the resistance and bankruptcy of cooperatives as compared to the traditional private companies. 8 eesc decision on "measuring social impact". int / 721. 12/10/2013 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 66 the eu's concern for the social factor has led to the inclusion of first-level indicators in the macroeconomic imbalance procedure (mip), where until 2014 there were 11 exclusively economic indicators, 5 external and 6 internal, and since 2015, there are 3 more social indicators (activity rate, youth unemployment and long-term unemployment). it is true that their non-compliance does not have a sanctioning value, but it can provide annual visibility of their evolution even when considered in relative figures, which makes them less alarming. in short, once we recognise the validity of se as the main factor for the development of the neediest groups and understand the concern of the states for the social factor, we must focus on obtaining the sufficient visibility of this market segment represented by social economy. 2. social economy and the third sector the third sector term has its origin in the united states and one of the first to mention it is levitt (1973)9, where the third sector is associated with the non-profit organisation (npo). these companies of anglo-saxon origin have the following five basic characteristics based on the approach of salamon and anheier, 1992: a private, formal, non-profit organisation with self-government and voluntary participation. from the definition of se showed above and the characteristics of the npo, we can extract the following table, where the common characteristics are grouped and the specifications that distinguish each group are showed below: 9 levitt, t.: the third sector – new tactics for a responsive society, division of american management associations, new york, 1973. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 67 table 2. similarities and differences between the npo and social economy source: made by the authors and based on j.l. monzón`s study (2006) according to monzón (2006)10, the non-profit consideration contemplates two approaches that are indivisibly included in its definition. the first refers to the exclusion of organisations that share benefits, but also requires, as a second approach, that their social objective should specify that they make no profit, which excludes cooperatives and mutuals from the concept. the democratic criterion affects the se sector in a unique way, since its requisite excludes entities like hospitals, universities, schools and other groups that do not meet the requirements of democratic operation but, nevertheless, are included in the npo format. however, the definition of the social economy oriented to the non-market finds a way towards social utility companies that are not governed by democratic criteria. the law 5/2011 recognises the democratic value as a guiding principle of social economy, which is why it restricts, at least for the moment, the orientation towards the nonmarket and separates npo companies from social economy entities. with respect to the third differentiating element, the service to the people, it represents a fundamental criterion in the se, since its objectives can benefit only the people that belong to the entity, whereas the npo always needs a social objective and excludes the personal objective. 10 social economy and related concepts: blurred boundaries and conceptual ambiguities in the third sector. j.l. monzón. ciriec magazine. 2006 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 68 the npo sector is positioned between the market and the state11, providing a third way to cover social needs that are not met and are only attended by these npo organisations, while social economy is part of private companies, which differ from capitalist enterprises in that they do not share profits. a first group of so-called not-for-profit companies can be found in the se definition proposed by chaves and monzón, referring to those whose objective is to share part of the benefits with partners, the rest being allocated to social objectives, and a second group, that of non-profit companies that do not have to comply with the requirement of democratic functioning and do not admit the sharing of benefits, that is to say benefits are allocated to the overall development of its social object. therefore, a unifying factor has been sought for in recent history and combined with the basic objectives of bringing together the concepts of the anglo-saxon third sector with that of social economy, but due to the fact that the principle of non-distribution of benefits (pndb) is maintained, this unifying possibility is remote for the moment. 3. the social enterprise this process of definition of the spanish social economy was consolidated with the approval of the law 5/2011 of march 29, on social economy, providing recognition, transparency and development of the sector. this law defines the allegiance to the private sphere in its article 2 and delimits some guiding principles in its article 4, where it establishes the primacy of people and social purpose over capital. it also specifies that there is no correspondence between the different contributions of capital and the representativeness of decision-making since it is established in line with its social purpose and the labour contributions of its members. it points out that the return of results must be for the partners and the social purpose of the entity, it also promotes solidarity both within the company and with society and declares independence with respect to the public powers. the section on social economy entities (article 5) of this law includes cooperatives, mutuals, foundations and associations and, on the other hand, labour societies, insertion companies, special employment centres, fishermen's association, agrarian transformation 11 salamon, l.m. and anheier, h.k. (1997) multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 69 societies and individual entities that operate with the principles of social economy defined in the law, as well as any other entity from the catalogue of social economy entities. the existence of the catalogue of social economy entities declared in article 6 as a unifying agent of the companies that form the se is relevant, although it is not specified who is in charge of it, nor how the inclusion procedure will be carried out, nor the processes of periodic revision for the maintenance of the entities in the catalogue. this law does not include the principle of free membership that has been postulated in all definitions of the social economy until now and is also exclusive with the non-profit status since in its guiding principles establishes the way in which benefits are shared. on the other hand, the law does not direct its action towards the general social welfare and, as mentioned before, it is insufficient due to the imprecision of the regulation in the catalogue of social economy entities. in fact, fajardo (2012) 12 estimates that: a) it must serve for the recognition of social enterprises within se b) it must be accessed by categories, c) it must provide statistical data about the sector and d) it must also serve to verify the compliance with the concept of social economy. in september 2015, some amendments to the law 5/2011 were included in the law 31/201513, the most significant being the inclusion in article 5 of special employment centres and insertion enterprises as entities of the social economy, extending this declaration to any social economy entity whose purpose is labour insertion of groups at risk of exclusion. the xunta de galicia approved the law of social economy of galicia, (law 6/2016) on may 4, 2016, where it amended and developed the state law 5/2011. it addressed the objectives of general interest, requested for some time, and also those of services in the benefit of members. in addition, it set up the council of social economy of galicia as advisory body of the social council, in order to develop the catalogue of social economy entities. all the sensibilities defined in the law are represented in this council, as well as the public sectors affected. a new process of visibility of the se began with this pioneering 12 fajardo, gemma. “the legal concept of social economy and social enterprise”. 2012. 63-84 13 law 31/2015 of 9 september, amending and updating the regulations on self-employment and adopting measures to promote self-employment and social economy. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 70 process in spain, since besides giving se legal coverage, it also allows to classify and control the entities that operate in social economy. the legal advance of social economy in spain is relevant and allows to think that in the near future its autonomous development, together with the advance of the catalogue of social economy entities, will help to consolidate the criteria and characteristics that define them. finding a common criterion concerning the third sector will be a more complex case though. 4. social enterprises and social economy enterprises. social enterprises are an important component of se, even though not the only one as social economy enterprises (sees) are also part of it, and it is clear that they have common characteristics but also distinguishing features. in the same way in which the spanish legislation has defined social economy, with its obvious aspects of improvement, it has not done so with the definition of social enterprises. the definition of the social enterprise from which we part is that of the sum of economic and social criteria. the economic criteria that define it can be found in its continuous activity of goods or sale of services, high degree of autonomy, a significant level of economic risk and a minimum number of workers, while the social criteria refer to having an explicit aim to benefit the community, an initiative undertaken by a group of citizens, power of decision not based on ownership of capital, participatory nature and a limitation in the distribution of benefits 14. it is the european economic and social committee15 that proposes the following definition of social enterprises in 2011: “companies for which the social or societal objective of the common good is the reason for the commercial activity, often in the form of a high level of social innovation, those where profits are mainly reinvested with a view to achieving this social objective, and where the method of organisation or ownership system, based on democratic or participatory principles or oriented to social justice, reflects their mission.” 14 defourny, j. & nyssens, m., “social enterprise in europe: recent trends and developments”, working papers series, no. 08/01, liège, emes european research network, p. 5, pp. 37 y 38. 15 com (2011) 682 final. "the initiative for social business – building an ecosystem to promote social enterprises, in the core of economy and social innovation". 25.10.11 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 71 from the above definition, two possibilities are extracted: that our efforts should be dedicated to the social purpose of the people targeted by our mission, which could be called social utility, or that the business mission should be carried out with the people who are the object of the social destination, which could be called labour insertion. 16 in the progress of the definition, it is interesting to take into account muhammad yunus´ opinion, a nobel peace prize laureate and founder of the grameen bank (the bank of the poor), who defines se as a business without losses or dividends, fully committed to achieving a social objective, and which is included in the first classification of social enterprises – the one addressing the social purpose. the definition of the social enterprise proposed by emes17 is in the same line (diaz, m., marcuello, c. & marcuello, ch. 2012): social enterprises are private non-profit organisations that provide goods and services directly related to their explicit aim to benefit the community. they are based on a collective dynamics in a way that involves the different stakeholders, their governing bodies are autonomous entities and bear the risks related to their economic activity. according to monzón, marcuello, calderón (2013) "social enterprises are companies of the social economy in which the groups of people who benefit from their activity are vulnerable groups or at risk of social exclusion" and the objective should be a measurable social impact that, according to melinda tuan (2008) 18, "is the long-term and sustainable change, attributed to a specific action or a set of interventions". this means that there should be a way to measure the accepted social indicators through prestigious organisms and an adequately accredited international scene in order to assess the social balance and the achievements attained at a social level within the social economy. if the bases of measurement of the "for-profit" economy are admitted, something similar should be achieved in the third sector. social value measurement in third-sector companies is the current challenge. it is known that the classic system of economic benefits in companies is primarily oriented to shareholders and, therefore, is not useful for the social sector, where groups of influence do not perceive financial benefits. social value measurement has been developed through 16 fajardo, g. "the legal concept of social economy and social enterprise" 2012. 78. 17 european research network emes http://www.emes.net/ 18 tuan, melinda. “measuring and/or estimating social value creation: insights into eight integrated cost approaches. december 2008 multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 72 different methods. tuan (2008) uses one such method of value creation measurement that consist of the difference between the inputs and outputs over a given period of time, while olsen and galimidi (2008) use the chain of impact generation, thus setting a framework of reference within the value chain. as a result of the definitions, it can be seen that the social enterprises are not entirely included in the social economy and that, besides, there are some characteristics that differentiate them from social economy companies, as seen in the summary table below, where the non-profit concept is oriented to social enterprises, having social objectives as their aim. table 3. similarities and differences between sees and social enterprises. typologies common points differences measurement of results main financing social economy enterprises (sees) economic and social criteria economic selfsufficiency social enterprises social impact external financing public and private source: made by the authors therefore, the social impact is the way of measuring results in social enterprises. for this reason europe is moving towards a certain unification of criteria that allows comparing, ranking and evaluating these companies. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 73 5. the methodology of social impact measurement the european commission is aware of the difficulty of developing the measurement criteria. the commission requested a subgroup of the expert group on social entrepreneurship (geces) to develop social impact measurement in october 2012 for the distribution of european social entrepreneurship funds (eusef) and the eu programme for employment and social innovation (easi). the group obtained its results by the end of 2013, just before the beginning of the implementation of the previously mentioned funds whose duration covers the period 2014-2020. as a preliminary to the study developed by these experts, which will establish the measurement parameters, the fundamental terms that limit the process in social enterprises should be first defined, as shown in the table: figure 1. terminology of social enterprises in their operational process. source: geces subgroup on impact measurement, 2014, equivalent to that developed by evpa based on the double bottom line project of the rockefeller foundation all that does not represent a direct consequence of the activity of social enterprises should be excluded from the impact they have on the beneficiaries, that is to say, the result inputs • resources used activities • where resources are used outputs • impact of activity on beneficiaries outcome • change produced in the beneficiaries impact •change since the intervention multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 74 is obtained by also considering the following aspects: 1. the changes that would have occurred in the short term regardless of the intervention (dead weight) which is obtained with the contribution of other concurrent or circumstantial actors (alternative attribution); 2. the extent to which the results have displaced other equally positive consequences that could have happened (displacement) and those produced in the long term; 3. the loss produced by the passage of time (reduction or decrease). there are two orientations in the analysis of the impacts on social enterprises: one that derives from the point of view of the social investor (si) aligned with the eusef funds and another one related to the social project that corresponds to the easi. the evpa19 identifies five phases in the process of impact measurement, which are also assumed in the work of the geces subgroup and which are as follows: determination of objectives, where the scope and objectives of the measurement are defined and the specificity in the definition of the objectives will allow a better measurement of the impact. at this point it is interesting to identify the objectives based on the investor`s vision and the social project, since the issues to be solved are different: in the case of the investor and considering the logic of the impact measurement, aspects such as the reasons for the measurement, the resources to be allotted, the rigor of the analysis and the time available for its carrying out are essential, while, on the other hand, issues like identifying the social problems to be mitigated and the expected objectives and results are fundamental. however, from the social point of view, the problem that it seeks to alleviate is highly relevant, as well as the activities and necessary resources to achieve the results. the following phase is specified in the identification of the parties involved, which are basically limited to the beneficiary and the issuer. the agents' identity, their hierarchy and the level of commitment are relevant. the management of the agents' expectations during the progress of the project must be taken into account, adjusting objectives if necessary in order to achieve better results. the measurement of results supposes the development of the third phase, which considers the point of view of the social investor and where the theory of change plays a 19 european venture philanthropy association. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 75 relevant role20. both from the point of view of the investor and the social project, the outputs and the outcomes must be transformed into quantifiable objectives. outputs can be better controlled because they relate to the activity of the organisation, however, when talking about outcomes, the results are beyond their direct scope and difficult to measure. measurements of products should be based on standard indicators such as iris21 or the global value exchange22. however, in the case of the measurement of outcomes, the selection of the indicators should fit best the appropriate relevance principles and should be accompanied by at least two other indicators for each result to be measured, hence, this process involves the difficulty of measuring the deadweight or the displacement factors. the fourth step is the verification and evaluation of the impact, which evaluates if the proposed results have been achieved. different techniques can be used for the verification part such as the search for other similar social projects, databases, and published statistical data by collecting information from the agents involved, while for the assessment part, techniques for gathering the satisfaction from the groups affected by the impact assessment should be used. the last section deals with the follow-up and presentation of results, a follow-up that is represented by the search for compliance with the proposed indicators, as well as the time and financial resources allocated to the non-financial support of social projects. the presentation of results should be unique and should not vary depending on the different agents that will receive the report. it is the social reporting standard that works in this line. the impact measurement process establishes a continuous five-phase method that is adapted and corrected at the same time as the results and their deviations from the objectives are obtained. 20 the theory of change: the company decides the change and establishes the most appropriate measurement, arriving at a later consensus with the main interested parties. it usually uses shortand medium-term goals that are documented to achieve the long-term goal. www.theoryofchange.org. 21 iris: standardised report on the impact of investment that provides a common language. 22 the global value exchange is a database with indicators values and results. http://www.theoryofchange.org/ multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 76 figure 2. impact management process in five phases. source: evpa 2013 6. conclusions from the point of view of the need to create quality jobs in the europe of the precarious employment that surrounds us, the companies developing in the context of social economy have the advantage, but when considering the underprivileged groups, it is the social enterprises that prove to have a greater know-how in the labour reinsertion of these people. there are public and private funds that can participate in the financing of particular objectives and even in the creation of the social enterprises, but they need to work with reliable indicators of the return on investment, not in accounting terms, but rather in the sense of social balance where outputs and outcomes are valued. the valuation establishes a principle of agreement between the parties involved, being the company the one to propose the objectives and measurement criteria, while the investors will modify and accept these criteria. the five-phase process that is developed by evpa is considered valid for this social impact metrics and proves to be efficient enough for both parties. 1. definition of objectives 2. identification of parties involved 3. measurement of results 4. verification and assessment of impact 5. follow-up and presentation of results multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 77 the future development of the final impact reports, which are unique and unrealised for each individual investor, and their subsequent publication that enables visibility in the international community will recognise some criteria of unification, as a result of this "jurisprudence" in the reports on the social enterprises. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 78 7. references castro,r.,santero,r.,martinez,m.i.&guillo,n.(2013):“impacto socioeconómico de las empresas de economía social en españa. una valoración cuantitativa de sus efectos sobre la cohesión social”, ciriec-españa, revista de economía pública, social y cooperativa, 79, 35-58. chaves r. & monzon, j.l. (2012): la economía social en la ue, cese, bruselas chaves rafael, monzon jose luis et al (dir), (2000), economía social y empleo en la unión europea, ed. ciriec-españa comisión europea (2010): comunicación una estrategia para crecimiento inteligente, sostenible e integrador, com (2010) 3/3/2010 comisión europea. bruselas comisión europea (2014): comunicación balance de la estrategia europa 2020 para un crecimiento inteligente, sostenible e integrador com (2014) 130 final, 5 de marzo, comisión europea bruselas. comisión europea (2014): comunicación relativa a un marco estratégico de la ue en materia de salud y seguridad en el trabajo, com (2014) 332 final, 6 de junio, comisión europea bruselas. comisión europea (2015): comunicación programa de trabajo de 2016, com (2015) 610 final, 27 de septiembre, comisión europea bruselas. comisión europea (2015): comunicación un sistema de imposición de las sociedades justo y eficaz en la unión europea: cinco ámbitos de actuación fundamentales, com (2015) 302 final, 17 de junio, comisión europea bruselas. comisión europea (2016): comunicación iniciativa europea de computación en la nube: construir en europa una economía competitiva de los datos y del conocimiento, com (2016) 178 final, 19 de abril, comisión europea bruselas. consejo económico y social españa: informe análisis de la gobernanza económica de la unión europea. número 01 – 2016. informe elaborado a iniciativa propia por el consejo económico y social, conforme a lo dispuesto en el artículo 7.1.3 de la ley 21/1991, de 17 de junio, de creación del consejo económico y social. aprobado en la sesión ordinaria del pleno del día 24 de febrero de 2016. multidisciplinary journal for education, https://doi.org/10.4995/muse.2017.7329 social and technological sciences issn: 2341-2593 ballester-miquel et al. (2017) https://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 4 nº 2 (2017): 61-79 | 79 consejo economico y social, ces (2016): analisis de la gobernanza económica de la unión europea. edición 1ª marzo 2016. diaz, m., marcuello, c. & marcuello, ch. (2012): “empresas sociales y evaluación del impacto social”, ciriec-españa, revista de economía pública, social y cooperativa, 75, 179-198. enciso, m., gómez, l. & mugarra, a. (2012): “la iniciativa comunitaria en favor del emprendimiento social y su vinculación con la economía social: una aproximación a su delimitación conceptual”, ciriec-españa, revista de economía pública, social y cooperativa, 75, 55-80 freeman, r. (2010). “does inequality increase economic output?” in controversies about inequality. stanford, ca: stanford university press. gómez urquijo, l. (2014): “el encaje de la economía social en la nueva gobernanza económica y social de la ue”, ciriec-españa, revista de economía pública, social y cooperativa, 82, 233-252. guía práctica para la medición y la gestión del impacto. asociación española de fundaciones. european venture philanthropy association. 2015 medición del impacto social. dictamen del comité económico y social europeo sobre el tema. int/721. diciembre 2013 metodos propuestos para la medición del impacto social en la legislación de la comision europea y en la práctica relativa a fese y easi. subgrupo de geces sobre medición del impacto 2014 monzón, j.l., marcuello, c. & nachar, p. (2013): empresas sociales y economía social: propuestas metodológicas para la medición de su impacto socioeconómico. bruselas 22 febrero 2013. narillos roux, hugo (2012): “economía social: valoración y medición de la inversión social”. editorial esic. retolaza, j.l., ruiz-roqueñi, m., san-jose, l. & barrutia (2014): “cuantificación del valor social: propuesta metodológica y aplicación al caso de lantegui batuak. multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 86 environmental implications of universal rural road access program (urrap) roads in southwestern ethiopia: the case of jimma and buno bedelle zones hassen nagesso1 and milkessa edae2 1 sociology and social policy, jimma university, jimma, ethiopia. e.mail: hasseng2013@gmail.com 2 oromo folklore and cultural studies, jimma university, jimma, ethiopia. e-mail: milkesaedae@gmail.com received: 2017-11-06; accepted: 2018-03-03 abstract ethiopia has been undertaking numerous development programs and projects with the objective of taking a country to the middle-income countries in the near future. universal rural road access program (urrap) roads being constructed in jimma and bunno bedele zones are among the projects in ethiopia. although these projects are assumed to bring desirable positive change, there are sometimes negative consequences on environmental issues. accordingly, the objective of this study was to investigate the negative sides of these projects on environmental elements. household survey with randomly selected household heads and in-depth interview, focus group discussion (fgd) and observation for purposively selected samples were used to collect relevant and required data. thereby, data were analyzed quantitatively by using statistical package for social sciences (spss) and qualitatively through thematic method. the findings from the analyzed data show that rural roads being constructed by urrap have both positive and negative impacts on plant species, bird species, reptile species, mammal’s species, amphibians’ species, soil, and climate. finally, urrap is in dilemma and needs an extensive and frequent supervision, monitoring, inclusive, interdisciplinary and sound strategies and policies by responsible bodies at all expected levels and contexts. keywords dilemma, development, environmental issues, ethiopia, universal rural road https://doi.org/10.4995/muse.2018.8908 mailto:hasseng2013@gmail.com mailto:milkesaedae@gmail.com multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 87 introduction physical infrastructure like road is often indicated as a key input to economic growth both in developed and developing countries (world bank, 1994; roberts et al., 2006; karani, 2007). the world governments have a set vision to make public, economic and social services physically more accessible to all the people: rural and urban (era, 2012; arethun and bhatta, 2012). there remains a critical need to provide rural communities with transportation infrastructure and services that ensures permanent accessibility to social and government services (porter, 2003); reduces the physical costs of access to resources and markets (lulit, 2012); provides economic and business services (jacoby, 2000), decreases poverty (allport and anderson, 2011; bryceson et al., 2008); creates better opportunities for employment (karani, 2007); and helps in income generation (vandana and potter, 2008; willis, 2013; asher and novosada, 2017). historically, in ethiopia road, there were constant flows of passengers, animals or erosion in the past regimes, though there was a commencement during the italian occupation (tarekegn, 1987; bahru, 1991; keller, 2005). in other words, ethiopia has the lowest road density in the world, which is even far below the average african standards (arethum and bhatta, 2012). currently, with its more than 100 million, the second-most populous country in africa, it has been undertaking rapid expansion in road infrastructure (csa, 2015). it is alleged that the fundamental causes of poverty, isolation, powerlessness, vulnerability, unemployment, and high income inequality particularly in rural parts of ethiopia is the result of insufficient and also unequal access and custody of road (escobal, 2005; wondemu et al., 2012; stifel et al., 2012). consequently, after 2010, the government has launched a new universal rural road access subprogram to address the rural accessibility and connectivity (era, 2012). the effects of roads tend to be complex and also very context specific, and therefore hard to presuppose a direct relationship between road development and economic growth, as the conditions under which road development will lead to positive economic growth outcome are not sufficiently specified. tighes (2008) also argue that transport development may also bring in its wake, the problem of environmental degradation. ethiopia has several major ecological systems that support varied plant and animal species (fikadu and melesse, 2014). it is a country with great geographic diversity like rugged mountains, flat topped plateaus and https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 88 deep gorges incised river valleys and rolling plains (egziabher, 1988; wondie and temesgen, 2013). these diversified topographic features made the country to had indigenous and alien plant and animal species and soil in tropical africa (tolera et al., 2008). these species are essential for human survival and wellbeing. they provide us with food, oxygen, shelter, recreation, and spiritual sustenance, and they are the source for over 5,000 commercially-traded products, ranging from pharmaceuticals to timber and clothing. in addition, these resources help to maintain the fertility of agricultural land used as habitat for wild life, protect water resources and reduce the risk of natural disasters such as land slide and flooding (world bank, 2004), and helps as a site of tourism which might increase the income of a country (fikadu and melesse, 2014). throughout history, humans have used the environment resources of plants, animals, soils and others togaingreatsocio-economicdevelopment (tigabu, 2016) of which many of the methods are now being seen as unsustainable (abdelfattah, 2009). for example, forests undergo changes in various ways. its areas can be reduced either by deforestation or by natural disasters such as volcanic eruptions. as a result, the expanse of forest areas are declining across the globe, partly through logging activities and also due to conversion of habitats to croplands (agricultural expansion) accounting for up to 40% of ethiopian forest losses (pant and samant, 2007; tesfaye, 2007; winberg, 2010). in spite of the fact that deforestation was highly known in the northeastern part of a country (woldie and temesgen, 2013), nowadays it is expanding to southern parts, which has plenty of plant and animal species. because of the disappearance of forests, most of the mountainous sides are bare. valleys have been gullied; strips and streams which used to have water the whole year round are now mainly dry in dry season (abate, 2003). among other factors accelerating the decline of plant and animal species diversity in ethiopia, is the enlargement of public infrastructures in rural parts of the country and its policies are playing a great role (tesfaye, 2007). along with an advancement of rural road, agricultural investment, resettlement schemes, charcoal production, relentless expansion are having a profound and determinant effect on the woody plant resource availability (winberg, 2010; fdre, 2012; tigabu, 2016). sucoff (2003) also argue that corruption of government institutions in universal rural road access program (urrap) road construction is the leading factor for the loss of biodiversity. these activities directly or indirectly contribute to increased deforestation, soil erosion and loss of biodiversity in a country (tigabu, 2016). https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 89 there is a common thought that roads constructed under urrap create highly devastating pillars which cause environmental issues in ethiopia. however, there was no directly, explicitly, and scientifically conducted researches on the issues, particularly on study areas of this paper which are known by rich plant and animal species and fertile soils. finally, this research was intended to fill this gap within a short plan and be a source of achieving a sustainable development within a long plan. objectives of the study general objective the general objective of the study was to investigate environmental implication of urrap roads in southwestern ethiopia; the case of jimma and buno bedelle zones specific objectives (1) to find out the associated factors with urrap roads in degrading pillar1 environmental issues. (2) to identify the types of degraded plant and animal species by urrap roads. (3) to identify the consequences of changing pillar environmental issues as in the case of construction of rural roads under urrap. limitation of the study like every researches, this study is not free from limitations. it has conceptual/theoretical, methodological and knowledge limitations. since, the issue of road is interdisciplinary; the concepts which have been used in this study were not exhaustively listed, used and/ or identified. for example, “non-urrap roads”2 were not included in this study. besides, sample size used in this study was not too enough which extends to the limitation of sampling techniques. the less attention given for quantitative data in the study might be another limitation of the paper. directly or indirectly these limitations had association with the given time and budget. research methodology study setting and population the study area is in the oromia national regional state (onrs) of ethiopia, jimma and bunno bedele zones, south western ethiopia. the total population of jimma zone is 2,986,957, of which 1,498,021 are male and 1,488,936 are female. gera and gomma districts from among 20 districts of jimma zones and didessa district among ten districts of bunno bedele zones were randomly selected. interest of the researcher officially allowed time to conduct a research and distance of selected sites were the reasons for the selection of these 1 forest, soil, and animal species for the purpose of this study 2admassu et al. 2013: 27 https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 90 two zones. the other reason was that, official data describe that either oromia national regional state or federal government has been giving significant emphasis for these zones in constructing universal rural road, because of its plenty resources: coffee, honey, and fruits. research design the study at hand deployed qualitative approaches supported by quantitative. the philosophical foundation of the study was pragmatism. the reasons for the selection of pragmatic approach were: to use variety of data sources, to use multiple methods in the study at the same time or one after the other and to use multiple perspectives to interpret the results. cross-sectional design was employed to collect data at specific data. moreover, this study was descriptive, exploratory, and explanatory. methods of data collection household survey through household survey: demographic and socio-economic data; nature, types and changes of pillar environmental issues; accessibility to urrap roads; and feeling of household heads about urrap roads along with pillar environmental issues. in-depth interview in-depth interview was employed in collecting detailed data which are a complete picture of the objectives of this study from purposively selected respondents. the researchers selected those participants assuming that they have an experience on issues under study and can provide pertinent information. field observation observation was served as a complementary method to triangulate data collected through in-depth interviews and other methods. accordingly, the researchers observed about road construction, road structure, deforested areas, and soil erosion as a result of urrap road. focus group discussion (fgd) so as to get detail information, fgd was employed with purposively selected discussants. the criteria to select members to fgd were: living areas, age, and distance from rural road. accordingly, data were collected through fgd in all selected districts. https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 91 instruments of data collections the main data-generation instruments were structured questionnaires, check list and interview guide. the primary objective of structured questionnaire was to elicit quantitative information from household head. altogether, the contents of the questionnaire items covered issues on an association and extricable effect among an accessibility of urrap road infrastructure and pillar environmental issues. checklists were used for observation while interview guide was used to elicit qualitative information through deeper consultations from informants, key informants and discussants. sampling techniques and sample size simple random sampling technique appeared appropriate since the lists of the units studied was accessible. in it, each member of the population under study had an equal chance of being selected from a list of the population. the sample size of the respondents was determined depending on the formula of yamane (1967: 886) because it is simplified in the case of finite population. the formula considers 95% of confidence, and 5% margin of error. accordingly, 210 household heads were randomly selected. concerning, the sample size for qualitative methods: 18 individuals for in-depth interview, 15 key informants, and 3 fgds which is one in each district were selected. the total discussants of fgd were 27 without facilitators. overall, the sample size was determined by saturation points under all the discussed qualitative methods. reliability and validity reliability the aspects of inter-rater technique of reliability proved its usefulness in the context of the pilot testing. the inter-rater reliability assessed the reliability of research instruments by utilizing four interviewers per site. through cross-examination of the reflections of the pilot testers, the evidence was obtained. researchers also table 1. sex, marital status and religion of respondents (household survey, 2017). variable category percentage sex female male 11.9 88.1 marital status single married 2.9 87.6 widowed 9.5 total 100 https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 92 ensured internal consistency of instruments by the split-half correlation from plot-testing of 20 questionnaires. this was done by comparing the results of one half of instruments with the results from the second half which had been separated as first half and second half. validity both the face validity and content validity were assured to judge whether the instrument capture the core concepts in the study or not. besides, content validity was assured by giving the instrument for four experts in the area of study. then their comments and suggestions were included to verify the validity of the instrument. methods of data analysis quantitative analysis methods were used with the numeric data. both descriptive and inferential statistics were calculated. the descriptiveanalysisemphasizedonpercentages, central tendencies and graphic presentations. consequently, the interpretations that followed presentations were made through pie charts and frequency tables portraying numeric facts in finding chapter of the study. the chi-square test of association was tested for looking at the prevailing associations among the interactional variables. the qualitative data were transcribed, categorized, schematized and interpreted based on their respective contents and themes. the meanings, words, symbols and argumentative texts formed basic premises in the structures of reporting the sub-titles, sections and chapters. ethical considerations in conducting this study, an ethical considerations and safety measures were made. before going to the field the letter from jimma university was taken and given to the districts‟ administrative and other required bodies. after going to the field and contacted with respondents, the purposes and importance of the study was explained for the participants of the study and informed consent was obtained from each of them. thus, participants were given the authority to permit or refuse in the collection of data in any form; full right was deserved to withdraw at any time: to change ideas or to edit recorded materials. besides, the privacy of the participants was promoted, and they were informed that whatever information they provide will be kept confidential. results and discussion this chapter deals with data analyses, presentation, interpretation and discussion of the study. it attempts to answer the research objectives concerned with environmental implications of urrap road in southwestern ethiopia. specifically, it includes the presentation of: (a) demographic and socio-economic information of respondents; (b) urrap roads and environmental sustainability from the perspective of local people; and (c) possible effect of accessibility to urrap roads on pillar environmental issues. the analysis and presentation of thesepartspredominantlyandcomprehensively appeared from qualitative data and lightly supported by quantitative data. https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 93 demographic and socio-economic characteristics of sample respondents table 1 shows that majority of respondents were male (88.1%) and married (87.6%). this is consistent with the reality of the study areas that the majority of rural household head are male and married, which is consistent with the secondary data obtained from the districts showing that majority of the household heads‟ of study areas (gomma, gera, and didessa) were male (73%) and married (68.6%). sex and marital status are directly or indirectly associated with urrap road construction and usage in the study area. for example, all of the drivers of the vehicles in the study areas are male. married women are not allowed by their husband to get job opportunities of road construction, because of myriad factors including socio-cultural factors. table 2 shows that averagely, the age of respondents is 40 years old; the years he/she stays in education was five; and the size of household size was 6.4. urrap road: environmental sustainability there have been debate and struggle about the definition of environmental sustainability among the scholars for the past decades (moreli, 2011). the process of prioritization from social, economic, and environment is different from country to country. ethiopia has also been expanding various programs for the purpose of social and economic sustainability including urrap. however, majority of the respondents of this study (76%) believe that the concern had not been given for environmental sustainability, while urrap road construction have been expanding and implementing in a country in general and in rural areas in particular. accordingly, this subsection deals with the negative impact of urrap on environmental sustainability from the households perspectives. in case of pillar environmental issues like deforestation, soil erosion, loss of wild animals, and climate change have been selected. likely, the impacts of un-surfaced roads on ecological resources include direct habitat loss, facilitated invasion of weeds, pests and pathogens many of which are not indigenous as well as a variety of edge effects (habitat council, 1980). table 2. age, education level, and household size of respondents (n =210) (household survey, 2017). variable mean sd age 39.8 12.3 education level 4.91 4.1 household size 6.37 2.7 https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 94 what is environmental sustainability? before analyzing to the impact of urrap road on environmental elements, having some opinions about environmental sustainability from the perspective of people under the study might have a pertinent hint and clarifications for the following sub-topics. the discussion among the discussants of fgds reveals that ethiopia has indigenous values, principles, and laws of protecting their environment since ancient time. they believe that natural environment is what god has given for all human beings: past, current and future generations. that is why our grand fathers and mothers had been selectively and purposively utilizing natural resources through identifying animals and plants as sacred and/or not. the discussion of the discussants of fgd in gera district shows that our grand fathers and mothers had set rules along with identified sacred natural resources like some trees, water bodies, hills, mountains, and fields which have been serving in conserving and utilizing natural resources till today. their foot has been followed in using these limited resources without compromising the coming generation. generally, for these discussants, environmental sustainability is not the recent issue or new thing coming from other areas, „westerns‟. again, in the future, also these values and principles will have been working for ever in order to assure environmental sustainability. respondents3 attached environmental sustainability with mercy, altruism, peace, development, stability, wisdom, healthy, and civilization. they argued that in one way or another, environmental sustainability is the causes and consequences of these human life basic values. in words of an interviewee from gera district (male, 67 years old) “if the inverse works, the world will wait for destruction, disorganization, instability, chaos, and loss of self-identity. superficially, these problems are not concerned with environmental elements like that of our life and socio-economic satisfaction.” this implies that nowadays people are deviating from existing and an old value of keeping sustainability of environment by indigenous4 people. urrap road and soil erosion many respondents argue that during wet periods, perennial streams and major rivers start to flow. there are no adequate cross drainage structures, such as, culverts and water-crossings-drifts, causeways and bridges. thus, access can be prevented and often for long periods; three months is not uncommon. majority of respondents (94.5%) expounded as the maintenance for the road is very weak. an expert of rural road construction from gomma district (male, 30 years old) associates the problem of maintenance with lack of trained personnel in the private sector who can carry outthe appropriate type of maintenance works. leader from road authority from gera district (male, 32 years old) also authority, 2017). contended that the challenges for providing allweather rural road is a complex interaction between availability of the right materials, climate (wind, rainfall, and intensity), terrain demands, service standard demands, and providing appropriate and cost effective services. 3 survey respondents 4people who have been living in the study area for a long period of time and have their own way of life or who don’t come from another place. https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 95 researchers observed that in majority of urrap roads in the study area, there is no draining of water from the road surface quickly without letting it concentrate. many of them have potholes, ruts, and mud on the surface of the road which are symptoms of drainage problems. it is characterized as improper steepness and slope and shape of the road (figure 1). majority of a ditch system plugged debris causing water to flow onto the road surface. ditch water captured by the road surface can cause severe erosion and wash out the road completely (kocher et al., 2007). respondents also argue that rural roads are not appropriately designed for all-season use. they believe that these rural roads are assumed and considered to be used year-round and are intended to be in continuous service for the for-seeable future. however, the structure of the road is completely changed between the summer and winter. in addition, the construction makes the slopes on the roadsides vulnerable to landslides and erosions. generally, all these and other factors lead to soil erosion and complications in maintaining stable rural roads. similarly, final report (1997: 12) on the environmental impact analysis of the road sector stated that: one of the most important issues of environmental impacts in a road project is soil erosion. erosion is a function of the stage of construction (rehabilitation/ upgrading and new construction) and the physical environmental condition (geology, climate, soil, topography and surface and groundwater hydrology). sheet and rill erosion and formation of gullies happen both naturally but is also figure 1. roads with potholes, ruts, and mud (picture taken from gera district office of road https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 96 highly accelerated by road construction if fresh soil slopes are not properly and timely protected and if drainage arrangements are not efficient (figure 2). urrap road and deforestation respondents believe that during and after construction, urrap road has an impact on the terrestrial vegetation in the vicinity. they also argue that there were uncontrolled uses of wood for energy and for construction purposes during a road project which lead to significant destruction of forest resources. logging roads result into clearing about 50 acres for each square mile of commercial forests, degrading about 10 acres for every mile of road and each square mile of forest averages 5 miles of road (noss, 2004). one interviewee from didessa district, natural resource bureau (male, 34 years old) argued that “local communities have been engaging in deforestation during and after road construction.” during road construction, correspondingly, project staffs collected firewood. one respondent from gera district stated that “those who came for road construction from another area had been collecting and enchanting firewood in vehicles.” another expert from natural resources bureau of didessa district (male, 30 years old) argues that “throughout road construction there was voluminous deforestation in our district. deforestation has been leading to extinction of indigenous tree like “woddeessaa, qaraaroo, heexoo, oomoo, and birbirsa.” respondents of the study areas had various arguments about how and why road construction might lead to deforestation if not controlled. these factors can be categorized as the following: socio-psychological factors: displacement, conflict, corruption, low integration and local communities‟ motivation. these play a great role in deforestation if handled anomalously. an interviewee from didessa district (male, 56 years old) stated that “individuals who had been displaced from their farming land and living figure 2.slopes that resulted from urrap road (picture taken from didessa natural resources office, 2017). https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 97 house occupied another land instantaneously. to construct their house and palisade, they were cutting trees. this may gradually bulldoze to deforestation. haplessly, vendetta happens among local communities, clients, contractors, consultants, and another sectors on limited resources. accordingly, one interviewee from gomma district (male, 65 years old) said that: “have big ideas of road construction, clients and other project staff churned design out in my friend’s farming land. they cut down his entire coffee’s tree and displaced him out. i think they did it because of two interplayed reasons. these are: firstly, he came from another area and secondly, responsible bodies were corrupted. my friend started to cut down natural trees and enfold spacious land for his life. again, kebele leaders and neighbors accused him for this reason. both sides fall in belligerence and in-between forest is victim”. this implies that on one hand, new opportunities for income generation were welcomed but at the same time it could result in conflict or ill feelings depending on how resource extraction activity takes place. economic factors: marketing, imx in rural areas, expropriate, lack of public infrastructure like electrification and cost of road construction. principle road facilitates marketing and development. people sell their products and consume others products easily if there is road access. it is not uncommon that rural people sell the products of natural resources such as charcoal and timber. getting access to road has been instituting local communities and people from other areas in cutting down natural forests. in other words, discussants of fgd from didessa district claimed that since a road was in place in the study area, it has been providing access to people wanting to supply urban markets with wood products such as charcoal, fuelwood, bush meat, or construction materials, contributing further to deforestation. consistently, expert from gera district natural resources bureau (male, 34 years old) states that "road construction has been motivating people to entangle with forest products.” regarding data collected through survey, 67.3% of respondents believe that urrap roads have paved the way for deforestation since various investments being expanded along with all the newly constructed roads. together with urrap road, there was expansion of imx in the study areas. for example, almost all (98%) „kebeles‟ of the study areas, which have access to road, have at least one imx. the types of imx are: energy and stones digging, farming, goat and sheep breeding, wood work, metal work, and the like. all these economic activities need extensive land. thereby, intentionally and/ or unintentionally there were the cases of deforestation. an interviewee from gera district, tumaateessoo kebele (female, 43 years old) claimed: “the supersize of imx is causing deforestation. getting money and becoming richer and richer is a skeletal clandestine of life. low sector-sector and sector-people coalescence and unpracticed rules are the pressing and preponderant reasons. people are flinging to win their today’s life; incidentally, there are numerous scenarios such as deforestation, loss of animal species, climate change, etc.” ethiopia is the second poorest country in the world (hdi, 2014). predominantly, it has been relying on aid from western countries for a long period of time. it is characterized by ineffective and inefficient use of resources. consequently, public infrastructures services are known by low quality. road being constructed under urrap is not different. in other words, as of low budget there was no rationally and practically balancing the cost of forest and constructing road. correspondingly, surveyor from gomma district (male, 29 years old) expounded that “practically and virtually, the costs of cutting down trees to that of constructing road have not been benchmarked. parenthetically, they only have to finish off the road with small amount of money. it is believed that this may cause extinction of big and indigenous trees in our country.” the leader of the road authority of gomma district (male, 35 years old) likely inclined to his colleagues saying that https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 98 “everything have both advantages and disadvantages. to get one, we lose another. they have been plodding not to deforest hence are being bulldozed by our budget and landscape.” urrap road and loss of animal species the construction of some urrap roads penetrates sensitiveareas, whichcauseanimpediment to movement of wildlife and destruction of wildlife habitat. as a proposed road project could be new, therefore, the wildlife which presently inhabits the proposed project area may be forced to migrate to other areas. 70.1% of respondents argued that the urrap roads can cause the loss of animal species. interview with respondent from didessa district (male, 63 years old) indicates that the following are gradually decreasing in number. they are: “saasoo, jaldeessa, booyyee, weennii, qamalee, dhaddee, woongoo, doollee, bosonuu, yayyii, leenca, qeerransa, arayaa, and iyyaa. another interviewee from the same district (female, 43 years old) added that “boyoo, rumicha, and booqaa are indigenous birds which are leaving these areas to another area.” the data from fgd, in-depth interview, key informant interview and household survey were indicated as immediate causes for the loss of biodiversity along with road construction are as follows and these reasons are intractably associated with each other; change in one reason can result in change of another. inequitable road line selection: the stakeholders, who have been constructing road, select road line depending on only cost analysis. surveyor from gomma district (male, 29 years old) rationalized as poverty is influencing them to do so. another interviewee from didessa (male, 50 years old) also argued that: “they come to select road line without any material in their hand. there is some body from the district, contactor, kebele leader and others; stand on some place; and they say let it be this way. generally, they select only short distance in place of identifying the required characteristics of the road”. roads crossing areas with high water tables or wetlands may act like dams to block surface and subsurface water flows. this is especially true where large quantities of material must be added to raise the road above the land surface, and where new material must be added annually to keep the road elevated. discussants from fgd5 argue that under these circumstances, land on one side of the road can become much wetter than it was before the improvement, while land on the opposite side may be drier. this may adversely affect crop production, the composition of species in the ecosystem. vehicles pollution: vehicles sound and chemical pollution forces wild animals to live the area. they need an area free from disturbances. they also require to be kept hidden. thus, the sound of vehicles (clasci or other related sounds) has been acting against the principles of these animals. one respondent from natural resources office, gomma (male, 32 years old) confirmed that “wild animals run away from the sounds of human and vehicles during and after the construction of rural roads.” likely, karani (2007) also found that road construction and upgrading and vehicles pollution have a negative impact on bird species, reptile species, amphibian species, mammal species, and butterfly species. the effects of new road include pollution from moving vehicles, chemical spills from trucks, displacement wild animal species due to construction of new roads, disturbance of hydrology and the ecosystem among others. 5 fgd held with 10 individuals in didessa district (male = 6, female = 4, average age = 33.4) https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 99 expansion of farming land: along with the enlargement of rural road, there is expansion of farming land either by rural community and externals (called „investors‟). in other words, rural roads help people to engage in farming through modern technology. since there is no land out of function, the coming modern technology for farming demand the land occupied by another forest, wild animals and small creatures. besides, the technologies which are being used, it can also be the reasons for the running away of wild animals. consistently, 53% of the respondents believe that expansion of farming land alongwith the construction of rural road is causing the loss of wild animals. among the participants (r1 and r2) of fgd, nine discussants from gera district depicted this reality as follows: r1 (male, 32 years old) stated “displaced people need the land. land is limited, but demand for land is unlimited.” r2 (male, 43 years old): “after we have gotten the road, the tractor and other technologies of farming have been coming to this area. we think this has a potential impact on the wild animals to live in the areas.” change of households’ settlement: data from fgds discussants implies that along with the construction of road, there were people who moved to the edge or nearby to the road line by leaving their origin place. thus, they lived their former place and found another place for their house and other activities. this might have been directly or indirectly impacting wild animals. the chi-square test found that there is a significant association between the distance of households‟ house from road and their responses regarding either the wild animal species are leaving the areas as a result of the rural road enlargement with (χ2 (18) = 56.795, p = 0.000) at α = 0.05. conclusion urrap road have been constructed from 2012 in differentpartsofethiopia.its players are both government and people. the eager and enthusiasm of people to get road is greater than what the government has been undertaking. people need road institution with all acceptable definitions: process, methods, and outcomes. this means, people give balanced weight both for quantity and quality of roads. however, the government has been going on only quantitative dimensions. the connection and association of road institution with other social institutions was not given considerable and required value. in other words, they are standing alone. the problems resulted from road institution caused by the failure of all social institutions interconnection; road institution is also among social institution. consequently, urrap road has been inserting various problems into pillar environmental issues. it has resulted in soil erosion, deforestation, and loss of animal species. indigenous trees are being cut and indigenous animal species are been moved. as a result, climate change appeared in most parts of the country. inequitable road line selection, investment and farming land expansion, and corruption are among the main causes of these problems associated with urrap road construction in ethiopia. however, if managed, designed, supervised, customized and internalized, road would play a vital role in environmental sustainability. even though there are a large number of problems, urrap road has also been providing various services for rural people. it creates a job opportunities, increases socio-political participation encourages social network, facilitates production and consumption, increasestourism, andassiststhe expansion of other social services. recommendations recommendations for government bodies include: https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 100 (1) government bodies6 should integrate road institution with other social institution (family institution, education institution, religious institution, political institution, and health institution) comprehensively, explicitly, transparently, and officially. (2) government bodies should change what is on the document into practice. in other words, let the criteria for road line selection and environmental impact analysis be applied. in light with this, the government bodies should encourage monitoring and evaluation at all levels. (3) government bodies should customize imported modernization elements into the local people indigenous knowledge of keeping the environment sustainable. (4) by taking into account the aforementioned recommendations, government bodies should increase the resources7 for rural road construction. recommendations for policy makers include: (1) policy makers should come up with inclusive policy, which includes issues about pillar environment elements. the policy makers should cooperatively work with researchers from different disciplines from natural sciences and social sciences. acknowledgements the authors would like to acknowledge the research and postgraduate studies office of college of social sciences and humanities (cssh), jimma university for its material and other opportunities offered. finally, they would also like to thank their respondents without which this study would have been impossible. references abate a (2003). afloristic composition and structural analysis of denkoro forest, south wello. msc thesis presented to the school of graduate studies of addis ababa university. abdelfattah m (2009). land degradation indicators and management options in the desert environment of abu dhabi, united arab emirates. soil horizons 50(1):3-10. asher s, novosada p (2017). rural roads and structural transformation. jel codes: o12/o18/j43. allport rj, anderson rj (2011). managing strategic risk:the worldwide experience of metros. proceed. inst. civil eng. manag. procurement law164(4):173-180. arethun t, bhatta b (2012). contribution of rural roads to access toand participation in markets: theory and results from northern ethiopia. j. transp. technol.2(1):165-174. bahru z (1991). a history of modern ethiopia 1855-1991. 2nd edition. eastern african studies. addis abeba, ethiopia: addis abeba university press. bryceson df, bradbury a, bradbury t (2008). roads to poverty reduction? exploring rural roads' impact on mobility in africa and asia. dev. policy rev. 26(2):459 482. csa (2015). section bpopulation 6 road authority sector (office) from federal up to district level. 7 time, budget, personnel, technologies https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 101 egziabher tb (1988). vegetation and environment of the mountains of ethiopia: implications for utilization and conservation. mountain res. dev. 8(1):211-216. escobal j (2005). the role of public infrastructure in market development in rural peru. wageningen university. available at: http://mpra.ub.uni-muenchen.de/727/ era (2012). universal rural road access program. addis ababa. fdre (federal democratic republic of ethiopia) (2012). national strategy and action plan for the implementation of the great wall institute in ethiopia. fikadu e, melesse m (2014). endemic plant species composition and their status in boda dry evergreen montane forest, west showa, ethiopia. int. j. biodivers. conserv. 6(7):563-569. karani p (2007). environmental implications of the road network in south africa.development bank of southern africa (dbsa) jacoby h (2000). access to markets and the benefits of rural roads. econ. j. 110 (1):713-37. keller ej (2005). making and remaking state and nation in ethiopia. in: larémont rr, editor. 2005. borders, nationalism, and the african state. boulder, co and london, uk: lynne rienner publishers, pp. 87-134. kocher s, gerstein j, harris r (2007). rural road: a construction and maintenance guide for california landowners. agriculture and natural resources; university of california. lulit a (2012). impact of road on rural poverty: evidence form fifteen rural villages in ethiopia.international institute of social studies ma thesis, the hague, the netherlands. pant s, samant s (2007). assessment of plant diversity and prioritization of communities for conservation in mornaulareserve forests. g.b. plant institute of himalaya environment and development, himalaya unit, himachal pradesh, india. roberts, peter, shyam k, cordula r (2006). rural access index: a key development indicator. transport paper 10, world bank stifel d, minten b, koro b (2012). economic benefits and returns to rural feeder roads: evidence from a quasi-experimental setting in ethiopia. essp working paper 40, ethiopia strategy support program ii. sucoff e (2003). deforestation, environmental encyclopedia, at pp. 358-359. tarekegn a (1987). the history of the imperial highway authority (iha): 1951-1980 [ba thesis]. addis abeba, ethiopia: department of history, addis abeba university. tigabu dg (2016). deforestation in ethiopia: causes, impacts and remedy. int. j. eng. dev. res. 4(2):23219939. tesfaye a (2007). plant diversity in western ethiopia: ecology, ethno botany and conservation.university of oslo norway, presented for degree of doctor of philosophy. tighes d (2008). planning rural roads in developing countries. from <http://www.ruralroads.org/indexes.shtml> (retrieved on 1 september 2015). tolera m, asfaw z, lemenh m, erik k (2008). woody species diversity in a changing land scape, southcentral ethiopia highlands. agric. ecosyst. environ.128:52-58. vandana d, potter r (2008). the companion to development studies. london: hodder education. willis k (2013). theories and practices of development. new york: rout ledge. https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2018.8908 social and technological sciences issn: 2341-2593 nagesso and edae (2018) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 5 nº 1 (2018): 86-102 | 102 winberg e (2010). participatory forest management in ethiopia, practices and experiences food and agriculture organization sub regional office for eastern africa. addis ababa. wondemu k, john w (2012). rural roads and development: evidence from ethiopia. eur. j. transp. infrastruct. res. 12(4):417-439. wondie m, temesgen g (2013). threats of woody plant species diversity and their conservation techniques in ethiopia. euro. j. bot. plant sci. phytol. 1(3):10-17. world bank (1994). infrastructure for development‟, world development report. world bank. world bank (2004). international bank for reconstruction and development, the world bank responsible for the new millennium. washington, d.c. yamane t (1967). statistics: an introductory analysis. 2nded. new york: harper and rao. https://doi.org/10.4995/muse.2018.8908 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 46 application of innovative teaching-learning methodologies in the classroom. coaching, flippedclassroom and gamification. a case study of success c. gómez-ejerique, f. lópez-cantos* dpto. de ciencias de la comunicación. universitat jaume i, avda. vicente sos baynat, s/n, 12071 castellón de la plan, spain. * corresponding author: email: flopez@uji.es; phone: + 34 964729906 received: 2019-09-01; accepted: 2019-04-19 abstract the new teaching techniques based on avant-garde methodologies aimed at enhancing student learning are being promoted at all levels of education for a few decades, integrating the significant advances that are taking place in different scientific disciplines, including from the areas of psychology basic and group up to the most effective techniques of talent management and organizations. in the present work, we review the teaching-learning techniques and methodologies that we consider most efficient at present, with examples that show their suitability for current teaching. the application and effectiveness of coaching and gamification techniques and the innovative pedagogical strategies integrated in the so-called flipped classroom are analyzed. the results obtained with the application of these methodologies allow us to conclude that the analyzed pedagogical tools are ideal to increase the learning capacities and personal development of the students. keywords education theory, teaching methodologies, coaching, gamification, flippedclassroom. https://doi.org/10.4995/muse.2019.9959 multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 47 1. introduction the notable increase in the use of the terms gamification, flipped classroom, and coaching in the educational discourse of a few years to this part is a good example of the significant changes that are taking place within the institutions and among the different groups that they make up the educational system (hargreaves, 2004-2006, claxton, 2008). and this growing tendency indicates, even with the own chiaroscuros that accompany the application of these new educational methodologies as a result of their novelty and lack of institutional consolidation, that the new teaching-learning techniques are not a passing fad anymore. and, just as it is happening in other areas, such as in the business in which there is already talk of "executive coaching" normally, these new cutting-edge teaching tools are demonstrating their effectiveness in the learning processes and knowledge construction. the application of these new methodologies and techniques are intended, first and in general, to facilitate significant improvements in the students' cognitive and relationalcommunicative abilities and, therefore, to facilitate learning and growth processes. and the new teaching-learning techniques based on gamification strategies, in the so-called flipped classroom, as well as the talent and creativity management tools for the development of people and organizations based on educational coaching are here to stay. the new educational methodologies that are analyzed in this paper apply collective and down-up learning models and grow-up techniques from the implementation of enquirybased pedagogical strategies and tools (wang, 2012) and gaming-based learning (gros, 2007; chorney, 2012; albretch, 2012), as well as innovative methodological proposals such as those integrated in the flipped classroom (baker, 2000, sam and bergman, 2014, strayer, 2014) and coaching techniques (wolk, 2007, harvard business esentials, 2005; bandura, 1997) that are oriented, unlike classical methodologies, to promote the development and autonomous learning capacity of students. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 48 1.1. how do we learn? this question is the first question that all teachers should know how to answer when they enter a classroom. knowing first of all how we learn is essential to concentrate all efforts on achieving a meaningful and lasting learning over time, that creates solid foundations on which to move forward and that is able to keep the student motivated, responsible and able to learn significantly. the objective of education should not be other than to teach students to learn to learn, nothing to do with the commitment of the educational system to prepare them to pass exams and pass one test after another. learning should allow to acquire new knowledge and also new skills that serve the student to face the events of life in a successful way, in a broad sense, so that they can handle the new situations that are presented to them and develop their maximum potential as people. according to pérez (2004), educational theories that are more efficient today have tried to explain how we learn by overcoming traditional pedagogical models and approaching the educational process from new perspectives. the traditional teaching models focused all the responsibility of the educational process on the teacher considering the student a passive subject receiving knowledge. in this teaching-learning model, which, although obsolete is still common to find in our classrooms, it is only intended to transmit concepts that have to be memorized and without addressing the learning process as a whole. the reaction to these archaic forms of teaching began to take place a few decades ago from different perspectives. in latin america, for example, what was known as the pedagogical theory of liberation, promoted by freire, developed in the 60s of the 20th century in a very particular political context in which models of change and change are promoted. transformation and knowledge is considered a continuous process where the use of dialogue and peer interaction between teacher and student becomes important. no doubt an interesting and applicable proposal in our environment, beyond that in its origins was developed more as a political action than as a pedagogical theory, and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 49 therefore it is known as the "pedagogy of the oppressed". what interests us about this theory is that it defends that education should not interfere with creativity, curiosity and, much less, the development of students' potential. in a similar sense, other pedagogical models have responded to classical teaching methods, such as the one known as non-directive pedagogy model, which deals with human development from the perspective of the theories of evolution and the constant biological tendency towards personal fulfillment. inherent to each individual, giving special emphasis to the process of valuing life experience as satisfactory or not depending on its adaptation to the innate improvement objectives of each person. therefore, any experience that is perceived as contrary to this assessment, to improvement and constant updating, is considered negative by the individual and, therefore, the little flexibility of traditional pedagogies does not contribute to facilitate a good experience of the students . in what we are interested in this pedagogical current driven by rogers (1992), the role of the teacher must not constrain the development potential of students, on the contrary has to create the right conditions for these to unfold in all its breadth and, therefore, that a climate of understanding, acceptance and respect is favored in the classroom. classical methods of teaching are far removed from ontogenetic mechanisms and the processes of development and learning in the terms in which piaget expressed himself, as biologically conditioned and destined to an ever better understanding of reality and the environment in which develops each individual in a process of perpetual, dynamic and flexible adaptation (piaget, 1968). and, in this line, other forms of approaching the teaching-learning process that start from psychology, such as the so-called cognitive model (neisser, 1976; chi, glasser and farr, 1988), are also inscribed, in which it becomes special emphasis on the specificity of each individual and the necessary adaptation of pedagogical methodologies to the characteristics of cognitive processes in general, which are hardly tolerated by direct and exclusively rote-based teaching methods. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 50 nowadays, the theories with which we approach teaching-learning processes are based on structuralist models that, with different perspectives, understand that the systems of symbols and discourses that sustain the belief systems, attitudes and expectations that determine the way of acting of each person have both an individual and collective character and mould the culture itself as well as the development of each individual. the knowledge, therefore, as far as we are concerned, can be understood as an individual and cultural construction in which the educational process has a major role. in that sense, in education, the most essential is the way in which each student builds that knowledge but, over all, how the student uses it and the validity and effectiveness of the learned experiences in front of the vital situations he or she faces every day. therefore, the role of the teacher comes first to know how his or her students build knowledge and from there to design the strategies that can be implemented to collaborate with the student to develop his or her learning, always with flexibility and adaptability and taking into account the context in which each student builds his or her values and integrates his or her experiences. in sum, and without extending much more, the way in which we learn has nothing to do with directing and conditioning methodologies and, quite the contrary, these are totally counterproductive and harmful for the proper development of the teaching-learning process and, in consequence, to achieve the goals of properly preparing the student to face his or her vital journey by promoting his or her cognitive development and personal growth. so learning is a cognitive, social and cultural process, and it is more effective when approached in a contextualized manner and taking into account the student's own features. and the teacher should promote group work and interaction with the student, and should adopt the triple role of researcher, to know who the student is; as mentor, to direct his or her learning in the most appropriate way; and as facilitator, to enhance the capabilities of each of the students. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 51 1.2. then, how should we educate? do we teach 21st century students? times are changing but, as it often happens, the methodologies and practices have done so to a lesser degree and they scarcely suit to the pedagogical tools and technologies we use today. and teachers can, and they should introduce the broad set of educational resources available to them that have amply proven to be effective and with very positive effects. in addition, the teacher must necessarily be a good communicator, he or she must have emotional management capabilities and be prepared for the different situations that can occur in the classroom for which the emotional control of the adult is fundamental. it is true that many teachers have not received an adequate training, and currently only marginally and often almost by obligation they attend updating and teaching competence and skill improvement courses. but, although it is not an easy task, there is no doubt that the obvious deficiencies can become interesting opportunities of renewal and stimulus to make the classroom and teacher´s work a rewarding and enriching profession, with a some will and dedication. to achieve it, even being an obviousness, it is essential to assume that any person who is engaged in teaching must keep continuously being trained, must be conscious that learning is a life lasting task, and that in the daily practice the teacher must have a high knowledge of teaching tools and methodologies and apply the most appropriate innovation techniques among the many that are offered to us today, to each situation. it should not be necessary to insist on how inappropriate it is to continue defending master classes and traditional pedagogical methods, but the data that show their inefficiency are overwhelming. if we look at the pyramid of learning of cody blair (prieto gil, 2010), we clearly see that with listening we only retain 5% of the information that is transmitted to us (in the following 24 hours). multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 52 figure 1 learning and retention piramid if we continue to descend, we observe that the next strategy with a low learning effectiveness is reading, another of the very common techniques used in spanish classrooms. although it is somewhat more effective than listening, it is only 5% more effective. audio-visual contents, which cautiously and after many efforts a few decades ago entered in the classrooms, are more suitable for teaching, with a 20% retention rate, but even by themselves they do not have a significant effectiveness. the further down in the pyramid the retention rate increases, reaching 75% when the applied technique is learning by experience. according to this scenario, the teacher must move from being a passive subject, mere transmitter of knowledge to being a facilitator for learning. he or she should be able to create a trusting environment, an active learning environment, and he or she should be able to awaken the curiosity of the student, to propose challenges, to be updated and to use in his or her classes all the resources and innovative experiences that lead to multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 53 achieving a high quality education. therefore, bringing up the phrase we inaugurated this text with: "education is not the filling of a pail, but the lighting of a fire" 2. methodology in our study we have applied the coaching and gamification techniques and flipped classroom teaching methodologies in a real learning environment in the classroom in collaboration with the teachers responsible for the subjects. in our research we have analysed, as a priority objective, the effectiveness of the coaching techniques applied to education. to achieve it, we have selected a sample of 12 teachers who teach classes in the different eso1 courses (mandatory secondary education) in a concerted school in the city of valencia (spain), and we have trained them to apply this above mentioned techniques to analize this performance in the classroom. to analyse the application of gamification and flipped classroom techniques, a total of 49 students have been selected; (24 students) in an eso 3rd degree group and another (25 students) in an eso 4th degree group. the analysis of each of the techniques has always been developed following the below explained phases: to begin with, extensive personal interviews were conducted with the teachers who were invited to participate in the study. then, after that first personal interview, we determined the teachers’ predisposition to introduce the proposed techniques and we gathered their impressions in relation to the deficiencies that their students would have, according to their perception, as well as the general problems they meet in their teaching practice. 1 translation note. eso: spanish acronym for mandatory secondary education.in spanish current education system, kids start the first of the four degrees the eso has, at the age of 13 years old. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 54 afterwards but prior to the application of each of the analysed techniques, the students were asked to respond to a semi-structured questionnaire in the same terms that had been used with the teachers. finally, and after the application in the classroom of the methodologies to be analysed, both the opinions of the teachers and of the students were again collected, in the same terms used in the beginning, to analyse the influence and effectiveness that the tools applied had had. we have had the collaboration of the teachers in each of the sessions we have devoted to apply in the classroom the different analysed techniques, so that during an academic year the work dynamic was: we will now expose the features and pedagogical relevance that the different techniques are presumed to have in the classroom and that we have analysed in our study sample and in a real environment to determine its degree of applicability and effectiveness: initial questionaire training the teacher classroom feedback multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 55 2.1 coaching coaching is a relatively new discipline that broke into our vocabulary in the 80s, spreading with greater strength in our daily life in recent years. although it currently has great media notoriety, its origins are not entirely clear, since coaching is nourished by different sciences, disciplines, thinkers, just as there is no unanimity when defining the term itself. for our purposes, coaching can be considered a methodology whose objective is the personal and professional development of individuals. it is a process that occurs as an interaction between two people, the coach and the coachee (the client or person willing to make a change), in which the aim is to transform a current state (unwanted) into another state desired by the coachee in a specific context. it is important to bear in mind that coaching is in consequence focused on both the present moment and on the exploration and the vision of a desired future. to achieve this the coach uses different methods and tools that generate in the coachee perspective changes; conscience, commitment and responsibility boost; increase of the desire of learning, and development of the already existing capacities so as to facilitate an impulse to the action that will revert in reaching the proposed goals. the aim of coaching is to discover and release the full potential of the persons, to bring to light all the resources they possess to achieve their purposes. it understands that the person has the skills to achieve the desired goal and in the process the coachee will discover and use them to achieve his or her goals. coaching is therefore a process that invites us to experiment and to learn from our own experience. it invites us to use different ways to obtain our goals, to explore new ways of performing to reach our objectives. the coach is understood as a catalyst of the change process. as someone who is able to see the coachee not as what he or she is but as what he or she can become, and who understands that he or she has all the resources to produce that change. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 56 this said, we are going to address all those techniques which we have trained the teachers on, and which they can use to achieve more dynamic classrooms, more motivated students and thus to achieve meaningful learning. 2.1.1 coaching training given to teachers for their application in the classroom all these techniques should be integrated by the teacher before applying them in the classroom in order to be able to extract a better harnessing out of them, that is why the first sessions in the process were dedicated to it, prior to the beginning of the academic year and then we run control and reinforcement sessions throughout the teaching period. 1) creativity as esquivias (2004) states: "creativity is a concept that alludes to one of the most sophisticated cognitive processes of the human being ... it cannot be approached as a simple feature of human beings, it is undoubted that aspects such as: the mind, the cognitive processes that take place in it, the personality, the motivation, the emotions and the affective world play a singular component in this process." and he continues: "on the other hand, we are all creative to a greater or lesser extent and what is even more encouraging, we can all develop it ". to be creative is, therefore, to be curious, to get out of the marked paths, to be opened to the world and to be receptive to everything that surrounds us, to discover ideas that have the potential to unite and form something new. but also reorient our mind to unexpected directions, to be able to see new ways, to relate concepts that seemed to have nothing in common and, in summary, to be in the world in a certain way that encourages openness, exploration and changes. in this sense the teachers were trained to: multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 57 • observe as they had never done before. • increase the student's awareness. keep the student here and now. • practice active listening with the student and have a perception of the environment. • introduce anti-routine changes. • use the sleep-writing technique. • use the brainstorming technique. • use the six thinking hats technique, by bono. • use the lateral thinking. 2) powerful questions one of the tools that make a coaching process effective is the open questions or powerful questions. these types of questions are those that cannot be answered with a yes or a no. these are questions that invite us to deepen in the discovery, in the awareness, drive us to action or reflection. in general, they are questions that lead us to introspection and of course they invite us to look inwards or into the future. for this reason it is important that the teacher uses these types of questions with his students, to increase their curiosity, to invite them to reflect, so that they can investigate inside, although first it is convenient to do them as a teacher. teachers were given a battery of questions for their own reflection and its application in the classroom adapted to the students, such as: • how do i contribute with as a teacher? • how does teaching contribute to me? • what resources do i have now? multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 58 • how far would i like to go? • what resources do i need to achieve the given goals? • why do i want to teach? • how do i think i am doing so far? 3) marc smart goals it is very important to get goals set, as this increases motivation, therefore, the teacher was trained in techniques to motivate their students with very clear objectives that meet the requirements known as smart requirements: • s (specific). the objective has to be specific, the more the better. • m (measurable). the goal has to be quantifiable. • a (attainable). the goal has to be achievable. • r (realistic). the objective must be realistic. • t (timely). the objective must be time bounded. 4) achieve commitment to the objective teachers have also been insisted that students must commit to the objectives proposed at the beginning of the course. teachers in the classroom should regularly explore to what a student should say yes, and to what he or she should say no; that is, what they should give up in order to achieve their objectives. the same thing happens to the teacher, who should review at all times if some resignation needs to be made to achieve those goals, because choosing always entails a renounce. this coaching technique makes it easier for the teacher to make his or her students responsible for their own learning by making them aware of what he or she is saying yes to and what he or she is saying no to. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 59 5) working with the “saboteur” we call saboteur anything that prevents us from moving forward, that does not let us reach the goal and as a teacher we must prepare ourselves to know the saboteur, our own and that of our students. to try to find out in what ways the saboteur appears we suggest some questions like: • where do i let my energy escape? • at what point am i breaking my self-commitment? • how am i being too flexible? • when do the resistances appear? • what comfort zone do i have to abandon to achieve my goal? • how do i limit myself? we carried out this work with the teacher on a regular basis and once he became aware of what was preventing him or her from moving forward, we worked with that limitation to be able to permanently eliminate it. in the same way the teacher worked this aspect in the classroom with his or her students. 6) the wheel of life as a tool for measuring results. teachers have been trained in the use of this evaluation tool that can give us a good starting point of how the teacher and the student are, to set with them the group or individual objectives to be achieved. with it we get both an individual and joint vision that allows us to evaluate at all times the status and results throughout the process. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 60 figure 2 life wheel (source: personal excellence.co) 7) give the students an effective feedback finally, in our methodology, feedback is crucial to determine if the proposed objectives are being met and if corrections and improvements in the implementation of the teachinglearning plan need to be introduced. feedback to be effective must be positive. rebuke if possible in private and recognize and congratulate in public. 2.2 flipped classroom. the flipped classroom model arose when professors jonathan bergmann and aaron sams (university of colorado) began recording science classes for students who could not multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 61 attend to the classrooms. they realized that the marks of these students were very good, so they continued to record videos and extended it to the whole group of students in the classroom. nowadays it is already a widespread practice in many schools around the world. according to its promoters, sams and bergman (200), the flipped classroom technique is a pedagogical approach in which direct instruction moves from the individual learning space to the group learning space and the resulting space is transformed into a dynamic and interactive learning in which the teacher guides students as they apply the knowledge and they can actively participate in the task. it is a model that has been proven efficient, beyond fashion, since it allows the teacher to have more time to establish relationships with students in the classroom and improve interactions with them. for this reason the teacher will know better how each of the students works, so that the instructor can improve the teaching strategies in the classroom and can know what motivates the pupils, so that the students will feel that the teacher cares about their learning and this will increase their motivation. let’s remember that without motivation there no learning is possible. in the flipped classroom methodology, the student has worked individually, brings to the classroom some already clear concepts and –possiblyalso some doubts, so that in that group space the pupil can solve them and integrate better individually. the teacher is necessary to be a facilitator, an expert at that moment in the class. later the student will integrate the acquired knowledge and skills and from that moment the pupil will be able to carry out projects or solve more complex challenges. to achieve this, the student has had to prepare the material before arriving to the classroom and the teacher has to be very clear about what the student has to do during the class time. the classroom must be a learning space with a high interaction among students and between teacher and students; thus, we are talking about active learning. the students will work in groups or individually, so that several situations are generated in the multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 62 class and the teacher will interact with each of the students, being able to personalize the learning according to the needs of each one. 2.2.1 application of flipped-classroom methodology in the classroom with students. teachers were trained in this technique, stressing that its implementation required following good programming and preparation in order to have an impact on less work and a higher quality on their relationship with students and also on a better performance. the tool used was a video related to the topic to be addressed, selected by the teacher and the methodology was developed in the following application phases: 1. before the class: the teacher provided the students with audio-visual material to visualize and work on it at home individually, the day before the class session. 2. during the class: a group work space was established in which the study material was analysed, questions were answered, knowledge was settled and various related activities were carried out, together with the teacher. 3. after the class: a project was proposed based on challenges, and that involved the realization of teamwork in a complementary way to the performance of individual activities of greater complexity. 2.3 gamification. in the field of education there are several success cases related to the use of games (diaz and troyano, 2013). gamification is defined by zichermann and cunningham (2011as a process related to player thinking and game techniques to attract the users and solve problems or as the use of mechanisms, aesthetics and the use of thought, to attract people, encourage action, promote learning and solve problems. teachers have been insisted that the purpose of gamification is to influence the psychological and social behaviours of the player and to produce and create experiences that generate feelings of control and multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 63 autonomy, especially critical skills for the cognitive development in the educational stages the students analysed in the study are in. throughout the teaching period, we have also dedicated some sessions with teachers to train them in gamification techniques and in the use of the kahoot tool. 3. analysis and outcomes 3.1 case study 1. coaching. we worked during the academic year with the 12 teachers selected from the stage prior to the beginning of the classes. teachers who were part of the study sample were initially instructed in the life wheel and the leadership wheel in class, jointly analysing the strengths and areas to improve that they had to measure in their students. the following aspects were taken into account: responsibility for the study, discipline, individual studying hours, participation in the classroom, interest in the subject, concentration and active listening in the classroom, material organization, task planning, punctuality, relationship with their colleagues and relationship with the teacher. at the end of the course, after having followed with them both individual sessions and group sessions, both the life wheel and the leadership wheel were again passed to the students in the classroom, checking that the scores had remarkably increased in all the areas. the teachers, on the other hand, showed a high index of satisfaction and pointed out significant improvements in everything related to their feeling of personal and professional fulfilment at work. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 64 figure 3 performance improvement. as can be derived out of the responses of the 49 students who participated in the study, the items "relationship with the teacher", "responsibility for the study", "interest in the subject" and "participation in the classroom" had significantly improved in cases where the teacher was actively coaching in the classroom. 3.2 case study 2. flipped classroom. the 24 students in eso 3rd degree studying the subject of physics and chemistry made a practice of decantation in the laboratory, which is a physical method to separate the components of heterogeneous mixtures (in this case water and oil), and a filtration, which is a process by which an element is passed through a filter to separate its parts, being retained those that do not pass through its size and being filtered those that do pass through the filter space (in this case sand and water). before, the way of working was to take the students to the laboratory and to begin the explanation of the practice there. what it used to happen was that not everyone would multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 65 understand it at first, one or more class sessions were used to explain the procedure and even so, when it came to performing the practice, not everything was clear. as part of our research and under our supervision, the teacher applied the flipped classroom methodology for the same didactic unit that had been very difficult to learn to the students so far. before the practice and in their homes, each student watched a video where the two separation techniques that were going to be carried out in the laboratory were clearly explained. after the video they had as a previous task to answer a series of questions that were considered necessary to understand in order to perform the practice. this way, if the student after viewing the video did not know how to answer the questions, he would watch it as many times as he would need it until he would be able to answer them. on the other hand, they were provided with a written protocol of the practice, which was nothing more than what they had seen in the video but in written script. once in the laboratory they saw the videos again, they shared their opinions, they resolved the doubts and we made sure that they had understood well both the task to be carried out and the foundation of the practices, that is, the purpose of the practice. when everything was clear, the students in groups of four generally did the practice. the teacher was able to solve doubts and difficulties and they could even watch the videos again at that time because, with this methodology, the students have great autonomy in the laboratory because they are clear about what they have to do and how they should do it. by the end of the practice the students had to prepare a memorandum with the following structure: objective of the practice, methodology used, material used, results, drawings and also reporting everything that happened to them or that had surprised them during the execution of the practice. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 66 after using the flipped classroom methodology with the physics and chemistry class, we conducted a survey on the 24 students participating in the pedagogical experience and these were the results obtained: 1. do you think that the viewing time of the videos in your house increases the time you are in front of the computer screen? 85% answer no. 2. has the video helped you to better understand the practice? 100% answer yes. 3. have you always watched the videos before the practice? at the beginning, only 50% answered yes, but at the middle of the course 85% would have watched the videos. 4. have you easily accessed the videos on the internet? 100% answered yes. 5. has it been important for you to have your queries prepared in order to be more efficient in the laboratory? 85% answered yes. 6. did you need to check the video again during the lab practice? 35% answered yes. 7. the teacher was asked why he thought it was useful to apply flipped classroom in the area of sciences and the answer was that he or she could save explanation time in the laboratory, the practices could be done in the scheduled sessions, something that previously did not happen and that the accidents in the laboratory had decreased as a result of the previous visualisation of the practices. the joint analysis of the results with the head of the subject led us to conclude that, unlike the traditional methodology used until then, if the students watched the videos before the practice they would know ahead the method that was going to be use in the laboratory and for the teacher it was much easier to start the work in the laboratory and the teaching-learning process was much more effective applying the flipped classroom methodology that we had implemented. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 67 3.3. case study 3. gamification. we use gamification techniques in collaboration with the classroom teacher in eso 4th degree, during a year. we first trained him in the use of teaching-learning tools insisting on the fact that for any playful activity to provide meaningful learning it had to be well designed and with a good execution and conclusion. in this specific case the kahoot game was used as a learning tool in the subject sciences applied to professional activity. a series of questions were designed for each didactic unit and the game was carried out during the development of the class session in groups. at the end of each content block, the same questions were taken as an exam. the result during the course was that the 25 students passed the partial exams done at the end of the game, obtaining in the final exams of the subject surpassed by 95% of the students with results well above the average of the previous courses. both the teachers and the students answered the final questionnaire of the subject indicating that they had better disposition throughout the whole teaching-learning process, and all indicated that this methodology was much simpler and more effective to assimilate the contents of the subject, showing clearly that the introduction of gamification tools in the classroom is of great pedagogical utility and facilitates meaningful learning. 4. discussion and conclusions in this research, we have made a review of the joint application of coaching and gamification and flipped classroom techniques in the classroom and have proved that they enhance both the professional motivation of the teacher and the development of students and their academic performance. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 68 researches carried out in recent years show that once children enter school, no other factor is as important for their development as the quality of teachers (bruns and luque, 2014). our main conclusion is that the joint application of these pedagogical techniques facilitates the improvement of the students' relationship with the teacher, who determines the effectiveness of teaching. among all of them, we understand that coaching is a very useful tool for teachers as a constant improvement tool applied both to their professional practice and to improve the relationship with their students and facilitate their teachinglearning process. we also understand that the results and conclusions obtained in this research reinforce the idea that in the education system there must be a revolution both in the approach and in the techniques that are currently applied in the classroom, since they are still very aligned with the traditional teaching methodology. likewise, the continuous learning of the teachers cannot comprise isolated courses without connection among them, but they need to keep coherence and a mission that is to provide the teacher with valid tools for the students of the 21st century. in summary, it is necessary to promote these avant-garde techniques and teachinglearning methodologies aimed at promoting the development of the students to achieve greater responsibility of the pupils, promote the teacher-student relationship, promote teamwork and, promote significant improvements in the results of the learning process and in the development of students as individuals. 5. references albrecht, c.s. (2012). the game of happiness. gamification of positive activity interventions. maastricht university. maastricht, the netherlands. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 69 baker, j. w. (2000). “the classroom flip: using web course management tools to become the guide by the side.” selected papers from the 11th international conference on college teaching and learning. 2000. bandura, a. (1997). self-efficacy: the exercise of control. new york: w.h. freeman. bbva innovation edge (2012). gamificación, el negocio de la diversión, 3, 165. bruns, b. and luque, j.. (2014). profesores excelentes: cómo mejorar el aprendizaje en américa latina y el caribe. washington, dc: world bank. chi, m., glaser, r. and farr, m. (1988). the nature of expertise. ed. l. erlbaum associates. chorney, alan i. (2012). "taking the game out of gamification", dalhousie journal of interdisciplinary management, 8, 1-14. claxton, g. (2008). what's the point of school? rediscovering the heart of education. oxford: oneworld. diaz, j. and troyano, y. (2013). "el pontencial de la gamificación aplicado al ámbito educativo", iii jornadas de innovación docente. innovación educativa: respuesta en tiempos de incertidumbre. university of sevilla. esquivias serrano, m. (2004). "creatividad: definiciones, antecedentes y aportaciones", revista digital universitaria, 1 (5). <http://www.revista.unam.mx/vol.5/num1/art4/art4.htm> gros, b. (2007). videojuegos y aprendizaje. barcelona: graó. hargreaves, d. (2004-2006). personalising learning. pamphlet, series. london: specialist schools trust. harvard business essentials (2005). coaching y mentoring: cómo desarrollar el talento de alto nivel y conseguir mejores resultados. barcelona: editiones deusto. multidisciplinary journal for education, https://doi.org/10.4995/muse.2019.9959 social and technological sciences issn: 2341-2593 gómez-ejerique and lópez-cantos (2019) http://polipapers.upv.es/index.php/muse/ mult. j. edu. soc & tec. sci. vol. 6 nº 1 (2019): 46-70 | 70 kapp, k. (2012). the gamification of learning and instruction: game-based methods and strategies for training and education. san francisco: john wiley & sons. neisser, u. (1976). psicología cognoscitiva. editorial trillas: méxico. padilla, s.; halley, f. and chantler, j.c. (2011). "improving product browsing whilst engaging users", digital engagement, 11, 15-17. pérez rodríguez, p. m. (2004). "revisión de las teorías del aprendizaje más sobresalientes del siglo xx", tiempo de educar, 10 (5), julio-diciembre, 2004, pp. 39-76. prieto gil, a. (2010). «la pirámide del aprendizaje». e-innova bucm. <http://biblioteca.ucm.es/revcul/e-learning-innova/27/art1263.pdf> piaget, j. (1968). genetic epistemology. columbia university press. usa. rogers, c. (1961). el proceso de convertirse en persona. buenos aires: ed. paidós. sams, a., bergmann, j. et al. (2014). flipped learning network (fln). the four pillars of f-l-i-p. <http://flippedlearning.org/wpcontent/uploads/2016/07/flip_handout_fnl_web.pdf> strayer, j. (2014). “how learning in an inverted classroom influences cooperation, innovation and task orientation”, learning environments research 15.2, 171–193. wang, q. (2012). "coaching for learning: exploring coaching psychology in enquirybased learning and development of learning power in secondary education’, procedia social and behavioral sciences, 69, 177–86. <https://doi.org/10.1016/j.sbspro.2012.11.397> wolk, l. (2007). coaching: el arte de soplar brasas. buenos aires: gran aldea editores gae. zichermann, g. and cunningham, c. (2011). gamification by design: implementing game mechanics in web and mobile apps. cambridge, ma: o’reilly media (microsoft word 5_pe\361aloza_vargas) multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 64 http://polipapers.upv.es/index.php/muse/ big-data and the challenges for statistical inference and economics teaching and learning j. l. peñaloza figueroa*, c. vargas perez** * department of statistics and operational research ii av. filipinas s/n, universidad complutense de madrid, 28003 madrid, spain. jluispf@ucm.es; phone: + 34 913946746 ** department of applied economics iv av. universitaria s/n. universidad complutense de madrid, 28040 madrid, spain. cvargas@ucm.es received: 2016-02-03; accepted: 2016-06-26 abstract the increasing automation in data collection, either in structured or unstructured formats, as well as the development of reading, concatenation and comparison algorithms and the growing analytical skills which characterize the era of big data, cannot not only be considered a technological achievement, but an organizational, methodological and analytical challenge for knowledge as well, which is necessary to generate opportunities and added value. in fact, exploiting the potential of big-data includes all fields of community activity; and given its ability to extract behaviour patterns, we are interested in the challenges for the field of teaching and learning, particularly in the field of statistical inference and economic theory. big-data can improve the understanding of concepts, models and techniques used in both statistical inference and economic theory, and it can also generate reliable and robust short and long term predictions. these facts have led to the demand for analytical capabilities, which in turn encourages teachers and students to demand access to massive information produced by individuals, companies and public and private organizations in their transactions and inter relationships. mass data (big data) is changing the way people access, understand and organize knowledge, which in turn is causing a shift in the approach to statistics http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ mailto:cvargas@ucm.es multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 65 http://polipapers.upv.es/index.php/muse/ and economics teaching, considering them as a real way of thinking rather than just operational and technical disciplines. hence, the question is how teachers can use automated collection and analytical skills to their advantage when teaching statistics and economics; and whether it will lead to a change in what is taught and how it is taught. keywords new technologies, paradigm, logical reasoning, instrumental skills, scenarios, interactivity, modelling and simulation 1. introduction in this paper we intend to approach the subject of teaching and learning based on data and put on the table the widening gap between current and traditional ways of producing and processing knowledge (teaching and learning). in this gap it is still true that the challenges related to the area of deductive and inductive logic, systemic thinking, ways of internalization and organization of knowledge, scalability of learning and the diversity of conceptual and analytical skills required have been neglected, but all of them are necessary for the development of new theories and practical applications in the real world. the relevance of the issue lies in the set of challenges related to data identification, processing, analysis and interpretation, especially those cases with no clear structure, pending to be resolved. this challenges call into question the nature of current teaching methods and the design of the contents taught in different subjects in either academic or experimental fields. our modest contribution is drawing attention to the turning point that has led to the emergence of big data in teaching, particularly in the ways to teach conceptual and theoretical content in university classrooms. and also improving the diagnosis of limitations and updating the content of the subjects taught in university. our aim is also to make a record of the fact that teaching immersion in the piles of data and the analytics of big-data will only be possible if the outdated computer infrastructure of http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 66 http://polipapers.upv.es/index.php/muse/ universities, which accentuates the gap and limited progress in data-based teaching, is substantially overcome in a short period. in the second section of this paper we develop the problem statement; then we continue with the analysis of the state of the issue. section 4 is devoted to discussing a new way of content teaching. section 5 outlines the problems and teaching limitations present in big data environment and section 6 introduces the software related to teaching and learning. finally, last section presents our conclusions. 2. problem statement the unprecedented amount of data collected, stored and copied automatically from various sources, in addition to the continuous and rapid growth in volume (40% per year over the decade1), and also in availability and complexity, have prompted a set of challenges related to the identification, processing, analysis and interpretation of these data, especially of those who do not have a clear structure. these challenges call into question the nature of current teaching methods and the design of the various subjects taught in either academic or experimental fields. a modern approach to big data2, in the context of teaching and learning, not only appears as a stimulus for the development of new ideas on how to think and do activities, but also as an improvement in the understanding of how structured and unstructured systems behave in their different (economic, social, technological ...) dimensions which in turn leads to an increased infrastructure and stock of knowledge of the students (peñaloza and vargas, 2006). so, the acquisition of analytical, comprehensive and interpretive capabilities is required, as part of an "updating process based on data and computational developments" in the different teaching and learning dimensions. this happens because the interactions and 1 big data: ¿la ruta o el destino? iefoundation. oracle. 2 fuzzy term that involves not only technological tools but also automated analytical techniques to predict future developments (trends) or identify patterns of behaviour, plus scalable and complex interpretive capabilities, http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 67 http://polipapers.upv.es/index.php/muse/ transactions existing in the piles of data automatically collected in the real world generate enough information about the program content of the subjects taught in the different areas of knowledge. emergent big-data implementation in the teaching and learning field have been designed, almost exclusively, to address issues related to the identification and evaluation of the students` learning difficulties to predict their academic progress, with the illusory idea of generalizing a personalized teaching. this bias in its implementation in addition to a narrow view of big-data as a "set of technologies" have hampered the progress in incorporating the philosophy of big-data on the contents of the various subjects included in the curricula of the different academic levels and areas of knowledge. big data perceived as a "philosophy" goes beyond the technological challenges as it emphasizes factors related to the change in the way decisions are made, the scenarios are configured and analytical and interpretive processes are performed. scalability problems caused by the exponential growth of digitized data have conditioned the definition, goals and research areas in big-data, reducing it to a simple technological tool (3vs or 5vs definition3, or understanding it as the technological factors leadership4). it is also true that there has been a lack of attention to those challenges related to the space of deductive and inductive logic, systemic thinking, ways of internalization and organization of knowledge, learning scalability and the diversity of conceptual and analytical skills required. they are all necessary for the development of new theories and practical implementation in the real world, which includes real-time analysis as one of the factors to consider when teaching subjects content. obviously, all these challenges are part of a process open to scientific knowledge related to accessibility, creation, extraction, organization and interpretation, which is based on the 3 volume, variety and velocity. zikopoulos et al., 2015; olov schelén, ahmed elragal and moutaz haddara, 2015, diebold, 2012; laney, 2001. 4 postnote 468 july 2014 big data. parliamentary office of science & technology. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 68 http://polipapers.upv.es/index.php/muse/ analysis and treatment of the content nested in the piles of data and the logic of the sharing resources process (new forms of communication, variety of formats and structures). this new logic and way of generating scientific knowledge has challenged many of the established standards of scientific knowledge; as in statistical inference, to call into question the "sample-population" relationship and bring to light many of the limitations of statistics for dealing with unstructured data models; or in economic theory by questioning the validity of current economic and financial indicators used for strategic analysis and decision making in scenarios of increasing complexity and uncertainty. these problems and challenges are usually exacerbated by the scalability, complexity and speed of big data (schelen, elragal and haddara, 2015). other dimensions related to the incorporation of big-data on teaching content is the accessibility to non-personal information databases generated not only by the public sector but also by the different (national and international) transactions and other productive and economic activities including research projects funded by public money. greater accessibility and adequate distribution of resources (software, hardware and financing) would not only allow a greater use of data collected by government agencies, foundations, research institutes, business and social associations, plus businesses, etc. (i.e. accessibility policy of the cis centre to their databases surveys –microdata-) but it would also stimulate the full incorporation of big-data in the subject content taught, which would encourage the feedback from a mainly conceptual, analytical and interpretive learning, which in turn would lead to increased background knowledge of students and society in general. 3. analysis of the state of the issue the coexistence of teachers and students with numbers or data and data technologies is a fact that cuts across all areas of knowledge from exact science to social and behavioural http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 69 http://polipapers.upv.es/index.php/muse/ science, automatically creating and generating new data types and new structures of more complex data, which are growing exponentially (anderson, 2009; rubin, 2007). it is a fact that almost every aspect of daily life and of society are transformed into information or represented as numerical data (gould, 2010; muller et al., 2013); for example, the location of a person, the calories you consume or the web sites you visit, the searches you perform on the internet, the chats in which you participate, the e-mails you send, the services offered by a company, the customers of a supermarket, the government spending, etc. careful observation of this type of data and what you can do with them leads to a new way of understanding the data (as resources) and to establish the role they play in the analytical process and in the generation of new knowledge (müller, rosenbach and schulz, 2013). the rediscovery of the role of data substantially changes working environments, the way of thinking and doing calculations and the access to information, things that would simply not be possible to carry out with small amounts of data. today, models or schemes are based on the collection and analysis of large volumes of data and the interpretation of the value of those results. in the environment of big-data, the notion of what could be done with more than a billion individual profiles of citizens of a country, should not only be seen as a monetary value but also as a necessary social value. the generalization of mass data in today's society has not only changed the way to understand and organize knowledge, and made possible the creation of collaboratively knowledge (palomo et al., 2010), but has also significantly changed the context in which the content is taught and the way subjects are taught, that is, leading to data-based teaching and to shackled-to-data technologies. the configuration of the new big-data scenarios not only includes the economic and social dimensions but also the academic dimension, in which the current content teaching methodology (a-conceptual operating procedures, technical methods, and memorization) has no place in education systems based on large volumes of data and data technology that have emerged in the environment of big-data. one of the important issues in these http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 70 http://polipapers.upv.es/index.php/muse/ scenarios is to ask to what extent it is possible to create a new system of teaching and learning based upon the content analysis of large volumes of data collected from external sources using new computer technologies. the big-data system plays a stimulating role by integrating the roles of data consumer and content producer by just pressing a key (see the wiki technology educational projects of the uca5) and by generalizing the automation of data management and handling, including the development of useful operational algorithms for the development of conceptual and theoretical content of the various subjects taught in universities, for example, intelligent transport systems (its) (zhang, j. et. al., 2011 ) or in e-commerce, based on the analysis of large volumes of complex data (tkacz, e. and kapczyn'ski, a., 2009). in fact, we live immersed in the world of mobile phones, laptops, tablets, game consoles, and similar devices. and if you combine that technology with big-data analytics, a series of new possibilities arises, ranging from new ways to learn basic concepts and advanced theories to new ways of reading and calculation, as well as new ways of approaching the computing concepts. so, if it were possible to obtain accurate information of what is in the minds of students, perhaps we could improve academic performance and potentially help transform the classroom going from the 20th century to the 21st century. however, experimental implementation of big-data in the field of teaching and learning have been focused, almost exclusively, to deal just with aspects related to the identification and evaluation of educational problems and learning difficulties of the students to predict its evolution, and focusing on the collection of data from internal sources. this widespread practice is based on the correspondence between teaching methods, activities and /or materials developed by teachers and the preferences in learning styles of students, which essentially means to adjust teaching to the students preferred styles6 5 free software and open knowledge office, universidad de cádiz. http://www.uca.es/es/ 6 these methods include "individual tutoring" , "troubleshooting" , "demonstrations " , "practical exercise" and elaborative processing (ros garcía et al., 2008). http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ http://www.uca.es/es/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 71 http://polipapers.upv.es/index.php/muse/ (hederich m., gravini d. and camargo u., 2011; gravini d., cabrera p., avila m. and vargas g., 2009, garcía ros et al., 2008; bertoglia r., 2005). the pioneering experiments with big data developed in the field of education have focused on basic education, with the idea of connecting decision makers with students in many ways. for example, the cluster of altschool7 schools in san francisco has made experiments with automatic collection and processing of data, internally generated, and with data technologies to help teachers to improve their teaching methods and their students learning styles (bazaco, 2014; cukier8 and mayer-schonberger, 2013; datalogic adc, 2012; orellana lópez y sánchez gómez, 2006; lafuente and olsina, 2001). at the university level, experience with big data continues with this line of implementation and its use in all activities planned and developed. for example, "purdue" indiana university has developed a software product using regression methods to predict the results of their students based on the characteristics of their stocks of knowledge, in order to determine the shortcomings of their students and to keep track of their academic results. similarly, east carolina university, nc, university of louisville, ky, university of south alabama, al, and university of south florida, also experimented with an interactive and educational online system called "mymathlab9", designed by pearson-education to meet the needs, of teachers and students, of exercises and formal tests of knowledge and operational skills. this practice has led to neglect the impact of big data not only on how to present content but also on the actual content taught in different subjects in university curricula. the collection and analysis of vast amounts of data to capture the intrinsic value included requires the development of new technological and technical tools. the various data technological tools (selection, capture, analysis and visualization of big data), developed by computing specialists, go through the heart of a number of disciplines of knowledge such as economics, computer science, statistics, semantic language and market research, 7 a conglomeration of 4 schools located around san francisco , usa, that are dedicated to innovation. 8 kenneth cukier is the editor of “the economist” journal. http://www.economist.com/sections/science-technology. 9 these are tests which have an unlimited number of attempts in all tasks and available learning resources. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ http://www.economist.com/sections/science-technology multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 72 http://polipapers.upv.es/index.php/muse/ among others (mujeeb and naidu, 2015; christmann, badgett and licking, 1997) as well as transversely through a series of operational and analytical skills. this understanding of the role of data requires methods supplied by multiple disciplines aimed at discovering useful information in large semi-structured or unstructured databases (see table 1). table 1: data types in the big-data paradigm structured data semi-structured data unstructured data customer file emails person to person birthdate structured part: communications in social networks name addressee person to machine address receivers medical devices transactions in the month topic e-commerce point of purchase unstructured part: computers / mobile phones expenditure in a month message body machine to machine atttach sensors / security cameras gps devices source: maté jiménez, 2014 in these environments, based on data and data technologies (akerkar, 2014), in which a wide variety of methodologies and techniques provided by different (multidisciplinary) areas of knowledge are required, the areas of statistics and economics play an important role as they mix and/or merge directly or indirectly with almost all other areas of knowledge. for example, the development of models and statistical techniques has been stimulated mainly by demand from other scientific disciplines (peña, 1995 and 2014) and that is the origin of our interest to emphasize the impact of big data on these two areas of knowledge. 3.1 knowledge of inferential statistics and big data recall that our performance in the context of big-data occurs at the intersection between the present and the future, giving rise to a new paradigm in data analysis consisting of the http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 73 http://polipapers.upv.es/index.php/muse/ implementation of intelligence in the analysis, on structured, semi-structured and unstructured data, most often in real time, to extract value from them. the effects of massive amounts of data on real-world analysis, which correspond to the concept of big-data, puts into question the validity of analyses based on small samples and should be considered not as a mere trend but as a turning point in the way to build knowledge. in fact, it is causing profound changes not only in statistical inference teaching or general statistics but also in the field of quantitative and qualitative research (peña, 2014). the need for methods to understand the information contained in existing data and to extract value from it, has stimulated the development of statistics as a science that analyses the data and builds statistical models in order to explain reality. although current data used to be a small, homogenous and structured sample of the phenomenon under study (biometric, economic, social or experimental data), its updating in real time is actually non existent. however, the complexity of new databanks (heterogeneity of sources, measurements, formats, etc.) which are a mix of different populations and formats including images, graphics and/or text, highlights the limitations of current inferential statistics, which was created to deal with samples and specifically with small samples and not with a "sample" close to population size. this situation demands innovative and efficient processing solutions for improving knowledge and decision making (peña, 2014; zicari, 2012; mitchell et al., 2012.). the key points that have an impact on the conceptual and theoretical content and the way to teach them are paradigms of big-data, regarding scalability, speed and variety of data, which appear as a result of the proliferation of data technologies and automation (collection and analysis), since they cannot be studied using traditional statistical techniques invented to operate on structured, homogeneous and small-scale data (samples) (duboc, rosenblum and wicks, 2006; bondi, 2000; hill, 1990). http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 74 http://polipapers.upv.es/index.php/muse/ scalability is a difficult issue to incorporate in the field of statistics especially because of the frequent mistake of thinking that the management and troubleshooting at a small scale is enough to extrapolate traditional methods when the size of the problem increases, just by adjusting the same techniques to a larger problem, which leads to a dilemma between working with a sample or the entire population (or its more realistic approach: big-data). the speed and real-time updating of databases are factors that have a strong impact on the nature of the conceptual and theoretical content, since many of the tools and statistical models lose their effectiveness and generate biases and much noise, due to the existence of real time solutions. finally, the variety of sources and formats also affects the handling and the use of statistical tools, as we are dealing with unstructured or semi-structured databases with different types of formats (numbers, texts, graphics, videos, comments, etc.), so that statistics teaching should go beyond numbers and mechanical reproduction of probabilities, by mapping or following the traces left by the data, taking into account the conditions under which each observation arises (gould, 2010; deming, 1940). in general, modern data analysis is based on the belief that it is enough to gather large volumes of data of a phenomenon to answer any question about the phenomenon under study, provided that appropriate methods are applied and that we have a powerful operational and analytical computational algorithm. for example, multi-scale methods used in image processing; or the analysis of functional data searching statistical regularities; or the application of clustering techniques by similarities or metric measurements, etc. 3.2. economic knowledge and big data the challenges big-data proposes in the teaching and learning of economic theory are based on the use of large volumes of economic, financial, business, public management, government and leisure, as well as social data, among many others. the exploitation of this http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 75 http://polipapers.upv.es/index.php/muse/ large collection of data requires academics (professors and researchers) greater involvement in processing and analysis, including the interpretation of results. the forward leap to make big-data immersion in economics teaching come true requires a mentality shift and also experimental logic by teachers to tell the conceptual, theoretical and practical content of economics and business, based on data, in addition to the support of computers, automated calculation systems and computer applications. interest in massive data in the area of economics, particularly in areas such as government and industry, has been increasing in the last five years, even surpassing known issues such as data mining and structured data analysis. the rise of big data is causing a turning point both at the computing and statistical level, but above all, on making markets work more effectively and efficiently, leading the global economy to a substantial change in management models and the way business is done, as well as the ways to approach clients and customers (che, safran and peng, 2013). in many branches of economic activity, economic agents target these challenges and make use of big-data towards organizing and managing their own actions, tasks and developments. for example, governments use big data to map the effects of the implementation of its pro-cyclical or counter-cyclical policies or to monitor an economic shock (i.e. financial, oil or housing bubble crisis) and their effects on disposable income and/or consumption expenditure of individuals, families or companies. since economic processes are the direct and indirect result of the economic agents behaviour in the different dimensions of society (motives, attitudes and expectations versus spending, saving and investment), mapping techniques and automatic tracking of big-data system helps to identify the nature of economic crisis, like the present one, and its concrete manifestations in the real world, such as the failure of current government and business management models, the lack of ability to manage the expectations of investors and consumers and the absence of strategic management by businessmen and consumers (economic myopia) (warren, 2008). http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 76 http://polipapers.upv.es/index.php/muse/ big-data changes the perception of uncertainty –the core of content teaching in economic sciencein the sense that we have greater information and knowledge and better analytical tools, which are used to narrow uncertainty and transform it in a risk indicator. hence, the theoretical and empirical analysis of the attitudes and expectations of economic and social agents requires new (other than current) economic variables and new analytical data tools combined with psychological variables in the context of probabilistic analysis. the scenarios set by big-data in the predictive dimension are one of the strengths of economics teaching and learning; however there are certain difficulties for the incorporation of the the results of the analysis of large masses of data, collected in real time, into the programs of the subjects, as a way of approaching prediction of the future. for example, a predictive big-data analytics which gets, in a digital way, all the information in the market would have reduced the size of the property bubble and minimized its consequences. digitalization and automation of millions of jobs emerge not only as an inevitable trend, but as a deep change in labour relations, the way people work and the type of work they have. these processes affect on the one hand, the configuration and dynamics of the labour market and raises questions like: will there be enough jobs in the future to relocate and retrain people displaced by automation? or would it be possible to establish a wealth distribution system to regulate the labour market? and, on the other, they affect the management and functioning of those organizations or systems which were not able to change and align with the guidelines of the new big-data scenarios (gould, 2013 and cobb, 2007). in the business dimension, companies use big data to develop specific implementation in general work areas. for example, credit card companies (visa, mastercard, …) are interested in the rapid or real-time recognition of unusual patterns of behaviour in the use of credit cards (sudden charges of large amounts of money on a credit card at an unusual place for the customer) in order to warn the cardholder about that anomaly (müller, et. al, 2013). http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 77 http://polipapers.upv.es/index.php/muse/ overall, big-data can be applied to economic policy management, to microeconomic and macroeconomic theory, to companies like google, facebook, coca-cola, nestle, etc., and/or to a small business which is dynamic and flexible not only in its operations but also in its decision-making. a clear example is found in the statement by fabinger et al. (2015) who tell us that the use of large databases allows us to a better approach to the correct form of the demand curves, while acknowledging that the larger is the data set used, the more complex is the construction of the curves of empirical demand and the calculations involved. in other words, data is becoming the new currency on which both short and long term business and consumer decisions are based, and it is a clear improvement in overall productivity. 4. options for a new way of content teaching our basic proposal is founded on the idea that conceptualization and theorizing is the best way to provide the necessary basis and language, to develop analytical and interpretive methods, based on large bases of semi-structured and unstructured data, to solve problems and challenges of the real world. the collecting and analytical potential of big-data will be useful if it is able to stimulate the shift of focus from operational activity-based teaching to one based on data and data technologies, so that it is possible to make a forward leap from samplebased analytics to multiscale analytics and parallel effectiveness, which makes possible the immersion of theoretical concepts and models in big-data environments. the figures from gartner research (2015) support these ideas. in 2015, 4.4 million technology-related jobs will be created worldwide to support the management of large volumes of data (big data). however, limited access to large volumes of data by universities currently contributes to the generation of a skills gap in the workforce of countries or among a group of countries. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 78 http://polipapers.upv.es/index.php/muse/ from the statistics point of view the potential of big-data manifests itself as a system that uses information technology to collect and organize data and (statistical) analytical methods to study and interpret them. the aim is to generate numerical descriptions of relationships between data, automatically and in real time. exploiting the "causality" and correlation relationships among different objects usually involves the use of efficient approximation algorithms such as monotonic multivariate regression on a large scale10. nevertheless, its implementation has certain limitations related to the dynamics of updating the database in real time. new techniques to exploit the potential of big data, in structured or semi-structured architectures, are characterized by the study of the problems related to the presence of outliers and the configuration of clusters (cluster analysis, survival analysis, statistical learning, data mining, meta-analysis, and kkd and latent analysis, among others). in addition new multivariate time series methods are used to analyse dynamic data and independent principal components to analyse structured functional data. these challenges require a comprehensive review of the existing data management systems covering the scalability metrics as partitions, replicas, concurrency control and consistency of large semi-structured and unstructured databases. this happens due to the physical principle that stablishes that the exponential increase in the speed of data collection and information extraction is closer to real time, making the traditional principles and methods obsolete and ineffective for analysis and verification. from the point of view of economics, due to the ubiquity of data it is possible that, as part of an international trade course, we could automatically examine and map the effects of imperfect competition on trade and terms of exchange among countries, or similarly, we could track the impact and spread of economic shocks in the economy or group of economies, as part of the content of a macroeconomics course. 10 econometric models work by statistical causality relations, while the big-data system works by statistical correlations and associations, and not by causalities, since information is instantaneous. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 79 http://polipapers.upv.es/index.php/muse/ for example, a macroeconomic problem of great interest is measuring inflation. big-data could provide a new way to gauge the growth of prices. inflation is a simple concept, but continuous price increases are surprisingly difficult to measure. big data could easily carry out the process of collecting and monitoring products prices and consumption baskets over time. in theory, online prices and baskets could be tracked digitally. adobe, a data technology company, is trying to calculate the us cpi figure. the company collects sales data with an anonymous design of the websites that use their software. the amount of data available is huge: according to the company, it includes three-quarters of online spending made in the top 500 retail businesses in the united states. the company is using this ocean of information to compile or create a "digital price index" (dpi) to compete with the official measures of inflation. in the business dimension the potential of big data requires a new team of analysts who can bridge the gap between information technology and businesses, making use of the conceptual evolution and the real time decision-making approach. this requires a different use of resources configuration, based on data and software. among the limitations of the immersion of the teaching of economics in the piles of data are, in the first place, the speed increase which modifies the economic relations that occur among economic agents when moving from one level to other (macro -7 micro), due to the new forces acting at that level. in the second place, the semi-structured and unstructured nature of the data collected and stored which make many of the analytical tools of economic theory obsolete, both at the micro level and the macro level. in summary, the challenges in teaching and learning statistics inference and economic theory, in the process of immersion in the use of large volumes of data of any kind, require the full exploitation of the data. for this purpose, special skills are needed to lead the challenges involved, which can be exacerbated by the scalability, complexity or speed of big-data. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 80 http://polipapers.upv.es/index.php/muse/ 5. problems and limitations so far teachers and scientists have developed various technologies and algorithms to collect, capture, analyse and visualize data from large databases. these technologies and algorithms cross a number of disciplines such as economics, computer science, and statistics, and they also require certain analytical and operational skills. this means that multidisciplinary approaches are necessary to discover useful information from large volumes of data. generally, categorization of big-data based on data heterogeneity, results in certain semi structured and unstructured architectures. management and analysis of these architectures requires a dynamic and flexible computing infrastructure that universities do not currently own (most of them have an outdated computer infrastructure). storing big-data is a critical task that requires advanced technology such as hadoop and hdfs from apache project, which are beyond the power of universities and even worse for teachers, to implement the new way of conceptual and theoretical content teaching based in large volumes of data. 6. software for teaching and learning in recent years, there have been several solutions using advanced analytics in the software market for higher education, such as: big data for management companies, big-data for business and trade and big-data for teaching and scientific research, among others, all of them focused on automatic and analytical learning. this is the case of mapreduce which is an efficient analytical tool of large volumes of data consisting of a visual and graphical search of behaviour patterns using clustering and classification techniques, combined with some other conventional techniques such as dbms and indexing methods. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 81 http://polipapers.upv.es/index.php/muse/ the wide variety of applications, with potential uses in conceptual and theoretical content teaching and learning, can be classified as follows: big data tools for batch processing: karmasphere studio and analyst tableau jasper soft bi talend open studio sky tree server apache hadoop pentaho business analytics mapreduce: aide y a2de apache mahout dryad big data tools for stream processing: storm splunk s4 apache kafka sqlstream s-server sap hana big data techniques such as: statistical techniques machine learning techniques optimization methods classification and clustering techniques data mining techniques regression analysis techniques http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 82 http://polipapers.upv.es/index.php/muse/ 7. conclusions activities related to data collection and data analysis are not new as they have been done for thousands of years. what is new is the great number of advances in data technologies during the last twenty years, combined with the convergence of phenomena such as the internet, smartphones, tablets, mobiles, wi-fi, apps, social networks, the cloud, automatic learning, cognitive computing, etc., so we can assume that data technology will continue improving, resulting in a digital gap among individuals and among organizations in the framework of a competition based on data analytics. the automatic interpretation of the results obtained from big-data and the specification of their semantics and the preservation of the context in which they occurred, are some of the challenges to be faced today and in the near future, when we try to immerse teaching in the ocean of data and results (big-data). in this scenario various concepts of data classification and grouping arise, including both the context and the standardization of formats (warehouse) to carry out the extraction of relevant information and its transformation into knowledge in the big data environment. big data environment is an opportunity to reposition the teaching of statistics in the centre of the acquisition of knowledge, integrating concepts and creating new methodologies in line with the advances in data technologies. for that purpose professors and researchers need a mental shift and start working with massive data, doing big data-based studies in their areas of research. within the framework of economics teaching, big data is the new frontier for innovation, competition and productivity which cuts across all industries, and all production and public activities. by leveraging the large volumes of data it is possible to transform the knowledge we have about economic policy, the corporate image of a company, a business partner or the requirement of making intelligent decisions by public and private operators. with big-data we run the risk of carrying out data-based problem analytics without a prior assessment of the need to adjust the data tools applied to the phenomenon under study, as a result of the "conviction" of computer gurus (microsoft, java, paython, etc.) that direct http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 83 http://polipapers.upv.es/index.php/muse/ implementation of these analytical techniques to data, whether economic or not, is a sufficient condition to turn data into knowledge. universities are organizations where it is feasible to apply big data and data technologies to increase efficiency and quality of training processes. in essence, it is necessary to opt for a conceptual and interpretive teaching, which does not imply a detailed description of the facts or phenomena but an analysis in terms of cause and origin of the facts or phenomena of interest and its subsequent explanation. therefore, regardless of the complexity of the problem or phenomenon it is possible to find a fundamental explanation so that the underlying theory or models can be explained, investigated or replicated. the large number of software solutions around apache hadoop project is a sign of progress in resolving the various challenges that arise, as immersion generalizes and deepens in teaching and knowledge development. however, democratizing access to large databases remains a pending task in universities, without which it is not possible to meet the large demand for skilled analysts in the various areas of knowledge. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 84 http://polipapers.upv.es/index.php/muse/ references akerkar r. (ed.). (2014). big data computing. crc press. anderson, c. (2009). “living by numbers”. wired magazine. july 2009.new york: conde nast publications. cobb, george, w. (2007). the introductory statistics course: a ptolemaic curriculum. journal of technology innovation in statistics education vol. 1 no. 1. ucla. (available at: http://escholarship.org/uc/item/6hb3k0nz). cukier, kenneth and mayer-schönberger, viktor (2013). big data: a revolution that will transform how we live, work and think. john murray publishers. london, uk. dean, jefry and ghemwat, sanjay (2010). map reduce: a flexible data processing tool. communications of the acm, volume 53, issuse.1, january 2010, pp 72-77. dean j, ghemawat, s. (2008). mapreduce: simplified data processing on large clusters. communications of the acm, 51(1), 107-113. diebold, f. x. (2012). a personal perspective on the origin(s) and development of “big data: the phenomenon, the term, and the discipline”. draft paper dated november 26, 2012. duboc, leticia; rosenblum, david s.; wicks, tony (2006). a framework for modelling and analysis of software systems scalability. proceeding of the 28th international conference on software engineering icse '06. p. 949. isbn 1595933751. fabinger, michal and e. glen weyl, e. glen (2015). a tractable approach to pass-through patterns. link: file:///c:/users/jl/downloads/ssrn-id2575471.pdf (access 07/04/2016). garcía ros, r., pérez gonzález, f. & talaya gonzález, i. (2008). preferencias respecto a métodos instruccionales de los estudiantes universitarios de nuevo acceso y su relación con estilos de aprendizaje y estrategias motivacionales. electronic journal of research in educational psychology, 6(16), 547-570. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ http://escholarship.org/uc/item/6hb3k0nz) multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 85 http://polipapers.upv.es/index.php/muse/ gartner, eric savitz (2012). “10 critical tech trends for the next five years”. http://www.forbes.com/sites/ericsavitz/2012/10/22/gartner-10-critical-tech-trends-for-the next-five years/. gould, robert (2010). statistics and the modern student. international statistical review, 78, 2, pp. 297-315. uk. hederich martínez, c., gravini donado, m. & camargo uribe, a. (2011). el estilo y la enseñanza: un debate sobre cómo enfrentar las diferencias individuales en el aula de clase. in r. roig vila y c. laneve, c. (eds.), la pratica educativa nella società dell’informazione. l’innovazione attraverso la ricerca (pp. 213-222). alcoy-brescia, italia: marfil y la scuola editrice. kambatla k, kollias g, kumar v, grama a. (2014). trends in big data analytics. journal of parallel distributed computation, 74, pp. 2561-2573. laney, d. (2001). 3d data management: controlling data volume, velocity, and variety. stamford, ct: meta group research note, february 6, 2001. leedy, p. & ormrod, j. (2001). practical research: planning and design. 7th editon. upper saddle river, nj: merrill prentice hall. thousand oaks: sage publications. maté jiménez, carlos (2014). big data: un nuevo paradigma de análisis de datos. revista anales de mecánica y electricidad. nov-dec. http://www.revista anales.es/web/n_29/pdf/10-16.pdf. meyer-schonberger, viktor and cukier, kenneth (2013). big data: a revolution that will transform how we live, work and think. john murray publishers. london. uk company. mitchell, ian and wilson, mark (2012). linked data. connecting and exploiting big-data. white-paper, march. fujitsu uk. link: https://www.fujitsu.com/uk/ images/linked-data connecting-and-exploiting-big-data-(v1.0).pdf. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ http://www.forbes.com/sites/ericsavitz/2012/10/22/gartner-10-critical-tech-trends-for-thehttp://www.forbes.com/sites/ericsavitz/2012/10/22/gartner-10-critical-tech-trends-for-thehttp://www.fujitsu.com/uk/ multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 86 http://polipapers.upv.es/index.php/muse/ mujeeb, s. md. and naidu, l. kasi (2015). a relative study on big data applications and techniques. international journal of engineering and innovative technology (ijeit).volume 4, issue 10, april. issn: 2277-3754. müller, martin u., rosenbach, marcel and schulz, thomas (2013). living by numbers: big-data knows what your future holds. der spiegel no. 20. germany (translated from german by christopher sultan). peña, d., prieto, j. and viladomat, j. (2010) “eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure”, journal of multivariate analysis 9, 1995 -2007, 2010. peñaloza f., juan luis y vargas p., carmen g. (2015). “construction and evaluation of scenarios as a learning strategy through modelling-simulation”. in: multidisciplinary journal for education, social, and technological science. vol. 14, num. 1. pp. 40 – 62. doi: http://dx.doi.org/10.4995/muse.2015.2245. peñaloza f., juan luis y vargas p., carmen g. (2006). ¿qué debe cambiar en el aprendizaje de la estadística en las ciencias del comportamiento?. xiv jornadas de asepuma y ii encuentro internacional, 21 y 22 de septiembre de 2006. facultad de ciencias económicas y empresariales, badajoz, españa. schelén, olov, elragal, ahmed and haddara, moutaz (2015). a roadmap for big-data research and education. technical report. luleá university of technology. isbn 978-91 7583-275-3. tkacz, ewaryst and kapczyn´ski, adrían (2009). internet: technical development and applications, springer, 2009. warren, kim (2008). strategic management dynamics. john wiley & sons, ltd. england. williams, carrie (2007). research methods. journal of business & economic research. volume 5, number 3 – march. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/ http://dx.doi.org/10.4995/muse.2015.2245 multidisciplinary journal for education, social and technological sciences http://dx.doi.org/10.4995/muse.2017.6350 eissn: 2341-2593 peñaloza and vargas (2017) mult. j. edu. soc & tec. sci. vol. 4 nº 1 (2017): 64-87 | 87 http://polipapers.upv.es/index.php/muse/ zhang, junping, wang, fei-yue, wang, kunfeng, lin, wei-hua, xu, xin, chen, cheng (2011). data-driven intelligent transportation systems: a survey. ieee trans. intell. trans. syst. 12 (4), pp. 1624–1639. zicari, r. v. (2014). big data: challenges and opportunities. in akerkar r. (ed). big data computing. crc press. zikopoulos, p., deroos, d., bienko, c., buglio, r., & andrews, m. (2015). big data beyond the hype: a guide to conversations for today’s data center. new york, ny: mcgraw-hill. education. http://dx.doi.org/10.4995/muse.2017.6350 http://polipapers.upv.es/index.php/muse/