Microsoft Word - 2_bares_et_al Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 16 Using Interactive Response Systems in Economics: utility and factors influencing students’ attitudes Bares López, Lydia1*, Fernández Pérez, Ana Mª1, Ferrándiz León, Esther1, Flores Varo, Mª Esther1, León Rodríguez, Mª Dolores1 Department of Economics, University of Cadiz, Avda. Enrique Villegas Velez 2, 11002 Cadiz, Spain. * Corresponding author: Email: lydia.bares@uca.es; Phone: + 34 956015472 Received: 2016-03-09; Accepted: 2016-11-12 Abstract The European Higher Education Area (EHEA) involves changing traditional methods to promote innovative teaching experiences. This paper has two main aims: a) to show evidence of the use of Interactive Response Systems (IRS) to identify gaps in the understanding of the course contents and b) to investigate factors influencing students’ attitudes towards the use of IRS. The experience was developed through a collective tutoring session in the subject of Economics using IRS. Economics is a first-year subject in the Degree of Business Administration and Management offered by the University of Cadiz, which includes contents of Microeconomics and Macroeconomics and uses economic models to explain the function of the economy and the behaviour of economic agents. Results show that IRS technique allows detecting gaps in learning and comprehension. From our econometric estimations, we also identify two strongly significant variables affecting students’ attitudes towards IRS: gender and received explanations regarding the use of IRS. Variables such as first enrolment in the subject and the number of hours devoted to studying have a positive and significant effect on the attitude to IRS, but at a lower level of significance (from 5% to 10%). Keywords Teaching innovation, new technologies, Interactive Response Systems (IRS) http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 17 1. Introduction The transformations within current society require the training of professionals who can adapt to changes arising in a constantly evolving world. Leadership and communication, problem solving, team working, negotiation skills are needed to become a part of teamwork. Academia must accept this challenge and include the possibility in the teaching- learning process that students learn this set of transversal skills in line with the needs of society. The development of the European Higher Education Area (EHEA) and the implementation of The European Credit Transfer System (ECTS) involve a gradual change of the traditional teaching approach to promote innovative experiences in the learning-teaching process. New teaching methodologies involve a different perspective on learning, in which the merely expository teaching class must be changed towards new approaches, focusing on the interaction between lecturers and students with active participation on both sides. New technologies are called to play a role in this process, since they provide a wide range of possibilities and opportunities to improve teaching. Interactive response systems (IRS) are a prominent example of this. The IRS is a variant of a traditional technique that can be performed by the well-known method of cards, where teachers raise a number of questions or items to students, who previously have been given a set of cards on hand to answer with different colours or distinctions (a, b, c, or another set of options). An IRS is an electronic device connected to software that processes the student’s answer to a multiple-choice questionnaire immediately, graphically through diagrams. The responses are recorded individually and anonymously, allowing a feedback process for decision-making in the classroom. In this paper we present a case study based on the use of an IRS to solve doubts and detect lack of understanding on the subject of Economics in the Bachelor´s Degree in Business Administration and Management at the University of Cadiz. The aim of this paper is twofold: a) to show evidence about the usefulness of IRS to detect doubts and lack of http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 18 understanding of the course contents and b) to provide evidence on the factors affecting a favourable attitude of the students towards the use of an IRS for the detection of doubts and gaps. The paper is organised as follows. In Section 2, we present the literature review. Section 3 describes the methodology. Section 4 shows an econometric model to identify the factors that explain a positive attitude of students towards the use of the IRS to detect doubts and lacks in the learning of the subject. Section 5 presents the conclusion and discussion of the paper. 2. Literature review The implementation of new teaching methods that enhance students’ performance has been a major concern for academics in the last decades (Butler 1992), as reflected in the use of new technologies for teaching (Gül et al. 2010). Although it has been applied to different academic disciplines such as psychology (Morling et al. 2008; Dallaire 2011; Ludvigsen et al. 2015), health (Schackow et al. 2004; Hughes et al. 2011), sciences (Kay and Knaack 2009; Moss and Crowley, 2011) or accounting (Carnaghan and Webb 2007; Camacho-Miñano and Del Campo 2014), it cannot yet be considered as widely used. In this sense, Emenike and Holme (2012) argue, using a methodological analysis of binary data between users and non-users of the system for a sample carried out at faculties of chemistry, that this technology is still at an early user stage, in transition towards a more widespread use status taking the life cycle of technology adoption (technology adoption life cycle - TALC). In their work, Fallon and Forrest (2011) compare alternately the traditional use of cards and an electronic IRS in a sample of 70 students in prior review sessions to assessment tests of the subject. In their research they obtained evidence of the degree of preference of students for the IRS, considering the main advantage to be the possibility of preserving http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 19 anonymity of the responses. However, in line with other research, they did not find clear evidence to show significant advantage over results in the use of an IRS (Stowell and Nelson 2007; Lasry 2008). There are a broad number of contributions about the use of the IRS in teaching. King and Robinson (2009) review previous studies citing, among other, research focused on the use of the system and the evaluation of students’ perception about these kinds of experiences (Van Dijk et al. 2001; Fies and Marshall 2006; Caldwell 2007; MacGeorge et al. 2008). In their paper, King and Robinson (2009) provide the results of an experiment carried out on a sample of 145 students of the second course, which show that the majority of students appreciate the advantages and usefulness of using an IRS in the classroom. Their results, in line with other studies (Liu et al. 2003; Siau et al. 2006), suggest the effectiveness of the system in terms of its ability to promote and encourage student participation in the classroom, showing a rate of 84.61% participation by an IRS compared to 28.67% when the student must respond with an outstretched hand and 12.59% when a verbal response is requested on a voluntary basis. Even students who did not regard the IRS as particularly useful, reported greater participation when using it. However, in line with previous literature King and Robinson (2009) found no correlation between the use of the system and the improvement of the results of students. Additionally, Anthis (2011) considers the need of separating the effects of using interactive systems itself versus the possibility of contemplating the same type of questions to the students, answered by an outstretched hand. In this sense, the system's effectiveness would show better academic results in the students who used the IRS against those who participated in a traditional response system in the classroom. Their results have shown no evidence to confirm this approach. Meanwhile Palmer et al. (2005) and Gebru et al. (2012) confirm that the use of systems that allow feedback on the teaching-learning process (learning-teaching feedback) can increase student knowledge retention during the course, reducing their loss learning over time. According to the study of Han and Finkelstein (2013) students consider IRS as a http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 20 useful tool for their engagement and learning. Fallon and Forrest (2011) discuss the advantages of the IRS applied to review sessions prior to assessment tests of students. Dallaire (2011), in line with previous studies that have investigated the direct results of the system of learning (Kennedy and Cutts 2005; Stowell and Nelson 2007; Morling et al. 2008, Poirier and Feldman 2007), research on the effectiveness of the IRS in a sample of 151 students. Although the goal of their work is not to examine the improvement achieved by this system of learning, the results suggest that the degree of effectiveness is conditioned by the mode of use and the characteristics and student final marks. With respect to the drawbacks of implementing the IRS in the teaching methodology, the main disadvantages are the training needed for setting up the software and for its use by teachers; the time spent on the acquisition of these skills, and the acquisition cost of the equipment. Furthermore, it is a time-consuming activity because instructors should examine if this kind of experiences are properly awarded to students (White et al. 2011). In spite of the disadvantages, the results obtained about previous experiences in the literature and its widespread use, leads us to value their potential advantages over their disadvantages. We use the IRS as a complementary teaching tool in the classroom during a review session before the exam. 3. Methodology The implementation of an IRS in teaching has been a commitment of the Faculty of Economics and Business at the University of Cadiz to the introduction of new communication technologies in the teaching-learning process. This tool has been purchased for use in the subjects taught in the Faculty and allows students to use the personal response devices without having to pay any fee for their use. The next paragraphs show the methodology followed to carry out this experiment. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 21 3.1 Target students This innovative teaching experience is applied to Economics. Economics is a first-year subject in the Bachelor’s Degree in Business Administration and Management offered by the University of Cadiz (Spain). Economics is taught during the first semester within the framework of Economic Theory in the basic training module. The aim of the subject is to provide a basic knowledge to enable students to understand Microeconomic and Macroeconomic issues from both a theoretical and practical point of view. In general, economic models are related to graphical and mathematical analysis which students often find hard to understand. In this sense, students' academic results are not satisfactory in terms of marks obtained in the subject, so the lecturers consider the use of alternative teaching methods over traditional attendance and assessment that assist improvement in student performance. 3.2 Development of the experience The implementation of this innovative teaching experience followed four steps: 1. Technical procedure development. The system needs to install the software able to recognize the signal from the IRS in the control unit. 2. To design the items or questions for each topic in the course. These questions allow teachers to evaluate the level of academic achievement and learning by students. At the same time, these questions enable teachers to identify understanding gaps in the contents of the subject. To display the selected items we use the corresponding software presentations (the technology incorporated in the system supports PowerPoint or similar). 3. Review session. In this session, teachers pose a set of questions and students respond to them with their IRS. Students must select the correct answer by pressing an http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 22 alphanumeric code in their device. Previously, students must have been identified in the system (newer versions of the system allow to use devices like PDAs or mobile phones instead of traditional interactive remote control). For this, each remote control is given to a student, assigned a corresponding identifier. At the beginning of each item the teacher gives the students time to think. After that, the system is activated to capture the responses from each user. 4. The answers are instantaneously analysed by the system. The results are shown together for the whole group to preserve student anonymity. For each question the results are graphically available as a bar diagram, histograms, etc. Teachers have the option of knowing the identity of each user, if they want it for the evaluation of the experience for the possible marks of the results although this is not part of the objective of this experience. By taking into account the results obtained by students, teachers can improve the teaching- learning process, providing additional explanations to the issues raised to clarify concepts, and give appropriate explanations to resolve detected gaps in understanding for the achievement of the proposed objective of this experience. Following the division of the matter for the subject of Economics the set of items is structured into two thematic sections corresponding to Microeconomics and Macroeconomics, of ten questions each, reflecting faithfully the structure of the teaching program of the course. This experience is an experimental activity that has been developed in four groups of students. The number of students who were voluntarily involved in this activity was 84. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 23 4. Results Table 1 shows the results of the test using the IRS. The test consisted of asking students a set of 20 questions related to theory of Macroeconomics and Microeconomics, which is taught in the subject “Economics”. The total number of items have been classified into two categories: Type A) “STUDY” (12 items) and Type B) “COMPREHENSION” (8 items). The data in this Table shows the percentage of right answers in each group of classes for each item. Additionally, this table shows the average number of right answers in all items and its standard deviation as indicator of the degree of spread of the results. Table 1. Results of the experience by group of class. TYPE OF QUESTION ITEM A B C D AVERAGE (*) STANDARD DEVIATION STUDY (TYPE A) 1 97% 25% 82% 79% 71% 0.2277 2 40% 63% 91% 43% 59% 0.1763 3 90% 81% 82% 57% 78% 0.1020 4 17% 19% 36% 43% 29% 0.1095 5 43% 56% 27% 50% 44% 0.0891 6 63% 13% 64% 0% 35% 0.2861 7 45% 50% 73% 21% 47% 0.1396 8 73% 31% 91% 14% 52% 0.2952 9 100% 94% 100% 93% 97% 0.0334 10 100% 88% 100% 86% 93% 0.0669 11 100% 44% 55% 79% 69% 0.2006 12 18% 63% 0% 29% 27% 0.1822 COMPREHENSION (TYPE B) 13 81% 75% 91% 100% 87% 0.0881 14 68% 25% 45% 79% 54% 0.1896 15 7% 0% 9% 7% 6% 0.0286 16 70% 69% 36% 79% 63% 0.1352 17 82% 25% 91% 43% 60% 0.2621 18 91% 13% 64% 7% 44% 0.3372 19 55% 56% 36% 14% 40% 0.1503 20 45% 13% 27% 50% 34% 0.1392 Average 64% 45% 60% 49% 54% 0.0766 (*) Percentage of right answers to each question As shown in Table 1, from the twelve Type A items related to the study, only questions 3, 9 and 10 have shown a high percentage of right answers (above 75% of total answers). The http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 24 share of correct answers for the rest of Type A items is relatively low, particularly in items 4 and 12, which suggests a lack of dedication of students to study the subject. The results for Type B items (comprehension) confirm low rate of success (below 75%). Especially remarkable is the low percentage of right answers to item 15 (6%). The only exception is item 13, in which the rate of right answers is 87%. It is worth mentioning that this item refers to an issue taught in the first lesson of the subject. Therefore, this result may suggest that students have just started to study the subject. This is confirmed by the overall average of right answers, that is 54%. Our results show relatively small dispersion among groups. From the results, we conclude that the use of IRS is a useful way to detect students’ lack of study and comprehension and other related issues. First, it has enabled teachers to detect that students have not prepared the subject sufficiently to pass the subject. This enables teachers to inform students of the need to increase their study of the subject. Second, the results obtained imply the need to repeat this experience after students have devoted enough time to study, so they can detect genuine lack of comprehension by students. Third, assuming reasonable results for Type A items, the experience shows that using interactive response system enables the detection of lack of comprehension of the contents. Fourth, the detailed analysis of the results obtained by group of class, so teachers can focus in the solving of these concerns. Also, it has served to facilitate the participation of students in the review class and obtained immediate feedback. This is an important issue, because traditionally there has been extremely low participation of students in these sessions. 4.1 Effort perception and usefulness of the experience according to students We surveyed students attending the collective session about two main issues: 1) the effort they had made to study the subject; and 2) the usefulness of the experience. For that purpose, students were asked to fill in a questionnaire with a set of questions in the Likert http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 25 scale (ranging from 1 to 5 points, being 5 “a lot/very useful/strongly agree”). Additionally, this scale was transformed into a binary variable that equals 1 when the answer is 4 or 5; and 0 in other case. The questionnaire was administered in two rounds: before and after the final exam. This enabled us to detect whether the results obtained in the exam shape students' perceptions. The number of attendees at the tutorial session was 84. 73 students completed the survey before the exam and 60 answered after the exam. Note that filling the survey was not compulsory and thus, some students decided not to complete it. Table 2 shows students' opinions about the usefulness of this experience to consolidate knowledge and to detect doubts in the subject. In general, the mark granted to most of the items is very high. This suggests that students appreciate the use of this tool to consolidate and detect doubts. However, our results suggest a different perception between the evaluation of the usefulness before and after the final exam of the subject. Focusing on the percentage of students that consider this initiative as highly valuable, it dramatically decreases when students are surveyed after the final exam. This may indicate that students are open and motivated to the use of this teaching tools but their perception is moderated and conditioned by the mark obtained in the subject. The opinion of students about teacher explanations was very positive (84.9% rated this item 4 or 5). According to the results, 78% of students consider the IRS useful or very useful to detect lack of comprehension and 75% of them consider that it facilitated the study of the subject. Additionally, Table 3 shows the rating of students to the number of hours devoted to studying the subject. Interpretation of these results must bear in mind that this survey was carried out during the last week of class, very close to the date of the final exam. Only 19.2% of students reported that they had devoted a high number of hours to study the subject before attending the tutorial class. When taking binomial values it arises that only 42.5% of students report having prepared the subject enough to participate in the tutorial class. Taking into consideration that students’ perception about this question may be http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 26 overestimated, these results contribute to explain the small percentage of right answers showed in Table 1, specially in Type A items, confirming the usefulness of this experience to obtain information about the level of study of the subject by students. Besides this, when questioned after having been informed about their final mark, 16.7% of students argued that they have devoted the maximum time possible to prepare this subject. This percentage increases to 40% when using binomial values. These results indicate that students’ opinion about their own effort is moderated when they are informed of their final mark. Table 2. Perception about the utility of the experience. LIKERT BINOMIAL 1 2 3 4 5 TOT 0 1 TOT What is your opinion about teachers’ explanations of the use of SRI? Pre- test Post- test 1.4 0.0 4.1 8.3 9.6 28.3 37.0 36.7 47.9 26.7 100 100 15.1 36.6 84.9 63.4 100 100 What is your opinion about the utility of the tool to reinforce the contents taught in the subject? Pre- test Post- test 0.0 0.0 4.1 5.0 20.5 23.3 31.5 41.7 43.9 30.0 100 100 24.6 28.3 75.4 71.7 100 100 What is your opinion about the utility of this experience to facilitate the preparation and study of the subject? Pre-test Post- test 0.0 0.0 5.5 6.7 19.2 23.3 39.7 38.3 35.6 31.7 100 100 24.7 30.0 75.3 70.0 100 100 What is your opinion about the utility of the tool to detect doubts and lacks in the subject? Pre-evaluation Post- test 5.5 0.0 2.7 8.3 13.7 25.0 32.9 31.7 45.2 35.0 100 100 21.9 33.3 78.1 66.7 100 100 Number of observations: Pre-test: 73 Post-test: 60 Table 3. Perception about students effort. LIKERT BINOMIAL 1 2 3 4 5 TOT 0 1 TOT How much have you studied the subject before attending the tutorial class? Pre-evaluation Post-evaluation 1.4 3.3 20.5 13.3 35.6 43.3 23.3 23.3 19.2 16.7 100 100 57.5 60.0 42.5 40.0 100 100 Number of observations: Pre- test: 73 Post- test: 60 http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 27 4.2. Factors affecting students’ perception of the usefulness of the IRS To analyse the factors shaping students opinion about the usefulness of the IRS, we put forward a set of empirical models that are described in the following paragraphs. 4.2.1. Model and variables The dependent variable measures the students' perceptions about the usefulness of the IRS to resolve doubts and to identify gaps in the study of the subject. More specifically, we constructed a binary variable taking the value of 1 if student considers the use of the IRS useful and the value of 0 otherwise. We use a binary logistic regression owing to the fact that the dependent variable in the model is a dummy variable resulting from the choice between two options (1 or 0). Logit models use the standard logistic probability distribution and the interest lies primarily in the response probability, on the following form: P(y =1/x) = G(β0 + β1x1 + … +βkxk) = G(β0 +xβ). In the Logit models, G is the logistic function; P(y =1/x) the probability of something happening given the values of explanatory variables, y is a dummy variable taking values 1 or 0; x is a vector of (k-1) independent variables and β is the k x1 vector parameters. We are primarily interested in explaining the effects of each explanatory variable on the probability that a student considers the IRS useful to resolve and identify doubts. In Logit models coefficients β are not, by themselves, especially useful because of the nonlinear nature of the logistic function. The parameters do not display directly the effects of each explanatory variable on the response probability but, knowing the sign of the parameters, we can determine whether the explanatory variable has a positive or negative effect on response probability, but not the magnitude of the effect (Wooldridge, 2003; Cabrer et al. 2001). Our dependent variable is students’ perception of the usefulness of IRS to detect doubts (DOUBTS), which is a dummy variable that equals 1 when the student considers the IRS as a useful or very useful tool and 0 otherwise. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 28 Our explanatory variables are the following: - GENDER is a dummy variable that equals 1 for males and 0 for females. - FIRSTENROL is a dummy variable that equals 1 for students who are enrolled in the subject for the first time and 0, otherwise. -PASS is a dummy variable that equals 1 for students who have passed the subject and 0, otherwise. - MARK contains information about the mark obtained in the final exam. It ranges between 0 and 10. - EXPLAN is a variable measured in the Likert scale (from 1 to 5) that refers to the students’ evaluation of the explanations given by the teachers about the use of the IRS. - PRESTUDY is a variable measured in the Likert scale (from 1 to 5) that refers to students’ evaluation of the number of hours devoted to studying the subject before attending the tutorial class. - STUDY: is a variable measured in the Likert scale (from 1 to 5) that refers to students’ evaluation of the number of hours devoted to study the subject. Table 4 shows the descriptive statistics of these variables. Table 4. Descriptive statistics of the explanatory variables. AVERAGE MEDIAN MAXIMUM MINIMUM STANDARD DEVIATION GENDER 0.50 0.50 1.00 0.00 0.50 FIRSTENROL 0.88 1.00 1.00 0.00 0.32 PASS 0.30 0.00 1.00 0.00 0.46 MARK 3.78 3.59 9.60 0.00 2.33 EXPLAN 3.82 4.00 5.00 2.00 0.93 PRESTUDY 3.37 3.00 5.00 1.00 1.02 STUDY 3.77 4.00 5.00 2.00 0.99 No. observations 60 60 60 60 60 Source: Own elaboration http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 29 4.3. Results of the models The main results stemming from the regression analysis are shown in Table 5. Model 1 has been estimated including all the explanatory variables. The main difference among regressions 1a and 1b is that regression 1a shows the results considering the variables EXPLAN, PRESTUDY and STUDY as Likert type variables (from 1 to 5), while regression 1b takes the variables EXPLAN, PRESTUDY and STUDY as dummy variables (0 or 1). Except the variable MARK, which is a continuous variable, all explanatory variables are binary variables in both regressions. Regressions 1c and 1d are exactly the same as 1a but we removed the variables PASS and MARK respectively, the reason being the high correlation among them. The results show that: - The variables gender (GENDER) and explanations received about the use of the IRS (EXPLAN) are relevant in all the models for 1% and 5% levels of significance. This result suggests that women are more likely to consider the IRS useful than men. Besides, it also highlights the importance of giving proper information and instructions about the use of this tool. - The variable that refers to the time spent by students studying the subject (STUDY) is relevant but considering a 10% level of significance. This indicates that students, who are more engaged to the study of the subject, are also more concerned about the usefulness of the IRS. - FIRSTENROL is relevant for a 10% level of significance in all the models, except in Model 1b. This suggests that students who are enrolled in the subject for the first time are more motivated by the innovation in teaching. - MARK, PASS and PRESTUDY are not relevant to explain the dependent variable. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 30 Table 5. Results of the estimation of logit models. MOD 1a MOD 1b MOD 1c MOD 1d MOD 2 MOD 3 MOD 4 MOD 5 Constant -6.73*** -1.95 -6.46*** -6.42*** -5.70*** -4.98** -4.14** -4.83** GENDER -2.02** -2.49*** -1.92** -1.95** -1.81** -2.01*** -1.75** FIRSTENROL 1.66* 1.55 1.62* 1.61* 1.22 1.66* 1.60* PASS -0.73 -1.31 -0.14 0.27 -0.55 -1.23 -0.45 MARK 0.15 0.24 0.03 -0.02 0.11 0.24 0.18 EXPLAN 1.08** 1.82** 1.11** 1.11** 1.02** 1.08** 0.98** PRESTUDY 0.09 0.29 0.05 0.05 -0.04 -0.01 0.56 0.30 STUDY 0.68* 1.66** 0.67* 0.71* 0.47 0.67* 0.57 Log-likelihood -27.22 -27.46 -27.36 -27.36 -31.17 -28.71 -30.42 -30.42 McFadden R-Sq 0.29 0.28 0.28 0.28 0.18 0.25 0.20 0.20 Correctly predicted cases 81.7% 78.3% 78.3% 81.7% 75% 75% 75% 75% LR test (Chi-squared) 21.94 (0.0026) 21.46 (0.0032) 21.66 (0.0014) 21.67 (0.0014) 14.03 (0.0292) 18.97 (0.0042) 15.54 (0.0165) 15.54 (0.0165) * Level of significance: 10% ** Level of significance: 5% *** Level of significance: 1% In order to test the robustness of our results we present the models 2, 3, 4 and 5 in Table 5. These models sequentially remove the variables relevant in Model 1. The robustness checks show that the results do not significantly change when the different relevant variables in the analysis are removed from the model. This confirms the reliability of our results. 5. Discussion and conclusions This teaching initiative based on the use of the IRS has been demonstrated to be useful to detect students' lack in the study and comprehension of the subject as shown by students' opinions (78% consider the IRS useful or very useful to detect lacks of comprehension and 75% consider that it facilitated the study of the subject). Therefore, it becomes a useful tool for teachers to improve the learning process in several ways. First, teachers can inform students about the need to increase their study of the subject in order to pass it. Second, as teachers become aware of the concepts that students find difficult to understand, they can http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 31 plan their theoretical class in subsequent years to focus on clarifying them. Third, the use of the IRS improves the retrieval of information with respect to the traditional system of raising hands, because in the latter method, students are often reluctant to participate and it takes time to tabulate the data. Assuming that the use of the IRS is not a magical recipe to guarantee students' performance and that there are some technical difficulties in the use of this tool, we have nevertheless shown the usefulness of its use for students. The regression analysis of our data provides evidence on the factors affecting the attitude of students to the IRS for detecting lacks and doubts in the subject. The estimated model has shown that gender and explanations received about the use and usefulness of the tool exerts a positive and significant effect. Female students in general evaluated this item more positively than males. The rating of explanation received about the use of this experience has a positive effect on students’ attitude to the IRS. Lastly, if the level of significance is increased, the variables accounting for students who enrol for the first time in the subject and the number of hours devoted to study become relevant, indicating a positive effect on the students' attitude to the use of the IRS. The main limitation of this initiative is that the IRS has only been used in one session, but given the positive feedback obtained, teachers of the subject are considering its regular use throughout the course. Future research in other subjects could contribute to greater depth of understanding of the reasons that create a positive attitude towards new teaching methodologies in general, and the IRS in particular. There is also ground to examine the effect of these methodologies on students’ performance. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 32 6. References Anthis, K. (2011). Is It the Clicker, or Is It the Question? Untangling the Effects of Student Response System Use. Teaching of Psychology, 38(3), 189-193. Butler, J. A. (1992). Use of Teaching Methods within the Lecture Format. Medical Teacher, 14(1), 11-25. Camacho-Miñano, M. D. M. and del Campo, C. (2014). Useful interactive teaching tool for learning: clickers in higher education. Interactive Learning Environments, 1-18. Cabrer, B., Sancho, A. and Serrano, G. (2001). Microeconometría y Decisión. Pirámide, Madrid. Caldwell, J. E. (2007). Clickers in the Large Classroom: Current Research and Best- practice Tips. Life Sciences Education, 6(1), 9-20. Carnaghan, C. and Webb, A. (2007). Investigating the Effects of Group Response Systems on Student Satisfaction, Learning and Engagement in Acounting Education. Issues in Accounting Education, 22 (3), 391-409. Dallaire, D. H. (2011). Effective Use of Personal Response “Clicker” Systems in Psychology Courses. Teaching of Psychology, 38(3), 199-204. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 33 Emenike, M. E. and Holme, T. A. (2012). Classroom Response Systems Have Not “Crossed the Chasm”: Estimating Numbers of Chemistry Faculty Who Use Clickers. Journal of Chemical Education, 89, 465-469. Fallon, M. and Forrest, S. L. (2011). High-Tech Versus Low-Tech Instructional Strategies: A Comparison of Clickers and Handheld Response Cards. Teaching of Psychology, 38(3), 194-198. Fies, C. and Marshall, J. (2006). Classroom Response Systems: A Review of the Literature. Journal of Science Education and Technology, 15(1), 101-109. Gebru, M. T., Phelps, A. J. and Wulfsberg, G. (2012). Effect of Clickers Versus Online Homework on Students´ Long-term Retention of General Chemistry Course Material. Chemistry Education Research and Practice, 13, 325-329. Gül, H., Gül, S. S., Kaya, E. and Alican, A. (2010). Main Trends in the World of Higher Education, Internationalization and Institutional Autonomy. Procedia Social and Behavioral Sciences, 9, 1878-1884. Han, J. H. and Finkelstein, A. (2013). Understanding the effects of professors' pedagogical development with Clicker Assessment and Feedback technologies and the impact on students' engagement and learning in higher education. Computers & Education, 65, 64- 76. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 34 Hughes, C., Roche, A. M., Bywood, P. and Trifonoff, A. (2011). Audience-response Devices (Clickers): A Discussion Paper on their Potential Contribution to Alcohol Education in Schools. Health Education Journal, 72 (1), 47-55. Kay, R. and Knaack, L. (2009). Exploring the Use of Audience Response Systems in Secondary School Science Classrooms. Journal of Science Education and Technology, 18 (5), 382-392. Kennedy, G. and Cutts, Q. (2005). The Association between Students’ Use of Electronic Voting Systems and their Learning Outcomes. Journal of Computer Assisted Learning, 21(4), 260-268. King, S. O. and Robinson, C. L. (2009). `Pretty Lights´ and Maths! Increasing Student Engagement and Enhancing Learning through the Use of Electronic Voting Systems. Computers & Education, 53, 189-199. Lasry, N. (2008). Clickers or Flashcards: Is there really a Difference?. The Physics Teacher, 46, 242-244. Liu,T., Liang, J., Wang, H., Chan, T. and Wei, L. (2003). Embedding Educlick in Classroom to Enhance Interaction. In Lee, K., Mitchell, K. (eds.) Proceedings International Conference Computers in Education (ICCE), 117-125. Hong Kong (China). Ludvigsen, K., Krumsvik, R. and Furnes, B. (2015). Creating formative feedback spaces in large lectures. Computers & Education, 88, 48-63. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 35 MacGeorge, E. L., Homan, S. R., Dunning, J. B. (Jr), Elmore, D., Bodie, G. D., Evans, E., Khichadia, S., Lichti, S. M., Feng, B. and Geddes, B. (2008). Student Evaluation of Audience Response Technology in Large Lecture Classes. Educational Technology Research and Development, 52(2), 125-145. Morling, B., McAuliffe, M., Cohen, L. and DiLorenzo, T. M. (2008). Efficacy of “Clickers” in Large, Introductory Psychology Classes. Teaching of Psychology, 35, 45-50. Moss, K. and Crowley, M. (2011). Effective learning in science: The use of personal response systems with a wide range of audiences. Computers & Education, 56(1), 36-43. Palmer, E. J., Devitt, P. G., De Young, N. and Morris, D. (2005). Assessment of an Electronic Voting System within the tutorial setting: A randomised controlled trial. BMC Medical Education, 5:24. Poirier, C. R. and Feldman, R. S. (2008). Promoting Active Learning using Individual Response Technology in Large Introductory Psychology Courses. Teaching of Psychology, 34, 194-196. Schackow, T., Milton, C., Loya, L. and Friedman, M. (2004). Audience Response Systems: Effect on Learning in Family Medicine Residents. Family Medicine, 36 496-504. Siau, K.; Sheng, H. and Fui-Hoon Nah, F. (2006). Use of a Classroom Respond System to Enhance Classroom Interactivity. IEEE Transactions on Education, 49(3), 398-403. http://dx.doi.org/10.4995/muse.2017.5476 Multidisciplinary Journal for Education, Social and Technological Sciences http://dx.doi.org/10.4995/muse.2017.5476 ISSN: 2341-2593 Bares López et al. (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 16-36 | 36 Stowell, J. R. and Nelson, J. M. (2007). Benefits of Electronic Audience Response Systems on Student Participation, Learning and Emotion. Teaching of Psychology, 34, 253-258. Van Dijk, L. A., Van den Berg, G. C. and Van Keulen, H. (2001). Interactive Lectures in Engineering Education. European Journal of Engineering Education, 26(1), 15-28. White, P., Syncox, D. and Alters, B. (2011). Clicking for grades? Really? Investigating the use of clickers for awarding grade-points in post-secondary education. Interactive Learning Environments, 19(5), 551-561. Wooldridge, J.M. (2003). Introductory econometrics, a modern approach. Second ed. South-Western College Publishing, Cincinnati, OH. http://dx.doi.org/10.4995/muse.2017.5476