Goulding, R. R. J., Harlick, A. M., Kelly, R., & Oksanen, K. (2017). Impact of the quick response code based in class assignments on student engagement and retention. Papers on Postsecondary Learning and Teaching: Proceedings of the University of Calgary Conference on Learning and Teaching, 2, 42-50. IMPACT OF THE QUICK-RESPONSE CODE BASED IN-CLASS ASSIGNMENTS ON STUDENT ENGAGEMENT AND RETENTION Richard R. J. Goulding, Memorial University of Newfoundland Anna Maria Harlick, University of Calgary Richard Kelly, Memorial University of Newfoundland Karoliina Oksanen, Memorial University of Newfoundland The paper describes a teaching technique alternative to online-based student response systems. We explore the potential of quick response (QR) code sheets as a tool for formative assessment, feedback, as well as a way to increase class engagement, student participation, information retention, and as a method to develop communication skills. The technique is neither institution nor course bound and can be implemented in classes of numerous sizes and levels. While it resembles more traditional teaching methods than computerized student response systems, it is student centered and accommodates digital natives’ approach to information gathering. Keywords: student engagement, formative assessment, student response systems INTRODUCTION Introductory Physics courses at the university level are often offered as service courses and are attended by students who plan to pursue majors in other scientific disciplines. The courses are designed to cover a variety of distinct and interlocking topics, but rarely give an opportunity to explore those subjects thoroughly (Redish & Steinberg, 1999). This approach is often a compromise between the needs of different faculties and results in a fast-paced course that treats multiple topics superficially. According to “Public Perception of Physics” report (2008), many of the enrolled students are negatively biased against the subject and consider it uninteresting, complex, and too specialized for most people to understand. In addition to this negative preconception, the first-time exposure to the topics and deficiencies in mathematical skills contributes to the difficulty level of the course (Ornek, Robinson & Haugan, 2008). Consequently, motivating the students to come to class and work systematically poses a challenge both for the instructors and for the students themselves. The paradigm of student engagement was introduced almost 30 years ago (Johnson, Johnson & Smith, 1991) and has become a matter of concern following the availability and popularity of higher education (Kahn, 2014). Astin (1985) argues, that learning is proportional to the quality and quantity of students' involvement, defined as a process that occurs through the learning continuum. Active participation in educationally purposeful activities has since been shown to have both a positive and statistically significant effect on academic outcomes and a compensatory effect on first-year grades with persistence into the second year of education (Kuh, Coup, Kinzie, & Goneya, 2008; Wright, 2014). Blood and Neel (2008) also show that students enjoy actively participating and believe that the process itself aids in their education. Goulding, Harlick, Kelly, & Oksanen 43 METHOD DEVELOPMENT The original motivation for the creation of the in-class assignment tool was driven by two factors: to increase student engagement in large, multi-sectional introductory physics classes, and to create an assessment tool that would allow both students and instructors to monitor student progress and learning in the courses. Despite exploring many web and technology-based student response systems (SRS), we were not able to find one that fully enables the practice of problem- solving skills which are crucial in teaching sciences. While moving away from the traditional, lecture-based teaching format and adapting the delivery styles to accommodate the evolving learning needs of generation Z (McCrindle, 2017-II), we came up with a tool that creates an opportunity for students to actively participate in the course and allows them to encounter and attempt questions of a range of styles and difficulty levels before facing heavily weighted summative assessments. The product of the research and development was a paper-based tool, which employs QR codes to store and process student information rather than linking it to an activity. Mueller and Oppenheimer (2014) show that taking notes longhand allows for deeper processing of the information given. Di Vesta and Gray (1972, 1973) explain that the note taking process must be discriminatory, that is the more deeply the format of the information is transformed (rewording, analysis, systematization, summarizing and application occurs), the greater the benefits. The development of a response system that engages handwriting allowed practicing problem-solving skills, from question set up, through the proper procedure to the formatting. Moving away from electronic devices creates an environment in which written communication skills can be developed and strengthened. Asking students to write down responses forces the expansion of their comfort zones and exercises writing and problem-solving skills that are currently diminishing in the student population. The in-class assignments are short (to comply with the bits of information at the time), but the vessel in which information is transferred is independent of the electronic device. The assignments serve as a method that contributes to bridging the gap between the comfortable world of sound-bite information, short messages and immediate responses, and to less familiar land of long-answer questions and written communication. Additionally, the process engages learning channels that are underused in both traditional lecture and device-focused response systems. The educational content is the crucial part of the design, as formation of the questions should evoke compilation and processing of the material presented in class. The formative assessment technique that concentrates on the interpretation and processing of information, combined with discriminative writing rather than simple recall of information, allows for processing of the information on the more cognitive level. In the initial implementation stages of the technique, we discovered another opportunity for active learning as it allowed us to start a dialogue with the students about essential questions such as course content and methodology. In-class assignments were used to gather students' opinions and reflections about midterms, pace of the course, their study habits, and the in-class assignments themselves. PROCEDURE The teaching method has been routinely used since 2013 in classes of various sizes, exposing up to 600 first-year students in Physics courses to it each year. The method is not Goulding, Harlick, Kelly, & Oksanen 44 course-specific; therefore, it can easily be implemented in other large classes with no additional cost. Figure 1. Schematics of the original procedure of administering QR code based in-class assignments. The schematic of the initial procedure is presented in Figure 1. Prior to coming to class, students generate a sheet with a QR code that contains information that allows a course instructor to identify them and connect the submission to their e-mail address/dropbox. The format of the activity itself is like those incorporating other SRS. A question is presented, students work on the answer (communicating with each other, referring to their notes and other available resources) and submit their work in class. The sheets are then scanned, generating a file in portable document format. The document is then analyzed using software developed for the project. It analyzes the document, deciphering of the information, populating a spreadsheet with participants information, and assigning participation marks. The format of the spreadsheet can be adjusted to the requirements of the platform used for distribution and storage of the grades. ASSESSMENT AND DEVELOPMENT OF THE TECHNIQUE Two groups of students were surveyed about their experience with the QR code response systems. The first poll was conducted in two sections of Introductory Physics (IP) II class, during Winter 2015 semester, using QR code sheets. While responding very positively to the idea of the in-class assignments, students pointed out that they are missing the access to their submitted work and feedback on it. The sentiment was repeated in a survey conducted two years later among students who completed one or more courses that implemented this teaching technique. The data collected are presented in Table 1. The latter survey also was designed to identify specific improvements desired by the students. The ideas, presented by popularity are tabulated in Table 2. Goulding, Harlick, Kelly, & Oksanen 45 Table 1 Results of the polls investigating students' general opinion about the QR code based in-class assignments Positive/ Slightly Positive [%] Neutral [%] Negative/ Slightly Negative [%] April 2015 Poll [n=230] 74 18 8 April 2017 Poll [n=97] 83 10 7 Table 2 Improvements of the QR code based in-class assignments suggested by students during an online survey (n=97). Numbers represent percentage of students who selected given answer (multi-selection type question). Improvement Idea Popularity [%] Access to correct solutions 83 Ability to submit forms electronically (using portable devices) 58 Solution presented in class 52 Access to submitted forms 43 Access to statistics (correctness, submissions, text analysis) 41 Grading for correctness 13 The most recent improvement includes the ability to provide students with an electronic copy of their own work accompanied by a correct solution for feedback purposes. The return of student work is done either using a university affiliated e-mail address or via online platform associated with the course (e.g. D2L). It also allows for easy creation of a revision resource in form of the question archive for the students. Figure 2. Schematics of the procedure of administering QR code based in-class assignments after the adjustments. Goulding, Harlick, Kelly, & Oksanen 46 The schematics of the procedure are presented in Figure 2. The main achievement of this upgrade is that it addresses the two key issues identified in the analysis of the process: returning the submitted work to students and accessibility of a full solution. The returned work is accompanied by correct answer to the question/detailed solution to the problem, giving the student opportunity for formative assessment. This addition effectively closes the loop of the activity. Without limitations imposed by clicker-style student response systems, paper-based in- class assignments can include all kinds of questions, from exploratory and summary to hypothetical and relational. Going through the entire process of the activity and including the revision of their answers and self-evaluation creates an opportunity to develop learning skills (Palfrey & Gasser, 2010) and allows students an honest and non-judgmental assessment of their work. Access to the submitted responses opens a possibility for students to contact course instructors to seek more meaningful feedback aimed at faulty interpretations rather than the lack of information (Hattie and Timperley, 2007). STUDENT ENGAGEMENT AND RETENTION Engagement Although never intended to be an attendance measuring device, submitted in-class assignments reflect students` presence and participation in class, both considered educationally purposeful activities. In addition to the general opinions presented in Table 1, each survey contained additional questions that focused on the educational functions. The most positive response is associated with the small amount of participation marks awarded for submission of the in-class assignments. Students perceived these marks in two ways - as a benefit by those who generally participate in class and as an unfair burden by those who choose not to. Both groups recognized them as a motivating factor in actively attending classes. Another noticeable trend is the appreciation for the motivation to come to class and pay attention to the presented content. Other positive comments referred to the realization of active engagement, taking ownership for their written words, self-evaluation, and self-assessment. Student criticism of the QR sheets targeted the methodology itself (the format requirements), the lack of feedback (which prompted the updates to the procedure) and the difficulty level of the presented questions (more range required). Information Retention While the discussion on the correlation between class participation and the final grades obtained in the courses is very much alive (Ahlfeldt, Mehta, & Sellnow, 2005; Krause, 2005, Blood & Neel, 2008; Credé, Roch & Kieszczynka, 2010), data collected by authors in multiple first year physics courses show correlation between class participation (measured by percentage of in class assignments submitted) and average grade achieved by student, regardless of instructor. Goulding, Harlick, Kelly, & Oksanen 47 Figure 3. Correlation between the percentage of submitted in-class assignments and average final grade for students in multiple sections/classes of Introductory Physics I (left, n=750) and Introductory Physics II (right, n=727) classes in years 2013-2016. Dashed line represents best fit, taking into account the weights of the data points. Graphs in Figure 3 show correlation between the number of submitted in-class assignments and the final grade. The correlation is more apparent in IP II course. One of the reasons for this may be lack of a credit exclusion preventing an enrolment into the course by students who previously took Physics courses. Material covered in IP I repeats many concepts introduced in high school physics, therefore students’ perception and understanding may be heavily influenced by preexisting knowledge (Prosser, Trigwell & Waterhouse, 2000). This may affect the effectiveness and importance of teaching techniques used throughout the course. Taylor (2012) presents evidence of his own observation that promoting attendance through mark additions and deductions is a reliable and valid method of helping students achieve better grade results. Similarly, our results do not allow for drawing any cause and effect conclusions. They agree with Golding (2011) findings, who in his review on the role of attendance policies in large classes, concludes a positive correlation between attendance and performance, although admits that that is not always the case. According to Golding (2011), in- class assignments prove helpful in encouraging students to come to class and engage in the lectures, but they are not found to motivate students to work harder outside of the classroom. Goulding, Harlick, Kelly, & Oksanen 48 Figure 4. Comparison of the results form term tests assessing two-dimensional kinematics and Newton's laws in Introductory Physics I. Data represents ratio of percentages of students who scored within a given bin in Fall 2015 (open circles) and Fall 2016 (solid circles) to students who scored within the same bin in Fall 2012 (prior to introducing in-class assignments). Realizing, that proper assessment of information retention and understanding of the material covered in class requires more rigorous methodology, we want to draw attention to the results of the midterm test administered in IP I course. Figure 4 shows the variation in the ratio of percentages of students who received grades in certain ranges on a term test assessing two- dimensional kinematics and Newton's laws. While keeping in mind results in IP I course could be affected by students` preexisting knowledge, this test was selected for comparisons as during both Fall 2015 and Fall 2016 semesters, material tested was targeted with challenging, exploratory and action-type questions on in-class assignments. Negative ratio variations indicate that percentage of students falling into the grade range decreased, while positive ratio variation means higher percentage of students scored within that range. Figure 4 shows that during two recent years fewer students fell in the low-mark range, with no students falling below 25% grade during Fall 2016. At the same time, the number of students falling into the grade above 90% at least doubled during both Fall 2015 and Fall 2016. FURTHER DEVELOPMENTS Further developments of this teaching technique will explore two routes. The first one will use existing resources to administer other activities. This will include incorporation of the QR-code sheets and feedback system in the bi-weekly problem-solving sessions to deal with providing exam practice, proper solutions and real marking schemes for more demanding exam style questions as well as to administer pre-class reading quizzes and other forms of take-home one question assignments. The second route addresses the second most popular suggestion for improvement indicated in Table 2, that is, the ability to submit forms electronically using portable devices. This development is a future goal and will have to rely on additional improvement of the software. Goulding, Harlick, Kelly, & Oksanen 49 CONCLUSIONS A formative assessment tool alternative to online SRS was developed using quick- response code sheets. Technology is implemented in the process only as a tool to encode, gather and store information about students' participation. While the in-class assignment itself resembles more traditional teaching methods, the approach behind it is student-centered. The process also accommodates the digital natives' approach to gathering information and, subsequently, knowledge (Palfrey & Gasser, 2010), through allowing for grazing (information presented in class), “deep-dive'” (attempt to solve a problem) and creation of the feedback loop (submitted answer and solution to compare provided). The technique is neither institution nor course bound and can be implemented in classes of all sizes and levels. In addition to its ability to monitor student attendance and progress, it aided in opening an unused channel for communication with students. The tool can be used to administer in-class surveys regarding course material, teaching and assessment methods, and gather students’ reflections on their own progress. We found that students tended to be very honest in their written submission regarding every aspect of the course. Additionally, the use of hand writing in providing answers to all types of questions creates an unscheduled opportunity for students to practice their communications skills, composition of the solutions to long-answer questions and general abilities to organize and present material in the environment of formative assessment. REFERENCES Ahlfeldt, S., Mehta, S., & Sellnow, T. (2005). Measurement and analysis of student engagement in university classes where varying levels of PBL methods of instruction are in use. Higher Education Research and Development, 24(1), 5-20. Astin, A. W. (1985). Achieving educational excellence. a critical assessment of priorities and practices in higher education. San Francisco, CA: Jossey- Bass Publishers. Blood, E., & Neel, R. (2008). Using student response systems in lecture-based instructions: Does it change student engagement and learning? Journal of Technology and Teacher Education, 16(3), 375-383. Credé, M., Roch, S. G., & Kieszczynka, U. M. (2010). Class attendance in college: A meta- analytic review of the relationship of class attendance with grades and student characteristics. Review of Educational Research, 80(2), 272-295. Di Vesta, F. J., & Gray, G. S. (1972). Listening and note taking. Journal of Educational Psychology, 43(1), 3-14. Di Vesta, F. J., & Gray, G. S. (1973). Listening and note taking II: Immediate and delayed recall as functions of variations in thematic continuity, note taking and length of listening- review intervals. Journal of Educational Psychology, 64(3), 278-287. Golding, J. M. (2011). The role of attendance in lecture classes: You can lead a horse to water... Teaching of Psychology., 38(1), 40-42. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. Johnson, D. W., Johnson, R. T., & Smith, K. A. (n.d.). Active learning: Cooperation in the college classroom. Edina, MN, USA, Interaction Book Co. Kahn, P. E. (2014). Theorising student engagement in higher education. British Educational Research Journal, 40(6), 1005-1018. Goulding, Harlick, Kelly, & Oksanen 50 Krause, K.-L. (2005). Understanding and promoting student engagement in university learning communities. James Cook University Symposium 2005. Sharing Scholarship in Learning and Teaching: Engaging Students. Townsville City, Australia. Kuh, G. D., Coup, T. M., Kinzie, J., & Goneya, R. M. (2008). Unmasking the effects of student engagement on first-year college grades and persistence. Journal of Higher Education, 79(5), 540-563. McCrindle, M., (2017). Generation Z- Learning Styles. http://generationz.com.au/learning-styles/ Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: advantages of longhand over laptop note taking. Psychological Science, 26(6), 1159- 1168. Ornek, F., Robinson, W. R., & Haugan, M. P. (2008). What makes physics difficult. International Journal of Environmental and Science Education, 3(1), 30-34. Palfrey, J., & Gasser, U. (2010). Born digital. Understanding the first generation of digital natives. New York, NY, USA, Basic Books. People, Science and Policy Ltd (2007), Public Perception of Physics, Institute of Physics, http://www.iop.org/publications/iop/2007/page_39533.html Prosser, M., Trigwell, K., & Waterhouse, F. (2000). Students’ experiences of studying physics concepts: The effects of disintegrated perceptions and approaches. European Journal of Psychology of Education, xv (1), 61-74. Redish, E. F., & Steinberg, R. N. (1999). Teaching physics: Figuring out what works. Psychology Today, 52, 24-30 Taylor III, L. A., (2012). Cutting class harms grades. Journal for Innovation Education, 10(1), 49-61. Wright, J. (2014). Participation in the classroom: Classification and assessment techniques. Teaching Innovation Projects, 4(1), 1-11.