Microsoft Word - sherr_final.docx Journal of Effective Teaching in Higher Education, vol. 2, no. 2 Nudging Students to Succeed in a Flipped Format Gateway Biology Course Goldie Sherr, Bronx Community College of CUNY, goldie.sherr@bcc.cuny.edu Shylaja Akkaraju, Bronx Community College of CUNY Seher Atamturktur, Bronx Community College of CUNY Abstract. The intensive nature of gateway biology courses often results in poor student performance. This issue can be resolved with the usage of the flipped classroom method. Our application of this model to a human anatomy and physiology course increased the retention and passing rates. However, the beneficial effects of flipping a classroom can be substantially weakened due to student disposition. To target this, we used the flipped classroom model in conjunction with a series of nudges in the form of self-monitoring logs and reminder notifications. Students were responsible for tracking their attendance, punctuality, preparedness, and grades via a self-monitoring log that they had to complete each week. Reminder notifications were sent to students via text or email messages. While the flipped classroom method accompanied by both self-monitoring logs and reminder techniques can be a powerful learning model leading to higher student success rates, we found that consistent nudging via text messages appears to be pivotal in ensuring student success. Keywords: flipped learning, nudging, reminders, self-monitoring, student disposition, student success. Gateway science courses are some of the most demanding classes that colleges offer. Content is typically complex with topics building upon one another, so that new information cannot be mastered until the earlier information is fully understood. The cognitive load is generally high, making this very difficult for the students. Moreover, a student population that is lacking good study habits and a science background can present significant challenges to the instructor as well. The flipped learning method, by including regular formative assessments, has been shown to be a suitable option to resolve these issues within a community college setting (Akkaraju, 2016). The flipped classroom model is a learning methodology that veers away from standard textbook-style learning and revolves around a more student-centered learning process. The premise behind the flipped classroom method is that the instructor provides the students with the fundamental basics of a lesson prior to coming to class so that class time can be spent more effectively (Bergmann & Sams, 2014). Before coming to class, students are required to go over the information, which is presented to them via a learning module prepared by the instructor. Precious classroom time can now be used to go over deeper and more profound learning concepts as well as for group activities, discussions, and problem solving (Bergmann & Sams, 2012). In order to guarantee that students review the material, a formative assessment is recommended to be given at the beginning of each class. These low stakes quizzes can not only lead to better Nudging Students to Succeed 58 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 preparation, but can also increase student enthusiasm, confidence, and motivation (Warnock, 2004). Flipped learning lends itself nicely to gateway science courses, which often contain a vast amount of material that must be covered in a short period of time. Concepts are usually complex and are likely unfamiliar to students. The flipped classroom method thus allows students to enter the science classroom with a baseline of information that enables them to tackle the more difficult material in a less overwhelming way. Previous studies have shown that the application of the flipped learning method in Anatomy and Physiology II, a difficult STEM course with low retention and passing rates, greatly increased student performance, morale, and retention (Akkaraju, 2016). However, while the flipped classroom method has many benefits, it does have its limitations. If students display a poor disposition, then the flipped learning method can become unsuccessful. It is also important for the instructor using the flipped learning method to fully embrace the practice of formative assessments (Moss & Brookhart, 2009) by communicating regularly and predictably with the students. Inconsistent communication may hinder student progress. In order to counteract these potential obstacles, we propose that the flipped classroom model be performed in conjunction with a series of nudges in the form of reminders and self-assessments. A nudge is a suggestion that has the potential to alter the behavior of the recipient and influence their decision. In the field of behavioral economics, nudging is used in a variety of ways. One powerful way of nudging is via reminders (Sunstein, 2014). Nudges via text messages have been shown to be highly effective in positively influencing the behavior and decision-making process of students (Castleman & Page, 2015). Studies have also shown that students greatly benefit from both reminders and educational support given through a mobile platform of their choosing (Carmean & Frankfort, 2013; Narasimhan, 2016). Results showed that students who had nudging messages sent to their devices directly had greater academic performance than students who did not. Additionally, the study also revealed that the students in mathematics and economics courses who received nudges had overall better final grades, better passing rates, and higher completion rates (Carmean & Frankfort, 2013). Thus, these results indicate that a simple gesture, such as a reminder via email or education platform can have profound effects on learning. Another powerful way to positively impact study behavior would be to nudge students to self-monitor their progress throughout the semester. Self-monitoring is a key aspect of self-regulation, and it can potentially improve both student disposition and academic performance (Harris, Friedlander, Saddler, Frizzelle, & Graham, 2005). Self-regulation is a set of processes that promote student learning and success (Zimmerman, 2002). Ideally, self-regulation involves a forethought phase during which the student engages in goal setting and strategizing, followed by a performance phase during which the student focuses on tasks and self- monitors progress, and finally a self-reflection phase during which the student engages in self-assessment and makes decisions about future strategies (Zimmerman & Capillo, 2003). Students who perform the self-evaluation show an Nudging Students to Succeed 59 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 increase in motivation and self-efficacy, thus not only boosting performance, but also enhancing the belief in their own abilities to succeed (Schunk, 2003). According to assessment experts, the three main aspects of a dynamic student self- assessment process include self-monitoring, self-assessing, and self-correcting (McMillan and Hearn, 2008). Self-monitoring can help students to raise awareness toward goals, self-assessing helps keep them focused on a specific target or benchmark, and self-correcting involves the corrective actions they take to overcome setbacks and celebrate achievement. Some authors believe that self- monitoring itself comprises two aspects—self-recording and self-assessment (Graham, Harris, & Reid, 1992). The flipped learning method combined with a weekly self-monitoring log can aid students in their self-regulation process. Formative assessments provide students the means for continual awareness of progress (Nicol & Marcfarlane-Dick, 2006). As formative assessments are an integral part of the flipped classroom model and provide immediate feedback, the self-assessment aspect of self-regulation fits nicely into the framework of a flipped format course. We decided to make use of a self-monitoring log in our Anatomy and Physiology I course in order for students to keep track of their performance in formative assessments while targeting important areas of student disposition, such as regular attendance, punctuality, and sufficient preparation for each class session. These areas are particularly important to monitor as a lack in any one of them can render flipped learning, which relies heavily on proper learning habits, ineffective. Additionally, we supplemented this self-monitoring log with consistent reminders to students, either via email or through a mobile messaging application called Remind. We believed that nudges would be specifically useful to first year students who may not yet have proper learning habits. In this article, we detail the success of using the flipped learning format in conjunction with nudging via self-monitoring logs and reminder notifications. Methods We implemented the flipped classroom method in a gateway human anatomy and physiology course at Bronx Community College of the City University of New York where more than 95% of the student body is made up of ethnic minorities and 90% of the students need developmental instruction in one or more basic skill areas. More than half the student population is either at or below the poverty line (Bronx Community College of CUNY, 2019). Our study involved seven sections taught by three different instructors over a period of four consecutive semesters. In six of the sections (Sections A-F) students were nudged to complete a self-assessment log every week. They were also nudged to prepare for upcoming quizzes and exams either via email announcement using the Blackboard learning management system or via text messages using a mobile messaging application called Remind. In the seventh section (Section G), the students were only nudged via text messages and were not involved in the practice of weekly self-monitoring. A total of 162 students Nudging Students to Succeed 60 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 were enrolled across the seven sections with each section consisting of 21-26 students (Table 1). Table 1 Overall set up of all seven sections and student performance data Sections Use of Self- assessment log Use of Nudging via email Use of Nudging via (Remind) Instructor Number of students C minus and above F grade W grade A Consistent None Consistent X 26 24 0 1 B Consistent None Consistent Y 21 19 0 2 C Consistent Consistent None Z 21 11 1 7 D Consistent None Inconsistent X 24 13 0 7 E Consistent None Consistent X 26 19 1 2 F Consistent Consistent None Z 23 12 1 10 G None None Consistent Y 21 17 1 3 The Flipped Classroom Format The lecture portion of the Human Anatomy and Physiology I course lasts for about 3 hours and usually includes one or two short breaks. In our flipped classroom lectures, class always begins with a formative assessment in the form of a pre- lecture quiz. Quizzes are typically given for the first 20–25 minutes of class and are relatively straightforward. These quizzes assess proficiency in the lesson’s fundamental information. Quizzes are based on information found in the learning modules which students can access through the Blackboard learning management system. Quizzes are 20 points each and serve as a main source of motivation for students to come to class prepared, as mastery of factual knowledge is key to further understanding of increasingly difficult concepts. Each of these low stakes quizzes is designed as a desirable difficulty (Bjork, 2014), a strategy that challenges the student without overwhelming them. While preparation for a quiz requires some time and effort, it should not be so difficult as to elicit avoidance behavior on the part of the student. The quizzes are made up of 20 multiple choice, short answer, matching, or true/false questions and generally act as confidence boosters at the beginning of each lecture session. Students are continually made aware that the benchmark grade for these low stakes formative assessments is 80% and that this benchmark must be met by the student in order to do well on higher stakes exams. Because these quizzes are given at the beginning of each class, it is crucial that students show up to class on time in order to take them. Once the quizzes are completed, class time is spent addressing conceptual and procedural (or application) knowledge. The higher order cognitive processing was addressed through a variety of learning opportunities offered by the three instructors: acting out concepts (Bealer & Bealer, 1996), interactive notetaking (Akkaraju, 2018), quiz games (Simkin, 2013), and problem solving in small groups (Laal & Ghodsi, 2012). It must be mentioned here that apart from the exposure to pre-training learning modules and pre-lecture quizzes Nudging Students to Succeed 61 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 the rest of the lecture sessions were not uniform with each instructor adopting a different set of active learning exercises. Nudging via a Self-Monitoring Log The self-assessment component of the course is conducted after the completion of the formative assessment. Students are handed their own self-assessment log (Table 2), which they are required to personally fill out at the end of each class. Areas of disposition that are addressed on the log include those of attendance, punctuality, preparedness, attentiveness, and cell phone usage. The monitoring log also includes a section for formative assessment grades and whether or not those grades meet the benchmark. This fulfils the self-assessing aspect of the self- monitoring process. At the end of each class students are required to return their completed log to the instructor. This ensures that self-monitoring logs do not get misplaced or forgotten so that they can be handed back to the students during the following class. Students are responsible for filling out their log during every class, allowing them to continually track their learning behavior (self-monitoring), evaluate their own performance on quizzes (self-assessing), and reflect on their learning behavior and study strategies to improve performance (self-correcting). Students who are absent will have the date for that specific class crossed out on the self-monitoring log by the instructor, indicating that they were not present. Table 2 Example of the student self-monitoring log Nudging via reminder notifications Throughout the semester, instructors also supply a series of reminders, or nudges, to raise student awareness for important information regarding the course. These nudges primarily came in two forms: (a) email announcements via the Blackboard learning management system or (b) a mobile messaging app called Remind (Remind, 2014) that can send announcements in the form of a text message to the students. Using text messages is more convenient. The instructor creates the course within the Remind platform; students then download the free Remind app directly to their phone and sign up for their specific course without having to exchange any personal Nudging Students to Succeed 62 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 information with their instructor. The benefit of sending reminders through this application, as opposed to using traditional email, is that students are much more likely to receive the message. Since most students keep their phones with them throughout the day, the likelihood of their seeing the notification (which appears on the phone similar to the way a text message does) is much greater. These nudges serve as an additional aid for students, particularly those in their first year, to remind them of important class dates and information. Additionally, students can privately contact the instructor via email or directly via Remind, which has the option for the student to respond to their instructor also via a text message. Students can send back questions regarding the material, ask for confirmation of exam/assignment dates, or inquire about additional course information. These nudges, therefore, not only serve as reminders for students to maintain a good disposition but also act as an open door for students to communicate with their instructor when needed. In four (sections A, B, E, and G) out of the seven sections, instructors used Remind messages to nudge students about upcoming quizzes, exams, scheduling information, and any other important course reminders that the instructors wanted the students to know. The relatively informal medium of the Remind platform allowed instructors to send friendly and encouraging messages to students (Table 3). In an additional section (section D), the instructor also used Remind but only in a sporadic and unpredictable manner. Table 3 Examples of the general tone and context of text messages sent to students Reminders Encouragement Helpful information Hello Class! Lecture quiz 3 tomorrow! Please be on time. Cheers! Hello Class! Good luck with your preparation for tomorrow's Lecture Exam. Don't forget to bring your take-home exam. Get some sleep! Tomorrow we will be doing chapter 11 - muscles. The PowerPoint slides for lecture are also posted on blackboard if you want to print them. Hi everyone, this is a Reminder: Your Final exam is tomorrow at 10am in ME- 226. Also, I will be having an office hour today from 12– 1pm. Hello Class! The Lecture Practice Exam II is live on blackboard. Please practice until you get 100% Good luck! Happy Easter! Hello Class! Here is a quick guide to the practical portion of the lab exam including data for graphing. (accompanied by attachment) Hope you had a nice spring break. Reminder: you have a pre-lecture quiz on the skeletal system tomorrow. Your final grade is up on blackboard. Congratulations! You ALL passed! I am immensely proud of each one of you! Have a great Summer! I will be having my office hour today…It will be from 11:45– 12:45pm. Nudging Students to Succeed 63 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 Results Student performance and completion rates were collected and analyzed from all seven course sections. We established a C minus as the benchmark grade for this gateway science course because students scoring less than a 70% in this course usually struggle in subsequent biology courses. The impact of nudging via reminder notifications Our analysis showed that the seven sections fell into two groups based on whether or not they received nudging via reminder notifications in the form of text messages. Figure 1. The impact of regular nudges via text message on student performance Overall, our data showed that students enrolled in flipped learning sections combined with the practice of consistent nudges via text messages using Remind as in sections A, B, E, and G fared significantly better (p=.002) than students enrolled in flipped learning sections that either did not use text messaging to nudge students, as in sections C and F, or only used it sporadically, as in section D (Table 1 and Figure 1). Among the students that were gently nudged via Remind on a regular basis, about 84% passed with at least a C minus in the course; whereas among the students that were nudged via email or inconsistently via Remind, only 52% passed with at least a C minus in the course. 84.19020147 52.90717736 0 10 20 30 40 50 60 70 80 90 100 Consistent Usage of remind Inconsistent/No Usage of remind Success Rate p=0.002 Nudging Students to Succeed 64 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 Figure 2. The impact of regular nudges via text message on student retention We also found that gentle nudging via text message also significantly increased (p=.0018) student retention rates in this course (Figure 2). Figure 3. Comparison of success rates by section to the semester course mean Examining each section individually, we found the following: (a) nudging via email resulted in success and retention rates around the overall course mean (see Sections C and F in Figures 3 and 4); (b) inconsistent nudging via text message also resulted in success and retention rates at the overall course mean (see Section Nudging Students to Succeed 65 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 D in Figures 3 and 4); and (c) consistent practice of nudging via text message resulted in success and retention rates that were significantly above the overall course mean (see Sections A, B, E, & G in Figures 3 and 4). This clear difference in performance, based on the method of nudging, indicates that the type of medium that instructors choose to nudge students and how often the nudge is employed can greatly impact overall success. We found this to be true even when the same instructor applied nudges consistently in one section and inconsistently in a different section thereby eliminating instructor bias to some degree. With the progression of technology, there are a plethora of new educational tools readily available to both students and faculty on mobile devices. The usage of such mobile apps enables instructors to communicate with students more effectively and allows for the most reliable way to remind students of course information. Figure 4. Comparison of retention rates by section to the semester course mean Nudging via self-monitoring logs We found no direct evidence that self-monitoring logs had a significant impact on success and retention rates. However, when students (n=22) from one of the sections (Section A) were surveyed regarding the usefulness of the self-monitoring practice, their response was resoundingly positive. Participation in the survey was voluntary and anonymous. Questions ranged from whether the self-monitoring log helped students keep track of their progress and meet their benchmark grades to whether it was useful for punctuality. Most students found the self-monitoring log Nudging Students to Succeed 66 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 to be either “strongly effective” or “effective” in the various disposition areas targeted (Figure 5). Figure 5. Student perceptions of nudging via self-monitoring The impact of formative assessments on student performance In those sections with significantly high pass rates (Sections A, B, E, and G), the students also met the benchmark of 80% for the weekly pre-lecture quizzes (Figure 6) with students displaying positive behavioral traits in terms of preparedness, regular attendance, and punctuality. These sections also enjoyed 87% attendance and punctuality throughout the semester (Figure 6). Figure 6. The impact of nudging via text messages on preparedness, attendance, and punctuality 79.75 87.7705 0 20 40 60 80 100 Students with at least 80% average on quizzes Students with at least 80% attendance record Pe rc et na ge Average Quiz Grades and Attendance (80% or higher) for sections that nudges with texts 7 11 8 11 8 3 7 5 3 3 2 1 3 4 5 5 1 1 0 0 T R A C K I N G P R O G R E S S M E E T I N G B E N C H M A R K S P U N C T U A L I T Y S T A Y I N G U N T I L T H E E N D O F C L A S S Strongly Effective Effective Somewhat Effective Ineffective Not Sure Usefulness of the Self-Monitoring Log (N=22) Nudging Students to Succeed 67 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 Discussion We found that the flipped classroom format generally has a strong positive impact on students enrolled in a gateway science course largely due to its emphasis on weekly formative assessments (see Figure 6). Students who do well in these weekly quizzes are better able to handle deeper concepts and apply this knowledge to real world scenarios. Student performance in these formative assessments is therefore a strong indicator of overall performance in the course. However, the success of the flipped classroom format hinges on student disposition traits such as preparedness for each lecture session, punctuality, and attendance. Our main finding is that employing nudges via text messages one day ahead of each quiz has a powerful impact on student disposition and performance (see Figure 1). We believe that in order to be effective, the nudges must be predictable, consistent, timely, and in the form of text messages. Even though the self-monitoring log allowed students to reflect on their own disposition, it did not appear to have a direct impact on student performance (see Figure 4). We believe that self-monitoring was a nudge that may have been applied too late and after the fact, as it were. According to one of the proponents of the nudge theory, timing is critical when applying a nudge because it gives the recipient an opportunity to act in response to the nudge (Sunstein, 2014). In our model, self- monitoring did not help students with the forethought phase of self-regulation (Zimmerman, 2002). Another limitation of this study is that the self-monitoring log was not designed to address metacognitive thinking on the part of the students. In future semesters, we hope to elicit student input to improve the design of the self- monitoring log to make this practice both meaningful and effective in enhancing student self-regulation. Based on our results, we believe that the inclusion of a nudging component (in the form of text messages) within the instructional design of a flipped gateway science course can have a strong positive impact on student success. Conflicts of Interest The authors declare that there is no conflict of interest regarding the publication of this article. References Akkaraju, S. (2016). The role of flipped learning in managing the cognitive load of a threshold concept in physiology. Journal of Effective Teaching, 16(3), 28–43. Akkaraju, S. (2018). Handwriting to learn: Embedding a crucial study skill in a gateway science course. Journal of Effective Teaching in Higher Education, 1(1), 55–72. Nudging Students to Succeed 68 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 Bealer, J. and Bealer, V. (1996). Acting out immunity: A simulation of a complicating concept, American Biology Teacher, 58(6), 360–362. Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement. Eugene, OR: International Society for Technology in Education. Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. Arlington, VA: International Society for Technology in Education. Bjork, R.A. (1994). Institutional impediments to effective training. In R.A. Bjork and D. Druckman (Eds.), Learning, remembering, believing: Enhancing human performance (pp. 295–307). Washington, DC: National Academy Press. Bronx Community College of CUNY (2019). Self-study for the Middle States Commission on Higher Education, February 2019, p.ii. Retrieved from http://www.bcc.cuny.edu/wp-content/uploads/2019/02/bcc-msche-self- study-2-7-19-1.pdf Carmean, C., & Frankfort, J. (2013, December 6). Mobility, connection, support: nudging learners to better results. EDUCAUSE Review Online. Retrieved from https://er.educause.edu/articles/2013/12/mobility-connection-support- nudging-learners-to-better-results Castleman, B. & Page, L.C. (2015). Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low- income high school graduates? Journal of Economic Behavior and Organization, 115(C), 144–160. Graham, S., Harris, K.R., & Reid, R. (1992). Developing self-regulated learners. Focus on Exceptional Children, 24, 1–16. Harris, K., Friedlander, B. D., Saddler, B., Frizzelle, R., & Graham, S. (2005). Self- monitoring of attention versus self-monitoring of academic performance: Effects among students with ADHD in the general education classroom. Journal of Special Education, 39(3), 145–156. Laal, M. & Ghodsi, S.M. (2012). Benefits of collaborative learning, Procedia – Social and Behavioral Sciences, 31(2012), 486–490. McMillan, J. H., & Hearn, J. (2008). Student self-assessment: The key to stronger student motivation and higher achievement. Educational Horizons, 87(1), 40–49. Moss, C.M. & Brookhart, S.M. (2009). Advancing formative assessment in every classroom: A guide for instructional leaders. Alexandria, VA: ASCD. Nudging Students to Succeed 69 Journal of Effective Teaching in Higher Education, vol. 2, no. 2 Narasimhan, R. (2016, August 16). Small nudges=Large impact on Student Success [Blog post]. Retrieved from https://www.pearsoned.com/large-impact- student-success/ Nicol, D.J. & Marcfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. Remind. (2014). Remind school communication [Mobile application software]. Retrieved from http://Remind.com Schunk, D. H. (2003). Self-efficacy for reading and writing: Influence of modeling, goal setting, and self-evaluation. Reading & Writing Quarterly, 19(2), 159– 172. Simkin, M.G. (2013). Playing jeopardy in the classroom: An empirical study. Journal of Information Systems Education, 24(6), 203–210. Sunstein, C. (2014). Nudging: A very short guide. 37 J. Consumer Pol'y, 583, 1–7. http://dx.doi.org/10.2139/ssrn.2499658 Warnock, S. (2004). Quizzes boost comprehension, Confidence. The Teaching Professor, 5. Zimmerman, B. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64–70. Zimmerman, B., & Campillo, M. (2003). Motivating self-regulated problem solvers. In J. Davidson & R. Sternberg (Eds.), The Psychology of Problem Solving (pp. 233–262). Cambridge: Cambridge University Press. doi:10.1017/CBO9780511615771.009