International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 17, 2021 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 https://doi.org/10.3991/ijim.v15i17.23579 Zeynep Akcay Ozkan1(*), Sirin Budak2 1City University, New York, USA 2University of Wisconsin, Wisconsin, USA zakcayozkan@qcc.cuny.edu Abstract—This study is aimed towards understanding students’ preferences in online learning of College Algebra at a community college in the United States, before and after the Covid-19 pandemic. We analyze the results of Stu- dent Satisfaction surveys administered during Fall 19, Spring 20 and Fall 20 semesters. The classes were partially or fully online with synchronous and asyn- chronous components such as face-to-face and/or virtual meetings, pre-recorded video lessons, assignments through online homework platforms, and discussion board assignments. Our analysis show that students found face-to-face meetings most helpful, when classes were taught partially online. After the transition to online teaching, virtual meetings did not directly replace the face-to-face meet- ings, instead online homework platforms were preferred. We find that students ultimately started utilizing all components of online classes on an equal basis. Our results reflect students’ adjustment to online learning, taking more advantage from asynchronous course components and becoming active learners over time. Keywords—online math education, student satisfaction, online course components, virtual classes, student preferences, Covid-19 effects, math instruction, educational technologies 1 Introduction Technologies and pedagogies used in online mathematics education have been evalu- ated widely for their benefits and challenges from early childhood [1], [2] to higher educa- tion [3]–[5]. The result of integrating technology into mathematics instruction was overall positive. Studies done in blended algebra classes in high schools [6] and in developmental math courses at community colleges [7] showed that students could benefit from blended learning environments. However, some studies show significant differences in success rates between face-to-face courses and online/blended courses [8]. In a study of develop- mental algebra courses at a community college, grades and completion rates were lower for students in online or blended courses compared to students in face-to-face classes [9]. This reminds us that successful delivery of an online/blended math class depends on choosing the appropriate tools that would fit the student body and the course level that is being taught. This motivated us to survey students in partially online College Algebra classes in 4 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i17.23579 mailto:zakcayozkan@qcc.cuny.edu Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 Fall 19 to learn about their preferences, what components of instruction helped them learn the course material, to be able to revise the course instruction in the following semesters. When the Covid-19 pandemic has been accepted as a threat to public health, most (if not all) schools worldwide were forced to switch to online education in the mid- dle of Spring 20 semester. The advantages and challenges with this rapid transition have been explored widely from students’ and instructors’ perspectives [10], [11] in general, and specifically for mathematics instruction [12]–[16]. Institutions responded swiftly to this transition by offering different modalities of instruction including fully online (synchronous and/or asynchronous), hybrid (partially face-to-face and partially online) or mixed solutions (face-to-face instruction that allows students to join online). Instructors and students have expressed dissatisfaction with this transition and prefer- ence with face-to-face learning [16]. Difficulties with delivering the course content to students virtually have been brought up [15]. This motivated us to explore how students responded to this mandatory change, whether their preferences have changed from pre- to post-pandemic conditions. For this purpose, we administered surveys in Spring and Fall 20 semesters that were similar to the one which was given in Fall 19 semester. 1.1 Literature review on components of online/blended mathematics courses The College Algebra courses in which our students were surveyed, consisted of three asynchronous components in addition to the synchronous lectures (face-to-face or virtual). These are pre-recorded video lessons, online homework (online HW) plat- forms and discussion boards. Here we provide a literature review on each component. Benefits earned from video lessons have been discussed in many studies [17]–[20]. According to Bergmann and Sams [17], video lessons were helpful to students with dis- abilities and struggling students in mastering the course materials. It was also stated that it increased student-teacher interaction. Due to its flexibility, use of video lessons were considered as a great fit for non-traditional students who work with busy schedules. Barkley [18] mentioned that video lessons helped increase the motivation of at-risk stu- dents to persist. In a study, students enrolled in classes which are video based believed that they learned more and progressed further towards their learning goals compared to students in non-video based, traditional classes [19]. Ichinose and Clinkenbeard [20] also revealed that students taking college algebra in video based classes performed better on examinations compared to the students who were in non-video based classes. The use of online HW platforms is very common in college algebra classrooms. Stud- ies revealed both benefits and drawbacks of using online HW platforms. One of the com- monly mentioned benefits of using an online HW platform is that it provides immediate and individualized feedback to students which mostly enables them to correct their mis- takes, another benefit is the ability of the platform to create new versions of questions which encourages students to practice a problem further to improve their understanding [21]–[23]. The ability of learning the homework grade immediately and having a chance to correct their mistakes by receiving hints are two important aspects that increase stu- dents’ appreciation [24], [25]. In a recent study, students were found to have positive attitudes toward the use of online HW platforms and they were motivated to use different features of them [26]. In addition to these benefits, the randomized exercises (each student receiving different version of a problem) was argued to reduce the plagiarism cases [27]. Although there are benefits, the literature also mentions the disadvantages of using online HW platforms. Since these platforms allow multiple submissions, students might iJIM ‒ Vol. 15, No. 17, 2021 5 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 find the correct answer by trial-and-error submissions [28]. This whole process could create an emphasis on getting the correct answer instead of focusing on the process of solution [27], [29]. Magalhaes et al. conducted a meta-analysis (31 research studies since 2010) and investigated the effects on students’ performance of traditional paper-pencil homework in comparison with homework assigned through online HW platforms [24]. These studies differed in terms of their methods and instruments such as the duration of intervention, course domain and approaches to implement the online homework. Half of the studies did not show any difference between the traditional and online HW assign- ments in terms of student performance. Nine of them reported better performance in online HW assignments and one study showed traditional homework resulted in better student performance compared to online homework. Six studies mentioned mixed results. Another common practice in online courses is making use of discussion boards to improve the interaction between learner and instructor, learner and learner and learner and content. The study of Hong and Jacob [30] reported the benefits of using discus- sion board on critical thinking in a first mathematics course for engineering students. Another study conducted by Sliva [31] focused on the community building aspect of discussion boards. Baldwin and Sabry [32] stated the importance of discussion boards for increasing student engagement. Thomas and Thorpe’s study added that students and facilitators have been crucial regarding the structure of the discussion and the quality of the interaction and content [33]. Another important characteristic of productive discus- sion boards is the facilitator’s presence in the discussion [34]. In order to get maximum benefits from discussion boards and learn in online environment, an effective social, teaching and cognitive presence of facilitator is required [35], [36]. 2 Methods 2.1 Participants The surveys were administered in partially and fully online College Algebra classes at an urban community college in the United States. College Algebra is an entry level course with relatively low (around 45%) passing rates. The students enrolled in the PI’s and her two collaborators’ College Algebra sections were asked to participate in these surveys which were administered through the courses’ the learning management sys- tems (LMS), Blackboard. Total number of participants were n = 18 in Fall 19, n = 32 in Spring 20 and n = 47 in Fall 20. 2.2 Components of the courses Here we provide the details of the course components that were surveyed. Class meetings held either face-to-face or virtually were the only synchronous components of the classes. All other components were asynchronous. Face-to-face/Virtual Meetings: Instructors used meeting times for lecturing, answer- ing students’ questions, observing students’ progress and giving them feedback. Virtual meetings were held through Blackboard Collaborate. Video Lessons: Pre-recorded videos by the own instructors of the classes or others were provided to students for topics that were not covered during the face-to-face or virtual meetings through Youtube or Playposit sites. 6 http://www.i-jim.org Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 Online Homework Platforms: The online HW platforms used for the classes were MyOpenMath [37] or WeBWorK [38]. Instructors assigned weekly homework to stu- dents through these platforms. Discussion Board Review Assignments: These assignments were posted on Black- board’s discussion board before each exam and aimed at collaboratively preparing stu- dents to the upcoming exam. Sets of review questions were posted and students either chose or were assigned problems to work on and discuss the solutions between each other. Students were required to comment on each other’s posts. 2.3 Course components in each semester During Fall 19 semester, the survey was administered in classes that were taught partially online. The classes met on campus half of the time and rest of the materials were delivered asynchronously through pre-recorded video lessons. Instruction was supported by practice problems on online HW platforms and collaborative exam review assignments on discussion board. Therefore, the classes were composed of four main components; face-to-face meetings, practice on online HW platforms, pre-recorded video lessons and discussion board review assignments. First half of the Spring 20 semester were taught exactly in the same format as the Fall 19 semester, in partially online modality. However, after the college’s shutdown due to Covid-19, the classes switched to being fully online. All the synchronous com- ponents of the classes stayed the same, and the only change was that virtual meet- ings replaced face-to-face meetings. The surveys administered at the end of Spring 20 semester asked the students to evaluate the following four components of the course; face-to-face meetings, virtual meetings, practice on online HW platforms and discus- sion board review assignments (video lessons were not surveyed in this semester). Fall 20 semester was held fully online in the same format as the second half of the Spring 20 semester. The only difference with the Fall 19 semester was that the virtual meetings replaced the face-to-face meetings. Therefore, the classes were again com- posed of four main components; virtual meetings, practice on online HW platforms, pre-recorded video lessons and discussion board review assignments. Course compo- nents in each semester are summarized in Table 1. Table 1. Course modalities and components during the semesters in which the surveys were administered Fall 19 Spring 20 Fall 20 First Half Second Half Modality Partially Online Fully Online Content Delivery 50% Synchronous: Face-to-face meetings 50% Synchronous: Virtual meetings 50% Asynchronous: Video Lessons 50% Asynchronous: Video Lessons Assignments Online HW Platforms: Weekly Discussion Board Reviews: Before exams (4–5 times) 2.4 Surveys Based on the instruction modality being partially or fully online, versions of the same survey about satisfaction in course components were administered at the end of iJIM ‒ Vol. 15, No. 17, 2021 7 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 Fall 19, Spring 20 and Fall 20 semesters. The surveys included 5-point Likert scaled questions and an open-ended question. The answer choices ranged from Strongly Dis- agree to Strongly Agree with the score of each choice ranging from 1 to 5. All the survey questions were reviewed and approved by the Institutional Review Board and Likert scale questions are given in Table 2. Student satisfaction for a component was measured by collecting student responses in four categories. These categories are Help in learning (H), Willingness to use again (A), Ease of use (E) and Completion/participation (C) for the given course component. Ques- tions from Category H were included in each semester for each course component that was used during that semester. Based on the previous experience of the PI’s, increasing the number of questions in the surveys would significantly decrease the participation and completion rate of the surveys. Therefore, questions from other categories were included at least once during the course of the experiment but not included during all semesters. Questions from Category E, Category C and Category A were included in Fall 19, Spring 20 and Fall 20 semesters, respectively. Questions from these categories were included to support the data obtained from Category H questions. In this way, surveys in each semes- ter included questions from at least two categories for each of the course components that were used during that semester. Correlation analysis and Cronbach’s alpha calcu- lated for different categories supported the validity of the surveys during each semester. This design allowed collecting sufficient data to make comparisons for different compo- nents during a given semester and for a given component between different semesters, while keeping the number of questions was limited to ten for each semester. The only exception is the video lessons component for which only Category H ques- tion was asked during Fall 19 and no questions were asked during Spring 20 semester. Since transition to online teaching occurred during Spring 20 semester, students experi- enced both face-to-face and virtual meetings and questions from both components were included in the survey. For the sake of keeping the number of questions at a limit of ten, questions for the video lesson were eliminated. 3 Results The means and standard deviations for the survey questions are given in Table 2. We first compare the course components during a given semester. We then analyze how the mean response to a specific component changed over time. We performed the statistical analysis using Microsoft Excel. We used sample stan- dard deviation function to calculate the standard deviation and two sample t-tests assuming unequal variance to compare two data sets the results. We set the significance level to 0.05 in our results. 3.1 Ranking of course components in Fall 19 For each of the four components used in Fall 19, students were provided statements of the form “Given component helped me learn the material” and asked to choose their levels of agreement (H1, H2, H3 and H5). The means of responses in decreasing order were μ = 4.67 for face-to-face meetings, μ = 4.22 for discussion board assignments, μ = 4.11 for video lessons and μ = 4.06 for online HW platforms. Each of the four components were above 4, which show ‘agreement’ on average with all the statements. 8 http://www.i-jim.org Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 Ta bl e 2. Q ue st io ns in th e E nd o f S em es te r S ur ve y, to ge th er w ith m ea ns an d st an da rd d ev ia tio ns to th ei r r es po ns es C at eg or y Q St at em en t F al l 1 9 (n = 1 8) Sp ri ng 2 0 (n = 3 2) F al l 2 0 (n = 4 7) M ea n St d D ev M ea n St d D ev M ea n St d D ev H el p in le ar ni ng H 1 M yO pe nM at h/ W eB W or K a ss ig nm en ts h el pe d m e le ar n th e m at er ia l. 4. 06 1. 21 4. 19 0. 98 4. 49 0. 91 H 2 D is cu ss io n bo ar d re vi ew a ss ig nm en ts h el pe d m e le ar n th e m at er ia l. 4. 22 0. 73 3. 97 0. 90 4. 23 1. 03 H 3 T he fa ce -t o- fa ce m ee tin gs h el pe d m e le ar n th e m at er ia l. 4. 67 0. 59 4. 32 0. 75 – – H 4 T he v ir tu al c la ss es th ro ug h B la ck bo ar d C ol la bo ra te h el pe d m e le ar n th e m at er ia l. – – 3. 90 1. 11 4. 21 1. 06 H 5 V id eo le ss on s on Y ou tu be /P la yp os it he lp ed m e le ar n th e m at er ia l. 4. 11 0. 90 – – 4. 32 0. 91 W ill in gn es s to u se a ga in A 1 I w ou ld li ke to u se M yO pe nM at h/ W eB W or K fo r h om ew or k if I ta ke m at h cl as se s in th e fu tu re . 3. 94 1. 11 4. 27 1. 08 4. 58 0. 89 A 2 I w ou ld li ke to u se d is cu ss io n bo ar d if I ta ke m at h cl as se s in th e fu tu re . 3. 56 1. 20 3. 94 1. 21 4. 00 1. 08 A 3 I w ou ld li ke to a tte nd v ir tu al c la ss es if I ta ke fu rt he r o nl in e m at h cl as se s in th e fu tu re . – – 4. 04 1. 15 A 4 I w ou ld li ke to b e pr ov id ed v id eo le ss on s if I ta ke fu rt he r o nl in e m at h cl as se s in th e fu tu re . 4. 51 0. 75 E as e of u se E 1 It w as e as y to le ar n ho w to u se M yO pe nM at h/ W eB W or K . 4. 00 0. 69 – – E 2 It w as e as y to v ie w a nd a dd c om m en ts o n di sc us si on b oa rd . 4. 06 1. 06 C om pl et io n/ Pa rt ic ip at io n C 1 I c om pl et ed M yO pe nM at h/ W eB W or K a ss ig nm en ts o n tim e. – 4. 06 0. 93 – C 2 I c om pl et ed d is cu ss io n bo ar d as si gn m en ts o n tim e. 3. 91 0. 96 C 3 I a tte nd ed th e fa ce -t o- fa ce m ee tin gs b ef or e w e w en t f ul ly o nl in e. 4. 47 0. 68 C 4 I a tte nd ed th e vi rt ua l m ee tin gs th ro ug h B la ck bo ar d C ol la bo ra te . 4. 32 0. 83 iJIM ‒ Vol. 15, No. 17, 2021 9 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 However, the statement that “the face-to-face meetings helped learn the material” was agreed the most, which is significantly higher than the other three components (p-value ≤ 0.033). The standard deviation (σ = 0.59) was also the smallest for this ques- tion. There was no significant difference between the mean of the responses for the rest of the three components. We conclude that students benefited most from face-to-face meetings when the classes were taught partially online. For the two components on ‘practice on online HW platforms’ and ‘discussion board review assignments’ we also asked whether students would like to use them in their future math classes (A1 and A2). The agreement level to use online HW platforms again in the future was slightly higher (μ = 3.94) than the agreement level to use ‘dis- cussion board’ (μ = 3.56), but this difference wasn’t significant. Questions E1 and E2 assessed whether students found online HW platforms and discussion boards easy to use. Both of the components were found to be easy to use by the students, with average responses μ = 4.00 and μ = 4.06, respectively, and without any significant difference. 3.2 Ranking of course components in Spring 20 We first compare students’ agreement levels to the statements about whether a given component helped them to learn the material (H1, H2, H3 and H4). We find that the mean value of the agreement level was highest (μ = 4.32) for face-to-face meetings. This was followed by assignments on online HW platforms, with a mean of μ = 4.19. There was no significant difference between the agreement levels of the two compo- nents. The agreement level to the help of discussion board assignments (μ = 3.97) and virtual meetings (μ = 3.9) were also close, with no significant difference. The level of agreement for these two components were significantly lower with p-value = 0.047 and p-value = 0.043, respectively, than the agreement level for face-to-face meetings and not significantly different from the response to the help from assignments on online HW platforms. Therefore, we conclude that students benefited more from face-to-face meetings and assignments on online HW platforms than they benefited from virtual meetings and discussion board assignments. Questions C1 and C2 assessed whether students completed the online HW platform and discussion board assignments on time, respectively. Students mostly agreed that they completed both online HW platform (μ = 4.06) and discussion board (μ = 3.91) assignments. There was no significant difference between mean responses. Similarly, questions C3 and C4 assessed whether students attended face-to-face and virtual meet- ings, respectively. The means of responses were μ = 4.46 for face-to-face meetings, and μ = 4.32 for virtual meetings. There was no significant difference at the 0.05 level between mean attendance to face-to-face or virtual meetings. However, attendance to face-to-face meetings was significantly higher than completion of assignments on dis- cussion board (p-value = 0.005) and online HW platform (p-value = 0.029). In addition, attendance to virtual meetings was also significantly higher than completion of discus- sion board assignments (p-value = 0.035). We can rank the participation in these com- ponents from higher to lower as face-to-face meetings, virtual meetings, assignments on online HW platforms and discussion board assignments. 10 http://www.i-jim.org Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 Questions A1 and A2 asked whether students would like to use online HW platforms or discussion boards in their future math classes respectively. The agreement level to these components were not significantly different (μ = 4.27 and μ = 3.94, respectively). 3.3 Ranking of course components in Fall 20 When students were asked whether or not a given component helped them to learn the material (H1, H2, H4 and H5), the response to practice on online HW platforms had the highest mean of μ = 4.49, which was followed by video lessons μ = 4.32, discussion board review assignments μ = 4.23 and virtual meetings μ = 4.21. However, the differences were not significant at the 0.05 level. Since the responses showed agreement for each compo- nent, we conclude that students benefited from all four components on an equal bases. The results were slightly different when students were asked whether they would like to use a given component in their future math classes (A1, A2, A3 and A4). We found that the mean responses to practice on online HW platforms (μ = 4.58) and video lessons (μ = 4.51) were significantly higher (p-value ≤ 0.012) than the mean responses to discussion board assignments (μ = 4.00) and virtual meetings (μ = 4.04). Students agreed that they would like to use online HW platforms and video lessons more. 3.4 Combined results on each component When the responses to the question how much students benefited from online HW platforms (H1) compared, no significant differences are found from Fall 19 semester to Fall 20 semester. Students agreed at the end of each semester that they benefited from the platforms on an equal bases. However, when the question whether they would like to use them again in their future math classes (A1) are compared, an increase in the mean of responses is observed. The mean of responses to this questions significantly increased from μ = 3.94 in Fall 19 to μ = 4.58 in Fall 20 with p-value = 0.020. Students’ agreement level to whether they benefited from discussion board assign- ments (H2) or whether they would like to use them again in their future studies (A2) did not show any significant change over the three semesters. Similarly, their response to whether they benefited from virtual meetings (H4) did not change significantly from Spring 20 to Fall 20 semester (μ = 3.90 and μ = 4.21, respectively). And their response to whether they benefited from video lessons (H5) also stayed same on average from Fall 19 to Fall 20 (μ = 4.11 and μ = 4.32, respectively). When we compare the responses to Question H3 in Fall 19 and Spring 20 semes- ters, we see that the mean response decreased significantly from μ = 4.67 to μ = 4.32 with p-value = 0.042. We conclude that benefits of face-to-face meetings were more appreciated when the class was hybrid. After transition to fully online instruction, this appreciation level decreased. 3.5 Students’ comments The last question in the survey was an open-ended question which was stated as “Do you have any other comments or suggestions to improve how this course is taught?” iJIM ‒ Vol. 15, No. 17, 2021 11 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 Some of the responses included only appreciation of how the course was taught, some of the questions included positive comments together with some further suggestions and some of the comments included only suggestions or criticism. We categorize these questions as “full appreciation”, “positive with suggestions” and “suggestions and criticism” and provide the number and percentage of each type of responses in Table 3. Some students only responded with a “No”, “No comment”, or similar statements. We categorize them as “No response” and do not include them when finding the response rate in Table 3. The total response rate to the open-ended question has not changed significantly over the three semesters. Percentage of responses with suggestions including positive components and responses with only suggestions and criticism did not also show any significant difference over the three semesters. However, the responses that stated full appreciation increased significantly (p-value = 0.048) from 6% in Fall 19 to 19% in Fall 20. Table 3. Number and percentage of students’ categorized comments received in the End of Semester Surveys Fall 19 (n = 18) Spring 20 (n = 32) Fall 20 (n = 47) n % n % n % Full appreciation 1 6% 3 9% 9 19% Positive with suggestions 1 6% 2 6% 2 4% Suggestions and criticism 4 22% 5 16% 8 17% Total responses 6 33% 10 31% 19 40% The suggestions received from students mostly varied and did not focus on a specific component. The general themes were suggestions to how the meetings (face-to-face or virtual) were utilized, the amount of tutorials, examples or resources provided or other rules of the classes. We share here students’ comments on similar themes that appeared more than once during the three semesters. In Fall 19 semester, we received two comments that emphasized the benefit of face- to-face meetings. These are “Lecture in class is way much better, but in some cases the videos online help” and “Topics that aren’t covered in class and were only given through lecture videos should be addressed in class at a later time”. Both comments showed students’ preference in learning materials during face-to-face meetings. In Spring 20 semester, there were two similar comments that focused on the assign- ments through online HW platforms. These were “I learned a lot through MyOpen- Math” and “I think MyOpenMath videos under each question helped me a lot more than discussion board”. Both comments showed students benefited from the online HW platforms. Lastly, in Fall 20 semester, the two comments that were on similar themes were about the recordings of the virtual class meetings. Students commented that the virtual meet- ings should be recorded and the recordings should be provided to students-although 12 http://www.i-jim.org Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 they were being recorded and shared with students. One take home message from these comments is the direct conclusion that students would like to refer to the recording afterwards. And the second and less explicit conclusion is that instructors should make sure that students are aware of the available resources and are confident in using the LMS for assessing virtual class recordings and other resources. 4 Conclusions and discussion When the classes were taught partially online, and face-to-face meetings were part of the course structure (Fall 19), students preferred this component over the other (online, asynchronous) components. They stated that they benefited most from face- to-face meetings. When we look at their preferences during Spring 20, we see that they benefited more from face-to-face meetings and online HW platforms compared to other components including virtual meetings. Although student participation in face-to-face and virtual meetings did not differ, they stated more benefit from face-to-face meetings than virtual meetings. The benefit they received from the online HW platforms was also more than the benefit of virtual classes. A common practice that has been used in all levels of education since the transition to online education due to Covid-19 was to offer virtual meetings instead of face-to- face meetings. The idea was to deliver the course content during virtual meetings and the expectation was that virtual meetings would replace face-to-face meetings. How- ever, we found that the virtual meetings did not help as much as face-to-face meetings. Instead, assignments through online HW platforms were preferred among students compared to virtual meetings. We found that students expressed equal benefit from all components of the course in Fall 20, while stating that they would like to use online HW platforms and video lessons more in their future math classes. Comparing this finding with the students’ preferences in Fall 19 (face-to-face meetings) and Spring 20 (face-to-face meetings and online HW platforms) we understand that students preferences changed over time, in a way that reflects their adjustment to online teaching, as they start utilizing the asynchro- nous components more and taking advantage from all components of the course on an equal basis. This also reflects a shift in students’ preference from instructor-led learning through face-to-face meetings to self-paced, independent learning through asynchro- nous components, such as online HW platforms. A reasonable inference drawn from these results may be that students have learnt how to utilize all components of online courses in the absence of face-to-face meet- ings. Students’ habits and preferences changed with the effect of Covid-19. With the increased use of video lessons, virtual meetings and discussion boards, students might get used to know how to get maximum benefits from those resources and eventually they might become more independent learners. When we look at how much students benefited from online HW platforms during the three semesters, we see no difference over time. However, we see a significant increase in their willingness to use them again in their future classes. We can conclude that students started taking more advantage from online HW platforms, which made them prefer it in their future courses. On the other hand, their response to the benefits of face-to-face meetings decreased significantly from Fall 19 to Spring 20. Students stated iJIM ‒ Vol. 15, No. 17, 2021 13 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 less benefit from face-to-face meetings. This could be due to student’s preference in the environments where they can be active learners. Although instructors tried to replace face-to-face meetings with virtual meetings, they are more inclined to switch to direct lecturing during the latter [4]. Students could take more active roles during face-to-face meetings, but it may not be possible to achieve this during virtual ones. Students could be more active learners compared to their roles in virtual meetings, when they use online HW platforms since they get feedback from the system after they solve a problem. The conclusions we infer are also supported by students’ comments. We found sig- nificant increase in students’ comments that showed full appreciation of how the course was taught, from Fall 19 to Fall 20. This reflects students’ adjustment in online learning. When only half of the course was taught online, students preferred to learn the subject matter during face-to-face meetings and did not take full advantage of the online asyn- chronous components of the course. However, in Fall 20, when the courses were taught fully online, students’ expectations changed and they grew to be more responsible, independent learners, which was reflected in comments that showed full appreciation of the course. Common theme of comments received during Fall 19 and Spring 20 also reflected the shift in students’ preferences, with face-to-face meetings and online HW platforms being mentioned for their benefits in these semesters, respectively. In this paper, our data and focus were on students’ preferences for different compo- nents of an entry level mathematics course. We do not suggest that any specific com- ponent is superior to another component of the course. Instead, our belief is that online teaching should include as many variety of components as possible. The shift which was observed in our students’ preferences from face-to-face learning to learning from multiple components is a positive sign that students are learning to take full advantage of components of online courses. 5 Limitations and future implications The limitation with the findings we obtained from Spring 20 semester could be that the transition to online teaching was unplanned for both students and instructors, and happened in the middle of the semester. This is different than the Fall 19 and Fall 20 semesters, where students completed the semesters in the format that was promised to them in the beginning of the semester. Since this was a mandatory change, and not a preference, students’ satisfaction might have been affected from it. This study analyzed the short-term effects of the transition to online teaching due to Covid-19 pandemic. Our results indicated that students were able to utilize all com- ponents of online instruction during the last semester of the study. More data has to be collected in the future semesters to understand the long-term effects of this change. 6 Acknowledgements We thank the City University of New York for supporting this research project under the PSC-CUNY TRADA-51-111 Grant. We thank Dr. Dona Boccio for her contribu- tions in creating the surveys, administering them and providing feedback on the manu- script. We also thank Dr. Ewa Stelmach for administering the surveys. 14 http://www.i-jim.org Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 7 References [1] S. Papadakis, M. Kalogiannakis, and N. Zaranis, “The effectiveness of computer and tablet assisted intervention in early childhood students’ understanding of numbers. An empiri- cal study conducted in Greece,” Educ. Inf. Technol., vol. 23, no. 5, pp. 1849–1871, 2018. https://doi.org/10.1007/s10639-018-9693-7 [2] S. Papadakis, M. Kalogiannakis, and N. Zaranis, “Teaching mathematics with mobile devices and the Realistic Mathematical Education (RME) approach in kindergarten,” Adv. Mob. Learn. Educ. Res., vol. 1, no. 1, pp. 5–18, 2021. https://doi.org/10.25082/ AMLER.2021.01.002 [3] M. C. Borba, P. Askar, J. Engelbrecht, G. Gadanidis, S. Llinares, and M. S. Aguilar, “Blended learning, e-learning and mobile learning in mathematics education,” ZDM – Math. Educ., vol. 48, no. 5, pp. 589–610, 2016, https://doi.org/10.1007/s11858-016-0798-4. [4] J. Howard and J. Beyers, Eds., Teaching and Learning Mathematics Online. Chapman and Hall/CRC, 2020. https://doi.org/10.1201/9781351245586 [5] A. S. Drigas and M. A. Pappas, “A review of mobile learning applications for mathematics,” Int. J. Interact. Mob. Technol., vol. 9, no. 3, pp. 18–23, 2015, https://doi.org/10.3991/ ijim.v9i3.4420. [6] J. G. Smith and S. Suzuki, “Embedded blended learning within an Algebra classroom: A multimedia capture experiment,” J. Comput. Assist. Learn., 2015, https://doi.org/10.1111/ jcal.12083. [7] Maggie P. Fay, “Computer-Mediated Developmental Math Courses in Tennessee High Schools and Community Colleges,” CCRG 91, 2017. [8] D. Sauers and R. C. Walker, “A comparison of traditional and technology-assisted instructional methods in the business communication classroom,” Bus. Commun. Q., 2004, https://doi.org/10.1177/1080569904271030. [9] J. Ashby, W. A. Sadera, and S. W. McNary, “Comparing student success between develop- mental math courses offered online, blended, and face-to-face,” J. Interact. Online Learn., 2011. [10] R. Firmansyah et al., “Educational Transformation : An Evaluation of Online Learn- ing Due to COVID-19,” Int. J. Emerg. Technol. Learn., vol. 16, no. 07, pp. 61–76, 2020. https://doi.org/10.3991/ijet.v16i07.21201 [11] N. M. Almusharraf and S. H. Khahro, “Students’ Satisfaction with Online Learning Experiences during the COVID-19 Pandemic,” Int. J. Emerg. Technol. Learn., vol. 15, no. 21, pp. 246–267, 2020, https://doi.org/10.3991/ijet.v15i21.15647. [12] A. C. Carius, “Teaching Practices in Mathematics During COVID-19 Pandemic: Challenges for Technological Inclusion in a Rural Brazilian School,” Am. Sci. Res. J. Eng. Technol. Sci., 2020. [13] A. Khirwadkar, S. Ibrahim Khan, J. Mgombelo, S. Ratkovic, and W. Forbes, “Reimag- ining Mathematics Education During the COVID-19 Pandemic,” Brock Educ. J., 2020, https://doi.org/10.26522/brocked.v29i2.839. [14] E. M. Mulenga and J. M. Marbán, “Is covid-19 the gateway for digital learning in mathemat- ics education?,” Contemp. Educ. Technol., 2020, https://doi.org/10.30935/cedtech/7949. [15] Sabaruddin, Marzuki, and Khairunnisak, “Pandemic Covid-19: The Opportunities and Chal- lenges to Using ICT in Mathematics Learning,” IJEBD Int. J. Entrep. Bus. Dev. eISSN 2597-4785 pISSN 2597-4750, 2020. [16] A. J. Berkova and R. Nemec, “Teaching theory of probability and statistics during the covid- 19 emergency,” Symmetry (Basel)., 2020, https://doi.org/10.3390/SYM12091577. iJIM ‒ Vol. 15, No. 17, 2021 15 https://doi.org/10.1007/s10639-018-9693-7 https://doi.org/10.25082/AMLER.2021.01.002 https://doi.org/10.25082/AMLER.2021.01.002 https://doi.org/10.1007/s11858-016-0798-4 https://doi.org/10.1201/9781351245586 https://doi.org/10.3991/ijim.v9i3.4420 https://doi.org/10.3991/ijim.v9i3.4420 https://doi.org/10.1111/jcal.12083 https://doi.org/10.1111/jcal.12083 https://doi.org/10.1177/1080569904271030 https://doi.org/10.3991/ijet.v16i07.21201 https://doi.org/10.3991/ijet.v15i21.15647 https://doi.org/10.26522/brocked.v29i2.839 https://doi.org/10.30935/cedtech/7949 https://doi.org/10.3390/SYM12091577 Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 [17] J. Bergmann and A. Sams, “Flip Your Classroom Reach Every Student in Every Class Every Day,” Get Abstract Compressed Knowledge. 2014. [18] A. Barkley, “Flipping the College Classroom for Enhanced Student Learning,” NACTA J., 2015. [19] W. Swart and K. L. Wuensch, “Flipping Quantitative Classes: A Triple Win,” Decis. Sci. J. Innov. Educ., 2016, https://doi.org/10.1111/dsji.12088. [20] C. Ichinose and J. Clinkenbeard, “Flipping College Algebra: Effects on Student Engagement and Achievement.,” Learn. Assist. Rev., 2016. [21] C. Halcrow and G. Dunnigan, “Online Homework in Calculus I: Friend or Foe?,” PRIMUS, 2012, https://doi.org/10.1080/10511970.2012.694015. [22] R. P. Rodriguez and L. M. Smith, “Using an online homework management system in tax accounting: Does it advance learning?,” 2016, https://doi.org/10.1504/IJLT.2016.076062. [23] M. L. Lunsford and M. Pendergrass, “Making Online Homework Work,” PRIMUS, 2016, https://doi.org/10.1080/10511970.2015.1110219. [24] P. Magalhães, D. Ferreira, J. Cunha, and P. Rosário, “Online vs traditional homework: A systematic review on the benefits to students’ performance,” Comput. Educ., 2020, https://doi.org/10.1016/j.compedu.2020.103869. [25] J. L. Davis and T. N. Mcdonald, “Online, Handwritten or Hybrid Homework: What’s best for our students in the long run?,” J. Online Eng. Educ., vol. 6, no. 2, pp. 4–7, 2016. [26] D. Serhan, “Web-Based Homework Systems: Students’ Perceptions of Course Interac- tion and Learning in Mathematics,” Int. J. Soc. Educ. Sci., 2019, https://doi.org/10.46328/ ijonses.18. [27] S. Bonham, R. Beichner, and D. Deardorff, “Online homework: Does it make a difference?,” Phys. Teach., 2001, https://doi.org/10.1119/1.1375468. [28] G. Kortemeyer, “An empirical study of the effect of granting multiple tries for online home- work,” Am. J. Phys., 2015, https://doi.org/10.1119/1.4922256. [29] M. Mendicino, L. Razzaq, and N. T. Heffernan, “A comparison of traditional homework to computer-supported homework,” J. Res. Technol. Educ., 2009, https://doi.org/10.1080/ 15391523.2009.10782534. [30] K. Hong and S. M. Jacob, “Critical Thinking and Socratic Questioning in Asynchronous Math- ematics Discussion Forums,” Malaysian J. Educ. Technol., vol. 12, no. 3, pp. 17–26, 2012. [31] J. Sliva, “Developing a Mathematical Community Using an Electronic Discussion Forum in an Elementary Mathematics Methods Course,” Contemp. Issues Technol. Teach. Educ., vol. 2, no. 1, pp. 81–96, 2002, [Online]. Available: http://www.editlib.org/p/14517. [32] K. Sabry and L. Baldwin, “Web-based learning interaction and learning styles,” Br. J. Educ. Technol., 2003, https://doi.org/10.1111/1467-8535.00341. [33] G. Thomas and S. Thorpe, “Enhancing the facilitation of online groups in higher education: a review of the literature on face-to-face and online group-facilitation,” Interact. Learn. Environ., 2019, https://doi.org/10.1080/10494820.2018.1451897. [34] P. Shea and T. Bidjerano, “Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learn- ing environments,” Comput. Educ., 2010, https://doi.org/10.1016/j.compedu.2010.07.017. [35] D. E. Bair and M. Bair, “Paradoxes of Online Teaching,” Int. J. Scholarsh. Teach. Learn., 2011, https://doi.org/10.20429/ijsotl.2011.050210. [36] S. Dole and L. Bloom, “Online Course Design: A Case Study,” Int. J. Scholarsh. Teach. Learn., 2009, https://doi.org/10.20429/ijsotl.2009.030111. [37] “MyOpenMath,” 2014. https://www.myopenmath.com/info/aboutus.php. [38] “WeBWorK,” 1994. https://webwork.maa.org/intro.html. 16 http://www.i-jim.org https://doi.org/10.1111/dsji.12088 https://doi.org/10.1080/10511970.2012.694015 https://doi.org/10.1504/IJLT.2016.076062 https://doi.org/10.1080/10511970.2015.1110219 https://doi.org/10.1016/j.compedu.2020.103869 https://doi.org/10.46328/ijonses.18 https://doi.org/10.46328/ijonses.18 https://doi.org/10.1119/1.1375468 https://doi.org/10.1119/1.4922256 https://doi.org/10.1080/15391523.2009.10782534 https://doi.org/10.1080/15391523.2009.10782534 http://www.editlib.org/p/14517 https://doi.org/10.1111/1467-8535.00341 https://doi.org/10.1080/10494820.2018.1451897 https://doi.org/10.1016/j.compedu.2010.07.017 https://doi.org/10.20429/ijsotl.2011.050210 https://doi.org/10.20429/ijsotl.2009.030111 https://www.myopenmath.com/info/aboutus.php https://webwork.maa.org/intro.html Paper—A Comparison of Students’ Preferences in Online Algebra Instruction Pre- and Post-Covid-19 8 Authors Zeynep Akcay Ozkan is an Assistant Professor of Mathematics at Queensborough Community College of the City University of New York. She received her PhD in Applied Mathematics from New Jersey Institute of Technology and Rutgers Universi- ties with concentration on Mathematical and Computational Neuroscience. Dr. Akcay Ozkan’s research interests include Online Teaching of Mathematics. She has completed several workshops on online teaching and mentors faculty members as they develop their online or partially online courses. She has served in the eLearning Committee of the College and Math Department’s Best Practices in Teaching and Learning Commit- tee in chair and secretary positions. Sirin Budak is an associate professor of mathematics education at the University of Wisconsin–Stevens Point. Her research area includes representations, visualization, and the effect of technology in learning. Article submitted 2021-04-26. Resubmitted 2021-06-24. Final acceptance 2021-06-25. Final version published as submitted by the authors. iJIM ‒ Vol. 15, No. 17, 2021 17