Carpenter, Y., & Sullivan, E. R. (2017). The role of interactive digital simulations in student conversations about visualizing molecules Papers on Postsecondary Learning and Teaching: Proceedings of the University of Calgary Conference on Learning and Teaching, 2, 16-23. THE ROLE OF INTERACTIVE DIGITAL SIMULATIONS IN STUDENT CONVERSATIONS ABOUT VISUALIZING MOLECULES Yuen-ying Carpenter and Erin Rae Sullivan University of Calgary The visualization of chemical compounds in three-dimensions is a foundational skill in the study and practice of chemistry and related fields, and one which has the potential to be supported by interaction with virtual models. Here, we present a collaborative learning activity piloted in first-year chemistry which investigates if inquiry-driven interactive technology can contribute meaningfully to student conversations around this topic, and how students’ conversations and practices may shift when driven by feedback from an interactive simulation. Our initial observations from this pilot project suggest that students engaged in collaborative sense-making and discussion around key ideas throughout this activity. Students’ post-activity reflections also highlighted their positive experiences and increased confidence with the topic afterwards. The unique dynamics of these interactions lead us to propose a novel framing of interactive visualizations as participants rather than merely as resources in student learning conversations. INTRODUCTION The rise of digital technologies in the classroom offers students and teachers new modes of interaction, affording, for example, new ways to access encyclopedic knowledge, iterate on design, or create collaborative artefacts. Digital simulations have particularly flourished in the design and learning of spatial visualization skills, offering students the opportunity to interact fluidly with virtual three-dimensional objects. In inquiry-based science courses, the dynamics of this interaction ideally drives students towards open exploration and sense-making practices around the visualization, and away from strictly task-oriented or answer-making efforts. To this end, this paper introduces a pilot project in first-year chemistry, where students learned about visualizing molecular shapes through a collaborative, digital, simulation-based activity. Reflecting on both the design features and impacts of simulation-based activities, we report here our initial impressions of the ways in which dynamic feedback from a simulation shaped student conversations. CONTEXT The students: Our target audience was the students enrolled in their first-semester of first-year undergraduate General Chemistry, a large-enrollment course with up to 660 students split across three lecture sections in a given semester. While all students have completed high school chemistry as a pre-requisite, they nonetheless enter this course with a broad range of incoming abilities and interests, and, while the majority plan to major in a scientific discipline, most are not chemistry majors. Carpenter & Sullivan 17 The challenge: Each semester, on informal mid-semester surveys and formal student ratings of instruction, we asked students what course topic(s) they found “the most difficult to understand”. Semester after semester, one of the most frequently identified areas of challenge was the drawing or interpretation of 3-D molecular structures. Yet, students cannot avoid the reality that molecules are 3-dimensional. The practice of visualizing chemical compounds “off- the-page” is foundational for predicting and understanding their reactions both in first-year chemistry and in upper division courses in organic or bio-chemistry. More broadly, spatial reasoning has been identified as a key skill set in many STEM disciplines, including chemistry (Harle & Towns, 2011). Critically, spatial ability is not fixed; rather, it develops over a person’s lifetime and can be improved through targeted interventions (Uttal et al., 2013). Since such improvements correlate with positive and enduring impacts on student retention and success in STEM courses (Sorby, 2009), we felt that an intervention targeting first-year students could provide in-roads to strengthening their capacity across their university careers. LEARNING ACTIVITY A novel opportunity to support student learning around this topic came in the summer of 2016. Our university’s Teaching and Learning Institute had launched a call for proposals from instructional faculty looking to pilot new pedagogical practices by bringing their courses into one of the Institute's newly-built flexible learning spaces. We already knew that recent studies (McCollum, Regier, Leong, Simpson, & Sterner, 2014; Stull, Gainer, Padalkar, & Hegarty, 2016) indicated some positive impacts of virtual models on individual students’ chemistry- specific 3-D visuospatial skills, but we saw these new learning spaces as an opportunity to expand on this theme and leverage student collaborative practices (Springer, Stanne, & Donovan, 1999) to better support their development in this area. Specifically, we were eager to use the learning spaces’ collection of large 50-inch touchscreen displays for group collaboration around an interactive simulation. In the context of a collaborative learning activity, we hoped that a larger interaction area would offer greater shared access to these virtual resources, since past studies had shown students’ discussions and conclusions could be directly impacted by their access to these visualization resources (Wu, Krajcik, & Soloway, 2001). With Erin’s experience coordinating and teaching this general chemistry course at this institution over the past several years and Yuen-ying’s experience in interactive simulation design at the PhET Interactive Simulations project at the University of Colorado in Boulder, we wondered: given our distinct expertise, could we collaborate to develop an activity which leveraged these new touchscreens to help students develop their spatial reasoning and ability to rotate 3-D molecules? And so, after a successful proposal for use of one of these flexible learning studios, we designed and implemented a collaborative group activity for Fall 2016 around the PhET Molecule Shapes simulation (“sim”), available for free online at http://phet.colorado.edu. Activity structure: To maximize opportunities for students to interact with the touchscreens and with the sim, we designed the activity to be implemented during one of the bi- weekly course tutorial sessions, where roughly 100 students and 3 facilitators (course instructors and/or graduate teaching assistants) could collaborate in the learning space at a given time. Unlike the previous tutorial on the topic, which relied on students building upon coverage of this topic from lecture, the new activity was designed with a discovery-learning approach and there was no lecture coverage of this topic prior to the tutorial activity. Instead, prior to the tutorial date, students were asked to individually complete a short out-of-class activity with three key tasks: Carpenter & Sullivan 18 1. To self-report their recollection and confidence with this topic based on prior exposure in high school; 2. To test their baseline ability to mentally rotate objects in 3-D space, using selected assessment questions from a standardized instrument (Bodner & Guay, 1997); and, 3. To play with the PhET Molecule Shapes sim and, given a series of open-ended prompts, draw some initial conclusions that they could share with their peers during the tutorial. During the tutorial, students worked in self-selected groups of four, with every two groups sharing a single touchscreen that displayed two side-by-side web browser windows open to the Molecule Shapes sim (Figure 1). Figure 1: Room setup for tutorial sessions, arranged to accommodate two groups of 4 students per touchscreen. Limited availability of touchscreens meant that student groups took turns manipulating the sim, using two distinct sim windows to allow the groups to progress independently of each other; notably, we did not observe any significant conflicts or delays in group dialogue as a consequence of this screen sharing. The 1-hour tutorial activity asked students to use the sim and/or provided molecular model kits (http://www.molymod.com/) to answer a series of scaffolded questions around three key learning goals: 1. Molecular shapes and their features: To name molecular geometries and identify their key features, including bond angles. 2. Multiple representations: To draw and translate between multiple representations, including disciplinary drawings, and both physical and virtual 3D models. 3. Mental rotation and perspective: To draw and recognize realistic molecular geometries regardless of their orientation in space. Design features: The activity was designed around key principles previously suggested to best facilitate student orientation towards sense-making (rather than task-completion or answer-making). We leveraged the sim design itself towards this goal (Lancaster, Moore, Parson, Carpenter & Sullivan 19 & Perkins, 2013), as the controls were designed to be not only interactive, but intuitive, to facilitate students’ ownership of the sim without the need for explicit instructor direction or intervention. While the sim allowed students to create pedagogically significant molecules that would be impossible in the real-world (and difficult or impossible to create with their concrete physical models), the sim also provided productive constraints on their exploration which focused their attention on task- or topic-relevant manipulations (Barrett, Stull, Hsu, & Hegarty, 2015). While the pre-tutorial activity was designed to encourage student ownership by prompting open play (Podolefsky, Rehn, & Perkins, 2013), the setup of screens and tables during the tutorial itself was chosen to optimize multi-student access to the screen, allowing equal student ownership of the available resources. Activity scaffolding and prompts were designed to offer students only moderate guidance (Chamberlain, Lancaster, Parson, & Perkins, 2014), to encourage students to explore more sim features and feedback while still guiding them towards key questions in their conversations around this traditionally challenging topic. Assessments: Following the collaborative learning activity, students completed a 20- minute multiple-choice collaborative quiz, using the same resources that had been available to them during the tutorial activity itself (sim, model kit, tutorial notes). We chose to use Immediate Feedback Assessment Technique (IF-AT®) scratch cards for this in-class assessment, so that groups received instant feedback on the correctness of their chosen response, and could, as a group, use this formative feedback to readdress the question for partial credit (Merrel, Cirillo, Schwartz, & Webb, 2015). As a follow-up assessment, students were given one week to complete an out-of-class post-tutorial reflection online. Students were assessed on their proficiency at each learning goal with a single multiple choice question and asked, in each case, to rate their confidence on that question and learning goal. After receiving feedback on their performance on these assessment questions, students were asked to select the learning goals for which they felt the most and least confident overall. RESULTS AND REFLECTIONS In this preliminary analysis, we focused on assessing this tutorial activity in terms of student attitudes, as measured by their post-activity reflections and course evaluations, and student behaviours, as reported through facilitator observations during the activity. Student reflections: Student feedback on the tutorial activity was overwhelmingly positive, exemplified by comments on the anonymous end-of-semester instructional rating surveys such as the feedback from this student: The technology and space definitely helped, as I have difficulty thinking in 3D and it was only after using things such as Phet simulations and model kits that I truly understood what was going on. Group discussion was also helpful, as others would ask questions I didn’t know I also was confused about. This student, like many of their peers, not only highlighted the benefits of using both virtual and concrete models but also reflected positively on the collaborative learning environment. Likewise, in anonymous end-of-semester tutorial evaluations, 83% of students responded favourably when asked to rate whether their experience with this tutorial activity supported their learning, using a 4-point Likert scale (Figure 2). Carpenter & Sullivan 20 Figure 2: Student ratings on tutorial evaluations (n = 605, response rate = 94%). In terms of student learning, we recognize that the majority of students had some exposure to 3-D molecular shapes and their features (learning goal #1) during their early high school curriculum (Alberta Education, 2014), and some even self-identified as having sufficiently strong understanding to teach the basics of this topic to another student (Figure 3, 18% ‘Very Strong’). Figure 3: Students' self-reported recollection of this topic prior to the tutorial activity (n = 366 consenting students) Yet, even students who self-identified with a high level of incoming confidence felt they learned from the experience. In post-tutorial reflections, the overwhelming majority (97%) of students’ 45% 38% 11% 3% 3% Strongly Agree Agree Disagree Strongly Disagree N/A 0% 10% 20% 30% 40% 50% I found being in the new [learning space] with the touch screen technology supported my learning 18% 29% 40% 7% 7% Very strong Moderately strong Limited to bits and pieces Non-existent Uncertain 0% 10% 20% 30% 40% 50% Before the tutorial, students ranked their understanding of 3-D molecular structures as... Carpenter & Sullivan 21 (n = 342 consenting individuals) described specific material which they felt they learned or extended during the activity. We recognize that this feedback reflects not only students’ perceptions of the usefulness of the activity, but also their updated understanding of the performance expectations in this area. Indeed, while one student initially rated their own understanding as very strong prior to the tutorial, they subsequently commented that the activity itself was “hard”; in contrast, a different student, who identified as only being able to recall limited information on the topic prior to the tutorial, commented in their post-reflection that the activity had been primarily “a refresher course.” Nonetheless, nearly all students were able to concretely identify areas where the tutorial had a perceived positive impact on their learning. Facilitator observations: During the week of the activity, every one of the seven tutorial sections were filled with lively student group discussions. As an instructor, it was uplifting to float through the room and see students engaged with the topic and with sharing their understanding of what they saw being represented in the sim. One common behaviour we observed among student groups was a tendency towards turn-taking; we rarely witnessed a single student assume sole control of the touchscreen. Instead, students fluidly traded control of the sim; often, multiple students would approach the sim together and engage in back-and-forth dialogue, using the sim for feedback. We saw some student groups engage in heated debate over their conclusions; to our delight, it was rare for students to involve the instructors and facilitators in a call to authority. Instead, debates were more often resolved by further engaging with the simulation. We contend that the availability of a shared visualization facilitated better communication among students, since they are still novices in their use of the disciplinary language and representations (Kozma, 2000). More importantly, the dynamic feedback students received from the sim allowed them to iteratively test and modify their collective understanding of the topic. Seeing student discussions move forward as a result of sim interactions significantly shifted our perspective on the role of an interactive visualization in these collaborative activities. The sim was not merely an encyclopedic source of content, nor a visual aid that students used to convey their own messages; rather, the sim seemed to be a dynamic participant in the group’s conversations, providing momentum when groups might have otherwise been stuck and needing a facilitator’s help. It was, in fact, so rare for students to ask the facilitators a question that the facilitating graduate student teaching assistants complained that they were, “really bored; [since students] are not asking questions,” and that “it seem[ed] that [students] are all figuring things out and we [the facilitators] are not needed.” It was particularly surprising that students asked fewer questions of the facilitators, when the material had not yet been introduced in lecture prior to this tutorial, as it had been during previous semesters. In conjunction with these discussions, we also observed students increased use of gesture toward and around the simulation. In particular, we saw examples of groups aligning and orienting their physical molecular models with the corresponding virtual representations in the simulation, using both representations to help them complete representational and drawing tasks. Since our attention was not primarily directed towards answering student queries as it had been in previous semesters, we had substantially more opportunities to attend to student gestures and sense-making process (Flood et al., 2015), which allowed lecture instructors to adapt the content of the following lectures in response to these observations. Carpenter & Sullivan 22 CONCLUSIONS AND FUTURE WORK Student collaborative engagement with an interactive simulation shows promise for supporting the development of molecular visualization skills. The majority of students in this pilot study were enthusiastic about the learning activity, as demonstrated both by their engagement in-class and their post-activity comments. To further improve the effectiveness of this pilot tutorial, we have planned future iterations of both the activity scaffolding and post- tutorial reflection prompts, and we intend to begin to assess student learning more directly through specifically-targeted questions on their summative exams. To help us better understand the process by which students develop and refine their visualization competencies, we plan to collect screen capture and audio-recording of future student groups during their collaborative work. Finally, recognizing the limited availability of large touchscreens in even modern, technology-enabled classrooms, we intend to expand our study to investigate how both screen- size and student ownership may play a role in how groups interact, by studying a future semester wherein the activity is completed solely on student-owned devices. More broadly, we would argue that the described patterns of collaborative interaction around a simulation or dynamic visualization offers an opportunity to reframe interactive visualizations as participants in student learning conversations. We particularly encourage other instructors and instructional designers to leverage this framing in their brainstorming and design of new sim-based activities. In focusing attention on creating a conversation between an interactive tool and the student(s), we open up new ways to turn virtual resources into catalysts for sense-making. ACKNOWLEDGMENTS We are grateful to the Taylor Institute Learning Spaces committee for this opportunity, which could not have been imagined without the use of both their space and technologies. This project would likewise not have been possible without the support of the Taylor Institute staff, especially Isadora Mok-Kulakova, D’Arcy Norman, Brian Pshyk, and Robin Mueller. Finally, we appreciate the Department of Chemistry, as well as the other instructors and graduate teaching assistants in this course, for their unwavering adaptability in the face of sometimes untested waters. REFERENCES Alberta Education. (2014). CHEMISTRY 20–30 Program of Studies 2007 (Updated 2014). Retrieved from https://education.alberta.ca/media/3069388/pos_chem_20_30.pdf Barrett, T. J., Stull, A. T., Hsu, T. M., & Hegarty, M. (2015). Constrained interactivity for relating multiple representations in science: When virtual is better than real. Computers & Education, 81, 69–81. Bodner, G. M., & Guay, R. B. (1997). The Purdue Visualization of Rotations Test. Chemical Educator, 2(4), 1–17. Chamberlain, J. M., Lancaster, K., Parson, R., & Perkins, K. K. (2014). How guidance affects student engagement with an interactive simulation. Chemical Education Research and Practice, 15(4), 628–638. Flood, V. J., Amar, F. G., Nemirovsky, R., Harrer, B. W., Bruce, M. R. M., & Wittmann, M. C. (2015). Paying Attention to Gesture when Students Talk Chemistry: Interactional https://education.alberta.ca/media/3069388/pos_chem_20_30.pdf http://www.sciencedirect.com/science/article/pii/S0360131514002097 http://www.sciencedirect.com/science/article/pii/S0360131514002097 https://link.springer.com/article/10.1007/s00897970138a http://pubs.rsc.org/en/Content/ArticleLanding/2014/RP/C4RP00009A#!divAbstract http://pubs.rsc.org/en/Content/ArticleLanding/2014/RP/C4RP00009A#!divAbstract http://pubs.acs.org/doi/abs/10.1021/ed400477b Carpenter & Sullivan 23 Resources for Responsive Teaching. Journal of Chemical Education, 92(1), 11–22. Harle, M., & Towns, M. (2011). A Review of Spatial Ability Literature, Its Connection to Chemistry, and Implications for Instruction. Journal of Chemical Education, 88(3), 351– 360. Kozma, R. B. (2000). The Use of Multiple Representations and the Social Construction of Understanding in Chemistry. In M. Jacobson & R. Kozma (Eds.), Innovations in science and mathematics education: Advanced designs for technologies of learning (pp. 11–46). Mahwah, NJ: Erlbaum. Lancaster, K., Moore, E. B., Parson, R., & Perkins, K. K. (2013). Insights from Using PhET’s Design Principles for Interactive Chemistry Simulations. In M. J. S. Jerry P. Suits (Ed.), Pedagogic Roles of Animations and Simulations in Chemistry Courses (Vol. 1142, pp. 97–126). Washington, DC: American Chemical Society. McCollum, B. M., Regier, L., Leong, J., Simpson, S., & Sterner, S. (2014). The Effects of Using Touch-Screen Devices on Students’ Molecular Visualization and Representational Competence Skills. Journal of Chemical Education, 91(11), 1810–1817. Merrel, J. D., Cirillo, P. F., Schwartz, P. M., & Webb, J. A. (2015). Multiple-Choice Testing Using Immediate Feedback—Assessment Technique (IF AT®) Forms: Second-Chance Guessing vs. Second-Chance Learning? Higher Education Studies, 5(5), 50. Podolefsky, N. S., Rehn, D., & Perkins, K. K. (2013). Affordances of play for student agency and student-centered pedagogy. In 2012 PHYSICS EDUCATION RESEARCH CONFERENCE (Vol. 1513, pp. 306–309). AIP Publishing. Sorby, S. A. (2009). Educational Research in Developing 3‐ D Spatial Skills for Engineering Students. International Journal of Science Education, 31(3), 459–480. Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of Small-Group Learning on Undergraduates in Science, Mathematics, Engineering, and Technology: A Meta- Analysis. Review of Educational Research, 69(1), 21–51. Stull, A. T., Gainer, M., Padalkar, S., & Hegarty, M. (2016). Promoting Representational Competence with Molecular Models in Organic Chemistry. Journal of Chemical Education, 93(6), 994–1001. Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., & Newcombe, N. S. (2013). The malleability of spatial skills: a meta-analysis of training studies. Psychological Bulletin, 139(2), 352–402. Wu, H.-K., Krajcik, J. S., & Soloway, E. (2001). Promoting understanding of chemical representations: Students’ use of a visualization tool in the classroom. Journal of Research in Science Teaching, 38(7), 821–842. http://pubs.acs.org/doi/abs/10.1021/ed400477b http://pubs.acs.org/doi/abs/10.1021/ed900003n http://pubs.acs.org/doi/abs/10.1021/ed900003n http://pubs.acs.org/doi/abs/10.1021/ed400674v http://pubs.acs.org/doi/abs/10.1021/ed400674v http://pubs.acs.org/doi/abs/10.1021/ed400674v http://www.ccsenet.org/journal/index.php/hes/article/view/51152 http://www.ccsenet.org/journal/index.php/hes/article/view/51152 http://www.ccsenet.org/journal/index.php/hes/article/view/51152 http://aip.scitation.org/doi/10.1063/1.4789713 http://aip.scitation.org/doi/10.1063/1.4789713 http://www.tandfonline.com/doi/abs/10.1080/09500690802595839 http://www.tandfonline.com/doi/abs/10.1080/09500690802595839 http://journals.sagepub.com/doi/abs/10.3102/00346543069001021 http://journals.sagepub.com/doi/abs/10.3102/00346543069001021 http://journals.sagepub.com/doi/abs/10.3102/00346543069001021 http://pubs.acs.org/doi/abs/10.1021/acs.jchemed.6b00194 http://pubs.acs.org/doi/abs/10.1021/acs.jchemed.6b00194 http://psycnet.apa.org/?&fa=main.doiLanding&doi=10.1037/a0028446 http://onlinelibrary.wiley.com/doi/10.1002/tea.1033/abstract?systemMessage=Wiley+Online+Library+will+be+unavailable+on+Saturday+01st+July+from+03.00-09.00+EDT+and+on+Sunday+2nd+July+03.00-06.00+EDT+for+essential+maintenance.++Apologies+for+the+inc http://onlinelibrary.wiley.com/doi/10.1002/tea.1033/abstract?systemMessage=Wiley+Online+Library+will+be+unavailable+on+Saturday+01st+July+from+03.00-09.00+EDT+and+on+Sunday+2nd+July+03.00-06.00+EDT+for+essential+maintenance.++Apologies+for+the+inc INTRODUCTION CONTEXT LEARNING ACTIVITY RESULTS AND REFLECTIONS CONCLUSIONS AND FUTURE WORK ACKNOWLEDGMENTS