Microsoft Word - IEJEE_5_6_Zertuche International Electronic Journal of Elementary Education, 2012, 5(1), 79-92. ISSN:1307-9298 Copyright © IEJEE www.iejee.com How do Openers Contribute to Student Learning? ∗∗∗∗ Amber ZERTUCHE ∗∗∗∗∗∗∗∗ University of California, Berkeley, United States of America Libby GERARD University of California, Berkeley, United States of America Marcia C. LINN University of California, Berkeley, United States of America Received: 20 August 2012 / Revised: 24 October 2012 / Accepted: 24 October 2012 Abstract Openers, or brief activities that initiate a class, routinely take up classroom time each day yet little is known about how to design these activities so they contribute to student learning. This study uses technology-enhanced learning environments to explore new opportunities to transform Openers from potentially busy work to knowledge generating activities. This study compares the impact of teacher-designed Openers, Opener designs based on recent research emphasizing knowledge integration, and no Opener for an 8th grade technology-enhanced inquiry science investigation. Results suggest that students who participate in a researcher-designed Opener are more likely to revisit and refine their work, and to make significant learning gains, than students who do not participate in an Opener. Students make the greatest gains when they revisit key evidence in the technology-enhanced curriculum unit prior to revision. Engaging students in processes that promote knowledge integration during the Opener motivate students to revise their ideas. The results suggest design principles for Openers in technology-enhanced instruction. Keywords: Technology, Science Education, Teaching Practices, Classroom Assessment, Formative Assessment ∗ This article is based on the first author’s unpublished MA thesis entitled “How Teacher-led Openers Contribute to Student Learning in a Web-based Inquiry Science Environment.” This material is based upon work supported, in part, by the National Science Foundation under Grant DRL-0733299 [Logging Opportunities in Online Programs for Science (LOOPS)]. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. ∗∗ Amber Zertuche, Graduate School of Education, Tolman Hall, University of California, Berkeley, U.S.A. Phone: (415) 656-5840. E-mail: anzertuche@berkeley.edu International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 80 Introduction Openers, or brief activities that initiate a class, routinely take up classroom time each day yet little is known about how to design these activities so they contribute to student learning. Teachers often give mini-lectures to remind students of what they studied in the previous class. Or, teachers may assign a short writing assignment. Technology-enhanced learning environments provide teachers with new tools to transform Openers from busy work to valuable learning activities. Openers can engage students in knowledge integration activities such as making predictions, critiquing a claim, assessing peer essays, or reflecting on progress—all activities that have been shown to improve student learning (Chiu and Linn, 2011; Linn and Eylon, 2011; White and Frederiksen, 1998). This study compares the impact of teacher- and researcher-designed Openers to no Opener in an 8th grade technology- enhanced inquiry science investigation. Questions include: How effective are Openers? How does the design of the Opener contribute to student learning? And, what are effective post- Opener revision processes? Teacher-led Openers can play an essential role in the success of technology-enhanced inquiry science instruction. Several studies show that the quality of teacher implementation of technology-enhanced instruction predicts the impact of the technology on learning (Tamin, Bernard, Borokhovski, Abrami, and Schmid, 2011). In a second-order meta-analysis conducted on technology-enhanced instruction over the past 40 years, the authors conclude that the teachers’ ability to monitor student understanding and help students sort out their ideas is essential to successful technology implementation (Williams, Linn, M, Ammon, and Gearhart, 2004). This meta-analysis found that as teachers asked more knowledge oriented questions during the course of a technology-enhanced science investigation as opposed to procedural questions, learning gains improved significantly. Further, to realize the potential of technology-enhanced instruction, teachers need to help students critically analyze visualizations relative to the conceptual learning goal as students often overestimate their understanding of dynamic computer visualizations, particularly in Chemistry (Chiu and Linn, 2012). Technology-enhanced learning environments provide teachers with unique tools to structure effective Openers. The computer stores and organizes a record of each student’s work including their multiple revisions, and provides ways to make examples of the student’s work public to the whole class. This means teachers can purposefully select examples of the student’s work for class discussion. Effective selection of examples could prompt students (and their teachers) to monitor understanding, sort out ideas about difficult concepts, and revisit and refine their reasoning (Izsak, 2012; Linn and Eylon, 2011). Using Student Work to Design Openers Student work is collected in technology enhanced learning environments and can be used to design openers. Black and Wiliam (1998) reviewed hundreds of studies and came to the conclusion that the use of student work for formative assessment and refinement of instruction is one of the most powerful ways to increase students’ learning gains, particularly among low achieving students. Similarly, Shute (2008) reviewed work on formative feedback, defined as information communicated to the learner that is intended to modify his or her thinking or behavior to improve learning, and found that effective feedback is non- evaluative, supportive, timely, and specific. Gerard, Spitulnik, and Linn (2011) showed that when teachers use student work to refine their teaching, students’ learning benefits. These studies distinguish between activities providing verification, where students choose whether an answer is correct or not, and elaboration, where teachers provide cues to guide learners toward a correct answer. Gerard et al found that use of student work to design instruction is How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn 81 most effective when the resulting instruction provides advice on what to do to improve performance, rather than on comparisons to other students or on accuracy of responses. One example of a successful use of student work is for self- and peer-assessment. Research suggests that students can benefit from self and peer assessment activities when they recognize the desired learning goal, have adequate evidence to determine where the work stands relative to the learning goal, and have an understanding of a way to close the gap between the two (Black and Wiliam, 1998). Under these conditions, students can distinguish between their own ideas and the goal of instruction and strengthen their understanding. Taken together, these results suggest that effective Openers should elicit student work and provide hints or guidance. What are the challenges of formative assessment? Black and Wiliam’s review of formative assessment studies point out that, although formative assessment has proven to be useful to student learning, there are concerns about how to enact successful formative assessment (1998). One issue is that in order to implement formative assessment in a useful way, teachers must have easy access to evidence of students’ ideas and efficient routines to elicit students’ ideas during class so they can help students to distinguish among these ideas. This calls for assessment or discussion questions that prompt students to make their reasoning explicit and provide multiple entry points for students with various levels of understanding. Students must also be actively involved in the feedback they are receiving in order for it to have an effect on their learning. This requires significant changes in a traditional secondary science classroom. Traditional science classroom routines often center on teacher-directed lectures and demonstrations, leaving little space for students to reflect on their understanding and sort out the variety of ideas they hold about the topic gathered from everyday experiences, peers, and school curricula. Ruiz-Primo and Furtak (2007), for instance found that teachers’ formative assessment routines focused on procedural elements of inquiry learning, learning such as checking the students’ knowledge of the correct procedure and asking them to apply a procedure to a new situation, rather than knowledge generation. How to Design an Effective Opener in Chemistry? This study investigates the design of knowledge integration Openers. Typical classroom Openers usually reflect the absorption model of instruction and use the question, response, evaluation (QRE) approach. In this model, the teacher asks a question with one answer in mind and prompts until a student gives the answer or fills in the response if none is elicited. This theory of learning assumes that the task of the learner is to acquire the body of connections that an expert analysis of the subject matter reveals (Greeno, Collins, and Resnick, 1996). In contrast, the knowledge integration perspective on learning resonates with the Black and Wiliam’s findings (2011) and guides the design of the Openers, curriculum and assessments in this study. The knowledge integration perspective draws on findings from learning sciences research. Specifically, learners hold multiple conflicting ideas about scientific phenomenon as has been documented in numerous studies of student intuitions about science topics (diSessa, 2000). In addition, learners, often in collaboration with others, can deliberately sort out, link, and critique their ideas when making sense of new scientific phenomena and benefit from encouragement to engage in this process (Linn, Lee, Tinker, Husic, and Chiu, 2006; Linn and Hsi, 2000; Novak and Gowin, 1984; Slotta, Chi, and Joram, 1995). This means that providing opportunities for students to compare alternative ideas to their own, develop criteria for sorting-out and distinguishing among ideas, and reflect on their ideas can help them form coherent hypotheses or explanations. International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 82 The knowledge integration Openers in this study were designed to help students make connections among their ideas about chemical reactions. In chemistry, one of the most difficult things for students to learn is how chemicals react. Students often have difficulties translating between symbolic representations, molecular representations, and observable phenomena (Ardac and Akaygun, 2004). Particularly, students struggle to make sense of chemical phenomena at the molecular level (Johnstone, 1993; Krajcik, 1991). For example, many students think of chemical reactions as an instantaneous process without bond breaking and formation, while others think all the molecules break into atoms. In addition, prior studies demonstrate that students often isolate molecular visualizations rather than linking them to existing knowledge or everyday experiences and have difficulty interpreting stand-alone dynamic visualizations (Tversky, Morrison, and Betrancourt, 2002; Zhang and Linn, 2011). This study addresses this gap in learning chemical reactions with Openers that ask students to reflect upon and critique peers’ visual molecular representations of hydrogen and oxygen combustion. Methods Research Design This study investigates how a researcher-designed Opener, teacher-designed Opener, and a control condition (no Opener) contribute to students’ revision of work and understanding of chemical reactions. Three central questions guide this research: 1. Do Openers contribute to student understanding of chemical reactions? 2. How does the Opener design influence students’ learning outcomes? 3. What are effective post-Opener revision processes? Curriculum and Assessments The Web-based Inquiry Science Environment (WISE) is an open-source on-line learning environment that includes multiple standards-aligned science inquiry curriculum units. To engage students in knowledge integration processes, WISE projects guide students in collaborative activities with visualizations of scientific phenomena that are difficult to observe, such as molecular views of chemical reactions (Figure 1). Students investigate hypotheses, design solutions to problems, critique scientific claims, and build scientific models, scaffolded by guidance based on knowledge integration principles. Students in this study worked on the WISE Hydrogen Fuel Cell Car unit. This is a one week unit designed to teach students about chemical reactions, alternative fuels, and energy (http://wise.berkeley.edu/webapp/vle/preview.html?projectId=911). The project begins by asking students if they would rather buy a hydrogen or gasoline powered car. It also elicits their ideas about energy and adds ideas about conservation of energy. Then, gasoline combustion in cars is explored including the relationship between carbon dioxide, a product of gasoline combustion, and temperature changes over the last 200 years. Students then create their own energy story about cars. This story includes where the energy came from to power the car and any chemical reactions that are involved in their story. Hydrogen combustion is then explored using a dynamic visualization of hydrogen combustion (Figure 1). Then students are taught about the difference between exothermic and endothermic reactions and finally, students investigate a visualization of a hydrogen fuel cell to learn how this technology works. Students are then asked which kind of technology they would prefer when buying a car. How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn 83 Assessments are embedded throughout the WISE projects to help students and teachers monitor student understanding and progress as students interact with visualizations (Figure 2). The embedded assessments ask students to make predictions about the visualizations, sort out evidence, and link ideas together to explain their thinking. Students can also get hints to help them complete the tasks. Figure 1. One of the visualizations in WISE’s Hydrogen Fuel Cell Cars project models the hydrogen combustion reaction. Students can run, stop, reset, and ‘spark’ to start the reaction and explore the nature of chemical reactions. In the Hydrogen Fuel Cell Cars unit, an embedded assessment immediately follows the visualization of hydrogen combustion asking students to draw four frames of hydrogen combustion (Figure 2). This is meant to help students make sense of the dynamic visualization, recognizing features such as conservation of mass, bonds breaking, and the progression of the reaction. The Openers in this study focused on student work from this embedded assessment since in previous years this particular task was particularly challenging to students and yet still, understanding how to draw basic hydrogen combustion is critical to student understanding of chemical reactions. Figure 2. The WISE project screen has an inquiry map on the left, a navigation bar on top, and an embedded assessment in the center of the screen. International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 84 Teacher Assessment Tools WISE provides teachers convenient access to a record of student work as students progress through a WISE unit (Figure 3). The WISE grading tools allow teachers to view embedded assessment data by step, and the flag tool allows teachers to select key student work examples for display. Students can view flagged work at any time by clicking on a tab at the top of their WISE screen. This allows students to actively see their peers' work and be a part of the feedback process to improve their understanding. Students can revise their work based on teachers’ and peers’ comments. All revisions are logged in the grading tool so the teacher can measure the impact of their Opener, or comments, by viewing the change in students’ ideas from their original to revised work. Figure 3. WISE assessment tool with revision number on the left, including time stamp, and checkbox to flag work on the right. Study Participants Two 8th grade teachers and their 236 students from 1 public school participated in this study. The school has medium diversity (22% receiving free lunch, 5% ELL, and 27% non- white). Both teachers had over five years experience teaching with WISE and frequently used Openers in their regular and WISE instruction. Students were randomly assigned by class period to one of the three conditions. There were 78 students in the teacher-designed Opener, 128 students in the researcher-designed Opener, and 30 students in the control group condition. The uneven sample sizes were due to the uneven number of class periods that each teacher taught Physical Science that year. Opener Design The researcher-designed Opener engaged students in Knowledge Integration processes, as shown in Table 1. Activities included a small group discussion, voting, a whole group discussion, and then a closing summary by the teacher. The examples were selected to illustrate the range of conceptual errors in student representations of the chemical reaction. These related to (a) conservation of mass, (b) breaking of bonds, and (c) progression of the reaction. Revision Number Time Stamp Flag Work Checkbox How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn 85 The teacher-designed Opener alternatively focused on picking out what is good and bad about the student work and a lecture to try to re-teach chemical reactions from a different angle. Examples of student work were selected to illustrate responses that could be described as, “the good, the bad and the ugly”. Table 1. Description of Openers used in Teacher 1 and Teacher 2’s classroom using the Knowledge Integration Framework. Teacher-Designed Researcher-Designed Teacher 1 Elicit Ideas: Question on the board about how a balanced equation obeys the law of conservation of mass. Students fill out a chart on how much mass (amu) exists before and after reaction. Add Ideas: Students use physical model of molecules to break bonds and put back together. Teacher shows multiple types of reactions. Distinguish Ideas: Teacher shows four examples of student work and asks students if the example is good or bad. Integrate Ideas: ____ (~12 min) Students open WISE project to view four examples of student work in WISE project by clicking on the “Flagged Work” button. Each example has one unique link missing (Figure 4) Elicit Ideas: Teacher asks students to write down which drawing best represents the visualization of hydrogen combustion and use evidence to explain why. Students vote and teacher tallies votes. Add Ideas: Students discuss their choices in groups of 4 and revisit evidence in the visualization. Distinguish Ideas: Students reconsider their initial choice in light of their discussion with peers and revisit the evidence. Make a new vote. Integrate Ideas: Teacher tallies new votes and asks students to justify their choice. Teacher synthesizes criteria used to evaluate drawings, and instructs students to revise their own drawing. (~20 minutes) Teacher 2 Elicit Ideas: ____ Add Ideas: Used embedded assessment problem and a tree to house analogy to take students through the chemical reaction steps. Tree must break into parts, then recombine parts and build a house. Uses physical models of hydrogen and oxygen molecules to show students progression necessary to make water. Distinguish Ideas: ____ Integrate Ideas: ____ (~8 min) Figure 4. Examples of student work shown during the researcher-designed Opener. Link missing: conservation of mass Link missing: bonds breaking International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 86 Data Collection and Analysis Data sources include: the original student work on the embedded assessment, the revised student work on the embedded assessment after the Opener, WISE log files illustrating student navigation through WISE immediately after the Opener, pre and post tests administered immediately before and a 1-14 days after the WISE project, classroom video of teacher implementation of the researcher-designed and teacher-designed Openers, and teacher interviews. Knowledge integration rubrics were used to score the embedded and pre/post assessments. The embedded assessment rubric is shown in Table 2. Table 2. KI rubric for embedded assessment where students make step-by-step drawings of hydrogen combusting with oxygen KI Level Score Characteristics Example Invalid 1 Blank or I don’t know No links 2 No normative ideas conveyed but work has been done Simple, 1 link 3 Represents conservation of mass OR bonds breaking OR progression* Advanced, 2 links 4 Conservation of mass AND progression* OR Conservation of mass AND bonds breaking OR Progression* AND bonds breaking Complex, 3 links 5 Conservation of mass AND bonds breaking AND progression* of reaction * Progression must be apparent on all 3 transitions Pre and post questions were also scored using a KI rubric. The main ideas that students should understand in the pre- and post- test are that there is a reaction process that occurs and that H2 and Cl2 should start breaking bonds before forming new bonds. Results We examine the effects of Openers on student learning outcomes, and then explicate the contributing factors including Opener design and students’ learning practices. We focus on embedded assessments and pretest-posttest performance. We consider the actual implementation of the conditions and student performance as reflected in the log files. How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn 87 Embedded assessments To investigate the impact of the Opener versus no Opener on student understanding we compared students’ original (before Opener) and revised (after Opener) responses to the embedded assessments. Responses were scored using the knowledge integration rubric. Both teacher and researcher-designed Openers (n=105 pairs) were compared to the no Opener condition (n=30 pairs). Pairs who did not complete the embedded assessment before the Opener were excluded from the analysis (n=26 pairs). Although the aim was to facilitate the Opener after 75% of students completed the embedded assessment, this number was based on an automated progress screen that only monitored whether or not students submitted work at least once for this assessment. Once the researchers looked at the student’s work, it was obvious that, although work was submitted, many students did not finish their drawings and therefore these were not included in the data analysis. Responses were scored using the KI rubric. Time stamps from the WISE log files were used to identify students’ final work immediately before the Opener and their revised work on the day of the Opener. The analysis suggests that students who had an Opener, either teacher- or researcher- designed, made substantially greater learning gains than students who did not have an Opener. As shown in Table 4, students who had an Opener (M = .29, SD = .78) doubled the mean gain score of those students who did not have an Opener (M = .13, SD = .51) on the embedded assessment. There was no significant difference between conditions in students’ pre-Opener scores. Students who had an Opener (M = .68, SD = .48) were significantly more likely to revisit and revise their work than students who did not have an Opener (M = .33, SD = .48), t(131) = -3.5, p < 0.001. In the Opener condition, revision was twice as likely as in the no Opener condition. Table 3. Pre to Post Gain Scores with and without Opener. N Pairs Who Revised Their Work Mean Gain (SD) Rate of Revision (N Pairs who Revised/Total Pairs) Opener 105 .29(.78) 66% No Opener 30 .13(.51) 33% The Openers were particularly effective for students who demonstrated at least a basic understanding on the embedded assessment prior to the Opener (Table 7). Having an Opener had a significant effect on students who began with at least a partial understanding, or level 3 on the knowledge integration scoring rubric (M = .71, SD = .69), t(26) = -2.07, p = .05. Students who began with partial understanding developed their ideas into a basic understanding (level 4) after the Opener. In contrast, students who began with partial understanding and had no Opener continued to demonstrate only partial understanding after revising their work (M = .18, SD = .60). The large effect of the Opener on level 3 students is partially due to the examples of student work selected for critique in the researcher and teacher-designed Openers. The selected examples that were shown during the Opener illustrated characteristics of level 3 understanding, making them most accessible to this population of students. Since student work was chosen this way, this Opener was unwittingly designed to increase the learning gains of students with an already basic understanding of chemical reactions. Students with non-normative ideas, or level 2 understanding, made modest gains with or without an Opener, as shown in Figure 5. Level 2 students may need examples aligned with their own ideas to improve to partial or high level understanding. For instance, showing International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 88 student work with no conservation of mass or bond breaking would lead to a discussion where students point out that both are missing from the drawing. Alternatively, showing a common level 2 student work may show this population of students that their attempt is acknowledged and that they have direct feedback for improving their drawing. Pre Opener: No Links Present Post Opener: No Links Present Figure 5. Examples of students’ work with a level 2 understanding before and after an Opener. Table 6. Pre to Post Opener Gain Scores Distributed by Pre-Opener Scores Pre- Opener Score N Pairs Who Revised Their Work Mean Gain With Opener(SD) N Pairs who Revised Mean Gain Without Opener (SD) 2 10 .4(.70) 5 .4(.89) 3 17 .71(.69) 11 .18(.60)* 4 42 .21(.42) 10 0(0) *p=.05 Pre-test to Post-test effects. The pre- and post- tests were administered before students began the week long project and after they finished the project. In spite of the gains found for the specific item addressed by the Opener, there was no difference on the pre/post tests between students who had an Opener (M = 1.74, SD = 2.50) and those who did not (M = 1.55, SD = 2.16), t(255) = -.511, p = .61. This is not surprising since the Opener treatment was only 10-20 minutes out of the 5-7 hour long project. Perhaps several Openers over the project length would have been able to affect the pre test to post test results. Also, one of the teachers in the study did not give the post test until 2 weeks after the project was completed. Although the schedule was controlled, the teacher ultimately has the authority on when to give the post test. This delay may have masked any immediate effects of the Opener on the post test. Because of these time related anomalies, student performance on the embedded assessment is a better representation of the immediate effects of the Openers. How does the Opener design influence students’ learning outcomes? The Opener designs, described in Table 2, differed primarily in their support for distinguishing and integrating ideas. In both Openers, students were presented with evidence regarding a chemical reaction and prompted to think about conceptual features of a chemical reaction. The researcher-designed Opener had additional components to support integration of ideas. After considering the new evidence, students were guided to reconsider How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn 89 their initial views in light of the evidence presented, and refine their criteria for conceptual features of a chemical reaction. These additional components made the researcher- designed Opener longer (approximately 20 min) than the teacher-designed Opener (approximately 10 min). Since the length of both Openers was difficult to predict ahead of time, the students in the teacher-designed Opener did not have an additional task to account for the 10 minute different time on task. Students who participated in the researcher-designed Opener were significantly more likely to revise their work on the embedded assessment (M = .75, SD = .44) compared to students who participated in the teacher-designed Opener (M = .56, SD = .50), t(101) = -1.98, p < 0.05 (Table 8). There was no significant difference in learning gains between the two conditions (Table 9). The researcher-designed Opener was not completely implemented as planned. The voting process in the researcher-designed Opener was a new activity for students and did not work as anticipated. In some classes, students directly copied the examples of student work that received the most votes even though none of the student work examples were in fact “correct”. Although teachers reminded students that none of the student work examples were correct, this could have been emphasized and the teacher could have checked that students truly did understand that examples were not to be directly copied (have a student repeat it back, whole class response etc). Table 7. Student KI Score (Maximum Score 5) of Embedded Assessment Before and After the Opener Treatment n Revised Total Teams Avg Rate of Revision Pre Opener Post Opener KI Score Gain Teacher Designed Opener 22 39 .56(50) 3.88 4.26 0.38 Researcher Designed Opener 48 64 .75(.44) 3.59 3.95 0.36 Table 8. Student Pair KI Score (Out Of 5) for Pre and Post Test n Pre Test Post Test KI Score Gain Teacher Designed Opener 78 2.69 3.54 0.85 Researcher Designed Opener 121 2.58 3.30 0.72 Log files. To analyze post-Opener learning practices we looked at log files. Two students from each period, one with the highest gain and one with the lowest gain, were selected for log file analysis in order to get an equal distribution of student learning practices from each condition. Students’ learning practices may explain why some students made greater gains on the embedded assessment post Opener than others. The WISE log files show that the biggest contribution to learning gains was the time students spent revising their initial work, and revisiting relevant evidence in the WISE unit. The students who made the greatest improvement revisited an evidence page immediately after the Opener or spent more time revising their original work than other students (Table 10). An evidence page could be the dynamic visualization of hydrogen combustion or notes that students wrote about what a International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 90 good drawing of hydrogen combustion should include. This suggests the value of providing students with access to evidence to sort out their ideas during an Opener. Table 10. Comparison of Students Who Had a Gain in Their KI Score By At Least 1 and Those Who Had No Gain. After The Opener, Students With a Gain Either Spent More Time on the Embedded Assessment or Revisited an Evidence Page After the Opener. Percentage Results Are Aggregated for Each Condition. Learning Practice Gain (n=6) No Gain (n=6) Revisited evidence page 67% 33% Revisited embedded assessment for more than 1 minute 100% 50% Both 67% 33% Discussion This study illustrates how Openers can improve student learning compared to not using Openers. Both the teacher-designed and researcher-designed Openers used in this study encouraged students to review their ideas. The researcher-designed Openers encouraged students to distinguish ideas and reflect, and resulted in a greater propensity to revise answers than did the teacher-designed Openers. These results for Openers resonate with research showing the benefit of giving students feedback based on their responses to assignments (Black and Wiliam, 1998; Shute, 2008). Openers supported students to reflect upon their initial ideas, reconsider evidence, and refine their views. Openers were particularly useful for students who began with a partial understanding of a concept. When students benefitted from Openers, they took advantage of the evidence presented in the Opener, revisited a dynamic visualization or another evidence source, and reflected on the new information. The findings from this study suggest the following design principles for Openers: 1) Openers can reinforce normative conceptual ideas by drawing attention to the distinction between student ideas and normative views. Getting evidence from peers, visualization, teacher, or other classroom resource can help students understand complex ideas. This evidence helps students close the gap between what they know now and the normative view of the phenomena studied. 2) Openers help students when they occur soon after a topic is introduced and direct attention to specific ways to improve their ideas. 3) Openers succeed when they are non-evaluative and support students to explore evidence or views of their peers. For example, teachers can support students by giving them the opportunity to distinguish among the ideas held by the group of students in the class. Giving students a chance to appreciate conflicting views held by classmates and use evidence to sort them out helps students integrate their ideas. 4) Openers should be short. The Openers in this study were much longer and had more teacher involvement for most Openers. It may work better to have students review examples of student work as homework, then discuss with their partner at the very start of class. This would free the teacher up to take attendance and give more time for the peer and class discussion. • • • How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn 91 Amber ZERTUCHE. Received Masters degree and Single Subject Credential in Science and Math Education from University of California, Berkeley in 2012. She is a first year Physics teacher at Burton High School in San Francisco Unified School District since 2012. Before becoming an educator, she helped engineer the James Webb Space Telescope’s primary mirrors. Libby GERARD. Postdoctoral fellow with the Technology-Enhanced Learning in Science center in the Graduate School of Education at the University of California, Berkeley. She completed her PhD from Mills College in 2008. She studies the relationship between professional development, technology- enhanced instruction and student knowledge integration in inquiry science. Libby is particularly interested in how to use technology-enhanced assessment data to support teacher and principal decision-making to improve inquiry instruction and student learning. Libby leads the TELS professional development activities including summer institutes, sustained inquiry based communities, and teacher mentoring. Marcia LINN. Professor, Graduate School of Education, University of California, Berkeley. Mbr: National Academy of Education. Fellow: American Association for the Advancement of Science, the American Psychological Association. Elected: Chair of the AAAS Education Section, President of the International Society of the Learning Sciences. Awards: National Association for Research in Science Teaching Award for Lifelong Distinguished Contributions to Science Education, Council of Scientific Society Presidents first award for Excellence in Educational Research. References Ardac, D. & Akaygun, S. (2004). Effectiveness of multimedia-based instruction that emphasizes representations on students’ understanding of chemical change. Journal of Research in ScienceTeaching, 41(4), 317-337. Berland, L.K. & Reiser, B.J. (2011). Classroom communities’ adaptations of the practice of scientific argumentation. Science Education, 95(2), 191–216). Black, P. & Wiliam, D. (1998). Inside the black box: raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148. Chiu, J. & Linn, M. (2012). The Role of Self-monitoring in Learning Chemistry with Dynamic Visualizations. In Metacognition in Science Education (pp. 133–163). Dordrecht: Springer. diSessa, A. (2000). Changing minds: Computers, learning and literacy. Cambridge, MA: MIT Press. Eylon, B. & Linn, M. (1988). Learning and instruction: An examination of four research perspectives in science education. Review of Educational Research, 58(3), 251–301. Gerard, L.F., Spitulnik, M., & Linn, M.C. (2011). Teacher Use of Evidence to Customize Inquiry Science Instruction. Journal of Research in Science Teaching, 47(9), 1037-1063. Greeno, J.G., Collins, A.M., & Resnick, L.B. (1996). Cognition and learning. In D.C. Berliner & R.C. Calfee (Eds.), Handbook of educational psychology (pp.15-46). New York: MacMillan. Johnstone, A.H. (1993). The development of chemistry teaching: A changing response to changing demand. Journal of Chemical Education, 70(9), 701-704. Krajcik, J. (1991). Developing students' understandings of chemical concepts. In S. Glynn, R. Yeany, & B. Britton (Eds.), The psychology of learning science (pp. 117-147). Hillsdale, NJ: Erlbaum. Linn, M.C., Clark, D., & Slotta, J.D. (2003). WISE Design for Knowledge Integration. Sci Ed, 87, 517-538 Linn, M.C. & Eylon, B.S. (2011). Science Learning and Instruction: Taking Advantage of Technology to Promote Knowledge Integration. New York: Routledge. Linn, M. C. & Hsi, S. (2000). Computers, teachers, peers. Hillsdale, NJ: Erlbaum. Linn, M.C., Lee, H.S., Tinker, R., Husic, F., & Chiu, J.L. (2006). Teaching and Assessing Knowledge Integration in Science. Science, 313, 1049-1050. International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012 92 Novak, J., & Gowin, D. (1984). Learning how to learn. New York: Cambridge Books. Slotta, J. D., Chi, M. T. H., & Joram, E. (1995). Assessing the ontological nature of conceptual physics: A contrast of experts and novices. Cognition and Instruction, 13(3), 373–400. Tversky, B., Morrison, J. B., & Betrancourt, M. (2002). Animation: Can it facilitate? International Journal of Human-Computer Studies, 57, 247-262. Tamin, R., Bernard, R., Borokhovski, E., Abrami, P., & Schmid, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4-28. White, B.Y. & Frederiksen, J.R. (1998). Inquiry, Modeling, and Metacognition: Making Science Accessible to All Students. Cognition and Instruction, 16(1), 3-118. Williams, M., Linn, M. C., Ammon, P., & Gearhart, M. (2004). Learning to Teach Inquiry Science in a Technology-Based Environment: A Case Study. Journal of Science Education and Technology, 13(2), 189-206. Zhang, Z. & Linn, M. C. (2011). Can Generating Representations Enhance Learning with Dynamic Visualizations? Journal of Research in Science Teaching, 48(10), 1177-1198.