Microsoft Word - blanton.docx Journal of Effective Teaching in Higher Education, vol. 4, no. 1 Learning by Design: Rethinking Curriculum Alignment through Collaborative Inquiry Morgan V. Blanton, Appalachian State University, blantonmv@appstate.edu Linda C. Pacifici, Appalachian State University Abstract. This article reports on an application of a SoTL research process for two teacher education instructors in an undergraduate teacher education course, Teaching and Learning in the Digital Age. After engaging in content analysis of student work samples derived from a Digital Sabbath assignment, the instructors asked more questions about how well the assignment aligned to course objectives and intended learning outcomes. Collaboratively, they determined how the Digital Sabbath assignment could be revised to better meet the stated course objective. The authors anticipate that assignment revisions could result in improved student learning outcomes for the teacher education students. Insights from the student responses and course materials are discussed as well as implications for teaching and learning in a teacher education program. Keywords: teacher education, curriculum, Scholarship of Teaching and Learning (SoTL), Digital Sabbath “Changing the status of the problem in teaching from terminal remediation to ongoing investigation is precisely what the movement for a scholarship of teaching is all about” (McKinney, 2007, p. 29). We, as two university instructors, found ourselves at this very crossroad realizing that a course learning goal was not met through a specific learning activity. We analyzed student learning products from a Digital Sabbath assignment, which required preservice teachers to take a break from social technology for an extended period of time. Upon reflection, we noted a discrepancy between an assignment’s learning outcomes and the course goals. The research question that guided our inquiry was, “To what extent is the Digital Sabbath assignment producing intended outcomes that lead to mastery of Course Objective 2?” In this article, the authors briefly describe the assignment results as the stimulus for examining it as a curriculum alignment issue. We used the Scholarship of Teaching and Learning’s content analysis and Wiggins and McTighe’s backward design as our primary methodological approaches (Bishop-Clark & Dietz- Uhler, 2012; Chick, 2019; McKinney, 2007; Wiggins & McTighe, 2005). A Problem Emerges The initial purpose for our collaboration was rooted in our desire to learn about the potential differences across five years of student projects in our Teaching and Learning in the Digital Age course. We thought that five years and significant changes in social technology would have an impact on how students viewed learning and learners in the digital age. However, through collaborative reflection regarding our assignment results, we realized that the greater issue was not Rethinking Curriculum Alignment through Collaborative Inquiry 2 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 necessarily that our students experienced the Digital Sabbath assignment differently between 2012 and 2017. The problem was more importantly about the discrepancy we noticed between the assignment outcomes and our intended learning goals. It was not until we analyzed student learning data collaboratively that we realized the learning outcomes of the Digital Sabbath assignment did not meet our expectations or assumptions of Course Objective 2. The lack of alignment between our assignment instructions and our course objectives became apparent. Therefore, we refocused our attention to curriculum design instead of ending the study after analyzing the Digital Sabbath assignment data findings. Conceptual Framework Scholarship of Teaching and Learning (SoTL) is defined as “the study of teaching and learning and the communication of findings so that a body of knowledge can be established” (Bishop-Clark & Dietz-Uhler, 2012, pg. 1; Chick, 2019). As educators, SoTL provided us with a roadmap for inquiring and exploring questions we had about our teaching and our students’ learning in a more formalized and structured manner. Further, Fanghanel (2013) suggests that SoTL “offers a space for critique and endorsement of practice” (p. 60). Our use of the SoTL research methodology enabled us to think more deeply about our learning activities. The backward design model provided a guide for us in which lesson planning begins at the end of an instructional cycle with the learning goals (Wiggins & McTighe, 2005). Learning activities are the vehicle to reach the learning goals and come after the learning goals are identified. We applied the backward design model to our course in order to determine when and how our learning outcomes and intended learning objectives became misaligned. Figure 1 illustrates how we envision the teaching/instruction framework of learning goals, assessment evidence, and learning plan working together with the SoTL process. The backward design steps are anchored with the teaching process in an iterative manner. We applied the model in three stages. Stage One focused on examining the desired results, Stage Two considered the assessment evidence, and Stage Three scrutinized the learning plan. Figure 1 Our Conceptual Framework Rethinking Curriculum Alignment through Collaborative Inquiry 3 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 Stage One: Desired Results Teaching and Learning in the Digital Age is a teacher education undergraduate course that explores topics related to emerging technologies and teaching in a 21st century world. The course is required in the professional core curriculum in a teaching education program at a regional comprehensive university in the southeast United States. The Digital Sabbath assignment was intended to align with Course Objective 2, “Express the changing nature of learners and learning in the digital age.” Stage Two: Assessment Evidence To determine how well our students met Course Objective 2, we asked students to keep an activity log detailing how they spent their time without digital technology. In addition, we gave them reflection prompts to guide their thinking after the Digital Sabbath experience was completed. Their responses in the activity logs and in their reflections were used to determine the learning outcomes and how well they mastered Course Objective 2. Stage Three: Learning Plan Our preservice teachers participated in a Digital Sabbath assignment during the first month of the semester. The purpose of this assignment was for students to experience and gain insights by comparing and contrasting life with and without technology. Considering Course Objective 2, “Express the changing nature of learners and learning in the digital age,” we wanted our students, as future teachers, to understand and articulate how this experience related to them and their future students. The hours without digital communication were required to be consecutive and students were encouraged to choose a normally busy time of day in which they were typically on their devices. Methods If the Digital Sabbath assignment was created to align with Course Objective 2, “Express the changing nature of learners and learning in the digital age,” then how did our students’ activity logs and reflections indicate how well they met this learning goal? Though we believed the Digital Sabbath experience was valuable to our students as individuals in a digital society, did it put them on a trajectory to better express the changing nature of learners and learning in the digital age, especially as future classroom teachers? Based on these uncertainties, we decided to engage in collaborative reflection that examined the Digital Sabbath assignment by analyzing the design and alignment to our course objective. Therefore, the research question that guided our collaborative work was, “To what extent is the Digital Sabbath assignment producing intended outcomes that lead to mastery of Course Objective 2?” Rethinking Curriculum Alignment through Collaborative Inquiry 4 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 Research Design Chick (2019) describes two project designs used in SoTL research: “What Is” and “What Works” projects. Our research design best fits into Chick’s description of a “What Is” project because we engaged in a systematic process of investigating our course assignments as teacher scholars. The “What Is” focus became the centerpiece of our Conceptual Framework, and backward design became the systematic process through which we examined the assignment and learning outcomes. In “What Is” projects, scholars provide “thick descriptions, snapshots of learning dynamics, or other detailed analyses of what student learning looks like, or what’s happening during the moments of learning” (2019). Shifting our focus to our own instructional practices, we examined the findings of our Digital Sabbath assignment through the lens of curriculum design (McKinney, 2007). By collecting additional data from our assignment instructions and course objectives, we hoped to determine how well the assignment resulted in intended learning outcomes. Data Collection Using backward design as a guide, we collected and analyzed data from the student responses, course objectives, and assignment instructions to better understand the instructional alignment, or lack thereof. We organized student learning data from the reflection and activity logs and began comparing those learning outcomes to Course Objective 2, “Express the changing nature of learners and learning in the digital age.” In addition to our original data from student responses, we also used data from the assignment itself. We examined the instructions to understand the context students worked in as they completed the assignment. Data Analysis To analyze our data, we reviewed the course objectives, assignment instructions, and the nature of the students’ written responses. We engaged in document analysis which included cross-walking course objectives with intended assignment outcomes. Further, we coded for common themes in the students’ reflections and cross-walked those themes with Course Objective 2. Findings In Stage 1, we identified the learning goal, and moved ahead to Stage 2 of the backward design model. In Stage 2, the assignment learning outcomes were closely reviewed. The primary learning goal in the Digital Sabbath assignment was for students to understand the changing nature of learners and learning in the digital age. Assignment instructions were written directing students to reflect on their experience without digital technology. As teacher educators, we hoped the students would express how the Digital Sabbath experience informed them as future teachers. When designing this assignment, we assumed that if students reflected Rethinking Curriculum Alignment through Collaborative Inquiry 5 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 on their experience and extended their thinking to the classroom, then this course objective would be partially met by completing this assignment. Course Objectives and Learning Outcomes Alignment “The SoTL puts the focus of the academic enterprise on students’ learning and urges the instructor to investigate, document, and present the results” (McKinney, 2007, pg. 101). We collected data from students’ activity logs and reflections. We coded these data for themes (Elliott, 2018). Descriptions and reflections related to the social and emotional impact of a Digital Sabbath were the most prevalent learning outcomes. Increased face-to-face interaction and improved mindfulness were noted as common themes in the data and within coded social reactions to the Digital Sabbath experience. Socially, the participants indicated an increased time spent in face-to-face interaction and "mindfulness" (focusing on the moment) as a result of taking a break from their devices. The increased presence was described as an “in the moment” connectedness that had been previously missed because of excessive technology usage. Often distracted by their phones, the participants learned that they were better able to attend to the present. They also learned that living in the digital society increased the distraction they experienced by technology and realized they were missing opportunities to interact with those around them. We labeled the themes of strategic planning, self-realization, and busyness as emotional learning outcomes. These themes emerged within the context of participants' feelings of anticipating or feeling bored during the Digital Sabbath assignment. In order to combat possible boredom, participants intentionally and proactively planned when they would complete the Digital Sabbath assignment Additionally, these students planned where and what they would do during the hours without their technology. Prior to completing the assignment, students did not realize how dependent they were on their cell phones for daily tasks and interactions until they were in the midst of the assignment or as they reflected upon their experience afterward. While these social and emotional responses to the assignment were valuable, we questioned how well these learning outcomes aligned to Course Objective 2. When analyzing their responses, we looked for instances in which they referenced their future students or themselves as classroom teachers. We discovered that the Digital Sabbath assignment outcomes did not indicate that students reflected on themselves as learners or future teachers as often as we expected or assumed. Rather, their reflections indicated a comparison of personal use and nonuse of digital and communicative technologies. Analysis of the 2012 student responses led us to realize that two of the twelve student responses mentioned classroom or teaching implications. However, both of these implications for classroom practice focused more on replicating the experience with their future students. One student noted, “I think it would be a good idea for teachers to encourage their students to take time off from technology,” and mentioned that some students “are so attached to technology that Rethinking Curriculum Alignment through Collaborative Inquiry 6 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 they cannot even go without it for an hour and a half long class.” Another student in 2012 reflected, “When I become a teacher, I will definitely teach my students how to use the technology necessary to succeed in the world, but I will also teach them how to distance themselves from technology. I will show them how to interact with nature instead of playing video games, or read books for the fun instead of watching television. Learning to love life without technology is well worth the sacrifice.” Both of these students predicted that their future students will use technology often and will need to be forced to take a break from their use of technology devices. However, they did not apply their thinking about their students as learners who also enjoy using technology. Technology was villainized in both of these examples and the focus was on excessive use instead of learning and learners. The other ten students in the 2012 group did not mention their future classrooms or students at all in their reflections. In the 2017 class, only three of the seventeen students extended their responses to include future classroom connections. Two students noted how often technology today is used for instruction or for students’ engagement with instruction. For example, one student reflected, “I realized while I was trying to do my homework in the first hour almost everything requires me to use the Internet to gather information. So, going into a classroom, I need to be able to show my students how to gather information.” Another student realized, “We also use technology as a resource for children in schools. Instead of just normal pencil and paper, it is also a great tool to help them learn different approaches of learning instead of just the normal techniques that we use.” Both of these students recognized how much they use technology as learning tools as students in the digital age and suggested that their future students will also need to know how to use it for educational purposes as well. Another student realized after engaging in the Digital Sabbath that communication is being negatively affected by technology. She asserted, “We depend so much on our technology that we don't know how to communicate with each other face-to-face. I think that's one of our biggest problems, and we need to try to fix it. As teachers we need to learn how to communicate with parents, other teachers in our schools, and even our principal, to solve problems. We need to talk to each other about how to improve our schools and classrooms. I think disconnecting ourselves from technology could help that especially for our generation because we’ve grown up with it.” Although she did not mention students or instruction, her reflection was relevant to herself as a future educator and to the profession as a whole. Because face-to-face Rethinking Curriculum Alignment through Collaborative Inquiry 7 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 communication was one of the stronger themes of the original study, her reflection is also related to other students’ responses, although they did not specifically mention communication in education in their reflections. Through a learning outcomes analysis, we acknowledge that a few students in each of our courses reflected on teaching and learning. Yet the majority of the students did not make the teaching and learning connection as part of their reflections. We wondered why this assignment was not resulting in the expected learning outcomes intended for this course for future educators. We continued our research and looked closer at the assignment provided. Analyzing the Learning Experience Stage 3 of the backward design model enabled us to go deeper with our curricular analysis. We began with an examination of the assignment instructions that were provided to both the 2012 and 2017 courses. While the 2012 and 2017 assignments were quite similar, there were two main differences in the instructions: time and response platform. The 2012 class observed a twenty-four-hour Digital Sabbath which allowed for 6 hour increments and did not have to be consecutive time away from technology. In contrast, the 2017 class engaged in the Digital Sabbath for only five consecutive hours total. In Figure 2, we have represented this difference by using the following notation: [XX-hour period]. Further, in 2012, students were asked to produce “visual, textual, or photographic representation of your experience using a minimum of 10 photographs/text entries with written text for each.” The 2017 class was instructed to “Write a blog post and create a video reflection where you reflect deeply on your experiences. Use a minimum of 5 original photographs, each with written captions and embed your video in your blog.” The remaining instructions and reflection prompts we used to guide students’ work on this assignment are provided in Figure 2. The differences between 2012 and 2017 instructions are identified using brackets and a contrasting font. Rethinking Curriculum Alignment through Collaborative Inquiry 8 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 Figure 2 Integrated Assignment Instructions Through a review of the assignment instructions, we learned that most of our reflection prompts focused on their personal responses to the Digital Sabbath experience. We provided only one prompt that related to instructional implications, and it was at the end of the suggested list. Further, the list of reflection prompts were simply suggestions, not prescribed components of the assignment. Having just a list of suggested reflection prompts gave students significant latitude regarding how they chose to reflect. However, when given loose parameters, many students chose to reflect on the personal implications instead of education-related implications. Unlike the personal response prompts, we did not provide multiple reflection prompts to guide them to that end. Discussion Based on research findings, we discovered that our assignment did not align as successfully with our course objectives as we originally assumed. The collaborative process of SoTL along with using the backward design model to guide data analysis produced useful results. We realized that some of our expectations of this assignment were left unspoken. This reflection process revealed assumptions we Rethinking Curriculum Alignment through Collaborative Inquiry 9 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 had about the assignment. We uncovered our assumption that students would intuitively extend their thinking to their future profession with one reflection prompt. Yet, after examining our assignment instructions and student responses, we now realize the limitations of our instructions. Explicit instructions written towards the course objective goal could have operationalized our assignment expectations. Ensuring our written directions matched our expectations could have improved the assignment outcomes and would have better aligned with our Course Objective. We believe we lost sight of the need for explicit instructions with this assignment. Explicit instruction is a common pedagogical practice (Marin & Halpern, 2011); however, when teaching adults, it is important to remember that adults benefit from explicit instructions, too. Furthermore, to better align with Course Objective 2, we learned to be more intentional about aligning our assignment instructions. Improved intentionality leading students to focus on the changing nature of learners and learning could have improved overall alignment and learning outcomes. Developing personal awareness regarding the impact of technology use was important and perhaps a first step. Yet our students did not extend or articulate their thinking to consider how their relationship with their device related to the changing nature of learners and learning in their future classrooms. To improve the assignment, we believe we need to revise our instructions so students have the opportunity to examine their personal responses separately from their professional responses. These students needed time and space to sort through their social and emotional use of technology before trying to think outside of themselves to their future students. In retrospect, perhaps a more effective way to frame the reflection prompts would have been to categorize them so students chose prompts to help them reflect both personally and professionally. To improve clarity, instructions for the assignment could be organized into two parts. The first reflection section could focus on personal impact. Personal impact includes but is not limited to their social, emotional, and physical reactions when taking a break from their devices. A second section could guide students to consider implications for themselves as teachers, and their future students as learners in a digital age. After the personal response, we could have provided them with multiple prompts to guide them to consider implications for themselves and their future students. By dividing the reflection into two sections, this revision to our assignment could support reflection and insights in a more intentional and substantial manner. Being more explicit about the expectation to extend their thinking as teachers in their future classrooms is imperative for this assignment to align with Course Objective 2. We learned students may not engage in this type of thinking on their own. Students in the course need specific questions to guide their reflection. Our original directions provided one prompt related to education, and data analysis led us to understand that students needed more guidance. The following prompts could be used in the assignment. These prompts would lead students to more thoughtfully and intentionally consider classroom implications. We could include prompts such as the following: Rethinking Curriculum Alignment through Collaborative Inquiry 10 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 ● How does this experience help you better understand learners in this digital society? ● Knowing what you now know about your relationship with your digital device(s), how do you think learners’ needs have changed as a result of the wide-spread use of digital technology? ● What implications does your experience have on your future teaching methods? Further, since the activity log was part of the assignment, we believe it caused confusion. Perhaps this activity log enabled students to focus their reflections more on what they did with their time instead of extending their thinking to classroom implications. The resulting misalignment with Course Objective 2 could be in large part because of how we organized the instructions and listed required artifacts. Students believed the main purpose of the assignment was digital self-awareness and reflection upon their activities. The organizational structure of the assignment was insufficient in guiding students toward our learning goal due to the assignment’s organizational structure. Being more explicit with prompts that guide students to extend their thinking beyond the activity log to the root issues would have been a better way to organize the assignment. Conclusion Providing explicit instructions is a best practice (Marin & Halpern, 2011); however, as instructors in higher education, we sometimes take for granted that adults benefit from explicit instructions as well. Taking time to ensure the written instructions match our unspoken expectations of the assignment is a reflective step that should not be overlooked. The SoTL process, which focuses on instructor critical analysis, theorized reflections, and thoughtful enactment of student learning, supported our desire to rethink and review our Digital Sabbath assignment (Bishop-Clark & Diez-Uhler, 2012). This type of research activity guided our efforts to learn more from the student learning outcomes and the questions that remained. It is important for instructors at all levels of education, especially those in pre-service teacher education, to engage in curriculum review and revision on an ongoing basis. Through a continuous improvement mindset, we need to scrutinize our own instructional practices in a way that improves our instruction. A continuous improvement mindset also serves as a model for our pre-service teachers. Having conversations with our students about the effectiveness of our instruction, in addition to the course learning goals we have from the throughout the course, sets the stage for them to become reflective practitioners as well. Curriculum design, review, and revision does not have to occur in isolation. Working with course-alike colleagues to collaborate in the process is beneficial. Collaboration was key for this endeavor because we drew on each other’s background knowledge and experience with curriculum design to improve our professional practice. Rethinking Curriculum Alignment through Collaborative Inquiry 11 Journal of Effective Teaching in Higher Education, vol. 3, no. 2 Engaging in SoTL with a colleague deepened our reflective practices in a number of ways. We were able to collaboratively analyze student work, encourage curriculum critiques, and learn from each other’s knowledge base and prior experience. This collaborative work enabled us to thoughtfully process student learning outcomes and used them to inform our collaborative curriculum revisions. The research from this inquiry project reinforces the value for ongoing curriculum review in higher education. Close examination of course goals in alignment with course assignments can occur in course redesign. This review process affirmed the need for explicit assignment instructions in our study. Ongoing, more frequent curriculum review and redesign prevents the error of engaging students over a five- year period in an assignment that was not adequately aligned to the intended course goals. Conflicts of Interest The authors declare that there is no conflict of interest regarding the publication of this article. References Bishop-Clark, C., & Diez-Uhler, B. (2012) Engaging in the scholarship of teaching and learning: A guide to the process, and how to develop a project from start to finish. Stylus Publishing. Chick, N. (2019). Scholarship of teaching and learning (SoTL). https://my.vanderbilt.edu/sotl/ Elliott, V. (2018). Thinking about the coding process in qualitative data analysis. The Qualitative Report, 23(11), 2850–2861. https://nsuworks.nova.edu/tqr/vol23/iss11/14 Fanghanel, J. (2013). Going public with pedagogical inquiries: SoTL as a methodology for faculty professional development. Teaching & Learning Inquiry, 1(1). 59–70. https://www.jstor.org/stable/10.2979/teachlearninqu.1.1.59?seq=1#metada ta_info_tab_contents Marin, L. M., & Halpern, D. F. (2011). Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and Creativity, 6(1), 1–13. McKinney, K. (2007). Enhancing learning through the scholarship of teaching and learning: The challenges and joys of juggling. Jossey-Bass. Wiggins, G. P., & McTighe, J. (2005). Understanding by Design. Expanded 2nd ed. ASCD.