Just-in-Time Teaching A. Gavrin Abstract Just-in-Time Teaching (JiTT) is an innovative pedagogy that enables faculty to increase interactivity in the classroom and engage students in learning. By creating a feedback loop between students' work at home and the classroom setting, time on task is improved in both quality and quantity. This paper includes an introduction to JiTT and evidence of its effectiveness. It concludes with a discussion of our efforts to disseminate JiTT since it was developed in the 1990s. Introduction Students, faculty and administrators disagree on many things, and the future role of technology in education is one of these. Most agree that technology will change education; the disagreements lie in questions of how, when, and for good or ill. In this article, I will describe a rare creature: a use of technology in education that is widely recognized as beneficial by all interested groups. I have three goals in this paper. I begin by giving a rough outline of the JiTT method, including its early development and the educational principles in which it is grounded. I also describe how we established the effectiveness of the method, both in improving educational outcomes and in improving students' attitudes about their classes. I then turn to a discussion of how this method has spread through the academic community since it was first developed in the late 1990s. The pedagogy is known as "Just-in-Time Teaching" or JiTT (Novak, Patterson, Gavrin, and Christian 1999; Novak 2006). This name is intentionally reminiscent of the "Just- in-Time" manufacturing process pioneered by Toyota in the 1970s (Monden 1998). Gregor Novak and I developed JiTT at IUPUI, in collaboration with Evelyn Patterson of the United States Air Force Academy. JiTT is applicable to any field of study and may be adapted to cover a full range from developmental to graduate classes. It is a strategy founded on several principles of pedagogical best practice (Chickering and Gamson 1987). JiTT encourages students to be well-prepared for class and promotes active learning. It helps faculty to identify their students' strengths, weaknesses, and learning styles. JiTT also encourages writing as an integral part of the learning process. We evaluated JiTT using a variety of qualitative and quantitative measures and sought to create a complete picture of our successes and failures. In the course of this evaluation, we have considered students' success rates in our classes, students' scores on measures of cognitive gains, self-reported data on study habits, attitude surveys, and student focus groups. This article includes examples drawn from our assessment 9 10 efforts. The results are positive and significant. In the introductory physics sequence at IUPUI, the introduction of JiTT resulted in a 40 percent decrease in the number of students who dropped the class or received a grade of "D" or "F" (Gavrin, Watt, Marrs, and Blake 2004). JiTT has also been shown to improve students study habits and to result in measurable cognitive gains (Marrs, Gavrin, and Novak 2004). When we developed JiTT at IUPUI and the Air Force Academy, we were teaching introductory physics courses. Since that time, JiTT has been widely adopted in the United States, and it has scattered adherents in Canada, Mexico, Israel, and several European countries. Although physics is still the discipline with the largest number of JiTT users, this preference is a bare plurality, with many instructors using JiTT in the sciences, social sciences and humanities. We are aware of over two hundred faculty who have used JiTT in over twenty academic disciplines at over one hundred institutions. Indeed, we no longer have an accurate count of the numbers of faculty using JiTT worldwide. Overall, we regard the dissemination of JiTT to be a success, though a moderate one. We are still working to expand adoption of JiTT by other faculty members. Principles of JiTT The JiTT method succeeds through a fusion of high-tech and low-tech elements. The high-tech elements center on our use of the World Wide Web (Web) to deliver curricular materials and expand communications among faculty and students. On the low-tech side, we stress classroom interactions among students, faculty, and student mentors. The underlying process is for faculty to make rapid adjustments to address students' problems using feedback from the Web and the classroom. We have reversed the common notion that technology should be used to replace or expedite classroom methods. Rather, we use information technology to improve the classroom activities themselves. In a JiTT course, work that students do at home is used to leverage the time they spend in class. The Web is used as a communications tool. The key is a series of assignments called "WarmUp Exercises." Typically, there is some material students ought to read before a given class period. WarmUp Exercises is an online assignment due before class that asks students to answer several open-ended, conceptual questions about the material that the instructor will discuss in class. Even in a physics or math class, the WarmUps should be conceptual questions that require written responses, not mathematical calculations. In my classes, the WarmUp is due two hours before class, but others who use JiTT vary this delay from one to twenty-four hours. One way to look at this is as a "reading quiz." Many faculty members give brief quizzes at the beginning of class to encourage students to come prepared. WarmUps accomplish much the same thing, but with several advantages. One obvious advantage is that no class time is used administering the quiz, but other advantages are far more important. Giving the reading quiz as a WarmUp-online and in advance-encourages students not only to read, but to think about the ideas in the reading, to connect them to their prior knowledge, and to apply them to a brief problem. When we first developed the notion of Warm Up Exercises, this was all we had in mind-a method to encourage students to come to class prepared. When we started to read the answers students gave, we realized that the exercise was far more powerful. We found that students had profound difficulties understanding some ideas, whereas they easily grasped others. Based on this observation, we began adjusting the amount of time we spent in class on various topics, giving more time to the areas that students had not understood. This is the origin of the phrase "Just-in-Time Teaching." We make adjustments to the classroom presentation "just in time" for class based on the results of the Warm Up Exercises. Now the analogy to manufacturing is clear. In JIT manufacturing, goods are produced in small quantities just-in-time to meet the needs of the distribution system, and raw materials and parts are ordered in small quantities just in time for them to be used in production. Similarly, Just-in-Time Teaching is more responsive to students needs, and classroom time does not focus on topics that students learn easily. Students come to class better prepared for the subject, and faculty come to class better prepared for their students. These advantages alone justify the costs of implementing JiTT: some basic technology infrastructure and the faculty time required to develop WarmUps. However, there is a way to take the method even farther. I usually take about a half hour before class and prepare one or two overhead slides with excerpts from the students' answers to the WarmUp questions. I bring these slides with me to lecture and use them as a "scaffold" for my presentation. Instead of lecturing from my notes, I explicitly begin from my students "current state" and work to bring them from that state to my desired end. When I select the excerpts for use in class, I include work from many students, touching all students at least occasionally throughout the semester. I present the excerpts without attribution to a particular student, and I always make a positive comment about the work, even if I bring it up to highlight a mistake. This completely changes the mood of the class. Rather than attending a lecture that starts from "scratch," my students participate in a discussion of what they need to refine their understanding of the subject. The class is inherently learner-centered and discourages passive note taking. Creating and managing a meaningful discussion of new ideas is never an easy task. JiTT allows me to start the discussion with my students' own words. Even in a large lecture setting, I can often initiate a lively discussion with many students participating. Students who prefer to "think a bit" before making comments can be included, as can students who are too shy to raise their hand. We often refer to this classroom setting as an "interactive lecture." One way to look at JiTT is as a feedback loop linking the classroom with all other learning environments the students use. The Web connects what students learn in the library, home and elsewhere to the interactive lecture. The classroom experience is 11 12 informed and improved by their work outside. What they learn in class becomes the basis of the next reading and Warm Up assignment. Assessment of JiTT IUPUI is an urban university with many of the problems typical of such an institution. Almost all of our students commute to campus, and the vast majority work at least part-time. Many are the first in their family to pursue a college education. As a result, IUPUI is deeply concerned with retention of students at every level. We calculated the DFW rate (the percentage of students earning a "D," "F," or withdrawing from class) for five courses taught using JiTT in physics, mathematics, biology, and chemistry. In all cases but one, JiTT had a substantial, positive effect on student success in these courses. As shown in Table 1, when JiTT is introduced students' success rates soar. This increased success rate may be attributed to many factors, including increased interaction among faculty and students, which has been identified by Astin ( 1993) as a crucial condition for success in college. The improvement in performance may also be due to improved study habits. One of the key features of JiTT is that students must read and consider new ideas before coming to class. As a result they are far better prepared. Further, because JiTT courses tend to have more frequent assignments than non-JiTT courses, students are encouraged to spread their work more uniformly. Students' self-reported behaviors in a JiTT class in biology include increased preparation and a reduced tendency to get behind then "cram" at exam times. The only class in which JiTT has not had a significant effect is the introductory chemistry class, where the result is positive, but is too small to ascribe to the introduction of JiTT with any certainty. Table 1: DFW Rates Without and With JiTT Without JiTT WithJiTT Course Semesters DFW rate Semesters DFW rate Reduction in DWFrate Physics I 5 47% 14 28% 40% Physics II 5 32% 14 19% 41% Calculus II 6 45% s 32% 28% Survey of Biology 3 27% 4 19% 31% Chemistry - Intro 7 37% 2 34% 8% To measure the effect of JiTT on cognitive gains in biology, we used the results from a twenty-question pre-class and post-class test, calculating an average improvement on each question. Our measure of improvement is the average normalized gain defined by Hake (1998). We found that students showed an improvement of 17% on test questions about concepts discussed in class, but not necessarily reinforced by any additional activities, and they showed an improvement of 21 % on test questions reinforced by homework problems. In contrast, students tended to show an improvement of 51 % on test questions reinforced by either WarmUp questions or Cooperative Leaming activities, and an improvement of 64% on test questions reinforced by both WarmUp questions and Cooperative Leaming activities. We also assessed students' attitudes about the use of JiTT. They are overwhelmingly positive. In an Introductory Physics course, 88% of the students said Warm Ups were helpful. Students gave a range of responses to why JiTT was helpful including, "Since I have a tendency to put things off, the WarmUps were a great way to get me to read ahead for lecture," and "It helps you to get an idea of the main points you 're going to talk about that day in lecture." Dissemination of JiTT We have worked to make others aware of our efforts using many conventional (and some unconventional) routes. The dissemination of JiTT may be regarded as somewhat unusual, in that the first significant publication regarding JiTT was a book, Just-in- Time Teaching: Blending Active Learning with Web Technology, published by Prentice Hall in 1999 (Novak, Patterson, Gavrin, and Christian 1999). In addition to the book, JiTT has been the subject of several peer-reviewed articles and conference proceedings (Simkins and Maier 2004; Marrs, Blake, and Gavrin 2003). We have also disseminated JiTT through electronic publications. The JiTT Web site, http://jitt.org, provides an overview of JiTT goals and methods and examples of JiTT materials. It also provides a link to a spreadsheet of faculty who have adopted JiTT, their institution, discipline, and contact information. Extensive JiTT materials are also available on the WebScience site, constructed by the author to disseminate the results of an NSF-sponsored project to adapt JiTT to chemistry, biology, and mathematics classes. Other Web sites that contain substantial discussions of JiTT include the project Galileo site at Harvard, the JiTT economics site at NCAT, and the geoscience-teaching site at Carleton College (Gavrin 2006; Mazur 2006; Simkins 2006; Guertin 2006). Most recently, Gregor Novak and Evelyn Patterson have gained funding through the NSF National Science Digital Libraries program to establish a Ji TT Digital Library (JiTTDL). The JiTTDL is under construction at http://jittdl.org. Another fruitful route for dissemination of JiTT is through talks and workshops. Along with my collaborators, we have spoken about JiTT at national meetings of the American Association of Physics Teachers (AAPT), the American Society of Engineering Education (ASEE), and the American Chemical Society (ACS). We have also given multi-day workshops at conferences hosted by Project Kaleidoscope, the NSF Chautauqua series, and a series of "New Faculty Workshops" jointly hosted by the American Association of Physics Teachers (AAPT), the American Physical Society (APS), and the American Astronomical Society (AAS). We have also given numerous talks at individual institutions. A partial listing of the talks and workshops we have given in recent years may be found on the WebScience project dissemination page (Gavrin 2006). 13 14 I recently conducted a survey of faculty in mathematics, physical science and engineering departments that I knew were using JiTT. The survey was conducted online with potential respondents being solicited using an email that contained a link to the survey site. In all, 162 faculty were asked to complete the survey. Of the 162 faculty members asked to participate, 52 responded, a response rate of 32 percent. The majority of respondents had two or more years experience using JiTT. Within the limited set of academic fields surveyed, physicists comprise a majority of JiTT users. (Of the 52 respondents, 27 identified their home department as a department of physics.) This is unsurprising, as JiTT originated in physics, and early dissemination was focused on the physics community. The distribution of respondents by academic rank is also revealing: 14 identified their rank as assistant professor, 17 as associate, and 12 as full professors; 4 identified themselves as lecturers or instructors; and 4 identified themselves in other categories, including high school teachers and IT professionals. This range nearly mirrors the distribution of ranks reported in the professoriate overall, though there is a slight under-representation of full professors (American Association of University Professors 2006). We asked respondents to "Briefly describe how you first heard about JiTT." The results are summarized in Figure 1. "New Faculty Workshop" refers to a series of three-day workshops for new faculty in physics and astronomy, hosted by the American Association of Physics Teachers (AAPT), the American Physical Society (APS), and the American Astronomical Society (AAS). These workshops have been held once annually since 1997, and one talk has been devoted to Ji TT each year. Project Kaleidoscope (PKAL) is a national alliance dedicated to undergraduate teaching and learning of science, mathematics and engineering. Since 1989, PKAL has hosted conferences and established faculty networks to advance its goals. JiTT has been featured at a number of these events. Online sources refer primarily to the JiTT Web site (http://jitt.org) and to information posted online pertaining to one of the workshops or papers. The results shown in Figure 1 show that word of mouth is the most common way for faculty to learn about JiTT. Notable among the mechanisms shown here are the New Faculty and PKAL workshops. Both of these methods target faculty members at the beginning of their teaching careers and seem to be particularly effective given that there have been far fewer of these than AAPT conference talks and workshops. The findings suggest that faculty are most likely to adopt new pedagogical strategies when they first begin teaching, before they invest a great deal of time and effort in developing traditional materials and methods. Figure 1. Information Sources JiTT From a colleague - AAPT conference - New Faculty Workshop - Project Kaleidoscope Other talk or workshop JiTI Book or article Online Other/don't recall -0 4 6 IO 12 14 16 # of Respondents We also asked respondents to list the positive and negative aspects of JiTT. Overall, the respondents stressed positive aspects over negative ones. We categorized 149 comments, of which ninety were positive and fifty-nine were negative. Many of the respondents' negative comments were linked to comments stressing the positives, e.g., "When TIME is tight, it can be rough to use this. However, skipping the preview leaves misconceptions uncorrected." We list this as a negative comment under "increased faculty workload." Nevertheless, it seems that the author of this comment did not intend for it to be entirely negative. Several respondents provided student comments that echoed this sentiment. For instance, one student wrote "Having the quizzes and concept problems daily was tough at times, but made me get more involved in the material." Another compared the JiTT experience to other courses: "I really like how you had pre-class quizzes that required us to read the chapter beforehand. That keeps us motivated to stay caught up in our schoolwork instead of falling behind like many of us have in other classes." Figures 2 and 3 summarize the positive and negative comments faculty made. It is noteworthy that the negative comments were far more homogeneous. That is, a single category (increased faculty workload) accounts for almost half of the negative aspects; no such dominant category was observed among the positive aspects. 15 16 Figure 2. Positive Faculty Comments - JiTT Improved classroom content or climate Students better prepared/read ahead !l Improved faculty understanding of students' capabilities l :~ ~ Improved student-faculty communication Improved student learning or attitude Students better focused or motivated Other 0 Figure 3. Negative Faculty Comments - JiTT 10 15 20 25 # of Respondents Increased faculty workload ······················- !:: . . "th" . ·--·--· ;.:::; D1fficulties w1 1mplementation ~ Increased student workload ·--·-· Other student objections •••• ·~ ~ Requires longer-range planning - z Other ·--·-· 0 10 15 20 25 # of Respondents 30 30 One of the most dramatic successes in our efforts to disseminate JiTT occurred at Erskine College, a liberal arts college in South Carolina. Erskine is a small school of forty-two faculty members and about 580 students. At present, between half and three- fourths of the faculty use JiTT to some extent and many use it in all of their courses. This extraordinary adoption of JiTT occurred during the late 1990s through 2001. I interviewed Prof. William Junkin, who was Dean for Learning and Technology and Professor of Physics at Erskine College. According to Dr. Junkin, JiTT became widely accepted at Erskine due to the convergence of several influences. Erskine benefited from a large grant from AT&T through the Foundation for Independent Higher Education. This grant provided substantial funds for professional development and the adoption of information technology at Erskine (along with Converse College, Eckerd College, and King College). Along with strong support from the administration, this gave faculty a "sense of pride in Erskine as a leading innovator in the use of technology." Another driving force behind the widespread adoption of JiTT was the decision to use JiTT as a common methodology in Erskine's Freshman Seminar. This is a course taken by all students and taught by all faculty members in rotation. Although each faculty member may "customize" his or her section of the seminar, some curricular elements are common to all sections. A faculty committee decided that all sections should use JiTT to some extent. As a result, the entire faculty had to gain at least a moderate familiarity with JiTT methods and technology. Many faculty members decided, having used JiTT in their seminar, to adapt the method to their regular classes. Conclusions Just-in-Time Teaching is a powerful pedagogical method that uses technology to enhance students' attitudes and academic performance. By using the Web as a communications tool, it allows faculty to link the classroom experience to students' work at home in a way that encourages both students and faculty to be better prepared for class. JiTT has been shown to improve students' cognitive gains and to improve student retention. Through a variety of publications, Web sites, and oral presentations, JiTT has spread widely among faculty members in the United States and elsewhere, and it is popular among the students and faculty who have used it. Faculty who have adopted it often feel that it takes more time than a traditional lecture, though they generally recognize that this extra time is beneficial to students. JiTT could be, however, more widely used by faculty in sciences and other disciplines. The question is how can we more broadly disseminate this pedagogy? References American Association of University Professors. The annual report on the economic status of the profession, http://www.aaup.org/surveys/03z/zrep.htm (accessed March 10, 2006). Astin, A. W. 1993. What matters in college: Four critical years revisited. San Francisco: Jossey-Bass. Chickering, A. W., and Z. F. Gamson. 1987. Seven principles for good practice in undergraduate education. AAHE Bulletin 39: 3-7. Gavrin, A. Webphysics, http://webphysics.iupui.edu/webscience (accessed March 10, 2006). Gavrin, A., J. Watt, K. Marrs, and R. Blake. 2004. Just-in-Time Teaching (JiTT): Using the Web to enhance classroom learning. Computers in Education Journal 14 (2): 51-59. Guertin, L. Just In Time, http://serc.carleton.edu/introgeo/justintime/index.html (accessed March 10, 2006). Hake, R.R. 1998. Interactive engagement vs. traditional methods: A six-thousand student survey of mechanics test data for introductory physics classes. American Journal of Physics 66:64-74. 17 18 Marrs, K. A., R. Blake, and A. Gavrin. 2003. Use of warmup exercises in Just-in-time Teaching: Determining students' prior knowledge and misconceptions in biology, chemistry, and physics. Journal of College Science Teaching (September): 42-47. Marrs, K., A. Gavrin, and G. M. Novak. 2004. Just-in-time Teaching in biology: Creating an active learner classroom using the Internet. Cell Biology Education 3:49-61. Mazur, E., Galileo, http://galileo.harvard.edu/galileo/sgm/jitt, (accessed March 10, 2006). Monden, Y. 1998. Toyota production system: An integrated approach to Just-in-Time. Norcross, GA: HE Press. Novak, G. M. Just-in-Time Teaching, http://jitt.org (accessed March 10, 2006). Novak, G. M., E. Patterson, A. Gavrin, and W. Christian. 1999. Just-in-Time Teaching: Blending active learning with Web technology. Upper Saddle River: Prentice Hall. Simkins, S. JiTT economics, http://www.ncat.edu/-simkinss/jittecon/jittpedagogy/ (accessed March 10, 2006). Simkins, S., and M. Maier. 2004. Using Just-in-Time Teaching techniques in the principles of economics course. Social Science Computer Review 22:444-456. Acknowledgements The National Science Foundation supported this work through awards DUE-9981111 and DUE- 0333646. Author Information Andrew D. Gavrin is Associate Professor of Physics and Associate Dean of Science at IUPUI. His research interests include magnetic materials and physics education. Andrew D. Gavrin Department of Physics, IUPUI 402 N. Blackford Street Indianapolis, IN 46202 E-mail: agavrin@iupui.edu Telephone: 317-274-6909 Fax: 317-274-0628 MU2006-12-010_page9 MU2006-12-011_page10 MU2006-12-012_page11 MU2006-12-013_page12 MU2006-12-014_page13 MU2006-12-015_page14 MU2006-12-016_page15 MU2006-12-017_page16 MU2006-12-018_page17 MU2006-12-019_page18