Microsoft Word - ouimette.doc Australasian Journal of Educational Technology 2009, 25(5), 731-747 Essential books in the field of instructional design and technology Jenelle Ouimette, Daniel W. Surry, Adrian Grubb University of South Alabama David A. Hall Jackson State University This article describes the results of a study to determine the books that instructional design and technology professionals believed were most important to the field. Participants in this study were 77 professionals from different areas of the field, including education, business, and government. The purpose of the study was to create a snapshot of the books that form the theoretical and practical foundation of the field of instructional design and technology at this time in the field’s history. A survey was conducted asking participants to rank the importance of books on a four-point scale from “profoundly important” to “unimportant”. The data were then analysed using descriptive and inferential statistics. Results indicate that the importance of a book varies widely, based on factors such as a person’s area of interest in the field, degree level, and age. Overall, however, the study found that 10 books were viewed as being among the most important by most respondent groups. This core group of books should be included in every instructional designer’s or technologist’s personal library. Introduction As Ely and Plomp wrote in their book, Classic writings on instructional technology (1996), most books, articles, or written works come about by a series of events or experiences by people within a field. This still holds true for the field of instructional design and technology (IDT). As theorists and practitioners in the field of IDT continue to work and conduct research, the knowledge base of the field grows, experiences increase, and the amount of literature continues to expand. As a result, IDT professionals are likely to find the number of high quality books growing at a continually increasing rate. While books obviously do not represent the entire body of literature, studying the essential books of a field can help to determine the major theories and concepts that serve as a foundation for that field (Braden & Sachs, 1983). The rapid growth of IDT has resulted in a broad field with a diverse theoretical and practical knowledge base. It is important for the members of any field, especially a growing and diverse one, to periodically reflect on the ideas, theories, practices, and beliefs that form the foundation of their field. One way to do this periodic reflection is to determine which books the members of field believe are most important and influential at a particular time in their field’s history. The purpose of this study was not to determine the “best” books or to list the only books one should own, but rather to take a snapshot of where the field is currently, and to provide a starting point for future discussion of the essential concepts and practices within the field. 732 Australasian Journal of Educational Technology, 2009, 25(5) Method The primary data collection method used in this study was an online survey. A multi- step process was used to develop and disseminate the survey. The first step in the process was to develop a list of all books that could be included in the study. To do this, we searched the inventory of a popular, web based book seller, Amazon.com, for relevant books using the keywords “instructional design”, “instructional systems”, and “instructional technology.” This generated a master list of 709 books. Following this, we attempted to shorten the list by removing any books that obviously did not belong on the list. To do this, two of the authors of this paper went through the list of the 709 books independently and marked books they felt obviously did not belong on the list. Whenever both authors agreed that a book obviously did not belong on the list, that book was removed. A total of 222 books were removed in this manner. The books that were removed were made up mostly of user guides for specific software programs and books from other fields, such as management, that included only a brief mention of one of the keywords. A total of 487 books remained on the list at this point. The next step in the process was to further refine the list. To do this we developed an online survey that included all of the 487 remaining books. We then asked 10 colleagues to go through the list and rate each of the remaining books. Each of the 10 raters held a doctoral degree in IDT or a closely related field. Each of the 10 was from the United States. Five of the ten were employed as college professors and five worked in either a business setting or in higher education in a non-faculty role. We asked each of the ten to log in to the online survey and respond to the question: “Which of these books do you think should be included in a reasonably well stocked academic library in the field of Instructional Design & Technology.” Respondents selected “Yes’ or “No” for each of the 487 books. As seen in Table 1, 388 books (79.7%) were selected as “Yes” by 3 or fewer raters. A total of 99 books (20.3%) were selected as “Yes” by four or more raters. Table 1: Number of books selected as “Yes” by raters Number of raters selecting “Yes” Number of books % 10 0 0 9 0 0 8 1 0.2 7 3 0.6 6 19 3.9 5 27 5.5 4 49 10.1 3 83 17.0 2 133 27.3 1 171 35.1 0 1 0 Prior to the ratings by our colleagues, we had decided to include any book selected as “Yes” by four or more raters in the final survey. This resulted in a total of 99 books being included in the final survey. For the final survey, we listed each of the 99 books and asked respondents to rate each book on a 4 point scale ranging from “profoundly important” (4) to “not important” (1). If respondents were not familiar with a book, they were allowed to skip the book without providing a rating or to select “Don’t know.” Books that were not rated or were rated as “Don’t know” were not included in Ouimette, Surry, Grubb and Hall 733 the statistical analysis for that respondent. Respondents also had the opportunity to add books they felt were profoundly important to the field that were not on the list. The survey also included a demographic section. All responses to the survey were anonymous. The final version of the survey was reviewed and approved by our university’s Institutional Review Board to ensure compliance with ethical standards for research involving human subjects. Once the final version of the survey was developed and approved, we made it available online using a commercial, web based survey production site, SurveyMonkey (http://www.surveymonkey.com). Following this, we solicited participation in the study by sending a message to two electronic mailing lists associated with the field of instructional design and technology. The first mailing list was a private list for people associated with our university, the University of South Alabama IDD mailing list. The other was ITFORUM, a large mailing list for instructional design and technology professionals from around the world (http://it.coe.uga.edu/itforum/). Prior to sending the messages to the lists, we contacted the moderator of each list, described the purpose of our study, and asked permission to send a request for participation to their list. Both moderators agreed to allow us to send such a request. The requests were sent in November 2007. The survey remained available online for one month following the day the messages soliciting participation were sent to the lists. Participants A total of 86 people responded to the survey. Of these, nine did not rate enough books, or did not provide enough demographic information, and were excluded from the sample. This left 77 usable cases. The first demographic question was age. Thirty-three of the 77 respondents listed the year in which they were born. The respondents ranged in age from 27 to 66 years old at the time the survey was completed. The mean year of birth was 1966 or 41 years old. As a result, we grouped the respondents into 41 years old and older and those younger than 41. The second demographic question asked the participants to list their country of residence. The majority of the participants (86%) resided in the United States. Other respondents resided in Canada (2.5%), China (1.2%), the Netherlands (1.2%), Turkey (3.8%), the United Kingdom (1.2%), and South Africa (1.2%). The third demographic question asked participants to state their primary area of employment. Respondents reported that they worked in a number of areas related to instructional design. Many of the participants (46.25%) were employed as college professors. Others (17.5%) were employed as instructional designers or listed their occupation as student (11.25%). The remainder of the respondents listed their occupation as researcher, media specialist, classroom teacher, performance technologist, or a mix of occupations. The fourth demographic question asked respondents to state their highest level of education. Of the 77 respondents, the majority (74%) either had or were working towards a doctoral degree. Others (23.4%) had or were working towards a Master's degree. A few of the participants (2.6%) had or were working towards a Bachelor’s degree. A large majority of the respondents (88.3%) reported that their highest academic degree was related to instructional design and technology while the remainder (11.7%) said their highest academic degree was not related to this field. Respondents were asked to state whether their interest in instructional design and technology was more of a technical interest or more of a theoretical interest. 734 Australasian Journal of Educational Technology, 2009, 25(5) Participants could choose from five different categories ranging from “much more of a techie” to “much more of a theorist.” None of the participants in this study rated themselves as “much more of a techie”. Forty-five respondents (58%) rated themselves as either “more of a techie” or “a mix of both.” Thirty-three respondents (42%) rated themselves as “more of a theorist” or “much more of a theorist.” For purposes of statistical analysis, we grouped the respondents into two groups based on those who described themselves as theorists and those who described themselves as a “techie” or a mix of both. We also asked the participants to select which areas of instructional design in which they felt they had the most interest or expertise. The areas were analysis, design, development, implementation, and evaluation. Respondents could select more than one area. Forty-six respondents stated they had interest and expertise in analysis, 53 in design, 34 in development, 30 in implementation, and 41 in evaluation. Data analysis Results After the survey closed, we downloaded the responses from the survey website and entered them into the software program SPSS for analysis. The first analysis we conducted was a ranking of the means of all of the books for the overall sample. The books with the 10 highest means are listed in Table 2. We should note that in all of the following tables, N represents the number of respondents who rated a particular book. Table 2: Top 10 books for the overall sample based on mean score (4 = Profoundly important, 3 = Moderately important, 2= Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 70 3.48 .79 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 61 3.37 .87 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D. Kirkpatrick, 2006 60 3.36 .82 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 64 3.35 .78 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 64 3.35 .82 Instructional design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories and models) by Charles Reigeluth, 1999 66 3.33 .77 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 66 3.3 .72 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 60 3.3 .88 Principles of instructional design by Robert M. Gagné, 1992 68 3.2 .86 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 59 3.2 .82 As shown in Table 2, the 10 books with the highest means were published between 1983 and 2006. Three of the books were published in the 1980s, 2 in the 1990s, and 5 in the 2000s. The highest rated book for the overall sample was The systematic design of instruction by Dick, Carey and Carey (2005). This book is one of the most commonly used textbooks in the field of instructional design and it was not surprising to learn it was the highest rated book. In fact, 9 out of the 10 books with the highest means were Ouimette, Surry, Grubb and Hall 735 either commonly used textbooks in the field of instructional design or were written or edited by well known authors in the field. The only book in the top 10 authored by individuals somewhat outside the mainstream of instructional design was Kirkpatrick and Kirkpatrick’s (2006) book on the evaluation of training programs. Another interesting result shown in Table 2 is that one book appears on both this list of top 10 books and a list published by Braden and Sachs over 25 years ago. The conditions of learning and theory of instruction by Robert M. Gagné was included in Braden and Sachs’s (1983) list of top four books in a number of categories. Gagné’s book is also found in the top four of our overall list. Other books found on Braden and Sachs’s list include Principles of instructional design by Gagné and Briggs (1974), Instructional design by Briggs (1977), and Handbook of procedures for the design of instruction by Briggs and Wager (1981). As pointed out 25 years ago, the “dominant influence of Gagné and Briggs is hard to ignore” (Braden & Sachs, p. 9). Because he authored or co-authored four of the ten books on this list of the field’s most important books, Gagné continues to have an immense impact on our field today. After determining the most highly rated books for the overall sample, we attempted to determine the highest rated books for each of the various demographic groups in the sample. Age was the first demographic variable we examined. As noted earlier, we divided the sample into two groups based on age. Table 3 shows the top ten most highly rated books for the younger group, born after 1966. Table 3: Top 10 books based on mean score for those born after 1966 (4 = Profoundly important, 3 = Moderately important, 2= Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 10 3.80 .42 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 13 3.69 .48 Computer-supported collaborative learning in higher education by Tim S. Roberts, 2005 6 3.66 .51 The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 18 3.66 .59 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 15 3.66 .48 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 14 3.64 .63 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 16 3.62 .50 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 13 3.61 .50 Trends and issues in instructional design and technology (2nd ed.) by Robert Reiser and John V. Dempsey, 2006 10 3.60 .51 Learning and individual differences by Robert M. Gagné 1967 10 3.60 .51 As shown in Table 3, Gagné and Medsker’s 1995 book The conditions of learning: Training applications was the most highly rated book for the younger respondents. In fact, four of the top 10 books for this age group were authored, or co-authored, by Robert Gagné. This result further demonstrates Gagné’s enduring influence on the field and suggests that his influence is likely to continue at least for the next generation of IDT professionals. 736 Australasian Journal of Educational Technology, 2009, 25(5) Table 4: Top 10 books based on mean score for those born in or before 1966 (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 15 3.60 .91 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 15 3.46 .63 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 15 3.40 .73 Instructional design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 13 3.38 .86 Designing effective instruction by Gary R. Morrison, Steven M. Ross, Jerrold E. Kemp and Howard K. Kalman, 2006 13 3.30 1.03 Instructional design competencies: The standards (3rd ed.) by Rita C. Richey, Dennis C. Fields and Marguerite Foxon, 2001 13 3.30 .85 Evaluating training programs: The four levels (3rd ed) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 14 3.28 .72 Preparing instructional objectives: A critical tool in the development of effective instruction by Robert F. Mager, 1997 12 3.25 .75 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 14 3.21 .89 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 15 3.20 .67 As shown in Table 4, The systematic design of instruction by Dick, Carey and Carey (2005) was the most highly rated book by the older group (those born in or prior to 1966). This was also the most highly rated book for the overall sample. Four books appeared in the top 10 most highly rated lists for both the younger and older respondent groups. These were The systematic design of instruction (Dick, Carey & Carey), Principles of instructional design (Gagné, Wager, Golas, & Keller, 2004), Evaluating training programs: The four levels (Kirkpatrick & Kirkpatrick, 2006), and Essentials of learning for instruction (Gagné & Driscoll, 1988). After we determined the rankings for each age group, we wanted to know if there were any statistically significant differences on the ratings of the books between the older and younger respondent groups. To do this, we conducted an analysis of variance on the mean rating for each of the 99 books on the final survey based on age group. This found 30 books with statistically significant mean score differences between the groups. Table 5 shows the 5 books with the highest mean score differences. As shown in Table 5, Computer-supported collaborative learning in higher education (Roberts, 2005) was the book with the highest mean score difference between the two groups. The younger group rated it 1.76 points higher than the older group. This may indicate that the topic of collaborative learning is more interesting or important to younger IDT professionals. More research is needed to further explore this finding. Another interesting thing that was discovered during the analysis of the data was that all but one book with a statistically significant mean score difference between age groups was ranked higher by Group Two, those participants born after 1966. The only book ranked higher by Group One, those born in or before 1966, was The guidebook for performance improvement: Working with individuals and organizations by Kaufman, Thiagarajan and MacGillis (1996). One hypothesis for this result is that the younger group places more importance on books in general than the older group. This greater importance on books could be a result of the younger group being less confident in Ouimette, Surry, Grubb and Hall 737 their abilities than the older, and presumably more experienced, group. More research is needed to determine why the younger respondents tended to rate books higher than older respondents. Table 5: Top 5 books with highest mean score difference between age groups (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book Mean for Group 1 (older) Mean for Group 2 (younger) F Sig. Computer-supported collaborative learning in higher education by Tim S. Roberts, 2005 1.90 3.66 26.28 .000 Hypermedia learning environments: Instructional design and integration by Piet A. M. Kommers, Scott Grabinger and Joanna C. Dunlap, 1996 1.70 3.11 14.59 .001 The performance consultant's fieldbook: Tools and techniques for improving organizations and people by Judith Hale, 2006 1.75 3.2 16.54 .002 The ROI fieldbook: Strategies for implementing ROI in HR and training by Patricia Phillips, Jack J. Phillips, Ron Stone and Holly Burkett, 2006 1.62 3.0 15.61 .002 The handbook of training technologies: An introductory guide to facilitating learning with technology -- from planning through evaluation by William J. Rothwell, Marilynn N. Butler, Daryl L. Hunt and Jessica Li, 2006 2.12 3.5 12.20 .004 Table 6: Top 10 books based on mean score for the Analysis Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The conditions of learning and theory of instruction by Robert M. Gagné, 1985 36 3.47 .65 The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 41 3.46 .77 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 36 3.44 .80 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 33 3.42 .79 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 39 3.38 .78 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 36 3.36 .83 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 36 3.33 .86 Instructional design competencies: The standards (3rd Edition) by Rita C. Richey, Dennis C. Fields and Marguerite Foxon, 2001 32 3.31 .82 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 36 3.27 .81 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 36 3.27 .74 After looking at differences in ratings between age groups, we sought to determine the books that were most highly rated for participants with interest or expertise in each 738 Australasian Journal of Educational Technology, 2009, 25(5) major area of instructional design and technology. The major areas included in the survey were analysis, design, development, implementation, and evaluation. For our study, we asked each participant to choose which area they considered their area of primary expertise or interest. Participants could select more than one area. Table 6 shows the most highly rated books for participants in the Analysis Group (n = 46). As shown in Table 6, The conditions of learning and theory of instruction by Gagné (1985) was the most highly rated book for the analysis group. One interesting result is that no book specifically focused on analysis was among the 10 most highly rated for the Analysis Group. This could indicate that there is no one widely cited book on the topic and may suggest that the analysis phase of instructional design and technology is an area of the literature that could benefit from new books. It is also interesting to note that Evaluating training programs: The four levels by Kirkpatrick and Kirkpatrick (2006) was the third most highly rated book by the Analysis Group. This seems to suggest that those interested in analysis also have an interest in the area of evaluation or that the content of the two areas share certain core theories and concepts. The top 10 most highly rated books for the Design Group (n = 53) are shown in Table 7. As shown in Table 7, the highest rated book for the Design Group was The systematic design of instruction by Dick, Carey and Carey (2005). This is not a surprising result as that is one of the most widely used instructional design textbooks. Other widely used design textbooks among the top 10 for the Design Group are two versions of Reigeluth’s Instructional design theories and models books (1983, 1999), Designing effective instruction by Morrison, Ross, Kemp and Kalman (2006), and Instructional design by Smith and Ragan (2004). Table 7: Top 10 books based on mean score for the Design Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 46 3.41 .80 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 39 3.41 .90 Instructional design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 44 3.36 .74 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 41 3.34 .88 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 43 3.27 .88 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 42 3.26 .82 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 39 3.25 .84 Designing effective instruction by Gary R. Morrison, Steven M. Ross, Jerrold E. Kemp and Howard K. Kalman, 2006 41 3.24 .66 Principles of instructional design by Robert M. Gagné, 1992 45 3.20 .96 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 40 3.20 .82 The top 10 most highly rated books for participants in the Development Group (n = 34) are shown in Table 8. As shown in Table 8, the highest rated book for this group was Ouimette, Surry, Grubb and Hall 739 Essentials of learning for instruction by Gagné and Driscoll (1988). This was also the 6th highest rated book for the Design Group. In fact, 9 of the 10 books in the Development Group’s top 10 were also in the top 10 for the Design Group. The only book among the top 10 for the Development Group not in the Design Group’s top 10 was The conditions of learning: Training applications by Gagné and Medsker (1995). This similarity between the results of the Design Group and the Development Group could indicate that there are no books specifically related to development that are widely seen as essential or that there was a significant overlap of respondents who had interest and expertise in both design and development. Table 8: Top 10 books based on mean score for the Development Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 26 3.46 .76 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 27 3.44 .80 Principles of instructional design by Robert M. Gagné, 1992 28 3.42 .99 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 23 3.39 .98 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 25 3.36 .90 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 23 3.34 .88 The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 21 3.33 .79 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 24 3.33 .96 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 22 3.31 1.04 The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 28 3.28 .93 The top 10 most highly rated books for the Implementation Group (n = 30) are shown in Table 9. As with the analysis and development areas, there was no book specifically on the topic of implementation among the top 10 most highly rated books for the Implementation Group. As shown in Table 9, the majority of the most highly rated books for the Implementation Group were more focused on the broad array of IDT processes. As with the other areas, this could indicate that there is a need for a book specific to implementation or that there was a high degree of overlap between respondents in the Implementation Group and the Design Group. The top 10 most highly rated books for the Evaluation Group (n = 41) are shown in Table 10. A somewhat surprising result is that Evaluating training programs: The four levels (3rd ed.) by Kirkpatrick and Kirkpatrick (2006) was only the fourth most highly rated book for the Evaluation Group. In fact, nine out of the 10 most highly rated books by the Evaluation Group were more general IDT books. Again, the likely cause of this is a high degree of overlap between respondents in the Design Group and the Evaluation Group. The results shown in Table 10 do indicate that the Kirkpatrick and Kirkpatrick book is seen as the most essential text specifically related to evaluation by IDT professionals. 740 Australasian Journal of Educational Technology, 2009, 25(5) Table 9: Top 10 books based on mean score for the Implementation Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 23 3.60 .78 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 20 3.60 .59 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 19 3.52 .69 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 23 3.47 .84 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 23 3.47 .66 The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 19 3.47 .61 Instructional design competencies: The standards (3rd ed.) by Rita C. Richey, Dennis C. Fields and Marguerite Foxon, 2001 19 3.42 .76 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 21 3.38 .74 Learning and individual differences by Robert M. Gagné, 1967 19 3.36 .83 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 22 3.36 .72 Table 10: Top 10 books based on mean score for the Evaluation Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The conditions of learning and theory of instruction by Robert M. Gagné, 1985 31 3.54 .67 The systematic design of instruction by Walter Dick, Lou Carey, and James Carey, 2005 36 3.47 .81 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 30 3.43 .77 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 35 3.37 .80 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 35 3.37 .73 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 33 3.36 .74 Principles of instructional design by Robert M. Gagné, 1992 38 3.28 .89 The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 29 3.27 .79 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 33 3.27 .83 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas, and John M. Keller, 2004 33 3.27 .83 After determining the most highly rated books for respondents with expertise in each area of IDT, we sought to determine if there were any differences between respondents with a doctoral degree and those without a doctoral degree. As stated earlier, a total of 57 respondents to the survey (74%) either held, or were working towards, a doctoral Ouimette, Surry, Grubb and Hall 741 degree. The remaining 20 (26%) did not hold a doctoral degree. The most highly rated books for the Doctoral Degree Group are shown in Table 11. Table 11: Top 10 books for Doctoral Degree Group (4 = Profoundly Important, 3 = Moderately Important, 2 = Moderately Unimportant, 1 = Totally Unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 55 3.49 .81 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 51 3.33 .84 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 51 3.31 .88 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 49 3.30 .93 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 50 3.30 .90 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D. Kirkpatrick, 2006 47 3.27 .87 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 54 3.25 .75 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 53 3.24 .78 Principles of instructional design by Robert M. Gagné, 1992 53 3.22 .93 Designing effective instruction by Gary R. Morrison, Steven M. Ross, Jerrold E. Kemp and Howard K. Kalman, 2006 52 3.19 .71 Table 12: Top 10 books for Non-Doctoral Degree Group (4 = Profoundly Important, 3 = Moderately Important, 2 = Moderately Unimportant, 1 = Totally Unimportant) Book N Mean SD Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 12 3.75 .45 Task analysis methods for instructional design by David H. Jonassen, Martin Tessmer and Wallace H. Hannum, 1998 6 3.66 .81 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 12 3.66 .65 Computers as mindtools for schools: Engaging critical thinking, (2nd ed.) by David H. Jonassen, 1999 9 3.66 .50 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 11 3.63 .50 The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 9 3.55 .52 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 11 3.54 .52 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 12 3.50 .52 Psychology of learning for instruction (3rd ed.) by Marcy P. Driscoll, 2004 10 3.50 .70 Principles of instructional design by Robert M. Gagné, 1992 14 3.50 .51 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 8 3.5 .51 742 Australasian Journal of Educational Technology, 2009, 25(5) As shown in Table 11, The systematic design of instruction by Dick, Carey and Carey (2005) was the most highly rated book for the Doctoral Degree Group, as it was for the overall sample. This is understandable since the majority of respondents fell into the Doctoral Degree Group. Table 12 shows the highest rated books for the Non-Doctoral Degree Group. As shown in Table 12, Evaluating training programs: The four levels by Kirkpatrick and Kirkpatrick (2006) was the most highly rated book for the Non-Doctoral Degree Group. Two other highly rated books for this group that did not appear in the top 10 for the Doctoral Degree Group were Task analysis methods for instructional design by Jonassen, Tessmer, and Hannum (1998) and Computers as mindtools for schools: Engaging critical thinking, by Jonassen (1999). These two books seem to be more focused on practical application than on theoretical concepts and their high ratings may suggest that the Non-Doctoral Degree Group has a more applied focus than the Doctoral Degree Group. Related to this, it is interesting to note that the highest rated book for the Doctoral Degree Group (Dick, Carey & Carey, 2005) was not among the 10 highest rated books for the Non-Doctoral Degree Group. There are a number of possible explanations for this. One possible explanation is that the Dick, Carey and Carey book may be more valued as a textbook or conceptual tool than as an applied resource for practicing IDT professionals. Table 13: Top 10 books based on mean score for respondents whose highest degree is in the field of Instructional Design and Technology (4 = Profoundly Important, 3 = Moderately Important, 2 = Moderately Unimportant, 1 = Totally Unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 62 3.50 .78 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 52 3.38 .86 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 58 3.36 .74 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 53 3.35 .90 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 55 3.32 .81 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 56 3.32 .85 Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 54 3.29 .88 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 58 3.29 .74 Trends and issues in instructional design and technology (2nd ed.) by Robert Reiser and John V. Dempsey, 2006 45 3.24 .90 Principles of instructional design by Robert M. Gagné, 1992 59 3.23 .89 As shown in Table 13, the highest rated book for respondents whose highest degree was related to IDT was The systematic design of instruction by Dick, Carey and Carey (2005). The highest rated book for respondents whose highest degree was not related to IDT (see Table 14) was Instructional design theories and models: An overview of their current status by Reigeluth (1983). It is interesting to note that the Dick, Carey and Carey (2005) book, which was the highest rated for the IDT Degree group, was not Ouimette, Surry, Grubb and Hall 743 among the top 10 for the Non-IDT Degree group. The small number of respondents in the non-IDT makes further analysis or discussion difficult. Future research in this area should try to include more participants with academic backgrounds unrelated to IDT. Table 14: Top 10 books based on mean score for respondents whose highest degree is not in the field of Instructional Design and Technology (4 = Profoundly Important, 3 = Moderately Important, 2 = Moderately Unimportant, 1 = Totally Unimportant) Book N Mean SD Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 6 3.66 .51 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 7 3.57 .53 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 7 3.57 .53 Principles of instructional design by Robert M. Gagné, 1992 7 3.57 .53 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 6 3.50 .54 Classic writings on instructional technology by Donald P. Ely and Tjeerd Plomp, 2001 5 3.40 .89 Instructional design competencies: The standards (3rd ed.) by Rita C. Richey, Dennis C. Fields and Marguerite Foxon, 2001 3 3.33 .57 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 6 3.33 .51 Principles of instructional design by Robert M. Gagné and Leslie J. Briggs, 1979 7 3.28 1.11 e-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning by Ruth Clark and Richard E. Mayer, 2002 7 3.28 .48 The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 7 3.28 .75 Table 15: Top 10 books based on mean score for the Theorist Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD Handbook of research on educational communications and technology by David H. Jonassen and Phillip Harris, 2003 25 3.60 .64 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 26 3.53 .64 The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 30 3.50 .57 Instructional-design theories and models: A new paradigm of instructional theory, Vol. 2 (Instructional design theories & models) by Charles Reigeluth, 1999 28 3.46 .69 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 25 3.36 .90 Psychology of learning for instruction (3rd ed.) by Marcy P. Driscoll, 2004 27 3.33 .87 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 27 3.33 .78 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 28 3.32 .90 Instructional design (Wiley/Jossey-Bass Education) by Patricia L. Smith and Tillman J. Ragan, 2004 25 3.28 .67 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 29 3.27 .70 744 Australasian Journal of Educational Technology, 2009, 25(5) As discussed earlier, we asked respondents to the survey to state whether they considered their interest in IDT to be primarily a theoretical interest or a technical interest. Approximately 42% of the respondents described themselves as “more of a theorist” or “much more of a theorist.” We grouped these respondents into the Theorist Group. Approximately 58% of the respondents described themselves as either “techies” or a mix of both “techie” and theorist. We combined these respondents in the Non-Theorist Group. The results for the Theorist Group are shown in Table 15. As shown in Table 15, the most highly rated book for the Theorist Group was Handbook of research on educational communications and technology by Jonassen and Harris (2003). This was not a surprising result as that book provides in depth discussion of the research related to most of the important ideas and concepts in the field. It was also not surprising to find that the Jonassen and Harris book was not among the top 10 most highly rated books for the Non-Theorist Group (see Table 16). It was somewhat interesting to find that The systematic design of instruction by Dick, Carey and Carey (2005) was the most highly rated book for the Non-Theorist Group. The Dick, Carey and Carey book was not among the 10 most highly rated books for respondents without a doctoral degree (Table 12) but was the highest rated book for the Non- Theorist Group. This suggests that there may be no relationship between a person’s highest degree and whether they consider themselves to be more of a “techie” or more of a theorist. That is an interesting possibility that requires additional research. Table 16: Top 10 books based on mean score for the Non-Theorist Group (4 = Profoundly important, 3 = Moderately important, 2 = Moderately unimportant, 1 = Totally unimportant) Book N Mean SD The systematic design of instruction by Walter Dick, Lou Carey and James Carey, 2005 39 3.46 .94 Principles of instructional design by Robert M. Gagné, Walter W. Wager, Katharine Golas and John M. Keller, 2004 35 3.37 .77 Instructional design theories and models: An overview of their current status by Charles Reigeluth, 1983 35 3.37 .87 The conditions of learning and theory of instruction by Robert M. Gagné, 1985 36 3.36 .79 Principles of instructional design by Robert M. Gagné, 1992 38 3.34 .87 Essentials of learning for instruction by Robert M. Gagné and M. Driscoll, 1988 36 3.33 .75 Learning and individual differences by Robert M. Gagné, 1967 33 3.30 .88 Evaluating training programs: The four levels (3rd ed.) by Donald L Kirkpatrick and James D Kirkpatrick, 2006 33 3.24 .93 Instructional design competencies: The standards (3rd ed.) by Rita C. Richey, Dennis C. Fields and Marguerite Foxon, 2001 29 3.24 .95 The conditions of learning: Training applications by Robert M. Gagné and Karen L. Medsker, 1995 32 3.21 .83 As mentioned earlier, respondents to the survey were given the opportunity to list any books they felt were profoundly important that were not included among the 99 books on the final survey. The 33 respondents to this question listed a total of 70 books, including an amazing array from diverse fields including management, psychology, computer science and organisational development. Only one book, however, was listed more than once: Training complex cognitive skills: A four-component instructional design model for technical training (Van Merriënboer, 1997) was listed three times. Ouimette, Surry, Grubb and Hall 745 Discussion Based on the results of this study, we believe there are three findings that are particularly interesting or important to the field of instructional design and technology. The first such finding is that the body of literature that forms the theoretical foundation of the field of IDT is both diverse and well defined. Not surprisingly, depending on what area of the field a respondent was interested in, different books were rated as being more or less important. For example, 23 of the 99 books in the final survey were rated among the top 10 most important for at least one demographic group. However, there were also a few books that were highly rated by most groups. Ten of the 99 books in the final survey were rated among the top 10 most important for 8 or more demographic groups. Not surprisingly, these 10 were the same books rated in the top 10 for the overall sample (see Table 2). These 10 books can, therefore, safely be classified as the most important, widely recognised, and enduring texts even in an ever growing, diverse, and changing field. The second interesting and important finding is that age played a major role in rating of the importance of each of the books. Younger respondents to the survey rated more books to be more important than did older respondents. Out of the thirty books found to have statistically significant mean score differences between the two age groups, the younger group ranked 29 books as being more important than did the older group. The older group only rated one book to be more important than the rating of the younger group. One hypothesis for this is that the younger respondents rely more on the literature than the older group, perhaps due to a lack of experience. Another hypothesis is that the younger group included more respondents who are still in school, or who are recent graduates and are, therefore, more influenced by the texts they used in school. The third interesting and important finding is that respondents who stated an interest or expertise in the various areas of IDT tended to have similar ratings. Those who stated an interest in the areas of analysis, development, implementation, and evaluation tended to rate general IDT books higher than books specific to their area of expertise. As mentioned earlier, this may be because respondents commonly stated expertise in several different areas, resulting in a high degree of overlap between the groups. However, it is also possible that the results show that the areas of analysis, development, implementation, and to a lesser degree, evaluation could benefit from the addition of newer books targeted especially for those areas. Much more research is needed to build on and extend the findings of this study. Areas for further research on this topic include involving more participants who consider themselves to have more of a technical interest in IDT as opposed to more of a theoretical interest. Because theorists were over represented in our sample, additional research will provide more insight into the differences between books rated highly by “techies” and theorists. There is also a need for more participants from outside the United States. The vast majority of participants in our study were from the U.S. and there are likely important differences in the rating of books by IDT professional in other countries. It may also be interesting to conduct research to determine if there are differences in the importance of books based on gender. We did not include gender as a demographic variable in this study. It may also be interesting to study the essential books from fields closely related to IDT, specifically learning sciences and performance technology. Because books represent only one part of the literature of a field, it may be 746 Australasian Journal of Educational Technology, 2009, 25(5) useful for future researchers in this area to study which articles or journals are seen as essential to IDT professionals. Researchers may wish to study the impact of the rapidly growing availability of e-books on rankings in the future. One possible limitation of the study is that respondents to the survey may have based their ratings of the books on the reputation of the authors without having read or even knowing much about the actual book. Therefore, there is likely a need for a study that employs an expert analysis method instead of a survey. Finally, instead of rating current books, future researchers could explore areas where there are gaps in the literature and identify topics on which more books could be written. The books listed in this study, while certainly not an exhaustive list of useful or important books, describe a field that has a robust and dynamic knowledge base. In addition, the books listed in this paper represent an interesting mix of classic works by well known authors and more recent works by emerging leaders in the literature. We believe this is a very positive sign for the future of our field. In conclusion, it is important for the members of any field to periodically reflect on the key ideas, theories, and practices that form the knowledge base of their field. In this study, we have attempted to contribute to this ongoing process of self reflection by determining which books are considered essential in the field of instructional design and technology at this point it its history. We believe additional research in this area will provide important insights into our field and contribute to the continued growth and refinement of instructional design and technology. References Braden, R. A. & Sachs, S. G. (1983). Basic references on instructional development. Paper presented at the annual meeting of the Association of Educational Communications and Technology, New Orleans, LA. ERIC Document Reproduction Service No. ED239579. http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED239579 Briggs, L. J. (1977). Instructional design. Englewood Cliffs, NJ: Educational Technology Publications. Briggs, L. J. & Wager, W. W. (1981). Handbook of procedures for the design of instruction. Englewood Cliffs, NJ: Educational Technology Publications. Carey, J., Carey, L. & Dick, W (2005). The systematic design of instruction. Boston, MA, Allyn and Bacon. Clark, R. C. & Mayer, R. E. (2002). E-learning and the science of instruction. Hoboken, NJ: John Wiley. Driscoll, M. P. (1994). Psychology of learning for instruction. Boston: Allyn and Bacon. Ely, D. P. & Plomp, T. (1996). Classic writings on instructional technology. Englewood, CO: Libraries Unlimited, Inc. Gagné, R. M. (1967). Learning and individual differences. Columbus, Ohio: Merrill. Gagné, R. M. (1985). The conditions of learning and theory of instruction. New York: Holt, Rinehart and Winston. Gagné, R. M. & Briggs, L. (1974). Principles of instructional design. New York: Holt, Rinehart and Winston. Gagné, R. M., & Driscoll, M. P. (1988). Essentials of learning for instruction (2nd ed.). Upper Saddle River, NJ: Prentice Hall. Ouimette, Surry, Grubb and Hall 747 Gagné, R.M. & Medsker, K.L. (1996). The conditions of learning: Training applications. Fort Worth, TX: Harcourt Brace. Hale, J. (2006). The performance consultant’s fieldbook: Tools and techniques for improving organizations and people. San Francisco: Jossey-Bass Pfeiffer. Jonassen, D. H. & Harris, P. (Eds.). (2003). Handbook of research for educational communications and technology (2nd ed.). Mahwah, NJ: Erlbaum. Kirkpatrick, D. & Kirkpatrick, J. (2006). Evaluating training programs. San Francisco: Berrett- Koehler Publishers, Inc. Kommers, P. A. M, Grabinger, S. & Dunlap, J. C. (1996) Hypermedia learning environments: Instructional design and integration. Hillsdale, NJ: Lawrence Erlbaum Associates. Mager, R. F. (1997). Preparing instructional objectives: A critical tool in the development of effective instruction (3rd ed.). Atlanta: Center for Effective Performance. Morrison, G. R., Ross, S. M. & Kemp, J. E. (2006). Designing effective instruction. New York: Holt, Rinehart and Winston, Inc. Phillips, P. P. (2007). The ROI fieldbook: Strategies for implementing ROI in HR and training. Improving human performance series. Amsterdam: Butterworth-Heinemann. Reigeluth, C. M. (1999). Instructional design theories and models: A new paradigm of instructional theory, Volume II. Mahwah, NJ: Lawrence Erlbaum Associates. Reigeluth, C. M. (Ed.) (1983). Instructional design theories and models. Hillsdale, NJ: Lawrence Erlbaum. Reiser, R. A. & Dempsey, J. V. (Eds.) (2006). Trends and issues in instructional design and technology. Upper Saddle River, NJ: Merrill/Prentice Hall. Richey, R., Fields, D., Foxon, M., Roberts, R., Spannaus, T. & Spector, J. M. (2001). Instructional design competencies: The standards (3rd ed.). Syracuse, NY: ERIC Clearinghouse on Information and Technology. http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ ERICServlet?accno=ED453803 Roberts, T. S. (Ed.) (2004). Computer-supported collaborative learning in higher education. Hershey, PA: Idea Group Inc. Rothwell, W. J. (2006). The handbook of training technologies: An introductory guide to facilitating learning with technology - from planning through evaluation. San Francisco: Pfeiffer. Smith, P. L. & Ragan, T. J. (2004). Instructional design. Columbus, OH: Merrill. Van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A four-component instructional design model for technical training. Englewood Cliffs, NJ: Educational Technology Publications. Jenelle Ouimette, University of South Alabama. Email: jmo303@jaguar1.usouthal.edu Dr Daniel W. Surry, Department of Professional Studies, College of Education, University of South Alabama. Email: dsurry@usouthal.edu Web: http://www.southalabama.edu/coe/surry.shtml Adrian Grubb, University of South Alabama. Email: agg602@jaguar1.usouthal.edu David A. Hall, e-Center@JSU, Jackson State University. Email: david.a.hall@jsums.edu