Evidence Based Library and Information Practice Evidence Based Library and Information Practice 2011, 6.1 4 Evidence Based Library and Information Practice Article Evidence or Evidence Based Practice? An Analysis of IASL Research Forum Papers, 1998- 2009 Marcia A. Mardis, Ed.D. School of Library and Information Studies College of Communication and Information The Florida State University, Florida, United States of America Email: mmardis@fsu.edu Received: 28 Aug. 2010 Accepted: 31 Jan. 2011 2011 Mardis. This is an Open Access article distributed under the terms of the Creative Commons-Attribution- Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective - Conferences are essential opportunities for professional development and for learning about research. This study analyses papers presented in the Research Forum track of the International Association of School Librarians (IASL) conferences to determine whether the amount of school library research reporting increased or decreased over time; who (i.e., what author roles and affiliations) has written about research; which countries were represented in the research articles; what topics were discussed in research articles; and what research methodologies were used. The aim was to determine the extent to which the Research Forum provides research evidence that relates to practice. Methods - This study continues the longitudinal analysis of published school library research begun by Clyde (1996) by analyzing Research Forum papers published in IASL conference proceedings from 1998-2009 and using the same approaches and metrics as previous studies by Clyde (e.g., 1996; 2002; 2004), Clyde and Oberg (2004), and Oberg (2006). Results - Conference paper topics, author origins, quantities, and research approaches remained static through the 11 years analyzed. The analysis reveals that the papers’ authors, methods, and topics reflected those found in previous studies of school library research. As well as replicating previous studies, the role of academic research at a practitioner-based conference was investigated. mailto:mmardis@fsu.edu� Evidence Based Library and Information Practice 2011, 6.1 5 Conclusions - Based on long-established imperatives from leaders in the profession, the IASL conferences provide both evidence and evidence based practice for school librarians from all over the world. However, when scholarly research is shared at practitioner venues, it is possible that school librarians may assume that research results constitute evidence based practice (EBP), not evidence upon which practice should be based. This distinction is important if considering that the purpose of academic research is to objectively inform, not to advocate a particular position or practice. The Research Forum can be a valuable venue for the presentation of empirical research findings and conclusions and objective program evaluations and provide a valuable complement to the evidence based practice descriptions shared in the Professional Papers portion of the conference program. It is argued that the Research Forum must be clear in its purpose: to present the results of research; to present effective practice determined by rigorous evaluation; or to present research-supported arguments for the support of school libraries. Through a reconceptualization of EBP, the paper demonstrates how EBP is both a method and a methodology for the presentation of school library research and practice in a conference atmosphere. Introduction Conferences are essential sources of professional development for school librarians (Alaimo, 2004; Genco, 2003). Often working in isolation, school librarians have few learning and professional growth opportunities that link directly to their work (Miller, 1999). With limited access to published research conducted in their field, school librarians may be left to make decisions without the benefit of objective evidence in which to ground to their practice (Clyde, 2003a). In an effort to promote evidence based practice, in 1998, the International Association of School Librarians (IASL) conference began providing participants with a variety of presentations that encompass trends and innovations in school librarianship through the sharing of best practices and original research (Clyde & Oberg, 2004). Each IASL conference identifies a unique overarching theme and subthemes; participants tailor their paper presentations to the specific themes of the conferences through professional papers presented in the Professional Papers track and research papers presented in a conference track called the Research Forum. Purpose of This Research This study analyzes, Forum papers published in IASL conference proceedings from 1998- 2009 based on the approaches used in previous studies by Clyde (e.g., 1996; 2002; 2004), Clyde and Oberg (2004), and Oberg (2006). Guided by a desire to discover the extent to which research in school librarianship constituted evidence based practice (Oberg, 2006), this analysis focuses on the extent to which LIS journals facilitated school librarians’ use of “‘good evidence’ including evidence from research and evidence from practice” (Clyde, 2003, p.26). To this end, data were analysed to determine whether the amount of school library research reporting increased or decreased over time; who (i.e., what author roles and affiliations) has conducted the research; which countries were represented in the research articles; what topics were discussed in research articles; and what research methodologies were used. The purposes of this study were to determine the extent to which an analysis of the Research Forum papers confirms the findings of Clyde and Oberg’s prior analyses of journal and conference papers between 1995 and 2006 and if the Research Forum papers affirm IASL’s mission “to provide an international forum for those people interested in promoting effective Evidence Based Library and Information Practice 2011, 6.1 6 school library media programs as viable instruments in the educational process” (IASL, 2007, ¶2) for, in the words of IASL founder Jean Lowrie (2011), “the beginning librarian as well as the person who does research…and to push the balance between research and practice.” Theoretical Perspective The needs of new practitioners provide an additional motivation for characterizing research in school librarianship: to determine the extent to which it can contribute to evidence based practice (EBP) (Oberg, 2006). EBP is an approach to practice from medicine, social work, public health, and related fields in which practitioners develop strategies to aid clients and make changes based on the findings of objective, scientifically based research. (McKibbon, 1998). EBP has formed the basis for evidence based librarianship (EBL), in which librarians make decisions about their practice based on objective research findings and evaluate the outcomes of those decisions in light of research, using evidence in a cycle of improvement (Booth, 2003). Through its Research Forum, IASL hopes to promote “the message that research studies are important in informing and advancing professional practice in school librarianship” (Lighthall & Haycock, 1997, p.xiii). Literature Review In an effort to characterize research in library and information studies (LIS), researchers have examined the contents of journals to identify trends in research topics and approaches; these researchers found that surveys and descriptive studies relating to information retrieval and library user education tended to dominate the research articles (Järvelin & Vakkari, 1993; Julien, 1996; Clyde 2002). The examined studies tended to describe the method (i.e., the “how to”) of research versus the methodology (i.e., the philosophical underpinnings) of research (Hjørland, 2000), leaving the reader to appreciate the results of the reported study but not the motivations behind it. Prior characterizations of school library research The role of research in school librarianship has long been debated. As early as 1950, school library leader Frances Henne lamented the lack of scientific research in the field and pointed out that: A lag exists between the theoretical opinion in the field and research. Most functions and objectives of school library practice have been formulated pragmatically on the basis of opinion and experience rather than on evidence produced in research. Too, most of these judgement values, even after long adoption and operation, have not been tested or evaluated through the medium of objective research (Henne, 1950, p.701). For decades, researchers (e.g., Lowrie, 1968; Clyde 1996) have been examining published school library research. Many of these studies (e.g., Clyde 2003b; Clyde & Oberg, 2004; Oberg, 2006) reported that researchers from the United States, Canada, and Australia authored the majority of published school library research (i.e., peer-reviewed journal articles and refereed conference papers) in all languages available in school librarianship. University faculty and doctoral students overwhelmingly provided authorship while practitioners, government officials, and other policymakers and stakeholders represented a scant number of authors (Lowrie, 1968; Clyde, 1996; Clyde & Oberg, 2004). Research approaches in school librarianship were typically focused on survey and qualitative methods like focus groups, observation, interview, and analysis of student work (Lowrie, 1968; Clyde, 1996; Clyde, 2003b). Investigations conducted in school libraries have predominantly centered on information literacy, reading, and instruction (Lowrie, 1968; Clyde, 1996; Clyde, 2003b; Oberg 2006). Utility of Characterizing Research Reasons for characterizing published research are manifold. Knowledge of the subject distribution of research in library and Evidence Based Library and Information Practice 2011, 6.1 7 information science can identify gaps for further research exploration (Blessinger & Hrycaj, 2010). Likewise, an understanding of the research approaches commonly used can identify overreliance on particular approaches or the need to equip researchers with methods more appropriate for the topics under investigation (Clyde, 2003b). The checks of the peer review process often act as an endorsement of a particular piece of published research (Blessinger, & Hrycaj, 2010). For university faculty and doctoral students, conducting and presenting objective research is part of the academic culture and research quality is primarily based on the execution of the method and analysis of the data; for practitioners, research quality is determined by the ease with which study’s conclusions can be enacted in their own environments and the benefits those actions yield. Consideration of the tension between the research objectivity and advocacy may be particularly important to the study of school library research (Clyde & Oberg, 2004). School Library Research and Evidence Based Librarianship The need to promote EBP is inherent in school library research. Historically, school library research has not been considered extensive, rigorous, or of high quality (Haycock, 1996; McClure & Bishop, 1989). This legacy is particularly troubling in an era when the “gold standard” of educational research has been promoted as the use of randomized controlled trials (RCTs) (What Works Clearinghouse, 2010), a method frequently used improperly in educational research (Brass, Nunez-Neto, & Williams, 2006), infrequently seen in LIS research (Järvelin & Vakkari, 1993) and rarely used in school library research (Gordon, 2006). Because RCTs involve the use of randomly assigned participants in control groups and treatment groups, to deprive children of a possibly beneficial educational intervention by including them in a control group raises ethical concerns as well (Morrison, 2001). Therefore, researchers have sought other methods to assess the soundness of school library research (e.g., Clyde, 2004) and its ability to inform practice that benefits children’s learning (Todd, 2009) to remain relevant in the educational policy context (Booth, 2003). To date, these alternate methods of establishing the soundness of research have been based on an investigation of the peer review process (Clyde 2004, 2006) and the evaluation of the outcomes of practice based on research (Todd, 2009). As the use of EBP has become more common in LIS, it has evolved into a type of cycle in which the outcomes of the evidence based decision have become the object of study and, in essence, project and performance evaluations have become data sources. Program and project evaluation, often in the form of action research, has been positioned as an inferential method of determining if a particular practice achieves a desired outcome, especially if that outcome is improved student learning. However, the use of EBL in school librarianship has been labeled as blurring the evidence based paradigm because research pursuits are often a response to a perceived threat to the profession or a need to advocate for an aspect of practice. This implicit motivation for undertaking research as EBP shapes the context in which results are examined and the ways in which conclusions are framed: [S]chool librarianship has forsaken two central tenets of EBP [evidence- based practice]...First, it has neglected the requirement for impartiality in the collection and interpretation of data. Second, it has diverted the focus of the evidence-based analysis away from client needs. The result is a topsy- turvy practice of collecting evidence for the express purpose of promoting school libraries as an effective educational intervention…Advocacy has no place in EBP. In fact, avoiding bias and prejudiced agendas is the main reason these practices developed in the first place (Lyons, 2009, p13). Evidence Based Library and Information Practice 2011, 6.1 8 Despite valid as concerns about the use of a strict definition of EBL in school librarianship, critics (e.g., Lyons, 2009) seem to have overlooked the incompatibility between scientific methods and the unique political and policy context of research in schools as well as the different uses for evidence in an applied science like LIS versus an original context like medical research (Booth, 2003). Proponents of school library EBL have explored and justified their adapted definitions (Clyde, 2006; Todd, 2009) based on this incompatibility. Aims The July 2006 issue of School Libraries Worldwide (SLW), IASL’s peer-reviewed research journal, contained an article that presented distributions of papers published between 1995-2006 (Oberg, 2006) in a replication of Clyde’s earlier studies (1994, 2002, 2003b, 2006) and Clyde and Oberg’s work together (2004). Oberg’s 2006 study was a quantitative analysis that focusing on the frequency of the occurrence of the phenomena specified in the research questions and an assessment of the degree to which SLW’s contents provided support for school library professionals engaging in evidence based practice. The study described here, repeats and extends Oberg’s questions and analyses. Research Questions The following research questions were investigated. These questions were also used by Clyde (2003a), Clyde and Oberg (2004), and Oberg (2006): 1. How have the numbers of research papers changed over time? 2. What were the roles, affiliations, and geographic origins of the research paper authors? 3. What were the research paper topics and methods? How have the topics and methods changed? 4. To what extent do the Research Forum papers constitute support for evidence based practice? The fourth research question required an interpretation of the relationship between EBP and research as well as of the Research Forum. The unique nature of this question necessitated that it be addressed in the Discussion and Conclusion section rather than addressed with the quantitative results. Methods Previous studies examined papers between 1995 and 2006. For this study, papers in conference proceedings from 1998 through 2009 were analysed, an overlapping 11 year time frame. Though Research Forum began in 1997, the papers were not assigned a Research Forum designation in the 1997 Proceedings. Only papers presented as part of the Research Forum track of the conferences were included in this analysis. Ultimately, 199 papers were included in the study sample. Description of the Analysis In the first phase of the research project, a working data file that contained metadata for each paper was created: Year; Title; Author; Role; Affiliation; Author Origin; and Conference Location. Then, additional metadata for each paper including was generated: Topic; Research Approach; Research Method. The papers’ abstracts and full text were analyzed to populate the topic field with controlled vocabulary based on the categorization of Oberg (2006). The research papers were designated as using a quantitative, a qualitative, or a mixed methods approach. The method used within each approach was described based on a controlled vocabulary of the research designs informed by Creswell (2006). The next phase of the project included an examination of the results in light of the research questions and the preparation of a research report. The Statistical Package of the Social Sciences (SPSS) was used to generate Evidence Based Library and Information Practice 2011, 6.1 9 frequency and descriptive statistics, charts, and tables in response to the research questions. In the final phase of the project, the coding of topic, research approach, and research method designations were validated through two additional checks by two doctoral students. Results Overall, the results of the analyses of the first three research questions were similar to the previous findings of Clyde (2003a), Clyde and Oberg (2004), and Oberg (2006) as detailed explorations of each research question illustrate. Numbers of Research Papers Over Time A total of 199 papers were published in the Research Forum proceedings between 1998 and 2009, with a mean of 18 papers published in each proceedings volume. Table 1 illustrates the number of Research Forum papers presented at each conference location and that conference’s percentage of the overall total. As Figure 1 illustrates, the number of Research Forum papers varied from conference to conference, but overall, the number of papers tended to increase with a peak (n=30) in the 2008 proceedings. The results suggested that although the number of research papers presented at the IASL conference varied significantly from year to year, the number of papers presented tended to increase over time. Roles, Affiliations, and Geographic Origins of the Research Paper Authors USA (n=60), Canada (n=24), and Australia (n=21) represented the largest number of first author origins, as Table 2 depicts. Because all but one jointly authored paper represented the same country collaborations and the sole jointly authored paper represented authors from USA and Australia, no statistical diversity was gained by counting additional authors. Table 1 Number of Research Forum Papers from 1998-2009 (N=199) Year (Location) Frequency Percent 1998 (Israel) 9 4.5 1999 (USA) 20 10.1 2000 (Sweden) 11 5.5 2001 (New Zealand) 16 8.0 2002 (Malaysia) 12 6.0 2003 (South Africa) 11 5.5 2004 (Ireland) 9 4.5 2005 (Hong Kong) 20 10.1 2006 (Portugal) 21 10.6 2007 (Taiwan) 13 6.5 2008 (USA) 30 15.1 2009 (Italy) 27 13.6 Total 199 100.0 Evidence Based Library and Information Practice 2011, 6.1 10 Figure 1. Numerical trend of Research Forum papers presented from 1998-2009 (N=199). Another way of looking at the geographic origins of the research papers is to categorize the authors by their IASL membership zones. The terms of IASL membership are based on country of residence and published gross national product indexes for that country, with international schools being in Zone A. Zone A countries are the most developed and Zone C countries are the least developed. The Appendix includes a list of countries in each zone. Zone A authors represented 93% (n=185) of the Research Forum papers in proceedings with Zone B representing 6% (n=12) and Zone C representing 1% (n=2) of the papers. The total number of authors for the papers was 308. University faculty or students wrote approximately 77% (n=238) of the research papers. School librarians authored about 6% (n=19) of the papers. The remaining author roles included librarians in and directors of other types of libraries (n=30 or 10%); principals, teachers, technology directors, and guidance counselors (n=9 or 3%); consultants (n=6 or 2%); and education officials (n=6 or 2%). Research Paper Topics and Methods For this question, the classification scheme for library and information science (LIS) topics developed by Järvelin and Vakkari (1990) as amended by Oberg (2006) was applied to the papers. Table 4 illustrates that the “Information skills and literacy” category was applied to the most papers (n=56 or 28.1). These papers reported research relating to the teaching and learning of information literacy skills and processes as opposed to papers coded as “Information seeking” (n=10 or 5%) which reported research pertaining to query formation and search strategy. “Information technology” was the topic of 10.6% of papers (n=21) while “Reading and reading promotion” papers (n=19 or 9%) reported efforts to increase interest and frequency in the reading of text presented in paper media. Papers that reported research relating to preparation of school librarians, “Education in LIS” comprised 8% (n=16) of papers while papers about “The profession” (n=14 or 7%) addressed topics such as professional standards, roles, and responsibilities as well as perceptions of in-service school librarians. Evidence Based Library and Information Practice 2011, 6.1 11 Papers relating to research on school libraries in the context of LIS, “Analysis of LIS” (n=12) information policy and librarianship, “Other aspects of LIS “(n=11), and the library program in the school environment, “Library and information science activities,” (n=10) comprised 17.5% of papers. Studies that pertained to the philosophy of and techniques for conducting research in school libraries were coded as “Methodology” and comprised 4.5% (n=9) of papers. The remaining 10% of papers addressed publishing children’s and professional literature, “Publishing” (n=7 or 3.5%), surveys of national conditions of school libraries “National survey,” (n=6 or 3%), “Censorship” of library resources (n=4 or 2%), “Library history,” (n=2 or 1%) and “Principal support” (n=2 or 1%). Three of Järvelin and Vakkari’s classifications, “Other studies,” “Scientific and professional communication” and “Information storage and retrieval,” did not apply to any papers. In an effort to gain insight into the potential professional applications of the research reports, paper topics were also matched to school librarian roles as espoused by Information Power (AASL & AECT, 1998) and affirmed by Empowering Learners (AASL, 2008), guidelines for school librarians supported by the American Association of School Librarians (AASL). These roles provided classifications for the possible practice application of the research papers’ content. As shown in Figure 2, research that was classified as reflecting the “Teacher” role (n=80) described studies in which the school librarian led students and fellow educators through the development, execution, and assessment of information literacy curriculum, instruction, and professional development. The second most frequent application of the research was to the “Program Administrator” (n=50) role. The role encompasses tasks relating to developing and evaluating school library programs, policies and procedures. The role also includes leadership, management, and advocacy for the school library program. Table 2 First Author Country of Origin (N=199) Author Origin Frequency Percent USA 60 30.2 Canada 24 12.1 Australia 21 10.6 Hong Kong 11 5.5 Taiwan 11 5.5 Israel 10 5.0 New Zealand 9 4.5 South Africa 9 4.5 Iceland 7 3.5 Portugal 4 2.0 Botswana 3 1.5 France 3 1.5 Japan 3 1.5 Malaysia 3 1.5 UK/England 3 1.5 UK/Scotland 3 1.5 Brazil 2 1.0 Croatia 2 1.0 Jamaica 2 1.0 Denmark 1 .5 Finland 1 .5 Indonesia 1 .5 Italy 1 .5 Netherlands 1 .5 New Guinea 1 .5 Spain 1 .5 Sweden 1 .5 UK/Wales 1 .5 Total 199 100.0 Evidence Based Library and Information Practice 2011, 6.1 12 Table 3 Authors by Year, Conference Location, and Origin Zone (N=199) Year (Location) Origin Zone Total Zone A Zone B Zone C 2009 (Italy) 24 3 0 27 2008 (USA) 29 1 0 30 2007 (Taiwan) 12 1 0 13 2006 (Portugal) 18 2 1 21 2005 (Hong Kong) 17 2 1 20 2004 (Ireland) 9 0 0 9 2003 (South Africa) 11 0 0 11 2002 (Malaysia) 10 2 0 12 2001 (New Zealand) 16 0 0 16 2000 (Sweden) 10 1 0 11 1999 (USA) 20 0 0 20 1998 (Israel) 9 0 0 9 Total 185 12 2 199 Table 4 Research Forum Paper Topics, 1998-2009 (N=199) Paper Topic Frequency Percent Information skills and literacy 56 28.1 Information technology 21 10.6 Reading and reading promotion 19 9.0 Education in LIS 16 8.0 The profession 14 7.0 Analysis of LIS 12 6.0 Other aspects of LIS 11 5.5 Information seeking 10 5.0 Library and information science activities 10 5.0 Methodology 9 4.5 Publishing 7 3.5 National survey 6 3.0 Censorship 4 2.0 Library history 2 1.0 Principal support 2 1.0 Scientific and professional communication 0 0.0 Information storage and retrieval 0 0.0 Other studies 0 0.0 Total 199 100.0 Evidence Based Library and Information Practice 2011, 6.1 13 In the “Information Specialist” role (n=46), the school librarian provides leadership and expertise in acquiring and evaluating information resources; in bringing an awareness of information issues to teachers, administrators, students, and others; and in modelling strategies for locating, accessing, and evaluating information. As an “Instructional Partner” (n=23), the school librarian collaborates with individual teachers to design and implement learning tasks and assessments as well as to integrate technology and information literacy into classroom content. As Table 5 depicts, the papers were distributed fairly evenly across topics from year to year. Most topics and numbers of papers relating to those topics were consistent across the data set. Exceptions were “Principal support” which reflected papers only in 1998 and 1999; “Methodology” which was reflected only in the 2005, 2006, and 2007; and “Information seeking” was which reflected in 2008 and 2009. Likewise, “National survey” and “Library history” were both only represented in 2006 and 2008. “Censorship” is the only topic that experienced a peak, decline, and resurgence with papers in 1998, 2000, 2008, and 2009. “Reading and reading promotion,” present during all conference years, peaked in 2009 (n=7). Figure 2. Professional roles (AASL & AECT, 1998) in Research Forum papers (N=199). Evidence Based Library and Information Practice 2011, 6.1 14 Table 5 Frequencies of Papers by Topic for Each Conference Year and Location (N=199) Paper Topic∗ Year (Location) 1998 (Israel) 1999 (USA) 2000 (Sweden) 2001 (New Zealand) 2002 (Malaysia) 2003 (South Africa) 2004 (Ireland) 2005 (Hong Kong) 2006 (Portugal) 2007 (Taiwan) 2008 (USA) 2009 (Italy) Analysis of LIS 0 0 0 2 3 0 0 2 0 0 2 3 Censorship 1 0 1 0 0 0 0 1 1 0 0 0 Education in LIS 0 2 2 2 0 3 0 2 2 1 1 1 Information Seeking 0 0 0 0 0 0 0 0 0 4 6 0 Information Skills & Literacy 1 6 5 5 4 4 4 5 8 2 4 8 Information Technology 3 5 1 3 1 1 1 1 1 3 0 1 LIS Activities 0 3 0 0 0 1 2 0 2 0 2 0 Library History 0 0 0 0 0 0 0 0 1 0 1 0 Methodology 0 0 0 0 0 0 0 4 4 1 0 0 National Survey 0 0 0 0 0 0 0 0 1 0 3 2 Other Aspects of LIS 2 0 2 0 0 0 0 1 0 0 2 4 Principal Support 1 1 0 0 0 0 0 0 0 0 0 0 Publishing 0 0 0 3 1 1 1 1 0 0 0 0 Reading & Reading Promotion 1 2 0 0 1 1 1 1 1 1 7 3 The Profession 0 1 0 1 2 0 0 2 0 1 2 5 ∗ The list of paper topics was initially defined by Järvelin and Vakkari (1990) and amended by Oberg (2006). Evidence Based Library and Information Practice 2011, 6.1 15 As Figure 3 shows, the majority of the papers (n=87 or 43.7%) used qualitative methods. Mixed methods (a combination of quantitative and qualitative approaches) was the next most popular, used in 21.1% (n=42) of the papers. Finally, quantitative methods were used in the remaining 19.6% (n=39) of papers. Other methods (e.g., policy analysis, literature review) constituted the remaining 15.6% (n=31). As Table 6 below illustrates, researchers used one method (Method I) or multiple methods (Method II). Some authors disclosed their research design and data analysis procedures; many did not. In these instances, research designs were deduced from an examination of the abstract and full text of the paper. Table 6 depicts frequencies of data collection approaches. In the instances a data collection method is followed by the word “analysis,” i.e., content analysis, log analysis, stat(istical) analysis, and meta-analysis, these approaches involved the analysis of existing data not collected by the researcher. Quantitative methods were used in 39 papers, as Table 6 shows. While these methods included experimental (n=2) and statistical analysis of existing data sets (n=3), most of the quantitative studies used surveys (n=34) to collect data. No researcher used more than one method in a quantitative study. As Figure 3 above showed, qualitative methods comprised the majority of the research approaches (n=87). The most popular qualitative data collection methods illustrated in Table 6 were interview (n=19) and content analysis (n=14). Interviews included one study participant, as opposed to focus groups (n=1) that included groups of participants. Questionnaires, which require participants to provide text answers to questions, unlike surveys that include numerical or scale responses, were used in 6 studies. Many qualitative studies used more than one qualitative method to collect data (n=33) with observation-interview (n=8) and questionnaire-interview (n=7) being two popular choices. Figure 3. Distribution of research approaches 1998-2009 (N=199). Evidence Based Library and Information Practice 2011, 6.1 16 Table 6 Frequencies of Research Approaches and Methods in Research Forum Papers, 1998-2009 (N=199) Research Approach Method II Total Method I Case study Content analysis Focus group Interview Observation Questionnaire Other Method I Policy analysis 3 3 Non-research 8 8 Meta-analysis 3 3 Lit review 17 17 Total 31 31 Mixed Methods Method I Survey 2 7 5 20 1 4 39 Stat analysis 0 1 0 0 0 0 1 Log analysis 0 0 0 0 0 2 2 Total 2 8 5 20 1 6 42 Qualitative Method I Questionnaire 6 -- 1 -- 7 1 1 16 Observation 9 -- 1 -- 8 0 0 18 Interview 24 -- 2 -- 0 4 3 33 Focus group 1 -- 0 -- 1 0 1 3 Content analysis 14 -- 0 -- 2 0 1 17 Total 54 -- 4 -- 18 5 6 87 Quantitative Method I Survey 34 34 Stat analysis 3 3 Experimental 2 2 Total 39 39 Evidence Based Library and Information Practice 2011, 6.1 17 The second most popular research approach was mixed methods (n=42), an approach that uses both quantitative and qualitative methods to collect data. The researcher integrates the data in the analysis phase (Creswell, 2006). Most mixed methods were survey-interview (n=20), survey-content analysis (n=7) and survey-focus group (n=5). Other research approaches were literature review (n=17), meta-analysis (n=3), and policy analysis (n=3). In a literature review, the researcher synthesizes results of previously conducted research to establish a position whereas meta-analyses use statistical methods to count and characterize research (Glass, 1978). Eight papers were not research, but were descriptions of programs and policy initiatives. Discussion In terms of the first two research questions, “How have the numbers of research papers changed over the last 11 years” and “What were the roles, affiliations, and geographic origins of the research paper authors,” the results of this study were not remarkably different than those of Clyde’s studies (e.g., 1996; 2002; 2003a), Clyde and Oberg (2004), and Oberg’s 2006 analyses of 11 years of research papers between 1995 and 2006. It should be noted that this study included the same number of years (1998-2009) as the previous studies but the number of research papers presented from year to year fluctuated, perhaps due to the proximity of the conference location to researchers but possibly due to organizers’ classification of papers into conference tracks. That is, because IASL does not have a set policy for the types of papers presented in the Research Forum (Oberg, 2010), some conference organizers may have chosen to place papers that described a program or practice into the Research Forum and some organizers may have chosen to place certain kinds of research in the Professional Papers track. Still, the overall number of research papers had a steady increase of almost 300% from 1998 to 2008. As in earlier studies, the results of this study suggest that authors are still largely American, Canadian, or Australian and university faculty, doctoral students, or Master’s students. With the predominance of scholars from wealthy Zone A countries among the authors, the concentration of geographic origins and professional roles calls into question the assumptions and applicability of research findings to practitioner situations in less developed nations. There were some differences pertaining to the third research question, “What were the research paper topics? How have the topics changed over the last 11 years?” For example, reading and reading promotion and national survey were two frequent topics in published school library research (Oberg, 2006) that were not seen frequently in the Research Forum papers. However, both sources included few articles about principal support and focused mainly on research relating to the day-to-day activities of the school librarian as a teacher, an instructional partner, and as an information specialist. The Extent to which Research Forum Papers Constitute Support for Evidence Based Practice This final research question was challenging to address. Clyde and Oberg did not define a set of steps they used to answer this question. Rather, they inferred whether the papers supported EBP through a consideration of the number of paper and the variety in their authorship and methods. While the direct answers to the stated research questions of this study are not particularly ground breaking and almost entirely repeat past characterizations of school library research (e.g., Clyde, 1996; Clyde 2002; Clyde 2003a; Clyde 2006; Clyde & Oberg 2004; Oberg 2006), more interesting are the possible conclusions one can draw in light of the extent to which the Research Forum can be used as a source of evidence based practice, the fourth research question. While this question was taken from Clyde (2006) and Oberg (2006), this researcher found a slightly different one in need of response: Should the Research Forum be a source of evidence based practice? Evidence Based Library and Information Practice 2011, 6.1 18 By definition, EBP is not research; it is practice guided by the belief that resolution of the research problem is valuable and that the solution should be based on evidence (Booth 2003; Todd, 2009). In Figure 4, Stokes’ (1997) model, “Pasteur’s Quadrant,” adapted by Mardis (2009), provides a useful illustration for the relationship between research and practice. Whereas EBP would occupy the bottom right quadrant labelled Research- based Practice, empirical research occupies the top two quadrants of the model. Research is pure basic research that is conducted without any intention for practical application of results. In contrast, practice-based research is research for which practice is the source of the research problem and/or means of data collection and research results are meant to be applied to practice. Practice that is not informed by research resides in the bottom left quadrant. Given the tension between objective, scientific research (the core of evidence-based practice) and data collection prompted by advocacy, Todd (2009) proposed a holistic model for evidence and practice in school libraries that encompasses many of the relationships explored by Stokes (1997) and Mardis (2009): Evidence for Practice - Focuses primarily on examining and using best available empirical research to form practices and inform current actions, and to identify best practices that have been tested and validated through empirical research. This is posited as the informational dimension of school library practice. Evidence informs practice. Evidence in Practice - Focuses on reflective practitioners integrating available research evidence with deep knowledge and understanding derived from professional experience, as well as implementing measures to engage with local evidence to identify learning dilemmas, learning needs, and achievement gaps to make decisions about the continuous improvement of the school library practices to bring on optimal outcomes and actively contribute to school mission and goals. This is posited as the transformational Figure 4. The relationship between research and practice in school librarianship based on Stokes (1997) and Mardis (2009). Evidence Based Library and Information Practice 2011, 6.1 19 dimension of school library practice. Evidence of Practice - As the measured outcomes and impacts of practice, is derived from systematically measured, primarily user-based data. It focuses on the real results of what school librarians do, rather than on what school librarians do. It focuses on impacts, going beyond process and activities as outputs. It established what has changed for learners as a result of inputs, interventions, activities, processes, and charting the nature and extent and quality of effect (p.89). Todd’s holistic model values the research/practice cycle and honors the many ways in which the effects of practice on student outcomes in school libraries can be measured without the use of a RCT. Critics of a holistic approach (e.g., Lyons, 2009) perhaps have oversimplified the need for school libraries to use experimental designs without considering ethical implications; taking into account the issues inherent in the use of RCTs in schools, Todd has broadened the definition of EBP to both method and methodology by acknowledging the essential advocacy philosophy underpinning school library research. By giving school library research a core methodology, Todd has provided something prior school library research has lacked (Hjørland, 2000). This view of EBP encompasses the many professional needs met at the IASL conference. When Todd’s levels of EBP are mapped to the Mardis (2009) model in Figure 5, they clearly mesh with aspects of research and practice in LIS. The findings of this study suggest that the Research Forum, then, meets the informational aspects of evidence for practice while the Professional Papers meet the formational outcomes of evidence of practice; for practitioners, each aspect of EBP illustrated in Figure 5 has function and value. In response to the fourth research question, the Research Forum is a source for evidence, which is an aspect of EBP. And, in response to the rejoinder as to whether the Research Forum should be a source of EBP, the Research Forum Figure 5. The relationship between research and practice based on the Holistic Model of EBP for School Libraries (Todd, 2009, p.89). Evidence Based Library and Information Practice 2011, 6.1 20 should certainly be a source of evidence, but the IASL conference as a whole is the source of EBP. Conclusion As a source of professional development and support, IASL’s conferences have many missions to fulfill. Based on the long- established imperatives from leaders in the profession to provide support for both research and practice, the conference provides evidence for practice, evidence in practice, and evidence of practice. The Research Forum can be a valuable venue for the presentation of empirical research findings and conclusions and objective program evaluations and provide a valuable complement to the evidence based practice descriptions shared in the Professional Papers portion of the conference program. The Research Forum must be clear in its purpose: to present the results of research; to present effective practice determined by rigorous evaluation; or to present research-supported arguments for the support of school libraries. Until the purpose of the Research Forum is clarified and stated, IASL participants may mistake practice for research and the quest to improve the quality of school library research by building on a core methodology will be frustrated. Still, the relatively static representations of researcher origins and roles as well as research approaches and topics suggest that the frontier of school librarianship remains relatively unexplored. As Ritchie pointed out, “In order for research to be read by practitioners, it needs to be accessible…to practitioners” (2009, p.31), and the Research Forum provides such access. However, the concentration of researchers from the United States, Canada, and Australia calls into question IASL’s ability to fulfill its mission provide an “international forum” (IASL, 2007) and whether research findings can apply to a range of national contexts. Armed with the tailored philosophical underpinning of the methodology of EBP as a core function of improving student learning in school libraries as well as the methods described in the Research Forum papers, perhaps more researchers will be willing to share their work and Research Forum will expand its representation. References Alaimo, R. (2004). Top six reasons to attend a conference. Knowledge Quest, 33(1), 11- 12. American Association of School Librarians [AASL], & Association for Educational Communications and Technology [AECT]. (1998). Information power: Building partnerships for learning. Chicago, IL: American Library Association. Blessinger, K., & Hrycaj, P. (2010). Highly cited articles in library and information science: An analysis of content and authorship trends. Information Processing & Management, 32(2), 156-162. doi: 10.1016/j.lisr.2009.12.007 Booth, A. (2003). Bridging the research- practice gap? The role of evidence based librarianship. The New Review of Library and Information Science Research 2003, 3-24. doi: 10.1080/13614550410001687909 Brass, C.T., Nunez-Neto, B., & Williams, E.D. (2006). Congress and program evaluation: An overview of randomized controlled trials (RCTs) and related issues. Washington, D.C.: Congressional Research Service, The Library of Congress. Retrieved from: http://www.opencrs.com/document/R L33301/2006-03-07/download/1005/ Clyde, L. A. (1996). Introduction to the collection. In L. A. Clyde (Ed.), Sustaining the vision: A collection of articles and papers on research in school librarianship (pp. 3-11). Castle Rock, CO: Hi Willow. Evidence Based Library and Information Practice 2011, 6.1 21 Clyde, L. A. (2002). Developing the knowledge base of the profession: Research in school librarianship. In D. Singh, A. Abdullah, S. Fonseka & B. de Rozarion (Eds.), School libraries for a knowledge society: Proceedings of the 31st annual conference of the International Association of School Librarianship and the sixth international forum on research in school librarianship, Petaling, Jaya, Malaysia, 5-9 August (pp. 59-75). Seattle: International Association of School Librarianship. Clyde, L. A. (2003a). Evidence-based practice in school librarianship: Evaluating the research evidence. Access, 17(4), 26-29. Clyde, L. A. (2003b). Research in school librarianship 1991-2000: Australia in an international setting. The Australian Library Journal, 53(2). Retrieved from http://alia.org.au/publishing/alj/53.2/f ull.text/clyde.html Clyde, L. A. (2004). Evaluating the quality of research publications: A pilot study of school librarianship. Journal of the American Society for Information Science and Technology, 55(13), 1119. Clyde, L. A. (2006). The basis for evidence- based practice: evaluating the research evidence. New Library World, 107(5/6), 180-192. doi: 10.1108/03074800610665194 Clyde, L. A., & Oberg, D. (2004). LIS journals as a source of evidence for evidence-based practice: The case of School Libraries Worldwide. Paper presented at the World Library and Information Congress: 70th IFLA General Conference and Council, 22-27 August, Buenos Aires, Argentina. http://archive.ifla.org/IV/ifla70/papers/ 051e-Clyde_Oberg.pdf Creswell, J. W. (2006). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage Publications. Genco, P. (2003). The International Association of School Librarianship: School librarianship without borders. Knowledge Quest, 31(5), 17-18. Gordon, C. (2006). A study of a three- dimensional action research model for school library programs. School Library Media Research, 5. Retrieved from http://www.ala.org/ala/aasl/aaslpubsa ndjournals/slmrb/slmrcontents/volum e9/actionresearch.cfm Haycock, K. (1996). Research in teacher- ibrarianship and the institutionalization of change. In L. A. Clyde (Ed.), Sustaining the vision: A collection of articles and papers on research in school librarianship (pp. 13- 25). Castle Rock, CO: Hi Willow. Henne, F. (1950). Libraries, school. In W. Monroe (Ed.), Encyclopedia of educational research, rev. ed. (pp. 701- 712). New York: Macmillan. Hjørland, B. (2000). Library and information science: practice, theory, and philosophical basis Information Processing & Management, 36(1), 501- 531. doi: doi:10.1016/S0306- 4573(99)00038-2 International Association of School Librarianship [IASL]. (2007, March 4). Information about IASL: Mission of IASL Retrieved January 22, 2010, from http://www.iasl- online.org/about/ Järvelin, K., & Vakkari, P. (1990). Content analysis of research articles in library and information science. Library and Information Science Research, 12(4), 395- 421. Järvelin, K., & Vakkari, P. (1993). The evolution of library and information science 1965–1985: A content analysis http://alia.org.au/publishing/alj/53.2/full.text/clyde.html� http://alia.org.au/publishing/alj/53.2/full.text/clyde.html� http://archive.ifla.org/IV/ifla70/papers/051e-Clyde_Oberg.pdf� http://archive.ifla.org/IV/ifla70/papers/051e-Clyde_Oberg.pdf� http://www.ala.org/ala/aasl/aaslpubsandjournals/slmrb/slmrcontents/volume9/actionresearch.cfm� http://www.ala.org/ala/aasl/aaslpubsandjournals/slmrb/slmrcontents/volume9/actionresearch.cfm� http://www.ala.org/ala/aasl/aaslpubsandjournals/slmrb/slmrcontents/volume9/actionresearch.cfm� http://www.iasl-online.org/about/� http://www.iasl-online.org/about/� Evidence Based Library and Information Practice 2011, 6.1 22 of journal articles. Information Processing & Management, 29(1), 129- 144. doi: 10.1016/0306-4573(93)90028-C Julien, H. (1996). A content analysis of the recent information needs and uses literature. Library & Information Science Research, 18(1), 53-65. doi: 10.1016/S0740-8188(96)90030-4 Lighthall, L. & Haycock, K. (1997). Introduction. In Lighthall, L. & Haycock, K. (Eds.) Information rich but knowledge poor? Research and professional papers p. resented at the 26th annual conference of the International Association of School Librarianship held in conjunction with the Association for Teacher-librarianship in Canada at the University of British Columbia Vancouver, British Columbia, Canada July 06-11, 1997. Seattle, WA: International Association of School Librarianship. Lowrie, J. (1968). A review of research in school librarianship. In H. Goldhor (Ed.), Research methods in librarianship: Measurement and evaluation, papers presented at a conference conducted by the University of Illinois, Graduate School of Library Science, September 10-13, 1967 (pp. 51-69). Urbana, IL: University of Illinois Graduate School of Library and Information Science. Lowrie, J. (2011, January 19). [What is the purpose of the Research Forum?] personal communication. Lyons, R. (2009). The call for evidence based practice: Speaking louder than words. Evidence Based Library and Information Practice (EBLIP), 4(3). Retrieved from http://ejournals.library.ualberta.ca/ind ex.php/EBLIP/article/view/6530/5537 Mardis, M. A. (2009). A gentle manifesto on the relevance and obscurity of school libraries in LIS research. Library Trends, 58(1), 1-8. McClure, C., & Bishop, A. (1989). The status of research in library/information science: guarded optimism. College and Research Libraries, 50, 127-143. McKibbon, K. A. (1998). Evidence-based practice. Bulletin of the Medical Library Association, 86(3), 396-401. Miller, S. J. (1999). Professional development for the library media specialist. Book Report, 17(5), 20-21. Morrison, K. (2001). Randomized controlled trials for evidence-based education: Some problems judging ‘what works.’ Evaluation and Research in Education, 15(2), 69-83. Oberg, D. (2006). School Libraries Worldwide as a source for evidence for evidence- based practice, 1995-2006. School Libraries Worldwide, 12(2), i-xii. Oberg, D. (2010, July 10). [What is the Research Forum?]. Ritchie, C. S. (2009). Filling the gap: Would evidence-based school librarianship work in the UK? Library and Information Science Research, 33(104), 26-35. Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation. Washington, D.C.: Brookings Institution Press. Todd, R. J. (2009). School librarianship and evidence-based practice: Progress, perspectives, and challenges. Evidence Based Practice in Library and Information Practice, 4(2), 78-82. What Works Clearinghouse [WWC]. (2010). WWC frequently asked questions Retrieved July 1, 2010, from http://ies.ed.gov/ncee/wwc/references/ idocviewer/Doc.aspx?docId=15&tocId =5 http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6530/5537� http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6530/5537� http://ies.ed.gov/ncee/wwc/references/idocviewer/Doc.aspx?docId=15&tocId=5� http://ies.ed.gov/ncee/wwc/references/idocviewer/Doc.aspx?docId=15&tocId=5� http://ies.ed.gov/ncee/wwc/references/idocviewer/Doc.aspx?docId=15&tocId=5� Evidence Based Library and Information Practice 2011, 6.1 23 Appendix. IASL Countries and Membership Zones Countries and zones are featured at http://www.iasl-online.org/about/joiniasl.html Zone A countries: International Schools (except local employees) are included in Zone A. Australia, Austria, Bahamas, Bahrain, Belgium, Brunei, Canada, China/Hong Kong, China/Macao, Cyprus, Denmark, Finland, France, Germany, Greece, Iceland, Ireland, Israel, Italy, Japan, Kuwait, Liechtenstein, Luxembourg, Monaco, Netherlands, New Zealand, Norway, Portugal, Qatar, San Marino, Saudia Arabia, Singapore, Slovenia, South Korea, Spain, Sweden, Switzerland, Taiwan, United Kingdom, UK/Bermuda, UK/British Virgin Islands, UK/Cayman, UK/England, UK/Gibraltar, UK/Falkland, UK/North Ireland, UK/Scotland, UK/Wales, United Arab Emirates, USA, USA/Virgin Islands. Zone B countries: Algeria, Angola, Antigua & Barbuda, Argentina, Azberbaijan, Barbados, Belize, Botswana, Brazil, Bulgaria, Chile, Cook Islands (New Zealand), Costa Rica, Croatia, Czech Republic, Dominica, Dominican Republic, Equatorial Guinea, Estonia, France/French Guinea, France/French Polynesia, France/Guadeloupe, France/Martinique, France/New Caledonia, France/Reunion, France/St. Pierre & Miquelon, Gabon, Grenada, Hungary, Jamaica, Kazakhstan, Latvia, Lebanon, Libya, Lithuania, Malaysia, Malta, Mauritius, Mexico, Micronesia (USA), Nauru, Neth. Antilles, Oman, Panama, Peru, Poland, Romania, Russia, Seychelles, Slovakia, South Africa, St. Kitts & Nevis, St. Lucia, St. Vincent & the Grenadines, Thailand, Trinidad & Tobago, Turkey, Turkmenistan, UK/Anguilla, UK/Monserrat, UK/Saint Helena, UK/Turks and Caicos, Uruguay, USA/American Samoa, USA/Guam, USA/Northern Marianas, USA/Palau, USA/Puerto Rico, Venezuela. Zone C countries: Afghanistan, Albania, Andorra, Armenia, Bangladesh, Belarus, Benin, Bhutan, Bolivia, Bosnia and Herzigovina., Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Central African Rep, Chad, China, Cocos (Keeling) Islands (Australia), Colombia, Comoros Islands, Congo, Cote d'Ivoire, Cuba, Dem Rep of Congo, Djibouti, East Timor, Ecuador, Egypt, El Salvador, Eritrea, Ethiopia, Fiji, France/Wallis et Futuna, Gambia, Georgia, Ghana, Guatemala, Guinea, Guinea-Bissau, Guyana, Haiti, Honduras, India, Indonesia, Iran, Iraq, Jordan, Kenya, Kiribath, Kyrgyzstan, Laos, Lesotho, Liberia, Macedonia, Madagascar, Malawi, Maldives, Mali, Myanmar, Marshall Islands (USA), Mauritania, Moldova, Mongolia, Morocco, Mozambique, Namibia, Nepal, Niue, Nicaragua, Niger, Nigeria, North Korea, Pakistan, Papua New Guinea, Paraguay, Philippines, Rwanda, Samoa, Sao Tome/Principe, Senegal, Serbia/Montenegro, Sierra Leone, Solomon Islands, Somalia, Sri Lanka, Sudan, Surinam, Swaziland, Syria, Tajikistan, Tanzania, Togo, Tonga, Tunisia, Tuvalu, Uganda, Ukraine, Uzbekistan, Vanuatu, Vietnam, West Bank/Gaza Strip, Yemen, Zambia, Zimbabwe. http://www.iasl-online.org/about/joiniasl.html� / Evidence Based Library and Information Practice Prior characterizations of school library research The role of research in school librarianship has long been debated. As early as 1950, school library leader Frances Henne lamented the lack of scientific research in the field and pointed out that: A lag exists between the theoretical opinion in the field and research. Most functions and objectives of school library practice have been formulated pragmatically on the basis of opinion and experience rather than on evidence produced in research. To... For decades, researchers (e.g., Lowrie, 1968; Clyde 1996) have been examining published school library research. Many of these studies (e.g., Clyde 2003b; Clyde & Oberg, 2004; Oberg, 2006) reported that researchers from the United States, Canada, and ... Utility of Characterizing Research Reasons for characterizing published research are manifold. Knowledge of the subject distribution of research in library and information science can identify gaps for further research exploration (Blessinger & Hrycaj, 2010). Likewise, an understanding... Therefore, researchers have sought other methods to assess the soundness of school library research (e.g., Clyde, 2004) and its ability to inform practice that benefits children’s learning (Todd, 2009) to remain relevant in the educational policy cont... However, the use of EBL in school librarianship has been labeled as blurring the evidence based paradigm because research pursuits are often a response to a perceived threat to the profession or a need to advocate for an aspect of practice. This impli... Research Questions The following research questions were investigated. These questions were also used by Clyde (2003a), Clyde and Oberg (2004), and Oberg (2006): How have the numbers of research papers changed over time? What were the roles, affiliations, and geographic origins of the research paper authors? What were the research paper topics and methods? How have the topics and methods changed? To what extent do the Research Forum papers constitute support for evidence based practice? Description of the Analysis In the first phase of the research project, a working data file that contained metadata for each paper was created: Year; Title; Author; Role; Affiliation; Author Origin; and Conference Location. Then, additional metadata for each paper including was ... The papers’ abstracts and full text were analyzed to populate the topic field with controlled vocabulary based on the categorization of Oberg (2006). The research papers were designated as using a quantitative, a qualitative, or a mixed methods approa... frequency and descriptive statistics, charts, and tables in response to the research questions. In the final phase of the project, the coding of topic, research approach, and research method designations were validated through two additional checks by two doctoral students. Numbers of Research Papers Over Time A total of 199 papers were published in the Research Forum proceedings between 1998 and 2009, with a mean of 18 papers published in each proceedings volume. Table 1 illustrates the number of Research Forum papers presented at each conference location ... Roles, Affiliations, and Geographic Origins of the Research Paper Authors USA (n=60), Canada (n=24), and Australia (n=21) represented the largest number of first author origins, as Table 2 depicts. Because all but one jointly authored paper represented the same country collaborations and the sole jointly authored paper repr... For this question, the classification scheme for library and information science (LIS) topics developed by Järvelin and Vakkari (1990) as amended by Oberg (2006) was applied to the papers. Table 4 illustrates that the “Information skills and literacy”... Discussion Appendix. IASL Countries and Membership Zones