J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston Forensic Science Information Seeking Behaviors: A Survey of Forensic Science Professionals. Sarah Bankston, M.S.1*; Laura Sare, M.S.1; Samantha J. Sawyer, M.S.2; and Jeffery K. Tomberlin, Ph.D.2 1University Libraries, Texas A&M University, College Station, TX 77843 2Department of Entomology, Texas A&M University, College Station, TX 77843 *corresponding author: sbankston@tamu.edu Abstract: Anecdotal frustrations expressed by former forensic science students indicated a potential need to improve education on information seeking research strategies to better prepare students for the workforce. This study describes the findings from a survey of forensic science professionals from eleven disciplines evaluating how they search for information related to their occupation. Over 300 individuals responded to the survey, with work experience ranging from less than a year to over fifty years. Open response questions created to gain insight into information seeking behavior were coded and analyzed. While few of the forensic science professionals surveyed (14%) search for research material daily, many (80%) need to find information on a monthly or weekly basis. The results indicate a need for guidance on easier methods to find information and ways to alleviate frustrations in acquiring information. Librarians, forensic science educators, and forensic science professionals can form partnerships that meet forensic scientist information needs from the classroom and into the workforce. Keywords: forensic science, information literacy, lifelong learning, open access to information, scholarly information Introduction The forensic sciences rely heavily on published literature for implementation and application of techniques in criminal investigations. However, access to such materials can be quite challenging given budgetary restrictions, restricted linkages with necessary resources, and/or simply keeping up with advancements as they are made. Such limitations can impact the ability of the forensic scientist to maintain flexibility and growth necessary to advance with the sciences. Historically, these resources have been gleaned from forensic science journals accessed through contracts with the publisher. However, with the development of the internet and open access journals, the diversity of resources available has increased substantially. Determining what information resources are appropriate, or not, can significantly impact evidence interpretation and application. This study was developed to identify frustrations in information seeking that previously had been voiced anecdotally by former students, now forensic science professionals (FSP), to the researchers. The survey was designed to generate data to determine if a need for education on research strategies and information access existed. The data provided would be used for developing education modules that could be used by FSP that focus on their research needs, in addition to informing librarians and forensic science educators of potential areas for curriculum development. The research questions were: 1) How are FSP accessing information and what are the barriers that they are experiencing? and, 2) are there educational opportunities to facilitate research strategies of FSP? This study describes the findings from a survey of members of a forensic science professional organization, consisting of eleven disciplines, on how they search for information related to their occupation. The researchers anticipate this study will serve as a starting point for a larger discussion about the information needs of the forensic science community and will provide ideas for the forensic science community to improve education about access to information resources. Several national reports have focused on the information needs of the forensic science community. The 2009 National Academies of Science report “Strengthening Forensic Science in the United States: A Path Forward,” noted that “forensic science practitioners require continuing professional development and training. Scientific advances in forensic science techniques and research in the forensic science disciplines are of interest to practitioners who must be aware of these new developments. Forensic science practitioners also may need to complete additional training for certification purposes or may desire to learn new skills as part of their career development” (1:218). The National Institute of Justice (NIJ) recognizes that forensic scientists, “must rely J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston on new technologies and scientific innovations to more effectively identify, gather and process evidence related to criminal activity” (2). This professional development and training, such as keeping up with new technologies and innovations, is encompassed in concept of lifelong learning. Lifelong learning is defined as: “…‘the aim of improving knowledge, skills and competence, within a personal, civic, social and/or employment-related perspective.’ Lifelong learning is therefore about acquiring and updating all kinds of abilities, interests, knowledge and qualifications from the pre-school years to postretirement. It promotes the development of knowledge and competences that will enable each citizen to adapt to the knowledge-based society and actively participate in all spheres of social and economic life, taking more control of his or her future plus valuing all forms of learning, including: formal learning, such as a degree course followed at university; non-formal learning, such as vocational skills acquired at the workplace…” (3:270) The 2015 National Commission on Forensic Science (NCFS) report stated credible scientific literature should be used in forensic practice and that “the open, peer-reviewed literature is what endures and forms a foundation for further advancements” (4:2). The NCFS also developed criteria to help evaluate the scientific validity of information (4:3). Two of the suggested criteria specifically address journal literature that is searchable via “free, publicly available search engines” or “databases that are available through academic libraries and other services” (4:3). Given this, knowing how to efficiently and effectively search these resources, as well as understanding routes for accessing the journal literature, are important issues. In the literature, one of the first known attempts to address the issue of accessing forensic science literature was Teitelbaum (5) who noted that, “Because there is no central repository for forensic science information, and because of the sheer number of disciplines under the forensic science umbrella, forensic scientists are often unable to locate material that is relevant to their needs.” (5:2) He goes on to outline six particularly useful sources and effective searching for each of these sources. Two workshops held at the American Academy of Forensic Sciences (AAFS) conference in 2016 and 2017 also addressed this issue. The initial workshop (6) focused on educating participants on methods for using the internet to locate such valuable resources, while the second workshop (7) emphasized scientific diversity and its interpretation in the court of law, as well as ways to search for and access scientific and legal information. In both cases, these workshops invoked the need for methods to provide forensic science practitioners a means to streamline and increase efficiency for finding and obtaining relevant literature. Other literature on improving information access includes a paper by Knoll (8) that is from the publishing side of forensic sciences and provides a useful introduction to open access publishing from the forensic psychiatry perspective. Since so little has been written about information seeking in the forensic science discipline, a broader search for information seeking behavior in the literature results in studies that either explore information seeking behavior in general, use specific methodologies, or only study novice learners, not experts in their field. Some exceptions include Pontis and Blandford (9) who focused on how science academics manage and explore information when asked to identify the current and upcoming authority figures in the field. Tangential to forensic sciences, Makri, Blandford, and Cox (10) explored how lawyers performed information searches using a “think-aloud” study, but the focus of the article was on the methodologies used rather than the outcomes of the interviews. Dinet, Chevalier, and Tricot (11) provide a useful overview of methodologies to consider when studying information seeking and Palmquist and Kim (12) also provide a methodological overview but from the library science perspective. Tenopir et. al wrote on scholarly reading patterns of university research faculty that was broken out by broad discipline but focused mostly on the format and amount read by faculty (13). The key lifelong learning strategy for this research study is partnerships between public authorities, education service providers, the business sector, associations, etc. (3:273). To explore ways to employ this strategy, specifically for forensic science professionals, the researchers created a survey as the first stage of an investigation of forensic science professionals’ information search strategies. The objectives of this survey were to discover how forensic scientists find and access information in their specific fields, in order to tease out strengths that could be shared throughout the forensic science community as a whole and identify knowledge gaps in search strategies that could be improved upon and brought to the community through learning modules. To inform the development of learning modules, questions such as the relevance of field or time in that field were used to determine if specific fields or stages in career would especially benefit from such modules. The focus of this study was to confirm if frustrations or barriers in information seeking practices of forensic scientists existed in order to identify educational opportunities on information seeking to support lifelong learning. Methods A survey was created in Qualtrics®, comprised of thirteen fixed-choice and open response questions. This survey was structured to provide feedback for the researchers to determine what education modules, if any, would benefit FSP’s access to information in the course of J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston their job or research. Because the forensic sciences are multidisciplinary the researchers, in addition to wanting to know where and how respondents conducted research, also wanted to see if experience or field was a factor in information seeking obstacles. An institutional review board study protocol was submitted and approved at the researchers’ institution. In February 2017 the researchers conducted a workshop at the AAFS where they conducted a pilot test of the survey instrument. The final survey was sent to AAFS on August 18, 2017 and widely distributed via the AAFS listserv (TABLE 1). The survey was open for responses until November 8, 2017. At the time of the survey, the AAFS had 6692 members (14). The AAFS member list was determined to be a good sample population due to its large membership and multiple disciplines. There was a total of 547 surveys started, with 375 completed and included in the dataset analyzed here. Surveys in which respondents did not answer at least half the questions were not included because these did not provide enough information for an analysis. Of the competed surveys, if a respondent skipped a question, the response was coded as No Answer. This is a response rate of 5.60% and is low, but the researchers were not using the survey results to make generalizations about the forensic science community. Rather, this survey was structured to gather evidence to see if the frustrations in information seeking they had heard from FSP that were former students occur among others in the forensic science community and if there were some educational opportunities that could be developed from the data. The survey instrument included open response and multiple-choice questions. Multiple choice response analysis was provided by Qualtrics®. All responses were downloaded to an Excel spreadsheet. Open response questions were post-coded using grounded theory methods. Creswell explains grounded theory is “a qualitative strategy of inquiry in which the researcher derives a general, abstract theory of process, action, or interaction grounded in the views of participants in a study.” (15:13 & 229). The open-ended questions were coded (by sentences or phrases) by the two researchers who conducted the qualitative research portion of the study. The two researchers coded independently using an open coding process and met to review codes and come to agreement on the final codes used. These codes were determined throughout the coding process, and not from pre-assumed categories, to follow Glaser and Strauss’ (16) grounded theory method to analyze the data with no preconceived hypothesis. Words and phrases that described topics of importance were noted, coded, and listed in an Excel spreadsheet. The initial codes were collapsed into broader categories using the constant comparative method, a process in which data, “are broken down into manageable pieces.” (17:7). Conceptually similar data were grouped together under a related heading and with further analysis developed into two major themes discussed later in this study. TABLE 1 Questions and response types included in survey Question Response option Q1. What is your highest level of education attained? Multiple choice Q2. What is your field and how long have you been in it? Open response Q3. Do you have any formal affiliations that allow you to free access to scholarly information (such as an agreement with a university)? Multiple choice – binomial Q4. How do you access articles you cannot get freely online? Open response Q5. How often do you search for scholarly articles? Multiple choice Q6. What search engines/databases do you regularly use to find information, and for what type of question? Open response Q7. How do you find what is published on a given topic? Open response Q8. How do you get the literature you need? Open response Q9. How often do you locate articles you would like to use but cannot access? Multiple choice Q10. Think back to the last time you needed to find information for your work. What did you need information on? Can you describe the process you used to locate the information you needed? Open response Q11. How do you currently document the literature you use for training, quality assurance, and accreditation purposes? Open response Q12. Does anything frustrate you about looking for information for your work? Open response Q13. Which of these items would you include in your definition of a scholarly article? Multiple choice Responses to Question 2, regarding field and time in field underwent two separate analyses. Field responses were coded into categories using the Section descriptions on the AAFS website and time in field responses were grouped into multi-year ranges. Analysis comprised of J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston determining the total number of individuals that responded with corresponding coding to determine percentiles. For Question 6, any response recorded (even if multiple items were reported in one response) was tallied in order to observe frequency of response. For example, if a response was “Google, Bing and Google Scholar,” all three responses were tallied for analysis as opposed to coding as “search engines.” Responses were then grouped by type of resource. Given the data collected were non-normally distributed and violated the assumptions of parametric statistical tests, a nonparametric test was used. A Kruskal- Wallis test was performed in R 3.5.1 “Feather Spray” in the base package (R Core Team 2018) to determine if particular responses could be attributed to a field, time spent practicing in the field, or both. The Dunn post hoc test was used following significant (P < 0.05) results of the Kruskal-Wallis Test to determine what was driving the significant result. For the questions that were open response, the coded data (based on methods outlined above) were used to determine if the responses were field, or time spent in field, specific, as these can give insight to who is encountering issues in obtaining literature and who is overcoming them. All responses noted as “Not Specified” were excluded from statistical analyses. Results All but one of the forensic science professionals who responded had college degrees, and of those, 81% indicated an advanced degree. Data in response to this question were highly variable; when the survey was first distributed, there were a few comments about the lack of a professional degree (MD/DVM) option. It was decided to add this category as an option for those who had not yet taken the survey (Q1). Respondents were asked their forensic field in an open response format (Q2), and given the breadth of answers, the framework of AAFS section titles was used to map responses. Respondents were largely mapped to the Pathology/Biology (25%) and General fields (19%) followed by Toxicology (15%), Criminalistics (12%), and Anthropology (10%). Time in field varied, and no group exceeded 17% of the total response population. However, individuals with less than 1 year and those with over 50 years in the field each represented less than 1% of the response population (TABLE 2). TABLE 2 Number (n = 375) of responses for field and time in field (Q2)* Field Number of Responses Percentage of Total Responses Pathology/Biology 92 24.53% General 71 18.93% Toxicology 55 14.67% Criminalistics 44 11.73% Anthropology 39 10.40% Engineering & Applied Sciences 18 4.80% Odontology 18 4.80% Questioned Documents 10 2.67% Not Specified 9 2.40% Psychiatry & Behavioral Science 8 2.13% Jurisprudence 6 1.60% Digital & Multimedia Sciences 3 0.80% No Answer 2 0.53% Time in Field (years) Number of Responses Percentage of Total Responses <1 2 0.53% 1-4 32 8.53% 5-9 49 13.07% 10-14 65 17.33% 15-19 50 13.33% 20-29 56 14.93% 30-39 47 12.53% 40-49 32 8.53% 50+ 3 0.80% Not Specified 37 9.87% No Answer 2 0.53% *Field was coded using AAFS sections and categorizing responses using the descriptions provided for each major group and time was grouped in ranges by year. When asked if respondents had any affiliations that allowed them free access to scholarly articles (Q3), majority responded Yes (58%). Those who answered No (42%) were then asked how they accessed articles that were not freely available as an open-ended question (Q4). When more than one access method was indicated, each method was coded (FIGURE 1). The most frequent methods of access were through colleagues (34% of responses) or paying for the article, subscription, or a professional membership that granted access (26% of responses). Nine percent of respondents indicated that they would not be able to access an article that was not freely available. Responses with unclear meaning, such as “Journal,” were coded as Not Specified, but these only comprised 5% of the total responses. J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston FIGURE 1 Accessing scholarly articles when not freely available and no institutional affiliation Only 6% of the forensic science professionals surveyed indicated that they never had to search for scholarly articles (Q5). While 14% of respondents search for scholarly articles daily, most need to find this type of material on a monthly (43%) or weekly (37%) basis. For those positions and disciplines that need to do research, a majority said they often could not access articles they identified as potentially useful and would like to use due to paywalls. Those that did not have problems accessing articles were usually affiliated with a university. Participants were also asked what search engines or databases they used regularly, and to indicate the types of questions they were trying to answer in the search (Q6). Only 33% of respondents answered both parts of the question, creating a limitation on information gathered in this question. While useful information was gathered about resources utilized, the responses could not be connected to particular types of research needs or examined in relation to time in field. Respondents most often regularly used Google, Google Scholar, PubMed/MEDLINE, Subscription Databases, Journal Websites (such as the AFTE Journal and Journal of Forensic Sciences), Professional Associations, and University or Public Libraries (FIGURE 2). When a specific subscription citation/article database was mentioned by name, it was coded as Subscription Databases versus a generic response such as ‘library search engine.’ Ambiguous responses such as ‘library search engine’ were coded as University or Public Libraries. Any mention of a professional association, whether specific or generic, was coded as Professional Associations. Any mention of a journal website, whether specific or generic, was coded as Journal Websites. Because of the frequency of responses of both Google and Google Scholar, these were broken out from the larger code categories to have their own category. Similarly, PubMed/MEDLINE were combined since the MEDLINE is the database behind PubMed web interface, and given the response frequency of this resource, it too was broken out from the larger code categories. These seven resources comprised 84% of the total responses. The remaining 16% of responses included specific federal agency websites, or websites such as Academica.edu, Wikipedia, YouTube, and LinkedIn. FIGURE 2 Frequently used search engines or databases To determine information search strategies and starting points, participants were asked how they find what is published on a topic in an open response question (Q7). Responses were coded, and the most frequent response was Search Engines (31%) such as Google, followed by Journals (13%) and Databases (13%). When the meaning was unclear, such as “most of the time” or “keywords,” responses were coded as Not Specified (14%). This question also generated unexpected responses such as email alerts, news sites, newsletters, and RSS feeds, though none of these responses exceeded 1% of total responses (FIGURE 3). FIGURE 3 How respondents find what is published on a given topic The ways in which respondents acquire the literature they need are incredibly varied (Q8). While there was no clear majority response, the most frequent way respondents obtain literature is by an online method (37%), through colleagues (11%), with others using university libraries (8%), other types of libraries (5%), and subscriptions (5%). J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston Online was listed both as a vague response (14% of respondents just answered “online”) because no specific resource was mentioned (FIGURE 4) and also combined with electronic format answers such as PDF, databases, Google, etc. Responses were coded Not Specified (4%) when the meaning was unclear, such as “research” or “computer.” Results from Question 8 are also discussed in the Kruskal-Wallis analysis at the end of this section. FIGURE 4 How respondents obtain literature When asked how often they were unable to access articles they would like to use (Q9), 25% of respondents indicated they never ran into access issues. Of those who never run into access issues, 48% responded earlier in the survey that they have a formal affiliation that allows for free access to scholarly information such as working at a university. The majority of respondents experience some type of access issue on a daily (4%), weekly (25%), or monthly (46%) basis. Respondents were asked to describe the process they use to locate information by recalling the last time they needed to find information for their work (Q10). While it was thought that this scenario-based question might uncover information seeking methods not yet revealed, it turned out the responses only served to further reinforce findings in the previous questions. Further analysis of the question did provide how FSP are using scientific research to inform their work. By far the most sought-after type of information was articles at 74%. Nine percent of those who responded ‘articles’ specified “free articles.” Textbooks and books were 20% of the responses, and materials like standards, data, specs, and conference proceedings were 6% combined. Of those respondents who included their research need as part of their answer, the most common response was to get current (18%), followed by methods (12%), court preparation (8%), and case investigation (8%) (FIGURE 5). FIGURE 5 Reasons for seeking information Determining how FSP document literature could be important for identifying future educational opportunities related to best practices for creating and managing a database of references for a lab or individual. Respondents were queried on how they document literature for quality assurance, accreditation, and training purposes (Q11). The majority responded that they used citations or some sort of reference management software (23%). Others used shared departmental folders or spreadsheets (13%), training/lab manuals and SOPs (11%), or saved physical or electronic copies of articles (11%). Nine percent of the responses were not specific enough to properly code, 6% responded that they do not document literature for these purposes, and 7% responded that such documentation was not applicable to their job. When asked if there were any frustrations when searching for information for their work (Q12), the majority of the responses related to problems accessing information (24%). Other frustrations included a combination of access and funding (13%), issues related to the search itself (8%), funding alone (8%), or the time it took to search (3%). However, 21% of the responses indicated no frustrations involved with their work, though 65% of those respondents indicated earlier in the survey that they have some sort of formal affiliation with an institution that grants access to scholarly information confirming access is not the only frustration respondents experience (FIGURE 6). FIGURE 6 Frustrations with searching for information J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston Participants were given a list of information types and asked which they would consider a scholarly article to help the researchers understand if/how information terminology used in libraries is understood by forensic science practitioners (Q13). One objective of this survey was to gather data for education modules being developed by the researchers and this question was used to gauge what language might be most effective or already commonly in use by FSP. Every respondent selected the answer, “peer- reviewed journal,” and 21% selected only this option. The most popular response was a combination of peer-reviewed journal, conference paper, and books (28%). After these two, the most numerous response was the combination of peer-reviewed journal and books (14%) (FIGURE 7). FIGURE 7 Sources included in respondents' definition of a scholarly article; PR – peer-reviewed article; CP – conference paper; B – books; TP – trade publication; W – website; NS – not sure The Kruskal-Wallis (KW) analysis was conducted to determine if the time FSP spent in their field, or their field had any relationship to information seeking behaviors. This analysis was to inform the researchers of any unique populations within the survey sample that could be targeted with the planned education modules. The KW analysis indicated that, per the survey population, how frequently an individual searches for scholarly articles (Q5) and [potentially] locates information (Q7) and documents literature (Q11) was driven by the length of time individuals were in any given field, while how someone finds information (Q6) is field specific. However, ways of accessing literature (Q8) were driven by time spent in that given field and potentially their specific field (SUPPLEMENTAL TABLE 1). The Dunn post-hoc test indicated several general patterns from significant results indicated in the KW test. First, individuals who spent over 40 years in their given field indicated they searched for literature more often than individuals who spent 30 years or less. Most fields relied on Google or Google Scholar for their preferred search engines; however, fields in Pathology/Biology and Toxicology used PubMed significantly more than fields comprising other AAFS sections. Finally, individuals who spent less than 5, 20-30, or over 50+ years in their respective fields relied on independent means to search for literature (e.g., searches in databases) where individuals who spent 5-10 or 30-50 years in a given field utilized a library or library service (e.g., University Library/Interlibrary loan/ Librarian). Discussion and Conclusion The survey confirmed the researchers’ primary research question that there are barriers encountered with accessing information for some survey respondents. Most FSP in this survey did not experience access barriers, but 42% reported they did not have affiliations that would make accessing scholarly information easier. This combined with a majority of FSP needing to conduct research on a weekly or monthly basis, and those needing to find information having access issues on a weekly and monthly basis, shows that some FSP could benefit from some basic information literacy research strategies. While FSP in this survey demonstrated their resourcefulness and found ways to access what they need despite time and access barriers, these barriers do exist. Understanding the difficulties in information seeking encountered by FSP is useful for forensic science educators as they develop their curricula. Forensic science educators can partner with librarians at their university library to integrate literature searching and evaluation skills into their assignments and courses. However, awareness of the complexity of the information landscape for FSP is important for addressing larger barriers to information access that might not occur to students until they are in their first professional position. From the qualitative data coding results, two themes were identified. The first theme is a need for guidance on easier ways to find information resources and particularly, for FSP without university affiliations, how to find gain access to these resources. The second theme is frustration in acquiring information. This theme arose from statements related to encountering paywalls, lack of funds to purchase information resources, and the time spent searching for and selecting information. These themes point to a need for practical solutions. While not the most frequent response, some FSP expressed frustration in how widespread forensic information can be, noting that there is too much information to search through or that there is not a central location to search for all information. However, a one-stop search solution that involves a single portal for access is not probable given the variety of disciplines, platforms, and sources of information used in the forensic sciences. While a one-size-fits-all search is unlikely, a more effective solution would be the development of a national library network for forensic sciences. A national library for forensic sciences would be instrumental in acquiring, making accessible, and distributing forensic science J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston literature, which would alleviate both the need for easier ways to find information resources as well as the frustrations with acquiring the information once found. Looking at the literature, both the National Library of Medicine (NLM) and the National Agricultural Library (NAL) provide models that could be used to develop a national library for the forensic sciences, or at the very least networking models for opening up access to literature for practitioners. Additionally, the library could develop interfaces for discovery and access, and training materials to help practitioners learn how to search for relevant literature more effectively. A logical agency to take on this role would be the National Criminal Justice Reference Service (NCJRS), which already hosts a virtual library. There would need to be government mandates for the NCJRS to take on this purpose but stepping into this role is not limited to a national library; a large, multi-disciplined, forensic science organization could also serve this function of coordinating training and access to literature. The NLM has a long history of supporting medical libraries throughout the country via grant funding to provide practitioners better access to biomedical literature. They developed the National Network of Libraries of Medicine (NNLM) which is a network of libraries sectioned into regions in order to more effectively work with practitioners in their areas (18). Similarly, NAL has worked to make agriculture literature available to practitioners through the development of the United States Agricultural Information Network (USAIN) and the Agriculture Network Information Center (AgNIC) (19, 20), both of which leverage partner libraries and librarians across the country to identify, preserve, and provide access to agriculture information. These models for distributed information access to practitioners provided by NLM and NAL can be explored for feasibility in developing a national library network for forensic sciences. Libraries and librarians can also help FSP by providing acquisitions expertise. Given that the information access problem does not appear isolated, city or county crime labs or criminal justice state agencies could work with local libraries to license resources, or work with the state library to see if they could become part of a library consortium with access to scholarly databases. Acquisitions librarians have worked with consortia models for a long time and could help labs and agencies understand potential models for licensing materials. The Center for Research Libraries is an example of a long-running international consortium (21). If the consortia model is not feasible, FSP, through membership with a local public library, could have access to databases where articles can be accessed directly, or requested via interlibrary loan. Academic libraries based in public universities are also good resources for subject specific research help as well as accessing specialized science databases. Most public academic libraries have ways for non-university patrons to use computers in the library. In the absence of a national library for forensic sciences or a national strategy for providing access to forensics literature, the development of continuing education opportunities to help FSP understand the limitations of existing sources of information as well as additional places to search to find credible, freely available information would be helpful. This could also include information on effective search strategies and best practices to help FSP become more efficient searchers. A focus on search strategy design and best practices can also mitigate the frustrations some survey respondents expressed with the actual search process. Learning how to effectively use keywords and Boolean operators to create search strings would limit the number of results as well as the time it takes to sift through results. Other solutions include better tool selection such as searching Google Scholar rather than Google to find scholarly material. Continuing education related to information searching and evaluation is critical, especially with the development of the Open Access (OA) movement that advocates for more scholarly information to be made freely available online. Many federal granting agencies such as the NIH, NSF, and NIJ now require researchers to make publications and/or data produced under grant funds to be openly available (22). While PubMed Central is a good example of a federal repository for publications from grants awarded through the NIH, forensic science researchers may be depositing their papers in university or other subject repositories to fulfill a grant mandate or university requirement. These OA mandates are a recent development and some FSP may not know about this option or that Google Scholar and Google’s Dataset Search indexes materials that are posted in university repositories. Continuing education related to identifying good resources for literature, regardless of whether it is OA or subscription-based, as well as learning how to identify predatory journal practices, is critical. Forensic science educators can request the assistance of university librarians to teach students about publishing models, creating data management plans for grant funding, and depositing research data in university repositories. Raising awareness of resources like the Directory of Open Access Journals (https://doaj.org), a list of high quality, open access, peer- reviewed journals that includes categories such as law, anthropology, medicine, and science, can prevent researchers and students from falling prey to predatory journal publishers. The survey results showed many respondents were searching for articles. Until publication practices shift to favor more OA publications, there needs to be an easier mechanism for FSP to purchase articles as needed for their research work. FSP new to the field were less likely to pay for articles they could not get freely online as opposed to FSP who have decades of experience. Paying for articles was often not an option because of workplace bureaucracy leading to information coming too late, or a workplace lack https://doaj.org/ J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston of budget. Some FSP reported that they paid for information resources out of their own pocket. Additionally, researchers at universities can work with their university library or subject liaison librarian to learn more about how to make their research more accessible by depositing pre-prints of manuscripts in institutional repositories. Researchers can also request funds to make their articles open access in the Gold OA model. This model charges the author a fee which allows the article to be open to all readers on the publisher’s database platform, not just those who subscribe to that database. Forensic science educators can incorporate the need for a line item in the grant request to include funding to make research more widely available in grant writing workshops for students. In reviewing the data from question 13, the researchers discovered that information seeking terms commonly used by librarians did not map easily to the ways FSP respondents used the same terms. For example, when librarians use database as a term, they are referring to specific resources such as PubMed or Web of Science, which are citation and article databases. Similarly, when asking FSP about what types of resources might fit in their definition of a scholarly article, it was enlightening to see the variety of answers. In the library and information sciences discipline, “scholarly article” is typically used to describe peer-reviewed journal article, and librarians often focus tutorials and instruction on identifying and finding peer-reviewed articles (23). However, it could be that FSP who responded might be including all things that might be considered scholarly, credible, or acceptable as sources in their field. For clarity, librarians need to find different words to use or better explain what they mean by these terms when educating students. At the university level, peer-reviewed articles are typically required for assignments, and librarians working with forensic science classes should work with professors to not only teach about peer-reviewed sources but also other credible and commonly used materials that would be used in the field. There also emerged a different approach to access between forensic science professionals and librarians, which could be an area of future exploration. It seems that FSP tend to think, ‘I need X information,’ and are very transactional based on a specific case or need. In contrast, librarians tend to teach information seeking strategies that are applicable to any discipline and providing access to all available information. Are librarians teaching forensic science students what FSP want or need in their future work research? Managing expectations was another issue that emerged for which librarians and FSP educators can provide guidance. The Kruskal-Wallis test demonstrated FSP in all fields and all times in field experienced similar frustrations in locating information, indicating that access to information was a universal frustration. Information seeking takes time and information is distributed through many resource formats such as journal articles, web sites, and books, among others. Given that information, especially forensic science information, on a particular topic can cross disciplines, FSP can approach research with these expectations in mind. In college courses, librarians often tout subscription article databases, but according to this survey, library instruction sessions should also include research avenues to open access resources for when FSP do not have access to university resources after graduation. Forensic science educators can work with their university librarians to develop effective library training sessions to provide students with research skills that relate to the situations they will encounter in their future jobs. To assist the information access needs of FSP out in the workforce, the research team created freely available education modules based on the themes developed from this survey, which have been made available at https://sites.google.com/view/forensicscience- openaccess/home as a model to demonstrate this type of training to FSP. This training is something a forensic science library network could develop and maintain. Limitations This study consisted of a survey that was distributed to a forensic science professional organization. While this is a large association, it is not the only one, and there was a low response rate. While the researchers obtained useful information for the goals of developing information literacy education modules and resources related to information seeking and access, more studies should be conducted to examine more closely the use of scientific literature among FSP. Specifically, future research could include a more targeted quantitative survey to tease out more specific issues uncovered by these preliminary findings, including funding, access, and quality of resources. Conclusion This research uncovered how some FSP have found ways to work around barriers to information access. Collegiality is important and useful for accessing information in the various disciplines. Many FSP contact authors directly for a copy of their article or ask for help finding information on mailing lists. Newer FSP in this survey relied heavily on colleagues to acquire information they could not get freely online. Educators teaching forensic science students can instigate networking skills before the students enter the workforce, and FSP workplaces can provide new FSP with introductions within these close-knit fields. In addition to emphasizing the importance of building these professional relationships, forensic science educators can work to integrate information access, searching, and evaluation (including https://sites.google.com/view/forensicscience-openaccess/home https://sites.google.com/view/forensicscience-openaccess/home J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston open access materials) into their assignments and courses to better prepare their students for their careers. This investigation into anecdotal claims points to a need for further investigation. This survey provides data to support the anecdotal evidence researchers encountered regarding frustrations with access to scholarly information. The research showed that while not all FSP who responded to the survey need assistance in information seeking, there could be benefit from guidance regarding finding quality resources, particularly via open access venues, as well as education related to search strategies to help mitigate issues with time and information overload. Forensic science educators can work with academic librarians to provide education and instruction on what types of research students might be performing when they are employed as an FSP so that they have adequate knowledge and reasonable expectations if they are no longer affiliated with a university that provides easy access to scholarly information. Other ways for the forensic science community to provide information access is to create a network of libraries that could help provide access and training, or to work with libraries to become consortia members to provide access to scholarly databases. The researchers hope this manuscript will lead to further discussions on the role open access can play in the forensic science community and their research needs. In addition to this manuscript, the study data has been made available at doi:10.18738/T8/2BAQEO to help continue the conversation related to providing forensic science professionals better access to scholarly information. Acknowledgements This research was funded by National Institutes of Justice grant 2016-R2-CX-0054. Opinions, points of view expressed in this research, and products discussed represent a consensus of the authors and do not necessarily represent the official position, policies, or endorsement of the United States Department of Justice, Office of Justice Programs, or the NIJ. The authors would like to thank Sarah Potvin for her review of an earlier version of this manuscript. References 1. Committee on Identifying the Needs of the Forensic Sciences Community, National Research Council. Strengthening forensic science in the United States: a path forward. Washington, DC: National Academies Press, 2009. 2. National Institute of Justice (NIJ). Impact of forensic science research and development. Washington, DC: U.S. Department of Justice, 2015. http://permanent.access.gpo.gov/gpo63341/248572.p df 3. Baporikar N. Lifelong learning in knowledge society. In: Ordóñez de Pablos P, Tennyson RD, editors. Impact of economic crisis on education and the next- generation workforce. Hershey, PA: Information Science Reference, 2016;263-84. 4. National Commission on Forensic Science. Scientific literature in support of forensic science and practice: Views document issued by the Scientific Inquiry and Research Subcommittee. Washington, D.C. U.S. Department of Justice, 2015. https://www.justice.gov/sites/default/files/ncfs/pages/ attachments/2015/02/25/scientific_literature_views_d ocument_as_adopted_1_30_15.pdf 5. Teitelbaum J. An improved forensic science information search. Forensic Sci Rev 2015;27(1):41- 52. 6. Butler JM, Teitelbaum J, Makar S, Malanowski A, Taylor MK, Wood MR. Information does exist beyond the first page of your Google search!: Tools and strategies for forensic science literature searching and use. Workshop presented at the 68th American Academy of Forensic Science Annual Meeting; 2016 Feb 22-27; Las Vegas, NV. 7. Tomberlin JK, Jordan HR, Bankston SK, Sare L, et al. Viewing research through different lenses: How to achieve success in court. Workshop presented at the 69th American Academy of Forensic Science Annual Meeting; 2017 Feb 13-18; New Orleans, LA. 8. Knoll JL. Open access journals and forensic publishing. J Am Acad Psychiatry Law 2014;42(3):315-321. 9. Pontis S, Blandford A. Understanding “influence:” an exploratory study of academics' processes of knowledge construction through iterative and interactive information seeking. J Assoc Inf Sci Technol 2015;66(8):1576-1593. doi:10.1002/asi.23277 10. Makri S, Blandford A, Cox AL. Investigating the information-seeking behaviour of academic lawyers: from Ellis’s model to design. Inf Process Manage 2008;44(2):613-634. doi: 10.1016/j.ipm.2007.05.001 11. Dinet J, Chevalier A, Tricot A. Information search activity: an overview. Eur Rev Appl Psychol 2012;62(2):49-62. doi:10.1016/j.erap.2012.03.004 http://permanent.access.gpo.gov/gpo63341/248572.pdf http://permanent.access.gpo.gov/gpo63341/248572.pdf https://www.justice.gov/sites/default/files/ncfs/pages/attachments/2015/02/25/scientific_literature_views_document_as_adopted_1_30_15.pdf https://www.justice.gov/sites/default/files/ncfs/pages/attachments/2015/02/25/scientific_literature_views_document_as_adopted_1_30_15.pdf https://www.justice.gov/sites/default/files/ncfs/pages/attachments/2015/02/25/scientific_literature_views_document_as_adopted_1_30_15.pdf J Forensic Sci Educ 2021, 3(1) 2021 Journal Forensic Science Education Bankston 12. Palmquist RA, Kim K-S. Modeling the users of information systems: some theories and methods. Ref Libr 1998;28(60):3-25. doi:10.1300/J120v28n60_02 13. Tenopir C., King DW, Christian L, Volentine R. Scholarly article seeking, reading, and use: a continuing evolution from print to electronic in the sciences and social sciences. Learn Publ 2015;28:93- 105. doi:10.1087/20150203 14. American Academy of Forensic Sciences. Membership Statistics. July 2017. 15. Cresswell JW. Research design: qualitative, quantitative, and mixed methods approaches. 2nd edition. Thousand Oaks: Sage Publications, 2003. 16. Glaser B, Strauss A. The Discovery of Grounded Theory. Chicago: Aldine Publishing Company, 1967. 17. Corbin J, Strauss A. Basics of Qualitative Research. 4th edition. Los Angeles: Sage Publications, 2015. 18. Speaker SL. An historical overview of the National Network of Libraries of Medicine, 1985-2015. J Med Libr Assoc 2018;106(2):162-174. doi:10.5195/jmla.2018.297 19. Frierson EG, Gardner M, McCarthy S, Blake PJ. Collaborative development of agricultural information services at the National Agricultural Library of the United States. Sci Technol Libr 2003;24(1-2):5-20. doi:10.1300/J122v24n01_02 20. Moberly HK, Gardner MA. Creating distributed information access: the Agriculture Network Information Center (AgNIC). Tech Serv Q 2000;17(3):33-46. doi:10.1300/J124v17n03_03 21. Center for Research Libraries. Accessed September 28, 2020. https://www.crl.edu/about 22. National Institutes of Health, Public Access Policy, https://publicaccess.nih.gov/; National Science Foundation, Today’s Data, Tomorrow’s Discoveries, 2015, https://www.nsf.gov/pubs/2015/nsf15052/nsf15052.p df; National Institute of Justice, Research, Development, and Evaluation Grant Award Requirements, https://nij.ojp.gov/funding/research-development- and-evaluation-grant-award-requirements 23. Examples of university library guides describing scholarly material: https://www.library.wisc.edu/help/research-tips- tricks/identifying-scholarly-articles/; https://www.library.illinois.edu/ugl/howdoi/scholarly /; https://library.yale-nus.edu.sg/start-your- research/how-to-identify-a-scholarly-source/; https://research.library.gsu.edu/c.php?g=115862&p= 754393 https://www.crl.edu/about https://publicaccess.nih.gov/ https://www.nsf.gov/pubs/2015/nsf15052/nsf15052.pdf https://www.nsf.gov/pubs/2015/nsf15052/nsf15052.pdf https://nij.ojp.gov/funding/research-development-and-evaluation-grant-award-requirements https://nij.ojp.gov/funding/research-development-and-evaluation-grant-award-requirements https://www.library.wisc.edu/help/research-tips-tricks/identifying-scholarly-articles/ https://www.library.wisc.edu/help/research-tips-tricks/identifying-scholarly-articles/ https://www.library.illinois.edu/ugl/howdoi/scholarly/ https://www.library.illinois.edu/ugl/howdoi/scholarly/ https://library.yale-nus.edu.sg/start-your-research/how-to-identify-a-scholarly-source/ https://library.yale-nus.edu.sg/start-your-research/how-to-identify-a-scholarly-source/ https://research.library.gsu.edu/c.php?g=115862&p=754393 https://research.library.gsu.edu/c.php?g=115862&p=754393 Methods Results Discussion and Conclusion