AISW Vol. 5, No. 2 172 ADVANCES IN SOCIALWORK Views of Reference List Accuracy from SocialWork Journal Editors and Published Authors Scott E.Wilks Christina A. Spivey Abstract:Objective: The study’s purposewas to answer two research questions: (1) In the opinion of social work journal editors, how important is reference list accuracy? and (2)Who is primarily responsible for the accuracy of reference lists published in social work journals? Method: A sample of 119 authors and 26 journal editors was surveyed to ascertain their views on the above questions and additional items. Results: Regarding the importance of reference list accuracy, editors’ responses (Likert scale) averaged between moderately and extremely important. Fifty-three percent of responding editors and 36.5% of authors reported that responsibility is shared between the editor/staff and manuscript authors; the remaining 47% and 63.5%, respectively, responded that responsibility falls upon manuscript authors. Responses from authors, mostly educators, revealed a greater-than-moderate importance (Likert scale) given to instructing students on the accurate construction of reference lists. Implications for social work education and journal publishing are discussed. Keywords: APA,publication, reference, reference list, reference list accuracy Journal article reference lists serve several significant functions: (1) they provide background information on a particular area of interest; (2) they are used in assessing faculty production during contract negotiations; (3) they are used in constructing citation indexes; (4) they are used to evaluate the rigor of an article; (5) they are used in the development of journal, publisher, and author rankings; (6) they help to establish the credibility of an author as a researcher; and, perhaps most importantly, (7) they are used in the identification and location of an article that the reader wishes to consult (Asano, Mikawa, Nishina, Maekawa, & Obara, 1995; Fenton, Brazier, De Souza, Hughes, & McShane, 2000; Foreman & Kirchoff, 1987; Peden, 1991; Sweetland, 1989; Taylor, 1998). As Ritchie (1995) stated, “[R]eferences should be cited to support the importance of the topic under inves- tigation, to trace the historical development of the topic, and to summarize the current state of affairs” (p. 2). Scott E.Wilks is Assistant Professor, Department of Sociology & SocialWork, Appalachian State University, Boone, NC 28608. Christina A. Spivey was a doctoral student at the University of Georgia, Athens, GA 30602. Copyright© 2004 Advances in SocialWorkVol. 5 No. 2 (Fall 2004) 172-181. Indiana University School of SocialWork. 173 In a study conducted by Spivey andWilks (2003), results indicated that five lead- ing social work journals suffered reference list error rates ranging from29% to 57%. This study serves as a follow-up; the authors have conducted a survey of social work journal editors and authors in regard to their views on the importance of ref- erence lists in scholarly work. The purpose of this study was to answer two research questions: (1) In the opinion of social work journal editors, how impor- tant is reference list accuracy? and (2)Who is primarily responsible for the accura- cy of reference lists published in social work journals? LITERATURE REVIEW Rates of Error Spivey and Wilks (2003) examined a stratified random sample of 500 references from the year 2000; references were obtained from volumes of the fivemost high- ly circulated social work journals (NASWPress, 1997):Clinical SocialWork Journal (CSWJ), Journal of Social Work Education (JSWE), Social Service Review (SSR), SocialWork (SW), and SocialWork Research (SWR). Each of the references includ- ed in the sample was verified for accuracy in six fields: article title, author name(s), journal title, pagination, volume, and year. The breakdown of the rates of error was as follows: (1) CSWJ, 57%; (2) JSWE, 47%; (3) SW, 42%; (4) SWR, 31%; and (5) SSR, 29%. The overall rate of error found in the investigation was 42% (i.e., 206 of the 500 references had at least one error; there were 262 errors found in total). A search of social work literature has revealed that this is the only study of reference list accuracy in this field. Other fields, however, appear to acknowledge the importance of reference accuracy through a rich knowledge base in their respective bodies of literature, including medicine (De Lacey, Record & Wade, 1985; Evans, Nadjari & Burchell, 1990), nursing (Taylor, 1998), and psychology (Faunce & Job, 2001;White, 1987). Causes of Error Several reference list accuracy researchers have offered explanations as to the causes of the high error rates cited above. Sweetland (1989) discussed several comprehensive reasons behind the error rates. They include (1) failure to check material against original sources; (2) failure to recognize or acknowledge the importance of reference lists; and (3) failure to recognize or acknowledge the importance of reference lists. Prior to Sweetland,White (1987) had cited a lack of clear education as one of the main factors contributing to reference errors and also posited that pressure to publish may lead authors to neglect their reference lists when preparing to submit manuscripts. Fenton, et al. (2000) also concluded that the “carelessness and misuse of language” led to inaccuracies in references (p. 42). Strategies to Minimize Error Authors have proffered numerous suggestions for addressing the causes of inac- curacies and decreasing error rates. Fenton, et al. (2000) proposed: • assessment of a certain percentage of a submitted manuscript’s references as part of the acceptance process; Wilks, Spivey/VIEWS OF REFERENCE LIST ACCURACY FROM SOCIALWORK JOURNAL EDITORS AND AUTHORS 174 ADVANCES IN SOCIALWORK • placing a limit on the number of references allowed for a submitted manu- script; • inclusion of the first page of each referenced article, or a complete copy of each reference article, in the manuscript submission package; • utilization of reference software, such as Endnotes; and • shared responsibility between authors, editors, and reviewers. Fenton, et al.’s study, particularly the final point above, served as an impetus for the current study. In addition, Foreman and Kirchoff (1987) and Taylor (1998) encouraged authors to check each reference against the original source, and Foreman and Kirchoff further suggested that authors review each draft of a man- uscript for errors to prevent them from becoming embedded in the work. Who is Responsible? The question of responsibility has been addressed by several researcherswhohave conducted investigations of reference list accuracy. Several of those researchers, including De Lacey, Record and Wade (1985); White (1987); Evans, Nadjari and Burchell (1990); Goldberg, et al. (1993); and Ritchie (1995) agree that the task of ensuring accuracy belongs to the author(s). As Foreman and Kirchoff (1987) stat- ed, “the author cannot be absolved of primary responsibility for assuring that ref- erences are both complete and correct” (p. 182). Key and Roland (1977) offered a different perspective, assigning responsibility to journal editorial boards. While the majority of researchers in this area have declared authors primarily responsible, several of these scholars recognize that there is a need for some degree of shared responsibility (that does not diminish the primacy of the author’s role). De Lacey, et al. (1985) wrote, “[E]ditors should accept some responsibility for accuracy” (p. 885), while Fenton, et al. (2000) stated, “[A] certain accountability lies within the editorial panel of the journal” (p. 43). Goldberg, et al. (1993) also holds the position that “some degree of editorial oversight appears to be necessary” (p. 1453). In addition, Evans, et al. (1990) advocated a stronger role for peer reviewers. The Role of Education White (1987) and Sweetland (1989) both questioned the role of (or lack of) instruc- tion as one of the causes of high error rates in reference lists. Furthermore,White stated, “[T]he problem is one that must be recognized and then solved by writers themselves, who also are, for good or ill, the major models for writers-to-be” (p. 291). As a means of redress, Sweetland encouraged the improved training of researchers (p. 301). Another researcher, Peden (1991), described a technique for educating students as both to the preparation of correct APA-style reference lists and to the importance of the accuracy of those lists. Peden exposed students to a method for teaching reference list construction that included a handout on APA references, six tests (three focused on identifying the type of material by its refer- ence structure and threewhich required students to actually construct references), and a posttest as to the students’ reaction to the instruction. In the posttest, stu- dents agreed that in comparison with learning the “importance of accuracy in research . . . it was equally important for them to learn accuracy in referencing” (p. 104). Peden concluded that, “developing and using this technique increased my 175 awareness and understanding of other aspects of writing beyond referencing . . . abilities that serve all of us well in our roles as authors, reviewers, and teachers” (p. 104). METHOD A purposive sample of authors was selected for this study: lead authors on articles published in the year 2000 volumes of the five journals reviewed in the Spivey and Wilks study (2003):Clinical SocialWork Journal, Journal of SocialWork Education, Social Service Review, SocialWork, and SocialWork Research. The vast majority of these authors were (at the time of the study) faculty members at various social work programs; thus, verification of mailing addresses was conducted through online websites of their respective employing academic institutions. Only those authors who were deceased or whose mailing addresses could not be verified via the Internet were excluded from the sample. A total of 119 authors were included in the sample. An additional purposive sample was selected for the study: editors of highly cir- culated, peer reviewed social work journals. Contact information for editors was available in their respective journals. The number of journal editors—26 in total—was limited due to various motives. For one, the opinions of reference list accuracy within the social work profession have never been gauged, thus justify- ing the exploratory nature of this study. As such, a small subset of the larger pop- ulation (all social work journal editors) was deemed appropriate (see Rubin & Babbie, 2001, p. 254) in this untapped area of social work research. Second, because journal circulation was an element involved in selecting the authors, for the sake of consistency, this component (specifically, wide circulation) was con- sidered in selecting the editors. The cutoff was 1,000, meaning that only editors from journals with a 1,000+ circulation were qualified for selection (NASW Press, 1997). Third, their journals must be focused on content relating entirely (or almost entirely) to social work, as opposed to journals centered with other human service professions (e.g., American Journal of Sociology or Psychological Review). Finally, the editors’ journals must subscribe to the APA publication style. To reiterate, after considering all of the aforementioned qualifications, the sam- ple size of journal editors was 26. There were two distinct questionnaire forms: one specifically for journal edi- tors, the other for authors. The editor questionnaire consisted of five questions geared toward measurement of the importance of reference list accuracy and determination of views on responsibility. The author questionnaire was some- what longer, 12 questions in total. Authors were also asked their opinions on responsibility; in addition, questionswere included to discover authors’ reference verification practices, whether or not they were university instructors, and whether, as instructors, they emphasized reference accuracy in their curriculum. RESULTS Response Rates As stated in the previous section, the study surveyed 26 social work journal edi- tors and 119 authors of published social work manuscripts. Eighty-nine of the Wilks, Spivey/VIEWS OF REFERENCE LIST ACCURACY FROM SOCIALWORK JOURNAL EDITORS AND AUTHORS 176 ADVANCES IN SOCIALWORK total 145 participants completed and returned the questionnaires, yielding a response rate of 61.4%. Isolating the two groups of participants, journal editors produced a higher response rate of 73% compared to the author response rate of around 50%. Next, we looked at descriptive statistics within editor and author questionnaires. EditorViews When asked about whom is primarily responsible for reference list accuracy, edi- tors had one of three options: editor/staff, manuscript authors, or shared respon- sibility. Approximately 53% of responding editors reported that such responsibil- ity is shared between the editor/staff and themanuscript authors. The remaining 47% responded that responsibility falls upon manuscript authors; as such, no editor reported that the primary responsibility of reference list accuracy rests solely upon her/himself or the editorial staff. Two subsequent items on the questionnaire centered on editorial practices. Regarding substantiation for reference list accuracy, only 21% of responding edi- tors reported that their journals maintain a formal procedure for verifying such accuracy in submitted manuscripts. Not surprisingly (in light of the previous statement), 21% of responding editors replied thatmanuscripts have been rejected for acceptance into their journals based primarily on reference list inaccuracies. The remaining items on the editor questionnaire were based on a 7-point Likert response scale, ranging from 1–not important to 7–extremely important. First, editors were asked, “How important is reference list accuracy?” Second, editors were surveyed about the influence of reference list accuracy on a manuscript’s probability of publication. For both items, responses averaged between moder- ately and extremely important. Exact numbers (means, standard deviations) are displayed in Table 1. AuthorViews The first item in the author questionnaire was identical to that in the editor ques- tionnaire, inquiring about primary responsibility for reference list accuracy. Compared to 53% reported by editors, only 36.5% of responding authors report- ed that the responsibility is shared between editors and authors. The remaining majority (63.5%) responded that primary responsibility falls squarely on the shoulders of manuscript authors. It is interesting to note that no one, editors or authors, responded that primary responsibility for reference list accuracy lies exclusively with editors. The next two items dealt specifically with reference list construction in manu- scripts. Authors were asked whether or not they use any type of specialized com- Item Mean Std. Deviation Importance of Reference List Accuracy 5.8 1.01 Influence of Accuracy on Publication Chances 4.1 1.43 Table 1: Importance/Influence of Reference List Accuracy 177 puter software in constructing reference lists; and, if usage was reported, authors were invited to document the specific software. Approximately 86% of respond- ing authors reported no usage of such software; the remaining 14% credited the use of the following software programs: ClarisWorks, Endnotes, Excel, and Prolite. The following two items focused on verification of reference list accuracy by checking against the original sources. In constructing their manuscripts, almost 80% of responding authors reported that they verify against the original sources. Less than half of the authors who responded, 43%, reported that any pressure to publish resulted in rushed verification of accuracy. In order to confirm that such pressure to publish does exist, instructors were asked to rate their feelings on this issue. Based on a Likert response method ranging from 1–no pressure to 7–extreme pressure, instructors as a group (n=59) reported a rather elevated amount of pressure to publish: mean=5.2, std. deviation=1.48. The remaining items of the author questionnaire were geared toward their roles (if applicable) as instructors. An overwhelmingmajority of authors who respond- ed, 93.7% to be exact, reported that they were an instructor at a college or univer- sity at the time of data collection. Those who confirmed their role as instructor were invited to complete the remaining items on the questionnaire; thus, the remaining statistics revealed in this section illustrate opinions only from the aforementioned respondents who disclosed their roles as academic instructors. Respondents were asked to indicate the length of time that had acted as an instructor. Based on three options, the data revealed the following: approximate- ly 71% reportedmore than 10 years; 27% reported 5-10 years; and 1% reported less than five years.When asked whether or not they correct inaccuracies in reference lists when grading students’ papers, 81% reported that they indeed correct stu- dents’ reference list errors. Going a step further, instructors were asked to specify what type of inaccuracies they checked for in students’ reference lists: specifical- ly, content and/or format accuracy. (As was clarified in the questionnaires, please note that examples of content accuracy include accurate spelling of authors names, correct journal title, and exact page numbers; examples of format accura- cy (APA) include hanging indent, double spacing, and proper italicization.) Over half of responding instructors reported checking for both types of accuracy with- in their students’ reference lists. Table 2 illustrates the specific percentages of aggregate responses for each method of correction. Additionally, instructors were asked about specific information they provide for their students regarding the APA Publication Manual. Options included one or more of the following: incorporating the manual as a required text, handouts on Wilks, Spivey/VIEWS OF REFERENCE LIST ACCURACY FROM SOCIALWORK JOURNAL EDITORS AND AUTHORS Option Frequency of Response (%) n Do not check 11.9 7 Check for content only 8.5 5 Check for format only 20.3 12 Check for both content and format 59.3 35 Table 2: Reference List Accuracy: Method of Correction in Students’ Papers 178 ADVANCES IN SOCIALWORK specific elements of themanual, and in-class presentations regarding elements of themanual. The largest portion of respondents, approximately 27%, reported giv- ing no information to students. The next highest percentage of respondents, 18%, reported using the manual as a required text in their classes. Table 3 reveals the percentages reported by instructors for each option. Lastly, using a 7-point Likert response system ranging from 1-not important to 7-extremely important, instructors’ opinions weremeasured regarding the overall importance of instruction to students on reference list accuracy. Aggregate data from instructors’ responses (n=59) revealed a greater-than-moderate importance given to instructing students on the accurate construction of reference lists: mean=5.0, std. deviation=1.52. DISCUSSION AND APPLICATIONTO SOCIALWORK PRACTICE The larger population of this study was easily identified: social work journal edi- tors and recently published authors in social work journals. Because enumerating them would be nearly impossible (especially the authors), a purpose sampling technique was employed (see Rubin & Babbie, 2001, p. 254). Purposive sampling carries the same limitations as most non-probability sampling techniques. In practice, according to Rubin and Babbie (p. 259), there is less likelihood that a non-probability sample will be representative of the population from which it is drawn compared to a probability sample. Also, the sample size may be consid- ered rather small. Justification for the sample size was explained in the Method section. A primary limitation of a mail survey is the possibility of low response rate (Grinnell, 1997). Low rates may threaten measurement validity. This study, which included follow-up mailings, yielded a response rate of about 61%. According to consistent views of social work researchers (Babbie, 1995; Rubin & Babbie, 2001), this constitutes a “good” rate. Nevertheless, a higher response rate (preferably over 70%) would have been preferred. There are others disadvantages to the self-administered survey. First, no assis- tance was available to any respondent if s/he needed clarification on a particular questionnaire item(s) (Grinnell, 1997, p. 352). Second, the questionnaire is useful Option Frequency of Response (%) n No information given 27.1 16 APA Manual as required text 18.6 11 Handouts 15.3 9 Presentations 11.9 7 Required text & handouts 1.7 1 Required text & presentations 1.7 1 Handouts & presentations 15.3 9 Required text, handouts, & presentations 8.5 5 Table 3: Information Provided to Students in Class only with those respondents who have the physical capacities to complete it. Third, as with any survey research, this study provided information with little context of social life (Rubin & Babbie, 2001, p. 381). Information was obtained regarding views of reference list accuracy; however, a deeper, richer understand- ing of editors’ and authors’ opinions of reference lists (as revealed through quali- tative data) remains unknown. To reiterate, the primary purpose of this study was twofold: (1) to learn the degree of importance social work journal editors attributed to reference list accu- racy, and (2) to learn who editors and authors believe is primarily responsible for this accuracy. The results of the survey showed that, based on the sample, journal editors consider accuracy of references important. The extent of this importance is mademore evident in the finding that reference list accuracy is at least moder- ately influential in the chance a manuscript will be accepted for publication. As to the second purpose, editors seemed to favor shared responsibility between authors and themselves, while the majority of authors responded that they and their fellow writers were primarily responsible for reference list accuracy. It is rather telling, however, that although accuracy of references was rated as more than moderately important and editors tended to fall on the side of shared responsibility, only a small minority of editors reported that their journals had a formal procedure for checking reference accuracy. The authors of the current study agree with themajority of editors who believed that responsibility should be shared. Ultimately, each actor, the author and the editor, has a role to play in the publication of a manuscript. It is the duty of the author to provide the editor with a sound, accurate manuscript (both in terms of content and references) and it is likewise the duty of the editor to provide con- sumers with a sound, accurate article. Inaccuracies within a publication cast doubt upon both the author and the journal, which is represented by the editor. Therefore, both participants in the publication process should have some degree of responsibility for the finished product. We would also encourage editors to consider instituting formal verification procedures, as this strategy has proven effective for journals in other disciplines (case in point: Canadian Journal of Anesthesia, see Bevan and Purkis, 1995). Regarding the survey of authors, secondary issues, such as pressures to publish, checking original sources, computer software, and education were examined. White (1987) questioned the role of pressure to publish by an author’s university of employment in rates of reference list error. On one hand, White’s supposition was supported by the current study’s findings: authors indicated that they felt more than moderate pressure to publish. On the other hand, less than half the authors reported that pressure to publish resulted in a rush to check references. The findings suggest that while there is pressure to publish, it has limited influ- ence on the time devoted to verification of references. Several authors (i.e., Foreman and Kirchoff, 1987; Sweetland, 1989; Goldberg, 1993; and Fenton, et al., 2000) have posited that failure to check references against the original sources has led to inaccuracies in reference lists. The results in this study fail to support this premise. The vast majority of authors stated that they do indeed verify references with the original materials. Amore sophisticated 179Wilks, Spivey/VIEWS OF REFERENCE LIST ACCURACY FROM SOCIALWORK JOURNAL EDITORS AND AUTHORS 180 ADVANCES IN SOCIALWORK study than the one done here, however, will be needed to conclusively determine the causal link, if any, between failure to check original sources and inaccurate reference lists. Fenton, et al. (2000) suggested the utilization of computer software as a strate- gy for reducing reference errors. Authors participating in this survey, however, indicated onlyminimal use of such software.We speculate that as computer tech- nology becomes further ingrained in our professional work, we will see an increase in the availability and employment of software in the preparation of manuscript reference lists. The utilization, impact, and effectiveness of software in scholarly writing should be an avenue of future investigation. Education is, without question, the most significant of the secondary issues addressed in this study. Those who have conducted research in the area of refer- ence list accuracy, including White (1987), Sweetland (1989), and Peden (1991), stressed the importance of proper instruction in reference list construction and the role instructors-as-authors have as models for future scholars. In the current study, authors who are also instructors clearly agree with the above-named schol- ars. They rated importance of reference accuracy instruction as more than mod- erately important. The majority provided some form of information to their stu- dents, whether it was in the form of presentations, handouts, the APAmanual, or some combination of the three. The majority also indicated that they check the accuracy of reference lists in student papers. The findings regarding education suggest at least two possible areas for further research: • What form of information provided or method of instruction is correlated with increase in reference list accuracy in student papers? and, • What impact does the provision of information have on students’ attitudes toward the importance of reference list accuracy? In a human service profession, such as the social work profession, where con- sumers of literature are faced with clients battling such issues as poverty, abuse, and mental illness, research pertaining to reference list accuracy may seem triv- ial. When we examine the purposes of reference lists, however, this triviality diminishes and their true importance is acknowledged. Reference lists are a tes- tament to the work of scholars, they are evidence of a knowledge base in a given field of interest; moreover, reference lists are a sign of the professionalism, pre- paredness, and hard work put forth by scholarly writers. Inaccuracies cast a shad- ow upon references and, by extension, their producers. For our students, our con- sumers, and our own professional credibility, we should make every effort to ensure the accuracy of our published work, including references lists. References Asano, M., Mikawa, K., Nishina, K., Maekawa, N., & Obara, H. (1995). Improvement of the accuracy of ref- erences in the Canadian Journal of Anesthesia. Canadian Journal of Anesthesia, 42(5), 370-372. Babbie, E.R. (1995). The practice of social research (7th ed.). Belmont, CA:Wadsworth. Bevan,D.R.,&Purkis, J.M. (1995).Citationerrorscanbereduced.CanadianJournalofAnesthesia,42(5), 367-369. 181 De Lacey, G., Record, C., &Wade, J. (1985). How accurate are quotations and references in medical jour- nals? British Medical Journal, 291, 884-886. Evans, J.T., Nadjari, H.I., & Burchell, S.A. (1990). Quotational and reference accuracy in surgical journals: A continuing peer review problem. Journal of the American Medical Association, 263(10), 1353-1354. Faunce, G.J., & Job, R.F.S. (2001). The accuracy of reference lists in five experimental psychology journals. American Psychologist, 56, 829-830. Fenton, J.E., Brazier, H., De Souza, A., Hughes, J.P., & McShane, D.P. (2000). The accuracy of citation and quotation in otolaryngology/head and neck surgery journals. Clinical Otolaryngology, 25, 40-44. Foreman, M.D., & Kirchhoff, K.T. (1987). Accuracy of references in nursing journals. Research in Nursing and Health, 10, 177-183. Goldberg, R., Newton, E., Cameron, J., Jacobson, R., Chan, L., Bukata,W.R., & Rakab, A. (1993). Reference accuracy in the emergency medicine literature. Annals of Emergency Medicine, 22(9), 1450-1454. Grinnell, Jr., R.M. (1997). Social work research and evaluation:Quantitative and qualitative approaches (5th ed.). Itasca, IL: F.E. Peacock Publishers, Inc. Key, J.D., & Roland, C.G. (1977). Reference accuracy in articles accepted for publication in the Archives of Physical Medicine and Rehabilitation. Archives of Physical Medicine and Rehabilitation, 58, 136-137. NASW Press. (1997). An author’s guide to social work journals (4th ed.).Washington, D.C.: Author. Peden, B.F. (1991). Teaching the importance of accuracy in preparing references. Teaching of Psychology, 18(2), 102-105. Ritchie, M.H. (1995). Proper use of literature review. Counselor Education and Supervision, 35(1), 2-3. Rubin, A., & Babbie, E. (2001). Research methods for social work (4th ed). Belmont, CA:Wadsworth. Spivey, C.A., &Wilks, S.E. (In press). Reference list accuracy in social work journals.Research on SocialWork Practice. Sweetland, J.H. (1989). Errors in bibliographic citations: A continuing problem. The Library Quarterly, 59(4), 291-304. Taylor, M.K. (1998). The practical effects of errors in reference lists in nursing research journals. Nursing Research, 47(5), 300-303. White, A. (1987). Reference inaccuracies in two counseling journals.Counselor Education and Supervision, 26(4), 286-292. Author’s Note: Address correspondence to: Scott E. Wilks, Ph.D., Assistant Professor, Department of Sociology & Social Work, Appalachian State University, 209 Chapell Wilson Hall, Boone, NC 28608, USA. E-mail: wilksse@appstate.edu Wilks, Spivey/VIEWS OF REFERENCE LIST ACCURACY FROM SOCIALWORK JOURNAL EDITORS AND AUTHORS