78 BARRIERS TO RECRUITING AN ADEQUATE SAMPLE IN RURAL NURSING RESEARCH Shirley Cudney, MA, RN, GNP1 Carol Craig, Ph.D., RN, FNP-C2 Elizabeth Nichols, DNS, RN, FAAN3 Clarann Weinert, SC, PhD, RN, FAAN4 1 Researcher, College of Nursing, Montana State University, scudney@montana.edu 2 Associate Professor, School of Nursing, Oregon Health Sciences University, craigc@oit.edu 3 Associate Professor, College of Nursing, University of North Dakota, Elizabeth.Nichols@mail.und.nodak.edu 4 Professor, College of Nursing, Montana State University, cweinert@montana.edu Keywords: Rural, Nursing Research, Sampling, Recruitment ABSTRACT Recruiting an adequate sample in rural nursing research has its unique challenges. In this article, nurse researchers share their adventures in recruiting study participants from four studies in the rural West. Distilled from these adventures, the authors present several issues to consider when developing recruitment strategies as part of the overall planning of a rural study. These include: (a) uniqueness of the rural culture and context, higher costs of recruitment, (b) necessity for rural- sensitive recruitment materials, (c) over-sampling, and (d) lack of local research infrastructure. INTRODUCTION In the quest for well-designed and executed research studies, decisions regarding the most effective designs, measurement tools, and methods of data collection and analysis have been debated and re-debated by nurse researchers over the years. Despite the profound influence the issue of recruitment of research participants can have on the success or failure of a project, it has been neglected in these discussions (Diekmann & Smith, 1989; Holden, Rosenberg, Barker, Tuhrim, & Brenner, 1993; Wray & Gates, 1996). The recruitment of participants is a crucial factor in the success of a research project but can be problematic if the target population is not easily approachable. In rural nursing research, the unique characteristics of the rural setting can pose significant challenges to the recruitment process (Nichols, 2003). For example, in rural areas, the numbers of those who meet study criteria may be limited and may be difficult to contact in remote locations on ranches and in small towns. Difficulty complying with Institutional Review Boards regulations, formidable distances to health care centers, relatively few health care resources, and inadequate transportation systems complicate the process of seeking potential study participants. Under these circumstances, creative recruitment methods are required. This article is based on the real life adventures of conducting studies with rural dwellers in Montana, Idaho, North Dakota, Oregon, South Dakota, and Wyoming. The nurse researchers share their challenges in recruiting rural study participants. From these experiences have been distilled several recruitment issues to be considered by researchers planning rural studies. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 http://www.montana.edu/wwwnu/ mailto:scudney@montana.edu http://www.ohsu.edu/son/ mailto:craigc@oit.edu http://www.nursing.und.edu/ mailto:Elizabeth.Nichols@mail.und.nodak.edu http://www.montana.edu/wwwnu/ mailto:cweinert@montana.edu 79 PARTICIPANT RECRUITMENT ADVENTURES IN RURAL AREAS Recruitment challenges for four research studies form the basis for discussion. Some were more successful than others in overcoming recruitment barriers. The lessons learned will be shared, thus meeting the obligation cited by Wray and Gates (1996) to share professional struggles as well as success stories. The four studies were multi-site team endeavors, funded extramurally, and conducted with adult rural dwellers who lived in communities of less than 20,000, more than 25 miles from major commerce centers, and/or in remote areas on farms or ranches. The titles of the studies were: (a) Health Care Choices (to determine the complementary therapy use of elders); (b) Living with Cancer: A Four State Study (to evaluate the psychometric properties of two fatigue scales in persons undergoing cancer therapy); (c) Spirituality: Rural Dwellers and Chronic Illness (to assess spiritual influences in chronic illness management); and (d) The Women to Women Project (an ongoing research-based computer outreach support and education intervention targeting chronically ill rural women). Adventure 1: Health Care Choices (HCC) Complementary and alternative therapies have been used by individuals for many years. Recent increases in the use of these approaches to manage or prevent health problems have focused on the actual extent of use of such therapies (Astin, 1998; Eisenberg et al. 1993; Eisenberg et al. 1998; Harris, 1987; Paramore, 1997). National studies of complementary therapy use have been extrapolated to sub-groups, such as rural dwellers, or the elderly, but little research has focused on the rural-dwelling elderly. The research team of four investigators from two rural, northern-tier states was interested in the elderly’s use of complementary and alternative therapies; what factors correlated with such use; what therapies were used; and for what purposes they were used. To answer these questions, the team decided to telephone interview rural elderly in North Dakota and Montana. The recruitment challenge was to obtain a sample large enough to support inferential statistical analysis as well as to ensure a representative sample. Using national survey data to determine expected values, statistical consultation and power analysis determined the necessary sample size to be 320. The specific recruitment goals were: (a) ensuring distribution of the sample across the two states and four different rural economic bases, (b) obtaining the names and addresses of the target population from which to draw the random sample, and (c) enticing randomly selected rural residents to participate in the telephone interview. Four strata were developed based on economic and geographic characteristics. From these strata, ten communities were randomly drawn. The number of participants to be recruited from each community was based on the proportion of residents older than 60 years of age in those communities compared to the population older than 60 in the state. Names, addresses and other biographical information about heads of households older than age 60 with mailing addresses in the selected communities were obtained from a commercial listing company, Senior Source. Senior Source listings were developed from a variety of public sources of information, so were deemed to be fairly comprehensive, although not exhaustive, listings of the target population. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 80 The research team employed a unique approach to gain access to the targeted communities and engage potential participants in the study. Initially, a logo and a simple, distinct project name were developed to place on all communications. The individuals who had been selected were given advance information about the study through letters of introduction and explanation, and an article about the impending study was placed in local newspapers. Local senior centers were asked to post flyers in prominent places. These promotional activities were timed to take place approximately two weeks before telephone interviewing began in a community. Data collection was arranged for an “off” time in the rural cycle of life, avoiding spring planting time, branding, harvest, and round-up times. Once the interviews were completed, all those whose names had been drawn were sent a “thank you” letter. One version was sent to those who were interviewed and another to those who were not interviewed, based on the interview quota for that community having been met. A short newsletter reporting study findings was mailed to the participants at the end of the study. To ensure efficient administration of the telephone questionnaires by experienced interviewers, trained students from the Social Science Research Institute at the University of North Dakota were employed. Interviewee participation may have been enhanced because the calls were identified as coming from the Research Institute at the University of North Dakota on caller ID systems, thus avoiding confusion with telemarketers. This process of identifying, recruiting, and interviewing participants took almost 18 months. The actual interviews were completed in two one-month time blocks. To obtain 325 completed interviews, 960 letters of invitation were mailed to heighten the visibility of the study within the community. The large number of letters reflects over- sampling to ensure adequate and appropriate participation in each target community. The participation rate for those who were actually contacted by telephone was 69.3%, a very satisfactory rate. Adventure 2: Living With Cancer: A Four State Study Fatigue, a subjective experience of tiredness (Winningham et al. 1994), is a most troublesome problem for people undergoing cancer therapy. For this reason, the research team tested mail survey measures that had the potential to discriminate levels of fatigue in a population of urban and rural persons with cancer. The aim of the study was to assess the reliability and validity of two scales: (a) The Fatigue Severity Scale (Krupp, Lauraceae, Muir-Nash, & Steinberg, 1989) and (b) The Visual Analogue Scale for Fatigue (Lee, Hicks, & Nino-Murcia, 1991). Four data collection sites were identified in order to obtain an adequate and diverse rural and urban sample. Investigators at the University of Vermont, operating base for the study, and Maine Medical Center, Portland, Maine, were responsible for the urban samples. The rural samples, one predominantly black and the other white participants in a sparely populated area, were recruited by co-investigators from Georgia Southern University and Montana State University–Bozeman. At each site, 130 individuals undergoing cancer therapy and 20 individuals without a diagnosis of cancer were to be recruited for a total of 520 and 80 respectively. The present discussion focuses on the challenge of recruiting the sample from a sparsely populated area. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 81 Eligibility included rural women and men, between the ages of 30-70, who were able to read and write in English. The Montana co-investigator centered her initial recruitment efforts in the largest city in the state since the health care facilities there provided cancer treatment for a very large rural area of eastern Montana and northern Wyoming. A key clinical nurse contact, known to the Montana co-investigator, was the president of the local Oncology Nurses Society, and was also a well-established Master’s prepared oncology nurse based at one of the two city hospitals. She routinely traveled with the hospital oncology team to small clinics throughout the area, was enthusiastic about the project, and encouraged other nurses to fully participate in recruiting participants. The oncology nurse and co-investigator worked together on a written data collection protocol to guide the selection of participants and assist clinic nurse collaborators. To determine rural residence, a list of all ZIP codes in Montana and Wyoming that did not qualify was also included. After being oriented to the study protocol by the oncology nurse, the clinic nurse approached potential participants at the time of their visit for treatment and, after describing the study, had them sign consent forms and data identification sheets. The nurse then gave a participant the attractive questionnaire packet that was designed following the Total Design Method (Dillman, 1978). The packet could be completed while waiting for treatment or filled out later at home, to be returned to the researchers in a self-addressed, stamped envelope. Upon receipt of the questionnaire in the research office, a “thank you” letter was sent to the study participant. Entering into the recruitment phase of the study, there were a number of reasons to believe that recruiting 130 participants would present few problems. First, the cancer treatment centers in the city had a flow of patients that appeared to have more than an adequate pool of eligible participants. Second, the oncology nurse was very energetic and committed to the project, and her presence in the health care community provided legitimacy to the research. Third, the recruitment burden on the clinic nurses was reduced by careful preparation of materials and a limited time commitment. Fourth, the questionnaire could be completed in about 30 minutes while the participant was waiting at the treatment center. The study was publicized by placing articles in all the daily and weekly newspapers throughout the state and by sharing a one-page description of the study with nursing students and the health care community. With this background, recruitment began in the three metropolitan cancer treatment centers on February 15, 2000, but only 24 useable questionnaires had been received by the end of June, necessitating additional strategies to enhance recruitment. Five additional clinical sites in Montana and Wyoming were added. The Montana co- investigator periodically called or sent notes to the nurse recruiter at each site, thanking her for participating and giving an update on the progress of data collection. In July the co-investigator traveled 1,030 miles to visit each of the data collection sites and spoke personally with the nurse recruiters, expressed gratitude for their help, answered questions, and encouraged them to continue recruiting. When they were called or visited, the cooperating nurses seemed enthusiastic about the project, but often said that they did not have clients who met the criteria or who wanted to participate. An additional strategy engaged two senior nursing students who assisted the nurse recruiters in the city. They communicated with the clinical sites, provided them with Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 82 adequate survey materials, and helped screen the patient lists for potential research participants. At first, the efforts seemed to be paying off, but recruitment again waned. No new efforts were initiated because the window of time for recruitment was closed, the holidays were approaching, and study resources were exhausted. While it was disappointing to collect only 50.7% of the target sample, the response rate was a respectable 84.6%. Data analysis was not completely compromised, since data were being collected in three other states. Recruitment for the cancer-free group (N=20) was somewhat more successful. The two senior nursing students contacted high school teachers, churches, conducted blood pressure checks at a small rural grocery store where they distributed the survey, worked with other students who had contacts in rural communities, traveled to two small communities in a 50 mile radius of the city, and used word of mouth. They were successful in getting 32 completed questionnaires from the 38 which were distributed, for a response rate of 84.2%. Adventure 3: Spirituality: Rural Dwellers and Chronic Illness Spirituality is an overarching human characteristic, an integrating force that allows us to be most fully human. Spirituality appears to have an effect on helping people to heal, to manage illness, and to find comfort during illness (Colantonio, Kask, & Ostfeld, 1992; Idler & Kasl, 1997; Morris, 2000 ). The extent of the connection for rural people, who have greater risk for poor outcomes, with chronic illness and spirituality is unclear. The chronic illness and spirituality project was a pilot study with a three-fold purpose to: (a) examine the association between spirituality and health behaviors, quality of life, and illness management; (b) explore the nature of spirituality among rural- dwelling people with chronic illness; and (c) investigate the psychometric properties of a new instrument to measure spirituality. The research team had a history of working with rural dwellers and was interested in exploring spirituality. Their definition of “spirituality” was: “A sense of meaning and purpose in life; connection to others, a higher power, or to nature; and a commitment to something greater than the self” (Craig, 2001). Participants were rural dwellers in Oregon and Montana, 18 years or older, with a chronic health condition, defined as any prolonged illness that does not resolve spontaneously and is rarely cured completely. A sample size of 240 was determined by power analysis. The recruitment challenge was to identify and enroll a rather large number of people with chronic illnesses who were living in very rural places. Parish nurses and church leaders seemed logical groups to approach because they had an interest in spirituality themselves. Senior centers were likely places where people with chronic illnesses could be found in large numbers. A flyer was developed with a description of the study and 204 of them were sent with a cover letter and business cards to rural parish nurses, churches, and senior centers in Montana and Oregon. Press releases were sent to religious newsletters in both states. Individuals could volunteer by calling a toll-free number. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 83 The response was dismal. Recruiting began in September, and by early November, only 12 people had agreed to participate. Because funding was limited, e-mail was used to send a message to 740 parish nurses. A press release was also submitted to all rural newspapers in both states, which generated a modest response. By February, 64 participants responded, but still far from the anticipated goal of 240. Next, telephone contacts were made with parish nurses, spiritual leaders, senior centers, and leaders of self-help groups, and the press release was sent to newspapers in larger towns, with the reasoning that rural residents often read papers from larger cities in their states. Personal contacts were not productive, but the newspaper response was encouraging. After re-evaluation, the research team decided that the prominent use of “spirituality” as the first word on news articles and letterhead might be discouraging people from participating. The news articles were rewritten with “chronic illness” as the headline and “chronic illness” was used in the letterhead instead of “spirituality.” This strategy increased response to some extent, but recruitment had to be terminated because of financial and time limitations. The recruitment period ended with a total of 111 participants. Adventure 4: The Women to Women Project For eight years, the focus of the Women to Women Project (WTW) has been the testing of a computer-based intervention for the delivery of social support and health education to rural, isolated, middle-aged, chronically ill women and evaluating its impact on the women’s potential to successfully adapt to their long-term illnesses (Cudney & Weinert, 2000; Weinert, 2000). WTW is being implemented in two phases. In the first four years (Phase One), 150 women were recruited to participate in the testing of a computer support group and off-line health education experience (five cohorts of 30 women each). In Phase Two, the study was revised to include an interdisciplinary research team, expand its scope from state to regional, and use an Internet-based health education experience. Based on a 20% attrition rate in Phase One of the project, the sample size was increased to 60 women per cohort (three groups of 20 per cohort) in Phase Two. The recruitment challenge for WTW was finding a sufficient number of women living in remote areas of five western states that met study criteria and were willing to commit to participate for 24 months. Inclusion criteria were: 35-65 years of age; chronic illness; and residence in a rural area of Montana, Idaho, North Dakota, South Dakota, or Wyoming. As recommended by Steger (1997), developing a protocol that described the recruitment procedure, time-line, and a checklist for recording the dates recruitment activities were initiated and completed was an essential first step needed to keep track of the many contacts involved. Since the procedure has been repeated seven times in both phases of WTW, and one more recruitment effort is in progress to complete the study, this protocol has proved to be invaluable. Recruiting the participant sample was initiated by one designated research team member, six to nine months in advance of the commencement of the computer phase of the project. It took considerable time to make all the necessary contacts through telephone calls, e-mail, FAX, and/or regular mail. Recruitment efforts were facilitated through newspapers, voluntary agencies (Arthritis Foundation, National Multiple Sclerosis Society, American Diabetes Association, Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 84 American Cancer Society, and American Heart Association), state agricultural extension services, schools of nursing, public and professional libraries, and word of mouth, for example, former participants in WTW and nursing students. An article in the Montana Nurses’ Association “Pulse” reached every nurse in Montana, and a recruitment notice in the WTW project biannual newsletter was mailed to all current and past participants, professional colleagues, and “friends” of the project. The newspaper announcement was disseminated to all daily and weekly newspapers in the state. In some states, the extension service offices distributed project information by contacting each of their local agents through e-mail list serves or mailing each an informational flyer. They also put an announcement in the PBS grapevine of their state. Some agents, who had local radio programs, broadcasted the information. Voluntary agencies participated in a variety of ways. Most included an item about WTW recruitment in their periodical newsletters. Some agencies were willing to participate in a personalized letter campaign, a strategy recommended by Diekmann & Smith (1989), although it is costly. In the most recent recruitment effort, one agency sent 183 letters netting four responses and another sent 361 letters with 18 responses, at a cost of $11.18 per response. Other agencies posted WTW information on their Web sites or placed WTW information in their agency resource book. Consistent with studies reported by Topp & Bawell (1992) and Wray & Gates (1996), the best response came from newspaper publicity, representing 42% of the women recruited for the most recent two cohorts. Voluntary agencies’ newsletters and personal letters were next with 25%, and the rest were evenly distributed between health professionals (14%), word of mouth (14%), and 5% unknown. DISCUSSION The importance of the sample to the credibility of the study findings demands that attention be paid, not just to how many participants the researcher must engage, but strategies the researcher can use to successfully recruit them. From the four recruiting “adventures” that have been presented, it is apparent that there are several issues to consider in developing recruitment strategies as part of the overall planning of a rural study: uniqueness of the rural culture and context, higher costs of recruitment, necessity for rural-sensitive recruitment materials, over-sampling, and lack of local research infrastructure. Uniqueness of the Rural Culture and Context At the outset, the researcher must internalize an appreciation for the uniqueness of the rural culture and context. These include the concept of “insider vs. outsider” and recognition of rural life cycles. “Insider” Vs. “Outsider.” Long and Weinert (1989) described rural dwellers as resistant to help or services from “outsiders” and more readily responsive to those “who know us.” In rural areas, the researcher may be seen as an “outsider” and techniques are needed to reduce this perception. Familiarity with the setting as demonstrated in the researcher’s timing, language, and contacts can help reduce suspicion. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 85 The “Living with Cancer” study applied Shreffler’s (1999) suggestion that involving local community leaders throughout the recruitment process, for example, incorporating their input into recruitment strategies. She encouraged meeting with them on a regular basis, and maintaining contact by phone and mail throughout the study, to help them develop ownership in the outcome, as well as minimize the “outsider” perception of the researcher. Nurses practicing in rural areas need to be enlisted as key members of the research team. Their presence provides a bridge between “insider” and “outsider,” and their enthusiasm can add momentum to the recruitment process and successful implementation of the study. Useful as these approaches are, it must be recognized that they also present inherent hindrances to the researcher’s attempts to provide ongoing supervision and encouragement in distant data collection sites. In this project, the clinic nurses and students were not involved in the study protocol until after the study was funded. Commitment by these “insiders” could have been enhanced by including them in the actual development of the proposal or by inviting them to serve as experts in the establishment of the study protocol. Cycles of Rural Life. Recognition of the importance of the participants’ physical and social environment is essential. This insight was illustrated by the HHC study when it ensured that the research demands on the participants’ time were not competing with rural life or work cycles. Higher Costs of Recruitment The “running out of resources” difficulties of Adventures #2 and #3 can be often be offset by careful planning in advance for the time and budget required for participant recruitment. The challenge of recruiting an adequate sample from small, geographically dispersed populations necessitates that proposals be crafted to convince reviewers who may not be familiar with rural research that more time and resources must be allotted for the recruitment phase. Recruitment almost always takes longer and costs more than expected because participants are difficult to locate when they are scattered over thousands of miles. The HCC study successfully overcame this hurdle by using a commercial listing that met study criteria. The other “adventures” were less fortunate. Enlisting the aid of rural clinics, parish nurses, churches, and senior centers was largely unproductive and finding sufficient numbers of chronically-ill women living in five sparsely populated Western states took months of intensive recruitment effort. Blending with the rural culture and finding adequate samples in rural areas remain a challenge and reinforce the fact that research in the rural context is unique. Developing Rural-sensitive Recruitment Information Developing recruitment information that is sensitive to rural dwellers helps to engage potential participants. The examples presented here support Dillman’s (2000) claim “that people must understand clearly what is wanted of them if they are to respond” (p.13). Most people do not understand health care or research language, so word choices should reflect concepts and words that are commonplace and not suggestive of taboo subjects. The experience with the term “spirituality” is a case in point. Highlighting the Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 86 chronic illness aspect and maintaining spirituality as a secondary focus was a more fruitful approach. Another strategy is to develop a logo that reflects the rural culture and to use a simple title to help establish the identity of the project. Follow-up communications such as “thank you” notes, as was done in all the studies, provide personal attention or social exchange (Dillman, 2000) that can be important in maintaining the individual for the duration of the study, as in WTW, or as a potential participant for future studies. Over-sampling Koop (2001) emphasized that recruiting research participants in sufficient numbers is crucial to the success of a research project, so over-sampling is a useful approach when working from listings of names. The strategy used in the HCC study was to sample three times the desired number of participants. This was critical in communities with small target samples because it facilitated data collection when individuals refused to participate, when telephone numbers were incorrect, or if people had moved or died. The very nature of “sparsely populated” in conjunction with the focus on obtaining rural samples for research means that each potential subject becomes increasingly precious and potentially over-studied. Lack of Local Research Infrastructure Rural researchers face problems when rural study areas lack local research infrastructures that provide for institutional review and comply with federal regulations. Recent changes in federal regulations dealing with human subjects are making the recruitment process in rural settings even more complex. By federal mandate, all health professionals who are participating in any aspect of a research study, including recruitment, must complete an online in-service study unit entitled “Human Participant Protections Education for Research Teams.” Most health professionals in rural settings have not completed this study unit and, therefore, are not eligible to assist researchers in the recruitment process. To add to the problem, small health care facilities do not have an Institutional Review Board to approve research projects. This, in effect, negates some of the strategies discussed earlier related to the involvement of key local health professionals and health care units. The federal mandate is a relatively new challenge that is being faced by rural researchers, such as the WTW team, in ongoing recruitment efforts. Recruiting participants for research studies is always challenging. The very nature of rural areas--sparsely populated, long distances, remote locations, the rural ethos– present unique challenges to the research team. Appreciation of the rural culture, careful planning, an adequate budget, sufficient time, creative recruitment strategies, and awareness of the potential stumbling blocks are the ingredients necessary for success. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 87 NOTES Health Care Choices (HCC) was funded by NIH/Office of Complementary and Alterative Medicine (1R15AT00095-03). The “Living with Cancer” fatigue measurement study was funded by NIH//National Institute of Nursing Research (1R15NR04872-01). Funding for the spirituality study was provided by the Center for Research on Chronic Health Conditions in Rural Dwellers, NIH/National Institute of Nursing Research (1P20NR07790-02). Funding for Women to Women, Phase One, was provided by the U.S. Department of Agriculture, NIH/National Institute of Nursing Research, SC Ministry Foundation, American Cancer Society, U.S. West, Montanans on a New Trac for Science, Montana State University College of Nursing, Arthritis Foundation, and Montana Chapter, National Multiple Sclerosis Society. Phase Two was funded by NIH/National Institute of Nursing Research (1R01NR07908-01). REFERENCES Astin, J. (1998). Why patients use alternative medicine: Results of a national study. Journal of the American Medical Association, 279, 1548-1553. [MEDLINE] Colantonio, A., Rask, S.V., & Ostfeld, A.M. (1992). Depressive symptoms and other psychosocial factors as predictors of stroke in the elderly. American Journal of Epidemiology, 136, 884-894. [MEDLINE] Craig, C. (2001). Spiritual health. In R. Daniels (Ed.), Nursing fundamentals: Caring and critical decision making (pp. 1489-1508). Clifton Park, NY: Delmar Cudney, S., & Weinert, C. (2000). Computer-based support groups: Nursing in cyberspace. Computers in Nursing, 18(1), 35-43. [MEDLINE] Diekmann, J., & Smith, J. (1989). Strategies for assessment and recruitment of subjects for nursing research. Western Journal of Nursing Research, 11, 418- 430. [MEDLINE] Dillman, D. (2000). Mail and internet surveys: The tailored design method (2nd ed.). New York: John Wiley. Dillman, D. (1978). Mail and telephone surveys: The total design method. New York: John Wiley. Eisenberg, D., Kessler, R., Foster, C., Norlock, F., Calkins, D., & Delbanco, T. (1993). Unconventional medicine in the United States. New England Journal of Medicine, 328, 246-252. [MEDLINE] Eisenberg, D., Davis, R., Ettner, S., Appel, S., Wilkey, S., Van Rompay, M., & Kessler, R. (1998). Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national survey. Journal of the American Medical Association, 280, 1569-1575. [MEDLINE] Harris, L., & Associates. (1987). Health information and the use of questionable treatments: A study of the American public (DHHS Study No. 833015). Washington, D.C.: U.S. Department of Health and Human Services. Holden, G., Rosenberg, G., Barker, K., Tuhrim, S., & Brenner, B. (1993). The recruitment of research participants: A review. Social Work in Health Care, 19(2), 1-44. [MEDLINE] Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=9605899 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=1442754 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=10673815 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=2675474 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=8418405 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=9820257 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=8153843 88 Idler, E.L., & Kasl, S.V. (1997). Religion among disabled and nondisabled persons II: Attendance at religious services as a predictor of the course of disability. Journal of Gerontology. Series B, Psychological Sciences and Social Sciences, 52, S306- S316. [MEDLINE] Koop, P. (2001). Reflections on research: Issues involved in the accrual of respondents. Canadian Oncology Nursing Journal, 11, 170-171. Krupp, B., LaRocca, N., Muir-Nash, J., & Steinberg, A. (1989). The fatigue severity scale. Archives of Neurology, 46, 1121-1123. Lee, K., Hicks, G., & Nino-Murcia, G. (1991). Validity and reliability of a scale to assess fatigue. Psychiatry Research, 36, 291-298. [MEDLINE] Long, K., & Weinert, C. (1989). Rural nursing: Developing the theory base. Scholarly Inquiry for Nursing Practice, 3(2), 113-127. [MEDLINE] Morris, E.L. (2000). The relationship of spirituality to coronary heart disease. Alternative Therapies in Health and Medicine, 7(5), 96-98. [MEDLINE] Nichols, E. (2003). A multi-state rural sample of elders: Challenges and solutions [Abstract]. Communicating Nursing Research Conference Proceedings, 36, 102. Paramore, L. (1997). Use of alternative therapies: Estimates from the 1994 Robert Wood Johnson Foundation national access to care survey. Journal of Pain and Symptom Management, 13(2), 83-89. [MEDLINE] Shreffler, J. (1999). Culturally sensitive research methods of surveying rural/frontier residents. Western Journal of Nursing Research, 21, 426–435. Steger. K. (1997). Recruitment of subjects in clinical trials. Research Nurse, 3(6), 1-13. Topp, R., & Bawell, K. (1992). Applied research briefs: A comparison of seven methods of recruiting older adults into an exercise study. American Journal of Health Promotion, 6, 327-329. Weinert, C. (2000). Social support in cyberspace for women with chronic illness. Rehabilitation Nursing, 25,129-135. Winningham, M., Nail, L., Burke, M., Brophy, L., Cimprich, B., Jones, L., et al. (1994). Fatigue and the cancer experience: The state of the knowledge. Oncology Nursing Forum, 21, 23-36. [MEDLINE] Wray, J., & Gates, B. (1996). Problems of recruiting participants for nursing research: A case study. NTResearch, 1, 366-373. Online Journal of Rural Nursing and Health Care, vol. 4, no. 2, Fall 2004 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=9403524 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=2062970 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=2772454 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=11565405 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=9095565 http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=8139999