Smith_psfahs,+682-Article+Text-4557-1-15-20210628+FINAL_setup_psf+8.30.21 Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 14 Strategies for Recruiting and Retaining Nurses for Rural Hospital Closure Research Jessica G. Smith, PhD, RN, CNE1 Kyrah K. Brown, PhD2 Melynda Hutchings, DNP, RN-BC, CEN3 Elizabeth I. Merwin, PhD, RN, FAAN4 1Assistant Professor, College of Nursing and Health Innovation, University of Texas at Arlington, jessica.smith2@uta.edu 2 Assistant Professor, Department of Kinesiology, College of Nursing and Health Innovation, University of Texas at Arlington, kyrah.brown@uta.edu 3Clinical Assistant Professor, College of Nursing and Health Innovation, University of Texas at Arlington, melynda.hutchings@uta.edu 4Dean and Professor, College of Nursing and Health Innovation, University of Texas at Arlington, elizabeth.merwin@uta.edu Abstract Purpose: Rural hospital closures result in the loss of lifesaving acute care for rural residents and threaten the health of rural communities significantly. Understanding short- and longer-term effects nurses experiences related to a rural hospital closure can pose a unique challenge, as nurses may relocate and become difficult to identify. In this paper, the authors describe the strategies used during a pandemic to recruit and retain rural nurse research participants who experienced a rural hospital closure. Strategies presented may help nurse researchers recruit a unique and difficult-to- reach rural nurse population during a pandemic. Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 15 Sample: The sample consisted of nurses who worked in a rural hospital that closed between 2014 to 2020 in one Southwestern state. Method: The authors describe multiple strategies used to recruit rural nurses such as public outreach, social media outreach, and personal outreach. Public outreach was used through professional organization email listservs and printed newspaper advertisements. Social media outreach included public posts online. Personal outreach to former rural hospital employees who were existing contacts was through personal social media messages, phone calls, and texts. Findings: The most effective method for recruitment and retention was personal outreach through rural hospital champions who sent personal messages using social media. The final sample included 10 nurses from two rural hospitals that closed. Five nurses reported about a hospital that closed in 2014, while five nurses reported about a hospital that closed in 2015. Conclusion: For successful recruitment, future researchers should use a direct, personal recruitment strategy to recruit and retain rural nurse research participants for studies about rural hospital closures. Other strategies for successful rural nurse recruitment for research include planning for an extended recruitment timeline, being flexible with participants regarding data collection procedures, cultivating research interest among rural nurses, and building a culture of research in rural settings. Keywords: rural, nursing, research, sampling, recruitment, hospital closure, COVID-19 Strategies for Recruiting and Retaining Nurses for Rural Hospital Closure Research Rural hospital closures significantly threaten the health of rural communities, as they result in the loss of lifesaving acute care for rural residents. Closures of rural hospitals have a disproportionate effect on rural residents who are, on average, older, poorer, and suffer more Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 16 chronic illnesses given their barriers to transportation and adequate health insurance (National Academies of Sciences, Engineering, & Medicine, 2018). The closure of rural hospitals also results in imposed mass resignations for trained rural acute care nurses. Understanding nurses’ experiences of a rural hospital closure, and its longer-term effects, can pose a unique challenge because nurses relocate and may be difficult to find after the hospital closure because records are not publicly available. In this paper, the authors describe strategies used to recruit and retain rural nurse research participants who experienced a rural hospital closure in one southwestern state. The authors' study was designed during fall 2019, it was funded in January 2020, and data collection occurred in May 2020 amid the COVID-19 pandemic. The authors make specific recommendations for nurse researchers aiming to identify, recruit, and retain rural nurse participants for research studies during a pandemic. The strategies and recommendations presented in this paper may help nurse researchers navigate the complexities of recruiting a unique and difficult-to-reach rural nurse population during a pandemic. Background Over the years, there has been a steady increase in nursing health services research (NHSR) in rural settings conducted by nurse researchers, including nurses as research participants (Cohen et al., 2021; Greiner et al., 2008; Squires et al., 2019). C. B. Jones and Mark (2005) established NHSR as referring to the subfield of health services research that focuses on the study of health care delivery and systems of care delivery involving nurses, the examination of structures, processes and outcomes of nursing care, and the evaluation of nursing practice innovations and new models of care delivery (Cohen et al., 2021, p. 2). Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 17 Rural nurses play a critical role as participants in NHSR because of their immense knowledge and experience within the rural hospitals, which are fundamentally different from urban hospital systems in terms of community characteristics and context, human and technological resources, and practice characteristics (Jackman et al., 2012; S. N. Jones, 2012). In the US, 20% of nurses reside and work in rural communities (Health Resources and Services Administration, Bureau of Health Professions, 2013). In general, rural nurses are directly involved in almost every aspect of ensuring hospital care delivery and quality, including the provision of direct patient care and education, data collection and reporting, designing and implementing quality improvement initiatives, and influencing organizational policies and procedures. Therefore, rural nurses as research participants can contribute to knowledge generation related to the structure, processes, and effects of rural healthcare services, and strengthen the NHSR literature (Squires et al., 2019). Previous research has identified psychosocial and work environment factors that may enable or impede nurses’ participation in research (Albers & Sedler, 2004; Björkström & Hamrin, 2001; Roll et al., 2013; vanGeest & Johnson, 2011). Regarding enabling factors, nurses are more likely to participate in nursing research when the topic under study interests them and when they work in environments where nurse managers and leaders foster a culture that values research (Björkström, Johansson, & Athlin, 2013; Higgins et al., 2010; Roll et al., 2013). Barriers to nurse participation in research include perceptions about research participation being outside the scope of their role as clinicians (Bullen et al., 2014) and having a limited understanding of the research process (Björkström & Hamrin, 2001; Björkström, Johansson, & Athlin, 2013; Higgins et al., 2010). Rural nurses may be less responsive to individuals who are viewed as ‘outsiders’ such as researchers who live outside of their community (Long & Weinert, 1989). Also, rural nurses are embedded within the cultural context of close-knit rural communities (Lee & McDonagh, 2018; Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 18 Long & Weinert, 1989). This community context is often characterized by lack of anonymity and extensive familiarity with others in the community (Lee & McDonagh, 2018). Therefore, nurses might be concerned about sharing information during the research process for fear of others finding out about the nature of their participation or for fear of inadvertently breaching patient confidentiality (Lee & McDonagh, 2018). Finally, rural nurses, especially those working in busy hospital settings, must contend with the mental and physical challenges of a busy workload (Björkström, Johansson, & Athlin, 2013; Govasli & Solvoll, 2020). Consequently, many nurses report having limited time and/or energy to participate in research during or outside work hours (Roxburgh, 2006; vanGeest & Johnson, 2011). Understanding the facilitators and barriers to nurses’ research participation is valuable because this information can inform the efficient design of studies seeking to collect primary data from rural nurse research participants. However, there is a need for more studies documenting the efficacy of processes and strategies used by nurse researchers to recruit and retain rural nurses as research participants (Broyles et al., 2011; Luck et al., 2017). To date, there are 11 published studies that describe and/or evaluate the processes and strategies used to recruit and retain nurses as research participants (Brewer et al., 2020; Broyles et al., 2011; Howerton Child et al., 2014; Coyne et al., 2016; Desroches, 2020; Hysong et al., 2013; Khamisa et al., 2014; Luck et al., 2017; Raymond et al., 2018; Stokes et al., 2019; vanGeest & Johnson, 2011). To the best of our knowledge, only one of these studies focuses explicitly on rural nurses as research participants (Brewer et al., 2020). Much of the literature on recruiting and retaining nurses as research participants describes face-to-face recruitment processes and strategies. In these studies, the research teams reported recruiting nurses from healthcare facilities or other organizational settings (Broyles et al., 2011; Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 19 Coyne et al., 2016; Hysong et al., 2013; Khamisa et al., 2014; Luck et al., 2017; Raymond et al., 2018). Few studies in the literature describe recruitment and retention strategies for Internet-based studies (Howerton Child et al., 2014; Desroches, 2020; Stokes et al., 2019), which are particularly useful in rural nursing research because rural communities tend to have more geographically diffused populations (Brewer et al., 2020; Cudney et al., 2004). Some of the effective recruitment strategies from Internet-based studies are similar to those reported in facility-based studies. These strategies include using more than one recruitment mode, identifying champions who have access to and trust with nurses, using snowball sampling techniques, and ensuring that data collection activities are flexible and convenient for nurse participants (Brewer et al., 2020; Broyles et al., 2011; Coyne et al., 2016; Hysong et al., 2013; Khamisa et al., 2014; Luck et al., 2017; Raymond et al., 2018). There are, however, some unique considerations for recruiting and retaining rural nurses as participants in Internet-based studies. While Internet-based studies in general offer the benefits of cost-effectiveness (Howerton Child et al., 2014), such studies conducted in rural contexts may result in greater costs in terms of time because of longer recruitment periods and fewer potential participants who are spread across vast geographic areas (Brewer et al., 2020; Howerton Child et al., 2014). Compared to facility-based studies, researchers conducting Internet-based studies have limited control over the study setting (e.g., being readily available to participants as they complete surveys) and depending on the nature of the study may not have access to a pre-existing contact list from which to recruit potential rural nurses (Brewer et al., 2020; Howerton Child et al., 2014; Stokes et al., 2019). Finally, researchers conducting Internet-based studies may encounter unique issues related to technological usability and issues related to spammers accessing and compromising online data collection forms (Howerton Child et al., 2014; Desroches, 2020). Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 20 Considering the unique challenges that come with Internet-based studies, there are several recruitment and retention strategies have been identified. One effective strategy is the use of social media platforms (e.g., Facebook (FB), LinkedIn) to create targeted ads using demographic data, to send direct messages to potential participants, and to post flyers in online interest groups (Howerton Child et al., 2014; Stokes et al., 2019). Nurse researchers have discussed the importance of tracking and reporting the number (and type of people) who see posts/ads, the number who engage with posts (i.e., likes, shares, comments), and the number of people directly recruited from posts/ads to inform modifications to online recruitment (Howerton Child et al., 2014; Stokes et al., 2019). On the other hand, one study focused on recruiting rural nurses across three states indicated that social media recruitment was effective in only one of the three targeted states (Brewer et al., 2020). A second effective strategy is testing online data collection tools for errors and usability (i.e., mobile phone viewing) to prevent potential participants from becoming frustrated and dropping out of the study (Desroches, 2020; Stokes et al., 2019). In response to rural nurses’ potential hesitation to participate in research due to issues related to confidentiality and anonymity, a third recruitment strategy involves using the consent form (and other recruitment materials) as a tool to overemphasize the research team’s plan to minimize risk associated with breach of confidentiality (Brewer et al., 2020). While little is known about what processes and strategies work for Internet-based studies recruiting rural nurses as research participants (Brewer et al., 2020), even less is known about how this works during the COVID-19 pandemic, which has created unprecedented challenges. Nurses’ work demands have been exacerbated by the COVID-19 pandemic. In the case of rural hospitals and communities, available evidence suggests that rural communities are at higher risk for hospitalization and severe illness among rural residents who tend to be older and have more Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 21 chronic conditions (Kaufman et al., 2020; Segel et al., 2020). During the COVID-19 pandemic, rural hospitals have faced financial (i.e., furloughs, reduction of staff hours) and operational (i.e., personal protective equipment shortages) challenges (Segel et al., 2020). The COVID-19 pandemic’s impact on hospital patient loads, finances, and operations has placed greater strain on nurses in rural communities, and on the limited time and energy that nurses may have had allocated for potential research participation. While Internet-based research offers many advantages (Howerton Child et al., 2014), during the COVID-19 pandemic there has been an increase in number of Internet-based studies. The potential of research fatigue, or populations growing tired of engaging with research, may present additional challenges for recruitment and retention in rural settings (Patel et al., 2020). There is a need for literature that can help guide nurse researchers who have had to adapt their research methodologies in the wake of COVID-19. To ensure the continued growth of NHSR in rural settings, further research is needed about strategies to recruit and retain rural nurses as participants (Brewer et al., 2020; Broyles et al., 2011). The aims of this paper are to (1) describe the process and strategies used to recruit and retain rural nurses in an Internet-based study designed to investigate the impact of rural hospital closures on nurses and residents during the COVID-19 pandemic and to (2) identify recommended strategies that may be of particular interest to nurse researchers conducting NHSR in rural settings during the COVID-19 pandemic. This paper will inform processes and strategies to effectively recruit and retain a hard-to-reach nurse population—rural nurses who worked in a hospital that has since closed. Finally, among the studies using social media to recruit nurses, only one of those studies used a qualitative study design (Brewer et al., 2020). Therefore, this paper will contribute to an understanding of the unique challenges and strategies involved in using social media to recruit rural nurses as research participants in a study using online surveys and qualitative interviewing. Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 22 Method The purpose of recruiting rural nurses as research participants was to understand their perceptions about the impact of hospital closures in rural communities. In the paragraphs below, the original study design and method is described first, followed by a description of modifications to the design and method needed to successfully recruit a sufficient sample. Original Study Design For this study, a hospital was classified as rural if it was in at least one of the following categories: (1) a Critical Access Hospital (CAH), (2) a Sole Community Hospital (SCH), (3) a Rural Referral Center (RRC) in a non-metropolitan statistical area (MSA), or (4) location in a county of 60,000 population or less. This classification is consistent with a state professional organization’s definition of rural. The organization is not named here for confidentiality. Licensed nurses who experienced a rural hospital closure in one Southwestern state between 2016 and 2020 were originally eligible to participate in the original study. Initial public outreach strategies included sending study advertisements to professional organizations, local newspapers, and posting advertisements in community settings (e.g., grocery stores, libraries). The original data collection plan was to conduct three focus groups with eight to ten staff nurses in each group, i.e., between 24-30 staff nurses, and five individual interviews with chief nursing officers (CNOs). Participants were to be asked to complete an online survey including 11 demographic items and 31 items about their perceptions of the rural nurse work environment before the hospital closure. Focus groups and interviews were planned to be either via videoconference or telephone conference. Participants’ verbal consent was to be obtained through a phone call, after the participant reviewed the consent form received through email. Reminder emails were to be sent about participating. The original data collection timeline was to be from May 2020 to August 2020. Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 23 Institutional Review Board (IRB) approval from the authors’ academic institution was received in May 2020. Results from survey and interview data will be shared in another paper. Modifications to the Original Study Design Due to challenges related to rural recruitment and COVID-19, the authors modified the study design, methods, and recruitment timeline. The authors made 11 modifications to their IRB study protocol. This section describes the need for design modifications. The COVID-19 pandemic necessitated that all recruitment be online through social media outreach, personal invitations, and email listserv invitations. After starting data collection in May 2020, the authors changed the design from focus groups to individual interviews for staff nurses because of slow recruitment and scheduling challenges due to increasingly busy schedules due to COVID-19. After a month of slow recruitment, the authors modified inclusion criteria to include nurses who worked in hospitals that closed between 2014 and 2020, adding two previous years to the original timeframe of 2016-2020. This decision was made after the third author (MH), who had personal connections from a rural hospital closure in 2014, expressed interest in helping our recruitment efforts. MH’s specific role in participant recruitment is described in detail in the description of recruitment strategies section. This authors also reasoned that potential participants who experienced closures in 2014 and 2015 were not vastly different from those who experienced recent closures. The study protocol was also modified to allow for recruitment of nurses from additional hospitals if more closures occurred in 2020, but to our knowledge, there were no additional closures in the one Southwestern state in 2020. The authors extended the recruitment period beginning in May to go through January 31, 2021 due to difficulties with participant recruitment and retention (e.g., some participants did not follow-up even after agreeing to participate). Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 24 Description of Recruitment and Retention Strategies Public Outreach Due to the pandemic, the authors decided against posting study advertisements in community settings. Public outreach involved the research team paying for one email to be sent to a large professional member organization, paying for advertisements in eight different newspapers across seven counties, and sending unpaid emails through three regional professional organizations. Social Media Outreach Initial efforts for social media recruitment using FB included unfunded advertising in FB special interest groups and on author’s personal pages. In June 2020, the authors reached out to the leaders of FB interest or cause pages related to nursing or rural health but found that not being their direct friend on FB resulted in our messages being sent to their spam folder rather than be notified of a message at the exact time it is sent. The authors also posted study advertisements on their personal LinkedIn pages, shared on Twitter, and were able to get professional organizations and their academic institution to post the advertisements on their Twitter and FBpages. The authors also used an existing research team FB page to pay for boosted advertisements as a strategy to recruit participants. Boosted FB ads are advertisements that are paid to be promoted on the newsfeeds of user profiles in a specific geographic area (i.e., counties where rural hospitals closures occurred) and to profiles with specific characteristics (i.e., profile indicates their job is at a hospital or nursing is listed as an interest). Boosted FB advertisements were less expensive and provided good reach and leads, while also offering potential participants the option to provide their contact information. However, the authors only requested email addresses in the contact form for the second, third, and fourth boosted FB advertisements. In the third and fourth FB advertisements, the option for those interested to indicate their phone numbers to contact them was added. This Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 25 strategy enabled the authors to text and call potential participants, as otherwise, emails could have been going to spam folders. The FB boosted advertisements settings were also modified to include counties that were adjacent to the counties with hospital closures on the third and fourth advertisements. Personal Outreach The authors updated the study protocol to include recruitment through personal contacts because of ongoing challenges in obtaining responses from our target population using public advertising outreach strategies. The first author (JS) sent personal messages to existing rural nurse or rural hospital contacts from her social media and personal network, which included 19 people. In August 2020, one author (MH) joined the protocol to serve as a champion for the study. This author (MH) posted the advertisement a rural nursing interest groups FB page and her personal FB page. This FB page had former employees of rural hospitals that closed, comprising 200 plus members. There were both nurses and non-nurses as members. The FB page comments were monitored, and the author (MH) personally messaged individuals that responded. The same author sent announcements to local nursing homes and home health agencies in areas local to rural hospital closures to attempt to identify any nurses that went to these facilities for employment after a rural hospital closure. Author (MH) sent private messages to members of a rural community (e.g., mayor, nurse leaders), including local government leadership, former rural hospital administrators, former rural hospital personnel, and a healthcare provider in primary practice. Retention Strategies The authors tested several participant retention strategies. During the initial stages of data collection, the authors experienced challenges in getting participants to return consent documents Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 26 prior to the scheduled interview and online survey. In response to this, the authors obtained IRB approval to obtain potential participants consent via email after reviewing the consent form. Originally, the authors reserved open time blocks during the work week to schedule interviews. After recognizing just how hectic potential participants’ schedules were, the authors told potential participants that they were open to doing interviews at whatever time they were available, including, in the evening and on the weekends. The authors also noticed that some confirmed participants were forgetting their scheduled interview even after an email reminder and calendar invitation. Having to reschedule a missed interview posed the risk of losing the participant to follow-up. Therefore, the authors began sending email and text message reminders within an hour prior to their scheduled interview. Findings In the sections below, the authors describe results from the study’s recruitment and retention strategies. This section shows tables to reflect more depth of information. Recruitment Outcomes Table 1 displays each of the three recruitment strategies, total costs and number of participants recruited. As shown in Table 1, this recruitment strategy was the most expensive and resulted in no participants. See Tables 1 and 2 for a detailed description of recruitment efforts throughout the duration of recruitment in chronological order. Table 1 Overview of Recruitment Strategies Recruitment Strategy Cost (U.S. Dollars) Number of Participants Recruited Public Outreach 1,686.04 0 Social Media Ads 215 0 Personal Outreach 0 10 Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 27 As shown in Table 2, FB Ads resulted in 17 leads (people who see the ad and sign up to be contacted). Of the 17 leads, two indicated initial interest in participating and were eligible after initial screening; however, neither followed up to emails reminding them to provide consent to participate. Therefore, none of the leads resulted in participation. Table 2 Facebook (FB) Ad Campaign Statistics FB Ad # 1 FB Ad # 2 FB Ad # 3 FB Ad # 4 Ad Objective Boosted Post Leads Leads Leads Duration 45 days 39 days 15 days 45 days Reach Total number, n 2,891 2,170 1,911 2,058 Age, % 18-24 5 4 1 4 25-34 14 12 7 8 35-44 22 27 20 18 45-54 24 29 27 20 55-64 20 21 28 22 65+ 15 8 17 28 Gender, % Female 100 100 100 84 Number of Leads 7 5 5 Post Engagements, n Link Clicks 3 57 48 Photo Clicks 42 Comments 1 1 Shares 23 12 Video Views 1,020 Total Engagements, n 97 1,077 70 Note. Lead form for FB Ad #2 and #3 included name and email. Lead form for FB Ad #4 included name, email, and phone number. Fifty-two potential participants were contacted using the personal outreach approach. The participation rate was 19% for potential eligible participants who were contacted through personal invitation throughout the data collection period (10/52=19%). One person contacted through personal outreach consented to participate but did not respond to scheduling an interview. Personal outreach results are presented in Table 3. Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 28 Table 3 Results for Recruiting Participants by Personal Outreach to Individuals Timeline Recruitment Method Total Cost ($) Contacted Participants (n) 7/2020 – 1/2021 Personal outreach (JS) to nurses from were existing contacts known to JS from a rural hospital that closed 0 19 5 8/2020 – 12/2020 Personal outreach by a “rural hospital champion” (MH) to nurses from existing contacts known to MH from a community in which a rural hospital closed 0 30** 5 12/2020 Personal outreach by 2 rural nurses from 2 different rural hospitals that did not close. These nurses did not have direct nurse contacts who were working at a rural hospital that closed during the 2014-2020 timeframe. 0 4 0 Note: *Number contacted refers to the number of private messages sent to individuals JS or MH knew through Facebook messages, phone calls, or text messages. **Not all individuals were nurses at the hospital that closed. Some were members of the community, stakeholders, or in other healthcare roles. Sampling Outcomes There were 13 potential participants who communicated interest in the study and were qualified to participate. Three individuals did not follow up to emails to proceed with next steps to participate. Two of these three individuals were recruited from FB leads and did not answer emails to proceed with consent. The remaining individual was recruited using the personal contact method and consented to participate but did not respond to further reminders. The retention rate was 76.9% (10/13 = 76.9%) and attrition rate was 23% (3/13=23%). The final sample recruited and retained throughout the study duration included 10 participants. The 10 rural nurse participants were from two different rural hospitals that closed between 2014 and 2020. Five rural nurse participants were recruited from one hospital that closed Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 29 in 2014, and five from another rural hospital that closed in 2015. One participant was a chief nursing officer. Specific strategies were found to be successful to retain this sample, including using multiple methods to obtain consent, planning to be flexible with scheduling interviews, and planning to be flexible with the interview process. These strategies will be discussed further in the recommendations section of the discussion. Discussion The purpose of this paper was to describe effective processes and strategies to recruiting and retaining rural nurses as research participants during a pandemic. The study’s target population, rural nurses who experienced a hospital closure, is a unique, difficult to reach population. For example, records about nurses who were hired at a closed rural hospital are not available for retrieval, although names of closed hospitals are available. The challenges associated with the recruitment and retention of this rural nurse population were further worsened by the COVID-19 pandemic. The findings presented in this paper contribute to the dearth of literature on strategies and processes in recruiting rural nurses as research participants (Brewer et al., 2020). One barrier to rural nurse participation in research included concerns related to colleagues within the small rural hospital community finding out what was shared in the interviews (Lee & McDonagh, 2018; Long & Weinert, 1989). For instance, some respondents expressed that there could have been concerns from her nursing colleagues about confidentiality even though the research team clearly conveyed confidentiality on recruitment materials. Rural networks are small, which is likely to contribute to these concerns. Another barrier was rural nurses’ busy workload (Björkström, Johansson, & Athlin, 2013; Govasli & Solvoll, 2020), but this expected barrier was exacerbated by the COVID-19 pandemic. The COVID-19 pandemic contributed to a more arduous process for participant recruitment and Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 30 participation. The pandemic required several changes to this study’s design to recruit and retain nurse research participants. This was understandable, as the target population could have been in the field managing patients with COVID-19 on the front lines, could have been pre-occupied about the risk of contracting COVID-19, and could also have been struggling with any number of situations exacerbated by the pandemic, including but not limited to furloughs, layoffs, and changes in care needs for elders or children under their care. A third potential barrier could have been potential participants being demotivated. Several rural nurse participants expressed how traumatizing the closure of their hospital was and it is possible that potential rural nurse participants did not want to relive those negative feelings. The study, described in this paper, was different from most prior studies on this topic because face-to-face or facility-based recruitment was not possible (Broyles et al., 2011; Coyne et al., 2016; Hysong et al., 2013; Khamisa et al., 2014; Luck et al., 2017; Raymond et al., 2018). Therefore, some strategies and approaches did not fit the context of the present study. For instance, the COVID-19 pandemic prevented the use of other methods that could have been useful for recruiting nurse participants, such as pursuing communications with open hospitals as a method of recruitment. Rural hospitals caring for patients were managing already overwhelming circumstances due to COVID-19, therefore, we did not pursue communications with these hospitals out of respect for those front-line nurses and administrators who were preparing for a surge in cases or experiencing the challenges of caring for COVID-19 patients. In terms of strategies for recruiting rural nurses during a pandemic, the most effective strategies identified from the study described in this paper included identifying champions who have access to and trust with networks of rural nurses and using snowball sampling techniques (Brewer et al., 2020; Broyles et al., 2011). Also, ensuring flexibility and convenience in interview Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 31 scheduling was essential in retaining rural nurse participants (Broyles et al., 2011). Our findings point to other effective strategies supported by existing literature such as testing online data collection tools or video conference tools to prevention participants from becoming frustrated (Desroches, 2020; Stokes et al., 2019) and using the consent form to overemphasize the commitment to confidentiality (Brewer et al., 2020). In contrast to existing literature (Howerton Child et al., 2014; Stokes et al., 2019), paid social media advertisement was not effective in recruiting and retaining rural nurse participants. Social media recruitment using FB posed unique challenges. If one is not a direct contact (friend) on FB with others who own FB groups that could be interested in this study, messages sent to group leaders could get sent to a spam folder and remain unread. This limits possibilities for advertising to other areas in the state with rural hospital closures. The settings are limited to what is posted in public, as well, and some former rural hospital nurses might not have indicators of being a nurse on their FB profile for privacy. Therefore, our boosted ad might not have been visible to all the possible nurses who experienced a rural hospital closure in our target population. Potential reasons paid FB advertising did not result in participants despite requesting for their contact information could be (1) not entering contact information correctly, (2) not responding to emails (likely because of emails going to spam folders), (3) not responding to phone calls or text messages due to not recognizing the sender, or (4) mistakenly signing up. It is possible that digital recruitment efforts could also have been hampered by other advertising efforts for other causes or campaigns related to COVID-19. People could have been suffering from advertising fatigue on a large-scale (Patel et al., 2020). The social media platforms were effective as a tool used by the study champion in reaching out to their respective networks of Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 32 rural nurses, however. It is important to note that tracking the social media advertisement metrics was valuable in helping the research team determine the cost-effectiveness of this specific strategy. Another important strategy was related to providing multiple modes of providing ‘informed consent’ (verbal consent, text consent) which was especially helpful for rural nurse participants who were too busy to sign and scan a returned consent form. Finally, one effective strategy for ensuring that rural nurse participants showed up to interviews was sending text message reminders within one hour of their scheduled interview. Again, due to their busy schedules during the pandemic, many rural nurse participants admitted that they almost forgot about their scheduled interviews. Recommendations for researchers encountering challenges conducting future research during a pandemic are presented in the following paragraphs. Recommendations for Recruiting and Retaining Rural Nurses for Research Based on the findings, the authors share strategies and implications for successful recruitment and retention of rural nurse participants (Table 4). Benefits for employing these strategies during a pandemic, which is a major unanticipated challenge, are also discussed. Table 4 Strategies for Successful Recruitment and Retention of Rural Research Participants Short-Term Strategies 1. Use a Direct, Personal Recruitment Strategy 2. Plan for an Extended Recruitment Timeline 3. Include Multiple Methods to Obtain Consent, 4. Plan to be Flexible with Scheduling Interviews 5. Plan to be Flexible with the Interview Process Long-Term Strategies 6. Cultivate and maintain a network of rural nurses who are interested in research 7. Build a culture of research in rural settings 8. Identifying and Mentoring Rural Hospital Champions for Research Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 33 Short Term Strategies Use a Direct, Personal Recruitment Strategy. Personal outreach was the single-most effective strategy for recruiting and retaining rural nurse participants for this study. Using personal outreach addressed the high costs of advertising for this project, which was an unanticipated barrier. Inviting a rural hospital “champion” with established connections with rural nurses to join our research team was paramount to recruiting participants for our study. Our rural hospital champion served as a direct contact with a network of former rural hospital nurses who could participate and attract participants. This more personal approach to recruitment seemed to yield a better response than ads targeted to the broader nurse audience in the state through list servs and newspapers. Plan for an Extended Recruitment Timeline. For successful recruitment of nurse participants for research, it is important for researchers to plan for an extended recruitment timeline. Our original participant recruitment period was planned to occur during a four-month timeframe. However, it required us nine months to achieve a sample of 10 participants. One other study reported recruiting rural nurses in the Appalachia region which consists of three states and six counties (Brewer et al., 2020). The authors reported achieving a sample of 15 rural nurse participants during a four month period using a combination of flyers, professional email lists, social media advertisement, and snowball sampling (Brewer et al., 2020).The present study’s sample of 10 could have been related to the unique nature of our study (i.e., seeking rural nurses who had experienced a rural hospital closure during a specific timeframe), the smaller population to recruit from given our inclusion criteria, and the conditions we were facing during the pandemic that made face-to-face recruitment not possible, requiring the sole use of online recruitment methods. Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 34 Prior to the data collection period, there was only one hospital closure in 2020 in our target state to our knowledge, whereas in previous years there had been multiple closures. There were no additional hospitals at high risk of a closure or in the process of closure, as far as we were aware. One potential future approach for a more successful recruitment phase would be to plan for an extended research participant recruitment timeline and monitor for rural hospital closures in progress, which could be discussed in news sources local to the rural area. It could be possible to collect data from nurses during a rural hospital closure, while it was fresh on everyone’s mind and of interest to everyone before moving on to another job, or before losing hope that a rural hospital could reopen and prosper. Include Multiple Methods to Obtain Consent. Changing the approach from obtaining phone-only consent to offering email consent, i.e., where a participant could email a yes to agree to participate, was helpful in obtaining consent. Including the option to obtain consent through email also increased the likelihood that a participant would participate in all phases of the study, including the survey and interview. Doing so meant additional considerations regarding the protection of subjects (e.g., having a protocol for permanently deleting email communications). Plan to be Flexible with Scheduling Interviews. Reminder emails were effective for ensuring participants made it through the entire process of scheduling the interview, taking the questionnaire, and participating in the interview. Sending reminder emails and text messages to participants within an hour prior to their scheduled interview was also effective. Given the challenges of obtaining a sample from this unique and difficult to reach population, it is important to be more flexible with interview hours, and more flexible with study design, as we changed our initial plan to include focus group to individual interviews only. We were also flexible with the Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 35 time interviews were held to respect participant needs if they worked night shifts, which helped increase the retention of participants throughout the entire study. It was more difficult to schedule an interview than it was for participants to complete the survey. For example, participants would complete the survey and then become hard to contact for scheduling an interview. Some potential participants consented to participate but did not respond to follow-up messages about taking the survey and scheduling a time to interview. It was challenging to keep participants engaged in participating or responding to messages related to scheduling and interview. It was also challenging to find a time to schedule for an interview for those who remained interested, although we remained flexible about when interviews were scheduled on our end. The difficulties our nurse participants had with scheduling an interview appeared to be because of having diverse availability and work schedules. Nurses from our target population were working at different institutions on different shifts, including night shifts. Also, nurses from our target population were especially busy in their current positions responding to patient care needs at the height of the pandemic. In addition, COVID-19 vaccinations were in high demand despite a short supply during the last month of our recruitment period, requiring individual efforts to search for a vaccination site and appointment. This was time consuming and high priority for many that could have deterred nurse participation in a research study. Plan to be Flexible with the Interview Process. Remaining patient, staying on the line, and communicating via email or text was an effective method to managing unanticipated interruptions that occurred during interviews (i.e., call drops, nurse having to leave interview to receive vaccination). This informed participants that the authors were still on and would like to continue the call. Sometimes unanticipated interruptions disrupted the flow of the participant’s thoughts as Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 36 they recalled information to share for the study. However, the authors’ method to address this was to continue at the stopping point at the re-scheduled meeting. For difficulties with communication on the phone conference platform (i.e., call drops), it is essential to remain patient and encourage participant to call back when technical difficulties occur and recur. Researchers may consider obtaining approval to use another video conference platform if needed. It was helpful to have both phone and video options for the interview, otherwise requiring video-only interviews could have resulted in fewer participants. Long-Term Strategies for Strengthening Rural Nursing Health Services Research For the long-term success of rural NHSR, the authors provide three additional recommendations (with pandemic-related considerations) based on results from this study to recruit and retain more nurses as research participants: (1) Cultivate and maintain a network of rural nurses who are interested in research opportunities and (2) Build a culture of research in rural settings, and (3) Identifying and mentoring rural hospital champions for rural nursing health services research. These recommendations have implications for organizational policies within rural healthcare settings. Cultivate and Maintain a Network of Rural Nurses for Research Opportunities For more expedient rural nurse recruitment for research, one idea is to build and maintain research-focused rural nurse networks willing to engage in research and contribute to its continued development. Cultivating such a ‘rural research network’ would require formalized partnership among rural health care providers, rural healthcare facilities, and nurse researchers. The rural research network could enable researchers to investigate changes over time due to the phenomenon of interest (i.e., rural hospital closures) and determine its effects. Findings could help researchers to design and test clinical, system, and policy interventions to improve patient care. Building a Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 37 network of rural nurses interested in research would result in more expedient communication with nurses who may be interested in participating in studies, even during a pandemic. For example, if nurses were part of an email list, opportunities to participate in research could be offered online when face-to-face interaction is not possible. Build a Culture of Research in Rural Settings It could be beneficial to build a culture of research to include rural nurses in the process of NHSR and educate them about how such research can help improve rural healthcare. This approach would need organizational policies that reward or reinforce nurses’ and nurse leaders’ interest in learning about how to participate in research as an investigator or participant. This could assure that concerns about confidentiality are not a barrier to recruiting and retaining rural nurses as research participants in future studies. Developing a culture of research among rural nurses could have broader implications for NHSR and contribute to a sub field of Rural Nursing Health Services Research (RNHSR). There was not an apparent rural nursing culture of research in place among our selected sample frame, which could have been a barrier to recruiting nurse participants. For example, during the initial phases of the research study, the authors learned that some rural nurses might be hesitant to participate because of concerns about their colleagues learning about their participation and opinions. This lack of understanding could be related to not being familiar with the commitment researchers have regarding confidentiality, and the processes set in place to ensure confidentiality (i.e., the IRB). Therefore, the authors were certain to overemphasize confidentiality during recruitment, scheduling, and during interviews. Online teleconference meeting invitations with toll-free call-in options were used and voice-only interviews were conducted to reduce technological burden and to honor participants’ concerns regarding confidentiality. Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 38 Creating and sustaining a culture of research in rural settings could result in greater trust regarding research studies, even during a pandemic. During a pandemic, there is little time for nurses unfamiliar with research to become educated about its purpose. If a culture of research was already established, then nurses would be more comfortable with the idea of participating in research efforts that are confidential and meant to help improve future patient care in rural settings. Cultures of research could result in rural healthcare leaders being more receptive to placing nurse researchers on meeting agendas at, for example, local nursing homes or home health agencies, to provide information about opportunities to participate in research. Identifying and Mentoring Rural Hospital Champions for Research Identifying a champion to help with recruitment efforts using a personal approach could be useful for future nurse researchers seeking rural research participants. Even though there were elements out of our control, such as the occurrence of the COVID-19 pandemic, rural hospital champions were effective in personal outreach to recruit and retain rural nurse research participants. Rural nursing health services researchers could identify and mentor nurses who are interested in becoming rural research champions to help others understand the research process and make it clear how research could influence future patient care. The benefit of rural nurse research champions, even during a pandemic, is greater interest in nurse participation in research. Conclusion Recruiting and retaining nurses for a study about rural hospital closures without face-to-face personal contact can be challenging, and more so during a pandemic. The single most effective strategy we employed was harnessing existing social ties with rural nurses through social media, as our sample was derived entirely from personal outreach recruitment efforts by MH and JS. It is therefore critical for rural nurse researchers to build and maintain relationships with rural nurse Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 39 social networks to help identify, recruit, and retain rural nurses to serve as research participants for rural nursing health services research. Acknowledgements Our study was supported by the Excellence in Research Award sponsored by the Texas Organization of Baccalaureate and Graduate Nursing Education (TOBGNE) References Albers, L. L., & Sedler, K. D. (2004). Clinician perspectives on participation in research. Journal of Midwifery & Women’s Health, 49(1), 47-50. https://doi.org/10.1016/j.jmwh.2003.09.013 Björkström, M. E., & Hamrin, E. K. (2001). Swedish nurses' attitudes towards research and development within nursing. Journal of Advanced Nursing, 34(5), 706-714. https://doi.org/10.1046/j.1365-2648.2001.01800.x Björkström, M. E., Johansson, I. S., & Athlin, E. E. (2013). An attempt to improve nurses’ interest in and use of research in clinical practice by means of network support to ‘facilitator nurses’. Journal of Nursing Education and Practice, 4(3), 58. https://doi.org/https://doi.org/10.5430/ jnep.v4n3p58 Brewer, E. P., Weierbach, F. M., Fletcher, R. A., Hall, K. C., & Nehring, W. (2020). The lived experience of nursing Appalachia: Sampling and recruitment. Online Journal of Rural Nursing and Health Care, 20(2). https://doi.org/https://doi.org/10.14574/ojrnhc.v20i2.619 Broyles, L. M., Rodriguez, K. L., Price, P. A., Bayliss, N. K., & Sevick, M. A. (2011). Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research, 21(12), 1705-1718. https://doi.org/10.1177/1049732311417727 Bullen, T., Maher, K., Rosenberg, J. P., & Smith, B. (2014). Establishing research in a palliative care clinical setting: Perceived barriers and implemented strategies. Applied Nursing Research, 27(1), 78-83. https://doi.org/10.1016/j.apnr.2013.10.007 Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 40 Cohen, C. C., Barnes, H., Buerhaus, P. I., Martsolf, G. R., Clarke, S. P., Donelan, K., & Tubbs- Cooley, H. L. (2021). Top priorities for the next decade of nursing health services research. Nursing Outlook, 69(3), 265-275. https://doi.org/10.1016/j.outlook.2020.12.004 Coyne, E., Grafton, E., & Reid, A. (2016). Strategies to successfully recruit and engage clinical nurses as participants in qualitative clinical research. Contemporary Nurse, 52(6), 669-676. https://doi.org/10.1080/10376178.2016.1181979 Cudney, S., Craig, C., Nichols, E., & Weinert, C. (2004). Barriers to recruiting an adequate sample in rural nursing research Online Journal of Rural Nursing and Health Care, 4(2), 78-88. https://doi.org/https://doi.org/10.14574/ojrnhc.v4i2.201 Desroches, M. L. (2020). Facebook recruitment of nurses as research participants: Methodological considerations. Applied Nursing Research, 54(2020), Article 151282. https://doi.org/10.1016/j.apnr.2020.151282 Govasli, L., & Solvoll, B-A. (2020). Nurses' experiences of busyness in their daily work. Nursing Inquiry, 27(3), Article e12350. https://doi.org/https://doi.org/10.1111/nin.12350 Greiner, D. S., Glick, D. F., Kulbok, P. A., & McKim Mitchell, E. (2008). Rural health nursing research review: Global perspectives. Annual Review of Nursing Research, 26(1), 261-294. https://doi.org/https://doi.org/10.1891/0739-6686.26.1.261 Health Resources and Services Administration, Bureau of Health Professions. (2013). The U.S. nursing workforce: Trends in supply and education. HRSA National Center for Health Workforce Analysis. https://www.ruralhealthinfo.org/assets/1206-4974/nursing-workforce- nchwa-report-april-2013.pdf Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 41 Higgins, I., Parker, V., Keatinge, D., Giles, M., Winskill, R., Guest, E., Kepreotes, E., & Phelan, C. (2010). Doing clinical research: The challenges and benefits. Contemporary Nurse, 35(2), 171-181. https://doi.org/10.5172/conu.2010.35.2.171 Howerton Child, R. J., Mentes, J. C., Pavlish, C., & Phillips, L. R. (2014). Using Facebook and participant information clips to recruit emergency nurses for research. Nurse Researcher, 21(6), 16-21. https://doi.org/10.7748/nr.21.6.16.e1246 Hysong, S. J., Smitham, K. B., Knox, M., Johnson, K-E., SoRelle, R., & Haidet, P. (2013). Recruiting clinical personnel as research participants: A framework for assessing feasibility. Implementation Science, 8(1), Article 125. https://doi.org/10.1186/1748-5908-8-125 Jackman, D., Myrick, F., & Yonge, O. (2012). Putting the (R)ural in preceptorship. Nursing Research and Practice, 2012, Article 528580. https://doi.org/10.1155/2012/528580 Jones, C. B., & Mark, B. A. (2005). The intersection of nursing and health services research: An agenda to guide future research. Nursing Outlook, 53(6), 324-332. https://doi.org/10.1016/ j.outlook.2005.09.003 Jones, S. N. (2012). The importance of nurses as participants of research. Home Health Care Management & Practice, 24(5), 261-262. https://doi.org/10.1177/1084822312447939 Kaufman, B. G., Whitaker, R., Pink, G., & Holmes, G. M. (2020). Half of rural residents at high risk of serious illness due to COVID-19, creating stress on rural hospitals. Journal of Rural Health, 36(4), 584-590. https://doi.org/10.1111/jrh.12481 Khamisa, N., Peltzer, K., Ilic, D., & Oldenburg, B. (2014). Evaluating research recruitment strategies to improve response rates amongst South African nurses. South African Journal of Industrial Psychology, 40(1), 1-7. https://doi.org/10.4102/sajip.v40i1.1172 Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 42 Lee, H. J., & McDonagh, M. K. (2018). Updating the rural nursing theory base. In C. A. Winters & H. J. Lee (Eds.), Updating the rural nursing theory base (5 ed., pp. 46-62). Springer Publishing. https://doi.org/https://doi.org/10.1891/9780826161710.0004 Long, K. A., & Weinert, C. (1989). Rural nursing: Developing the theory base. Scholarly Inquiry for Nursing Practice, 3(2), 113-127. https://pubmed.ncbi.nlm.nih.gov/2772454/ Luck, L., Chok, H. N., & Wilkes, L. (2017). Nurses as participants in research: An evaluation of recruitment techniques. Nurse Researcher, 25(2), 44-48. https://doi.org/10.7748/ nr.2017.e1546 National Academies of Sciences, Engineering, and Medicine. (2018). Achieving rural health equity and well-being: Proceedings of a workshop. The National Academies Press. https://doi.org/10.17226/24967 Patel, S. S., Webster, R. K, Greenberg, N., Weston, D., & Brooks, S. K. (2020). Research fatigue in COVID-19 pandemic and post-disaster research: Causes, consequences and recommendations. Disaster Prevention and Management, 29(4), 445-455. https://doi.org/10.1108/DPM-05-2020-0164 Raymond, C., Profetto-McGrath, J., Myrick, F., & Strean, W. B. (2018). Process matters: Successes and challenges of recruiting and retaining participants for nursing education research. Nurse Educator, 43(2), 92-96. https://doi.org/10.1097/NNE.0000000000000423 Roll, L., Stegenga, K., Hendricks-Ferguson, V., Barnes, Y. J., Cherven, B., Docherty, S. L., Robb, S. L., & Haase, J. E. (2013). Engaging nurses in research for a randomized clinical trial of a behavioral health intervention. Nursing Research and Practice, 2013, Article 183984. https://doi.org/10.1155/2013/183984 Online Journal of Rural Nursing and Health Care 2021(2) https://doi.org/10.14574/ojrnhc.v21i2.682 43 Roxburgh, M. (2006). An exploration of factors which constrain nurses from research participation. Journal of Clinical Nursing, 15(5), 535-545. https://doi.org/10.1111/j.1365- 2702.2006.01374.x Segel, J. E., Ross, H. I., Edwards, J. L., Braun, K. A., & Davis, L. A. (2020). The unique challenges facing rural providers in the COVID-19 pandemic. Population Health Management, 24(3), 304-306. https://doi.org/10.1089/pop.2020.0151 Squires, A., Germack, H., Muench, U., Stolldorf, D., Witkoski-Stimpfel, A., Yakusheva, O., Brom, H., Harrison, J., Patel, E., Riman, K., & Martsolf, G. (2019). The Interdisciplinary Research Group on Nursing Issues: Advancing health services research, policy, regulation, and practice. Journal of Nursing Regulation, 10(2), 55-59. https://doi.org/10.1016/S2155- 8256(19)30116-4 Stokes, Y., Vandyk, A., Squires, J., Jacob, J-D., & Gifford, W. (2019). Using Facebook and LinkedIn to recruit nurses for an online survey. Western Journal of Nursing Research, 41(1), 96-110. https://doi.org/10.1177/0193945917740706 vanGeest, J., & Johnson, T. P. (2011). Surveying nurses: Identifying strategies to improve participation. Evaluation and the Health Professions, 34(4), 487-511. https://doi.org/10.1177/0163278711399572