ADYNSKI _Other,+666-NEWArticle+Text-4565-2-15-20210716_graves+final+edits071721_formatted_Final APA edits Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 167 A Systematic Review of the Strategies to Address Health Worker Shortage in Rural and Remote Areas of Low- and Middle-Income Countries Gillian I. Adynski, PhD, RN1 Leah L. Morgan, PhD, RN2 1Postdoctoral Research Fellow, National Clinician Scholars Program, Duke University, gillian.adynski@duke.edu 2Assistant Professor, Queens University of Charlotte, morganL4@queens.edu Abstract Purpose: The aim of this paper is to examine the literature on strategies to address the maldistribution of health workers between urban and rural and remote areas of low- and middle- income countries. Sample: 16 peer-review articles that address strategies to recruit and retain health workers in rural and remote areas of low- and middle-income countries. Methods: A systematic literature review was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Findings: Embase, PubMed and Web of Science, were searched resulting in 731 articles, 16 articles were included after screening. Across the 16 articles, the main categories to address the maldistribution of health workers between urban and rural and remote areas were Use of Public Sector Employment, Rural and remote Employment Incentives, Student and School Based Strategies, and Health Infrastructure and Retention. Conclusion: There is not just one type of strategy that can successfully address the maldistribution of health workers between urban and rural and remote areas. To address population health in Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 168 countries suffering from maldistribution of health workers use of increasing the availability of public sector employment, increasing incentives for health workers to work in rural/ remote areas, partnering with health worker training programs and schools, and building up high quality infrastructure and other interventions to retain health workers in rural and remote areas are successful. Future research should evaluate interventions to recruit and retain health workers in rural and remote areas and look at population health as a result. The ultimate goal for recruitment and retention of health workers in rural and remote areas is improving population health and this has not been considered in past research. A Systematic Review of the Strategies to Address Health Worker Shortage in Rural and Remote Areas of Low- and Middle-Income Countries Skilled health workers (physicians, nurses, and midwives) are necessary for attaining and sustaining universal health coverage worldwide (Global Health Workforce Alliance & World Health Organization, 2013). Nearly all countries are challenged by worker shortages, skill mix imbalance, maldistribution, negative work environments and weak knowledge bases. However, these problems are worse in low- and middle- income countries (LMICs) who also face high disease burdens and emigration of health workers (Chen et al., 2004; World Health Organization [WHO], 2015). These LMICs are constantly losing the healthcare workers they do have because of low wages, poor working conditions, poor health infrastructure, and high burdens of diseases (Lehmann et al., 2008). The shortages of health workers in LMICs worsens in rural and remote areas (Johnston et al., 2020; Lehmann et al., 2008; WHO, 2016), as they have even less physicians, nurses, and midwives to provide care to residents. This paper defines rural and remote as those areas relating Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 169 to the country, or areas away from cities (Merriam-Webster Dictionary, 2021). In the World Health Report, it was determined that half of the world’s population lives in rural and remote areas, however only approximately 38% of the world’s nursing workforce and approximately 25% of the world’s physician workforce resides in rural and remote areas (Guilbert, 2006). In South Africa, 46% of the population lives in rural areas but only 12% of the doctors and 19% of the nurses live there (WHO, 2010). In contrast in urban areas in Bangladesh, 30% of the nurses are in four metropolitan districts where only 15% of the population lives (WHO, 2010). Past research has demonstrated that having higher physician and nurse to population ratios in rural and remote areas leads to better health outcomes (Bigbee et al., 2014; Fields et al., 2016). The knowledge and evidence to recruit and retain health workers is still low (Mbemba et al., 2016) and health workers policy makers are not consistently making evidence-based policy decisions to address this problem (Dolea et al., 2010). Despite some reviews of the literature (Lehmann, et al., 2008; Buykx et al., 2010; Mbemba et al., 2016; WHO, 2010) on recruitment and retention of health workers to rural and remotes areas, few have focused on specific interventions that work in LMIC countries, which may have less resources to devote to complex policy interventions. The purpose of this review is to focus specifically on LMICs to update a review on the evidence to recruit and retain health workers in rural and remote areas of LMICs. In order to grow the body of evidence that policy makers can choose from, this paper seeks to systematically explore the literature on recruitment and retention of health workers to rural and remote areas of LMICs and provide necessary updates on what is known about rural and remote retention and recruitment of health workers. A better understanding of current and updated recruitment and retention strategies will assist policy makers and key stakeholders in implementing interventions to strengthen the rural and remote health workforces in LMICs. While this paper Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 170 does not focus solely on the nursing workforce, nurses work in interdisciplinary teams and it is vital that nurse policy advocates who strive to provide the greatest access to care to rural populations understand the body of evidence about strategies for recruitment and retention of a variety of different healthcare cadres to rural and remote areas of low- and middle- income countries. Methods This review is guided by the Preferred Reporting Items for Systematic Reviews and Meta- Analyses (PRISMA) guidelines for conducting systematic reviews. This review searched for articles that reported empirical research to further knowledge about strategies to recruit and retain health workers in rural and remote areas of LMICs. To qualify for this systematic literature review, the studies had to consider strategies to address the problem of not enough health workers in rural and remote areas in LMICs, such as rural retention initiatives, school-level policies for recruitment for nursing and medical schools, and governmental laws and regulations. Articles were excluded if they did not describe or address a strategy or solution to address recruitment and retention of health workers in rural and remote areas. Articles were excluded if they did not meet the general definition of rural or remote areas. Articles that stated they were looking at rural or remote health workforces were included and their exact definitions of rural or remote were extracted. Articles were not included if they generally addressed recruitment or retention in both urban and rural areas, meaning articles were only included if they were addressing specifically the recruitment and retention of health workers serving rural or remote populations. To qualify for this review, an original, peer-reviewed research paper must have addressed a solution to the problem of maldistribution of health workers between rural and urban areas in LMICs. For the purpose of this literature review, health worker was defined as physician, nurse or Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 171 midwife, as they are cadres the World Health Organization (WHO) and the Global Health Workforce Alliance deem as key health workers in achieving universal health coverage (Campbell et al., 2013; Global Health Workforce Alliance & WHO, 2013; Guilbert, 2006). LMICs were defined by the World Bank as those who have a Gross National Income of $12,375 or less in 2019 (World Bank, 2019). Both quantitative, qualitative and mixed methods articles were included. Articles were excluded if they were not in English, did not meet our definition of healthcare workers, were not a peer-reviewed research paper, or did not include an intervention to recruit or retain health workers in LMICs. Due to the complexity of the search and the large number of synonyms for the included search terms, a simplified version of the search is included in this section, while the full search string with all synonyms is included in Appendix A. Searches were adjusted for each database. EMBASE, PubMed, and Web of Science were searched. The search was “middle/low income country” OR [list of all middle/low income countries]) AND “health worker” AND “distribution”. The search covered all years, in order to maximize the included and relevant articles. Next, two reviewers screened title, abstract and then subsequently the full text to see if articles were relevant. These reviewers screened texts independently then met together to concur on included articles. Covidence software was used to screen articles. This was followed by one reader extracting all articles, with 10% of the articles being extracted by a second extractor to confirm results. Articles were extracted for study design, health workers’ type, country, sample size, rural definition, intervention, and intervention evidence. Quality of all articles was assessed using the Mixed Methods Appraisal Tool (Hong et al., 2018). An assessment of quality was conducted in order to interpret the rigor and strength of the literature. However, no articles were excluded based on Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 172 quality in order to maximize the included and relevant articles. After assessing for quality, results of included articles were synthesized to interpret findings to answer the research question. Findings Database Search Results After conducting the search, EMBASE resulted in 238 articles, Web of Science in 288 and PubMed in 390. Once duplicates were removed, this resulted in 731 articles. After title, abstract, and full text screening, 16 articles remained. See Figure 1 for additional details. Figure 1 PRISMA Diagram. Records identified through database searching EMBASE (n =238) WEB of SCIENCE (n=288) PubMed (n=390) Total (n=916) Additional records identified through other sources (n =0) Records after duplicates removed (n = 731) Records screened (n =731) Records excluded (n =701) Full-text articles assessed for eligibility (n =30) Full-text articles excluded, with reasons Not in English (n=1) Conference Poster (n =1) Doesn’t meet HCW Definition (n=1) Thought Piece (n=4) No Intervention (n=6) Not observing distribution (n=1) Studies included in Scoping Review (n =16) Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 173 Overview of Articles Four articles used qualitative methods, 10 articles used quantitative methods, while the remaining two used mixed methods. Of the 11 quantitative methods, four used discrete choice experiments as their survey type. Most articles (n=10) were focused on countries in Sub-Saharan Africa, as shown in Table 1. The articles discussed different types of health workers: 13 discussed physicians, seven discussed nurses, and three discussed midwives. No articles specifically stated how they defined rural or remote for their study. The quality assessment of all articles can be found in Appendix B. Of the included articles, 15 were of high quality, with only one article not scoring as the others, as the article was not explicitly clear on the details of the intervention they were evaluating Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 174 Table 1 Overview of Included Studies Study Design Health Worker Country Sample Rural Definition Intervention Type Outcomes Figueiredo et al. (2019) Cross Sectional Descriptive Design Physicians Brazil 54 medical schools Not explicitly defined, generally defined as rural and underserved areas • Established Health Worker Recruitment • Student Recruitment Prevalence of Health Workers in Rural/Remote Areas Gross et al (2010) Cross Sectional Descriptive Design Nurses Kenya 18181 Nurses Not explicitly defined, generally defined at rural or remote areas • Public Sector Employment Prevalence of Health Workers in Rural/Remote Areas Hesketh et al. (2017a) Quasi Experimental Study Physicians China 104 village physicians Not explicated defined, generally stated as rural China • Health Infrastructure and Retention Prevalence of Health Workers in Rural/Remote Areas Kiwanuka et al. (2017) Qualitative Descriptive Nurses, Midwives Uganda 21 HCWs Not explicitly defined, generally defined as rural districts • Health Infrastructure and Retention Prevalence of Health Workers in Rural/Remote Areas Lagarde et al. (2012) Cross Sectional Descriptive Design; Discrete Nurses South Africa 377 Nursing Students Not explicitly defined, generally defined at rural areas • Established Health Worker Recruitment Willingness to Accept Employment in Rural/Remote Areas Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 175 choice experiment • Student Recruitment Leon et al. (2010) Cross Sectional Descriptive Design Physicians Tanzania 130 Medical Students Not explicitly defined, generally stated as rural area or rural job • Established Health Worker Recruitment • Student Recruitment Willingness to Accept Employment in Rural/Remote Areas Meliala et al. (2013) Cross Sectional Descriptive Design; Qualitative Specialty Physicians Indonesia 149 Specials Physicians Interviewed 15 Specialist Physicians Shadowed Not explicitly defined, generally defined at rural or remote areas • Public Sector Employment Prevalence of Health Workers in Rural/Remote Areas Munga et al. (2009) Qualitative Descriptive Physicians, Nurses Tanzania 21 Human Resource Managers and Health Workforce Policy Makers Not explicitly defined, generally defined as rural districts • Established Health Worker Recruitment • Public Sector Employment Prevalence of Health Workers in Rural/Remote Areas Nkomazana et al. (2015) Qualitative Descriptive Physicians, Nurses Botswana 15 focus groups 5-12 Policy Markers, Health Workers and Community Members in each Not explicitly defined, generally stated rural or remote rural. • Established Health Worker Recruitment • Health Infrastructure and Retention Prevalence of Health Workers in Rural/Remote Areas Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 176 Okeke et al. (2016) Difference in Difference Design Midwives Nigeria 7104 women who gave birth across 12 Nigerian states Not explicitly defined, generally defined as rural area or community • Public Sector Employment Population Health in Rural/ Remote Areas Robyn et al. (2015) Cross Sectional Descriptive Design; Discrete Choice Experiment Physicians, Nurses Cameroon 351 Medical Students, Nursing Students and Health Workers Not explicitly defined, generally defined as rural area or remote rural area • Established Health Worker Recruitment • Health Infrastructure and Retention • Student Recruitment Willingness to Accept Employment in Rural/Remote Areas Serneels et al. (2007) Contingent Evaluation Physicians, Nurses Ethiopia 219 nursing students; 90 medical students Not explicitly defined, generally defined as rural area • Health Infrastructure and Retention Willingness to Accept Employment in Rural/Remote Areas Sirili et al. (2019) Qualitative Descriptive Physicians Tanzania 20 recently graduated physicians Not explicitly defined, generally defined as rural area • Established Health Worker Recruitment • Student Recruitment Willingness to Accept Employment in Rural/Remote Areas Song et al. (2018) Longitudinal Observation Study Physician China 796 Poor Counties Not explicitly defined, generally defined as rural counties • Public Sector Employment Prevalence of Health Workers in Rural/Remote Areas Thoresen & Fielding (2010) Cross Sectional Descriptive Physicians; Nurses; Midwives Thailand 93 Healthcare Students Not explicitly defined, generally • Established Health Willingness to Accept Employment in Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 177 Design Qualitative defined as rural districts Worker Recruitment • Health Infrastructure and Retention Rural/Remote Areas Vujicic et al. (2011) Labor Market Survey; Discrete Choice Experiment Physicians Vietnam 292 Physicians Not explicitly defined, generally defined as rural areas • Established Health Worker Recruitment • Health Infrastructure and Retention Willingness to Accept Employment in Rural/Remote Areas Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 178 Strategies Used The strategies of the included studies fit into four categories developed by the authors for organizational purposes and synthesis of information. The categories were “public sector employment” (n=5), “established health worker recruitment” (n=9), “student recruitment” (n=5) and “health infrastructure and retention” (n=7). Studies observed outcomes that fit into the following three categories, again developed by the author for organizational and synthesis of information: “prevalence of health workers in rural and remote areas” (n=8), “willingness to accept employment in rural and remote areas” (n=7), and “population health in rural and remote areas” (n=1). Table 2 overviews the results by strategy type and study results. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 179 Table 2 Strategies to Recruit and Retain Health Care Workers in Rural and Remote Area’s Evidence Intervention Type Interventions Evidence Public Sector Employmen t The government used their own funds to hire healthcare workers to rural/remote areas • Increasing public sector midwives in rural areas increased use of antenatal care by 7.3 percentage points. However, they found no significant difference in maternal delivery outcomes or skilled birth attendance (Okeke et al., 2016) • 12% increase in nursing staff in the public sector • 204 of 2359 health facilities run by only Emergency Hiring Plan nurses • The most remote area province had a 37% increase in nurses (Gross et al., 2010) China’s Comprehensive Health System Reform that increased physician was found to increase physicians in rural underserved areas, however the improvement was not distributed between richer and poorer rural areas (Song et al., 2018) The government mandated health worker location and public service • Very few physicians spend the required time in their public hospitals, some as little as 1.5 hours a week. This was likely because public sector pay was such a little part of the physician’s overall incomes. • Physicians were non-compliant with the new licensure regulations, and would practice where they wanted to, whether they had a license to practice at that location (Meliala et al., 2013) Federal government places health workers in jobs in the district of their choice • Federal government was more effective at placing people in highly remote areas, than when the districts published the jobs themselves (Munga et al., 2009) Establish Health Worker Recruitment Car or Housing Allowance • In a focus group, housing and car allowance were mentioned as a recruitment study they believed would work for them (Nkomazana et al., 2015) • Lagarde et al (Lagarde et al., 2012) found that offering housing and a car allowance as a recruitment strategy both worked as a way to recruit people to a job and was cost effective. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 180 • Providing free housing to physicians in rural areas has a low impact and would only increase those willing to accept employment from 7% to 11 % (Vujicic et al., 2011) Salary Increase • Medical students (aOR=8.27, CI: 5.23-12.96, P<0.001), medical doctors (aOR=5.60, CI: 4.12-7.61, P<0.001) and nurse aids (aOR=4.29, CI: 3.11-5.93, P<0.001) all reported they would take a rural/remote job for 75% of the base salary as a bonus (Robyn et al., 2015) • Robyn et al (Robyn et al., 2015) they did not find significance for a salary bonus of 75% with nursing students or nurses. • Salary increases about 20% were found to be very expensive, and therefore not cost effective (Lagarde et al., 2012) • To reach Ethiopia’s goal of 80% of health workers in a rural post, you would need to increase physicians’ salaries by 83% and nurses’ salaries by 57% (Serneels et al., 2007) Guaranteed reassignment to Urban areas • Nursing Students (aOR=2.81, CI: 2.22-3.56, P<0.001) were more likely to take a job if they were offered a transfer to an urban area after their service was complete (Robyn et al., 2015) • This was supported by Munga et al. (2009), who found that many accepted rural positions working for the government with the hope of transferring to an urban area in their governmental job. Allow health workers to reside near their families • Family and communities’ commitments and social ties and a sense of belong were associated with HCWs seeking employment in rural areas, and especially nurses sought employment near their families (Thoresen & Fielding, 2010) Student Recruitment Recruit students from rural areas to attend medical and nursing schools • Graduating medical students over the age of 26 were much more likely to accept a rural by 30 percentage points (p<0.05) (Leon & Kolstad, 2010) • Graduating medical students with rural residing parents were 50 percentage points more likely to take a rural/ remote job (Leon & Kolstad, 2010) • Recruiting students from rural and remote areas is much more cost than any recruitment measures such as salary increase, car bonus, etc. (Lagarde et al., 2012) • Those with a rural background were less motivated for a medical career at the end of medical school (74% of rural background students reported being less motivated by the end of medical school than before medical school). (Leon & Kolstad, 2010) Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 181 aOR-Adjusted Odds Ratio • Urban born are less likely to be willing to take jobs in rural areas (Sirili et al., 2019) Create medical schools in rural/ remote areas having rotations in rural areas • Municipalities with new medical schools had an increase in physicians: inhabitant ratios greater than those in municipalities without a new medical school (Figueiredo et al., 2019) • Students who have a community health rotation were less likely by 34 percentage points (significance 5%) to accept a job in a rural area than students without a community health rotation (Leon & Kolstad, 2010) • Providing short term training courses in rural areas increased willingness to accept rural employment from 7% to 14% (Vujicic et al., 2011) Health Infrastructur e and Retention Improve rural/remote health resources and health facilities • Medical students, nursing students, medical doctors, and nurses all would take a job in a rural/remote if the facility had good infrastructure (Robyn et al., 2015). • The lack of resources in a district when the districts oversee recruiting health workers, also made it very difficult (Munga et al., 2009) • Improving equipment in rural areas improves a physician’s willingness to take a job in a rural area from 7% to 15% (Vujicic et al., 2011) • Physicians increased satisfaction with their salaries and other clinic salary changes, decreasing their turnover (Hesketh et al., 2017a) • Family proximity, community ties, job security and pension enhance retention, while higher cost of living and unpredictable employment increased turnover (Kiwanuka et al., 2017) Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 182 Public Sector Employment Public sector employment strategies were defined as any studies that included directly hiring health workers into the public sector and deploying and distributing them to rural and remote areas for permanent positions or temporary ones. Gross et al. ( 2010) reported on a government intervention, an Emergency Hiring Plan, that used a mix of government and aid organization money to hire nurses into the public sector of employment, allowing the number of nurses hired to be greater in areas of more need. This descriptive comparative study had a sample size of 18,181 nurses. The intervention had many successes such as: a 12% increase in nursing staff employed in the public sector, the nursing staff increased by 7% in hospitals, 13% in health centers and 15% in dispensaries, and 204 of 2,359 health facilities were staffed entirely by nurses hired by the Emergency Hiring Plan. The province with the highest prevalence of remote areas benefited the most with a 37% increase in the number of nurses per 100,000 populations (Gross et al., 2010). Similarly, in Nigeria, the federal and local government partnered together to create the Midwives Service Scheme, which hired midwives who were unemployed, retired or newly graduated to staff underserved communities. In regions that received the intervention, there was a 7.3 percentage point increase in antenatal care visits at clinics (Okeke et al., 2016). However, there was no significant difference in maternal delivery complication or use of skilled birth attendance between regions who received the intervention and those that did not (Okeke et al., 2016). Another intervention found in the literature was also a government-level policy change which compared the use of the federal government for hiring healthcare workers versus the district government having the hiring power. Munga et al.(2009) found through qualitative interviews that the centralized federal government was more effective at recruiting highly skilled healthcare Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 183 workers to remote areas. However, when district leadership recruited health workers themselves there was better retention (Munga et al., 2009). Meliala et al.(2013) evaluated the effectiveness of a government policy that required physicians work in public facilities for a certain number of days per week. In addition, the government awarded only a certain number of physician’s licenses per district, therefore distributing them out further. This policy change was not well enforced, with physicians only working 1-3 days out of the 5 days of work required. Additionally, the findings indicated that physicians would just practice where they wanted without being licensed, so this intervention was ineffective in distributing the physician workforce (Meliala et al., 2013). Finally, another study by Song et al. (2018) looked at the Chinese Comprehensive Health System Reform, which included training health workers to work in public facilities and requiring physicians to serve in rural areas in order to be eligible for a professional promotion. Through this policy change in China, the number of health workers did increase in rural areas. However, the number of health workers did not increase as much in poorer rural counties as in richer rural counties (Song et al., 2018). Established Health Worker Recruitment Established health worker recruitment strategy was defined as any incentive to recruit or encourage more established health workers to work in rural or remote areas. Within the literature, this section can be further broken down into the following four categories of established health worker recruitment: car/housing allowance (n=3), salary increase (n=3), guaranteed reassignment back to an urban area (n=2) and allowing health workers to reside near their families (n=1). Three studies observed the effectiveness of a car/ housing allowance as a recruitment intervention for health workers to take a job in a rural and remote area (Lagarde et al., 2012; Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 184 Nkomazana et al., 2015; Vujicic et al., 2011). The first was a qualitative study, with participants of both health workers and health policy makers in Botswana identifying that they perceived a car or housing allowance to be an effective intervention to recruit health workers to take a job (Nkomazana et al., 2015). Lagarde et al. (2012) observed the cost effectiveness of a variety of different recruitment interventions, finding that a car allowance in combination with offering study leave sooner was a cost-effective intervention to recruit heath workers to rural areas However, Vujicic et al. (2011) found that providing free housing to physicians in rural areas has a low impact and would only increase those willing to accept employment from 7% to 11% . Three studies looked at the use of salary increases or bonuses as a means of incentivizing people to take positions. Two of those studies used quality methods of a well-appraised measurement tool (Araujo & Maeda, 2013), a discrete choice experiment, with appropriate sample sizes and low attrition (Lagarde et al., 2012; Robyn et al., 2015). The third study used observed contingent evaluation to observe how much health workers’ salaries would need to increase for them to be willing to take a job in a rural and remote area (Serneels et al., 2007). Robyn et al. ( 2015) looked at the odds of a medical student, nursing student, medical doctor, nurse or nurse aid’s willingness to take a position in a rural and remote area based on several incentives. Medical students, medical doctors, and nurse aids all reported they would take a rural and remote job for 75% of the base salary as a bonus but salary increases did not increase nursing students or nurse’s willingness to accept a job in a rural area (Robyn et al., 2015). Similarly, Lagarde et al.(2012) found that salary increases of 20% higher than the current salary was found to be an expensive intervention. Finally, Serneels et al. (2007) found that to reach Ethiopia’s goal of 80% of health workers in a rural post, you would need to increase physicians’ salaries by 83% and nurses’ salaries by 57%. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 185 Another possible incentive was offering leave sooner to those who are working in rural and remote areas than for those who are not. Many governments have an amount of public service time required after graduating before a health worker can choose where they want to work. This was found to be a relatively cheap incentive, compared to incentives like a salary increase. This incentive provided that if workers took a job in a rural and remote area they could leave their position sooner to go to graduate studies. It was even more effective when it is combined with a car or a salary increase, but the costs go up (Lagarde et al., 2012). Similarly, another intervention was giving a guaranteed reassignment to urban areas, after a mandatory rural time was complete. Nursing students were more likely to take a job if they were offered a transfer to an urban area after their service was complete (Robyn et al., 2015). This was similar to the findings of Munga et al. (2009) who found that many accepted rural positions working for the government with the hope of transferring to an urban area in their governmental job. Finally, one study observed placing health workers in jobs near their families who reside in rural areas as an incentive to take a job in a rural area. Family and communities’ commitments, social ties, and a sense of belonging were associated with health workers seeking employment in rural areas, and especially nurses sought employment near their families (Thoresen & Fielding, 2010). Student and School-based Strategies Student and school-based strategies included any intervention that focused on factors of students (i.e. demographics or experiences in school) that increased likelihood of taking a position in a rural and remote area. Three studies observed interventions that recruited students to medical or nursing school from rural and remote areas, to see if they are more likely to return to rural and remote areas post-graduation (Lagarde et al., 2012; Leon & Kolstad, 2010; Robyn et al., 2015). Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 186 Health worker students whose families lived in rural districts were 50% more likely to take rural and remote jobs after graduation and students 26 and older were 30% more likely (Leon & Kolstad, 2010). Recruiting students from rural and remote areas is more cost-effective than salary increases or car bonuses (Lagarde et al., 2012). Although students from rural areas are more likely to be willing to accept rural jobs, 74% of those students from rural areas were found to be less motivated at the end of medical school then their peers (Leon & Kolstad, 2010). Next, the introduction of a community health rotation during medical school was evaluated as a possible way of incentivizing health workers to work in rural and remote areas. Leon & Kolstad(2010) found that in a study with 130 medical students in Tanzania, students who have a community health rotation were 34% less likely to accept a job in a rural area than students without a community health rotation, suggesting that this intervention was ineffective. This may have been because they did not have positive experience in their community health rotations (Leon & Kolstad, 2010). However, when Vujicic et al. (2011) used a discrete choice experiment with 292 physicians, they found that providing short term training courses in rural areas increased willingness to accept rural employment from 7% to 14%. Figueiredo et al. (2019) looked at an intervention in Brazil which set up medical schools in rural and medically underserved municipalities. When comparing the rural municipalities who received a new medical school and those that did not, they found that those who had a new medical school had a greater increase in physicians to inhabitant ratios, suggesting that locating medical schools in rural areas was a way to increase physicians in rural areas (Figueiredo et al., 2019). Health Infrastructure and Retention Health infrastructure and health worker retention category included any intervention that worked on retaining health workers in rural and remote areas. Five studies observed interventions Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 187 within this category. The main intervention discussed was improving rural and remote health resources and health facilities (Nkomazana et al., 2015; Robyn et al., 2015; Vujicic et al., 2011). A qualitative study of interviews with policy makers and health workers pointed to fixing resource deficits as a possible solution to rural retention (Nkomazana et al., 2015). Medical students, nursing students, medical doctors, and nurses all would take a job in a rural and remote facility if that facility had good infrastructure (Robyn et al., 2015). In addition, improving equipment in rural areas improves a physician’s willingness to take a job in a rural area from 7% to 15% (Vujicic et al., 2011). Two other studies observed rural retention. Physician’s job satisfaction was increased when their salaries increased, which in turn decreased physician turnover (Hesketh et al., 2017a). Family proximity, community ties, job security and pension enhanced retention while higher cost of living and unpredictable employment increased turnover (Kiwanuka et al., 2017). Discussion This systematic review explored strategies to address the lack of healthcare workers in rural and remote areas in LMICs. The literature was limited due to the lack of randomized controlled trials or quasi experimental research. We used systematic review methods to update past reviews on evidence-based strategies to recruit and retain health workers in rural and remote areas, while focusing on LMICs who may face unique resource challenges in creating workforce distribution policy interventions. Our results indicate some successful interventions as well as the need for further research to evaluate the implementation across contexts. Our results indicate that there is not one solution or perfect strategy that can address the lack of health workers in rural and remote areas of LMICs. Each strategy had its successes and its shortcomings. The next sections will address the successes and shortcomings of each of the Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 188 following intervention types: public sector employment, rural and remote employment incentives, student and school-based strategies, and health infrastructure and retention. Public Sector Employment Policy Recommendations The following is a list of important considerations for policy makers considering expanding public sector employment: 1. Using the resources of the federal government may be more effective in hiring health workers to rural and remote areas than local resources, as the federal government may have more access to advertising and recruitment strategies (Munga et al., 2009). 2. Even with public sector employment scale up, it is still necessary to maintain high job satisfaction or retention of these employees (Munga et al., 2009). 3. When rolling out increased public sector employment in rural and remote areas, it is important to monitor patient outcomes, since Okeke et al. (2016) found that increased midwives at public facilities did not improve maternal outcomes. 4. Public sector employment can be expensive on the part of the government and may not have the opportunity to be subsidized by external funding as it was in the Kenya’s Emergency Hiring Plan (Gross et al., 2010). Sustainability of external funding sources should be evaluated. 5. If using required hours per week with health workers in the public sector, it is important to have strict monitoring of these hours in order to ensure compliance (Meliala et al., 2013). Depending on the financial resources of a government considering addressing the geographical imbalance of healthcare workers, public sector employment could be successful. But Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 189 the government would need to make sure financial support to fund the jobs is available, as well as investing in ensuring good job satisfaction at these rural and remote jobs. Establish Health Worker Policy Recommendations The following are a list of important considerations for policy makers considering recruiting established health workers to rural and remote posts: 1. Increasing salaries were found to be effective by three studies (Lagarde et al., 2012; Robyn et al., 2015; Serneels et al., 2007). However, it should be noted that this was an expensive way to increase willingness to accept employment, compared with other incentives (Lagarde et al., 2012). 2. Offering a car or housing allowance increased participants’ willingness to accept employment but was much less expensive (Lagarde et al., 2012). Therefore, these interventions would be more cost-effective choices for policy makers. Various incentives to recruit health workers to rural and remote areas may be effective. Before implementing policy changes, cost should be considered because a salary increase incentive may be more expensive than other interventions that yield similar results. Student Recruitment Policy Recommendations The following are a list of important considerations for policy makers considering recruiting health worker students to rural and remote posts: 1. Recruiting and incentivizing student from rural and remote areas to attend medical, nursing, and midwifery schools may be an effective way to increase the health workforce of rural and remote areas, as these students are more likely to be willing to take a job in rural and remote areas (Leon & Kolstad, 2010). Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 190 2. Placing medical, nursing, and midwifery schools in rural and remote areas could be an effective way to increase health workforces in rural and remote areas, based on findings that indicated that medical schools located in rural and remote areas increased the number of medical doctors in those regions (Figueiredo et al., 2019). 3. Policy makers could consider increasing clinical rotations in rural and remote areas; however, these programs should be carefully monitored as the literature was mixed as to the effectiveness of this interventions. One study indicated that a rural clinical rotation increased willingness to accept jobs there (Vujicic et al., 2011), while a second indicated students were less likely to accept employment in rural areas after a clinical rotation there (Leon & Kolstad, 2010). Recruiting students from rural areas, locating health worker schools in rural areas and rural clinical rotations may all be effective ways of increasing rural and remote health workforces. These are highly cost-effective interventions in comparison to increasing salaries or creating emergency hiring plans to hire employees into the public sector. However, they will not work if there are no rural or remote job positions to take so they need to be paired with interventions to scale up employment opportunities in rural and remote areas. Health Infrastructure and Retention Policy Recommendations The following is a list of important considerations for policy makers considering improving health infrastructure and other interventions in order to promote health worker retention: 1. Improvements and investments in rural and remote health facilities to support health workers would likely be effective in decreasing health worker turnover (Munga et al., 2009; Robyn et al., 2015; Vujicic et al., 2011). Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 191 2. Other factors that decrease turnover are increased salaries (Hesketh et al., 2017b), family proximity, community ties, job security and a strong pension (Kiwanuka et al., 2017). These factors that relate to retention need to be addressed in conjunction with interventions that recruit health workers to rural and remote areas. Implications This literature review supports a comprehensive strategy that addresses one or more of the above facets of recruitment and retention of health workers (public sector employment, established health worker recruitment, student recruitment, and health infrastructure and retention) and will likely be the most successful methods to increase the number of skilled health workers available to deliver care in rural and remote areas of LMICs. This review was not the only review that supports multi-faceted strategies that address multiple factors of recruitment and retention. Lehmann et al. (2008), after conducting a literature review on recruitment and retention of health worker staff in rural and remote areas in LMICs, also concluded that there are many facets of attraction and retention of health workers to rural and remote areas. In a synthesis of literature on retention strategies for health workers in rural and remote areas Buykx et al. (2010) also supported that bundles of retention strategies were more effective than single faceted strategies worldwide. This overlapped with the findings of this literature review that supported the importance of addressing multiple facets of recruitment and retention rather than a single intervention. This literature review contributes to the state of the science regarding recruitment and retention of health workers in rural and remote areas by reflecting the need for multiple and integrative intervention approaches. The integration of these strategies can improve the efficacy and financial feasibility of interventions in order to effectively recruit and maintain health care Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 192 workers. The results reflect similar retention strategies from past literature reviews published a decade earlier. The majority of the included articles were published since the previous reviews published in 2010 and earlier indicating that the results have remained true over the last decade. In 2008, Lehmann et al. (2008) claimed policy makers are not always making choices based on evidence-based strategies, like the ones reviewed in this systematic review. This review of the literature indicated that since 2008 many researchers are utilizing WHO’s recommendations to monitor, evaluate, and disseminate recruitment and retention interventions in rural and remote areas (WHO, 2010). It is important for researchers to continue to publish work related to the outcome of interventions and strategies to recruit and retain health workforces in rural and remote areas so that knowledge can be shared, and evidence-based strategies can be used in more locations. Nurses are key policy advocates for rural populations around the world and understanding the need for policy-based strategies are key to developing interventions to support health workforces in rural areas globally. There are many implications for rural healthcare in LMICs based on the findings of this literature review. Rural healthcare facility managers and policy makers need to utilize the above technique to recruit and retain health workers to their facilities in order to have stronger health systems that can serve rural populations. These manager and policy makers can increase health worker salaries, offer car or housing allowances, encourage local medical, nursing and midwifery schools to complete rotations in their facilities, and invest in good work environments for employees. In resource limited settings and LMICs it may be difficult to make these types of investments; however, this paper supports that if these investments are made there is evidence of effectiveness in recruiting and retaining health workers. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 193 In conclusion, an intervention that addresses multiple facets of recruitment and retention is likely to cost a government significant money, but it may improve access to healthcare for the individuals who reside in rural and remote areas. It would be most effective to have seamless and effective coordination between the government, not-for-profit and private sectors in order to create effective interventions. This review supports that there are four facets of recruitment and retention of health workers to rural and remote posts in LMICs which are public sector employment, established health worker recruitment, student recruitment and health infrastructure and retention. This review supports multi-faceted interventions that are specifically designed for the settings they are being implemented in. Recommendations for Future Research Future research should measure population health indicators as an outcome. Only one article addresses population health as an outcome of the interventions discussed (Okeke et al., 2016). It should be noted that the goal of increasing the prevalence of health workers in rural and remote areas is to improve the health of populations in rural and remote areas and health care access of the populations of rural and remote areas. Nursing research often approaches problems in a patient and people centric way. Nursing research could lead the way on developing evaluations of policies and strategies to recruit and retain health workers in rural and remote areas that focus on looking at population health indicators as an outcome of success. Many of these studies looked at health workers willingness to accept employment as an outcome. These studies were hypothetically asking health workers and health worker students about their willingness to accept employment. More research needs to be done on the actual effectiveness of these interventions, rather than the hypothetical effectiveness of these interventions. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 194 Limitations The articles lacked homogeneity in strategies and geographical locations which made it hard to draw generalizable conclusions about this subject. In addition, these articles did not have specific definitions of rural and remote areas included in them, making it difficult to compare interventions to one another. Future research should explicitly define rural and remote areas in research to make research findings generalizable to other locations. To implement any strategies that this review points to, more research needs to be done on the specific area where these strategies are being considered and look toward unique strategies that meet the needs. Research will need to focus on region specific strategies that are feasible and cost effective. Experimental studies of these interventions will need to be conducted to evaluate if these strategies are effective in getting more health care workers in rural and remote areas, as well as measuring their impact in health outcomes. Conclusions This study showed some positives and negatives of different strategies that could address the lack of health care workers in LMICs’ rural and remote areas. Likely, in order to successfully address the needs of the populations in the world’s rural and remote areas many strategies will need to be implemented simultaneously, including increasing the capacity of public sector employment, increasing recruitment of establish health workers, increasing recruitment of health worker students and building health infrastructure to retain health workers in rural and remote areas. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 195 References Araujo, E., & Maeda, A. (2013). How to recruit and retain health workers in rural and remote areas in developing countries: A guidance note. http://documents1.worldbank.org/curated/en/273821468154769065/pdf/785060WP0HR HDC00Box377346B00PUBLIC0.pdf Bigbee, J. L., Evans, S., Lind, B., Perez, S., Jacobo, L., & Geraghty, E. M. (2014). RN-to- Population Ratio and Population Health: A Multifactorial Study. Journal of Nursing Regulation, 5(1), 11-17. https://doi.org/10.1016/S2155-8256(15)30094-6 Buykx, P., Humphreys, J., Wakerman, J., & Pashen, D. (2010). Systematic review of effective retention incentives for health workers in rural and remote areas: Towards evidence-based policy. Australian Journal of Rural Health, 18(3), 102-109. https://doi.org/10.1111/j.1440- 1584.2010.01139.x Campbell, J., Buchan, J., Cometto, G., David, B., Dussault, G., Fogstad, H., Fronteira, I., Lozano, R., Nyonator, F., Pablos-Méndez, A., Quain, E. E., Starrs, A., & Tangcharoensathien, V. (2013). Human resources for health and universal health coverage: Fostering equity and effective coverage. Bulletin of the World Health Organization, 91(11), 853-863. https://doi.org/10.2471/BLT.13.118729 Chen, L., Evans, T., Anand, S., Boufford, J. I., Brown, H., Chowdhury, M., Cueto, M., Dare, L., Dussault, G., & Elzinga, G. (2004). Human resources for health: Overcoming the crisis. The Lancet, 364(9449), 1984-1990. https://doi.org/10.1016/S0140-6736(04)17482-5 Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 196 Dolea, C., Stormont, L., & Braichet, J. M. (2010). Evaluated strategies to increase attraction and retention of health workers in remote and rural areas. Bulletin of the World Health Organization, 88, 379-385. https://doi.org/10.2471/BLT.09.070607 Fields, B. E., Bigbee, J. L., & Bell, J. F. (2016). Associations of provider-to-population ratios and population health by county-level rurality. The Journal of Rural Health, 32(3), 235-244. https://doi.org/10.1111/jrh.12143 Figueiredo, A. M., McKinley, D. W., Lima, K. C., & Azevedo, G. D. (2019). Medical school expansion policies: Educational access and physician distribution. Medical Education. https://doi.org/10.1111/medu.13941 Global Health Workforce Alliance, & World Health Organization. (2013). A Universal Truth: No Health without a workforce (World Health Organisation (WHO) Report, Issue. https://www.who.int/workforcealliance/knowledge/resources/GHWA- a_universal_truth_report.pdf?ua=1 Gross, J. M., Riley, P. L., Kiriinya, R., Rakuom, C., Willy, R., Kamenju, A., Oywer, E., Wambua, D., Waudo, A., & Rogers, M. F. (2010). The impact of an emergency hiring plan on the shortage and distribution of nurses in Kenya: The importance of information systems. Bulletin of the World Health Organization, 88(11), 824-830. https://doi.org/10.2471/blt.09.072678 Guilbert, J. J. (2006). The World Health Report 2006: Working together for health. Education for Health, 19(3), 385-387. https://doi.org/10.1080/13576280600937911 Hesketh, T., Wang, W. Y., Xu, Y. N., & Zhou, X. D. (2017). Effect of an incentive-led intervention to address the shortage of village health workers in China: A community-based, non- Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 197 randomised controlled trial. Lancet, 390, S5-S5. https://doi.org/10.1016/s0140- 6736(17)33143-4 Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., GagnonM-P, G. F., Nicolau, B., & O'Cathain, A. (2018). Mixed methods appraisal tool (MMAT), version 2018. IC Canadian Intellectual Property Office, Industry Canada. Johnston, K., Guingona, M., Elsanousi, S., Mbokazi, J., Labarda, C., Cristobal, F. L., Upadhyay, S., Othman, A.-B., Woolley, T., & Acharya, B. (2020). Training a fit-for-purpose rural health workforce for low-and middle-income countries (LMICs): How do drivers and enablers of rural practice intention differ between learners from LMICs and high income countries? Frontiers in public health, 8. https://doi.org/10.3389/fpubh.2020.582464 Kiwanuka, S. N., Akulume, M., Tetui, M., Kananura, R. M., Bua, J., & Ekirapa-Kiracho, E. (2017). Balancing the cost of leaving with the cost of living: drivers of long-term retention of health workers: an explorative study in three rural districts in Eastern Uganda. Global Health Action, 10, Article 1345494. https://doi.org/10.1080/16549716.2017.1345494 Lagarde, M., Blaauw, D., & Cairns, J. (2012). Cost-effectiveness analysis of human resources policy interventions to address the shortage of nurses in rural South Africa. Social Science & Medicine, 75(5), 801-806. https://doi.org/10.1016/j.socscimed.2012.05.005 Lehmann, U., Dieleman, M., & Martineau, T. (2008). Staffing remote rural areas in middle-and low-income countries: A literature review of attraction and retention. BMC Health Services Research, 8(1), 19. https://doi.org/10.1186/1472-6963-8-19 Leon, B. K., & Kolstad, J. R. (2010). Wrong schools or wrong students? The potential role of medical education in regional imbalances of the health workforce in the United Republic Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 198 of Tanzania. Human Resources for Health, 8, Article 3. https://doi.org/10.1186/1478- 4491-8-3 Mbemba, G. I. C., Gagnon, M.-P., & Hamelin-Brabant, L. (2016). Factors influencing recruitment and retention of healthcare workers in rural and remote areas in developed and developing countries: An overview. Journal of Public Health in Africa, 7(2). https://doi.org/10.4081/jphia.2016.565 Meliala, A., Hort, K., & Trisnantoro, L. (2013). Addressing the unequal geographic distribution of specialist doctors in indonesia: The role of the private sector and effectiveness of current regulations. Social Science & Medicine, 82, 30-34. https://doi.org/10.1016/j.socscimed.2013.01.029 Munga, M. A., Songstad, N. G., Blystad, A., & Maestad, O. (2009). The decentralisation- centralisation dilemma: recruitment and distribution of health workers in remote districts of Tanzania. BMC International Health and Human Rights, 9, 9. https://doi.org/10.1186/1472-698x-9-9 Nkomazana, O., Mash, R., Shaibu, S., & Phaladze, N. (2015). Stakeholders' perceptions on shortage of healthcare workers in primary healthcare in Botswana: Focus group discussions. PLoS One, 10(8), e0135846. https://doi.org/10.1371/journal.pone.0135846 Okeke, E., Glick, P., Chari, A., Abubakar, I. S., Pitchforth, E., Exley, J., Bashir, U., Gu, K., & Onwujekwe, O. (2016). The effect of increasing the supply of skilled health providers on pregnancy and birth outcomes: Evidence from the midwives service scheme in Nigeria. BMC Health Services Research, 16, 425. https://doi.org/10.1186/s12913-016-1688-8 Robyn, P. J., Shroff, Z., Zang, O. R., Kingue, S., Djienouassi, S., Kouontchou, C., & Sorgho, G. (2015). Addressing health workforce distribution concerns: A discrete choice experiment Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 199 to develop rural retention strategies in Cameroon. International Journal of Health Policy and Management, 4(3), 169-180. https://doi.org/10.15171/ijhpm.2015.27 Merriam-Webster Dictionary. (2021). Rural. In Merriam-Webster.com. Retrieved July 16, 2021, from https://www.merriam-webster.com/dictionary/rural Serneels, P., Lindelow, M., Montalvo, J. G., & Barr, A. (2007). For public service or money: Understanding geographical imbalances in the health workforce. Health Policy and Planning, 22(3), 128-138. https://doi.org/10.1093/heapol/czm005 Sirili, N., Frumence, G., Kiwara, A., Mwangu, M., Goicolea, I., & Hurtig, A. K. (2019). "Doctors ready to be posted are jobless on the street..." the deployment process and shortage of doctors in Tanzania. Human Resources for Health, 17(1), 11. https://doi.org/10.1186/s12960-019-0346-8 Song, S., Ma, X., Zhang, L., Yuan, B., & Meng, Q. (2018). Precision targeting for more equitable distribution of health professionals in rural China. Health Policy and Planning, 33(7), 821- 827. https://doi.org/10.1093/heapol/czy061 Thoresen, S. H., & Fielding, A. (2010). Inequitable distribution of human resources for health: Perceptions among Thai healthcare professionals. Quality in Primary Care, 18(1), 49-56. Vujicic, M., Shengelia, B., Alfano, M., & Thu, H. B. (2011). Physician shortages in rural Vietnam: Using a labor market approach to inform policy. Social Science & Medicine, 73(7), 970- 977. https://doi.org/10.1016/j.socscimed.2011.06.010 World Bank. (2019). World Bank Country and Lending Groups. In. World Health Organization. (2010). Increasing access to health workers in remote and rural areas through improved retention: Global policy recommendations. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 200 World Health Organization. (2015). WHO global disability action plan 2014-2021: Better health for all people with disability. World Health Organization. World Health Organization. (2016). Global strategy on human resources for health: workforce 2030. Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 201 Appendix A Search Strategy for PubMed: Middle/Lo w Income Country ("deprived countries" OR "deprived country") OR "deprived nation") OR "deprived nations") OR "deprived population") OR "deprived populations") OR "deprived world") OR "developing countries") OR "developing country") OR "developing economies") OR "developing economy") OR "developing nation") OR "developing nations") OR "developing population") OR "developing populations") OR "developing world") OR "lami countries") OR "lami country") OR "less developed countries") OR "less developed country") OR "less developed economies") OR "less developed economy") OR "less developed nation") OR "less developed nations") OR "less developed population") OR "less developed populations") OR "less developed world") OR "lesser developed countries") OR "lesser developed country") OR "lesser developed economies") OR "lesser developed economy") OR "lesser developed nation") OR "lesser developed nations") OR "lesser developed population") OR "lesser developed populations") OR "lesser developed world") OR "lmis") OR "lmis") OR "low gdp") OR "low gnp") OR "low gross domestic") OR "low gross national") OR "low income countries") OR "low income country") OR "low income economies") OR "low income economy") OR "low income nation") OR "low income nations") OR "low income population") OR "low income populations") OR "lower gdp") OR "lower gnp") OR "lower gross domestic") OR "lower gross national") OR "lower income countries") OR "lower income country") OR "lower income economies") OR "lower income economy") OR "lower income nation") OR "lower income nations") OR "lower income population") OR "lower income populations") OR "middle income countries") OR "middle income country") OR "middle income economies") OR "middle income economy") OR "middle income nation") OR "middle income nations") OR "middle income population") OR "middle income populations") OR "poor countries") OR "poor country") OR "Poor Economies") OR "Poor Economy") OR "poor nation") OR "poor nations") OR "poor population") OR "poor populations") OR "poor world") OR "poorer countries") OR "poorer country") OR "Poorer Economies") OR "Poorer Economy") OR "poorer nation") OR "poorer nations") OR "poorer population") OR "poorer populations") OR "poorer world") OR "third world") OR "transitional countries") OR "transitional country") OR "Transitional Economies") OR "Transitional Economy") OR "under developed countries") OR "under developed country") OR "under developed economies") OR "under developed economy") OR "under developed nation") OR "under developed nations") OR "under developed population") OR "under developed populations") OR "under developed world") OR "under served countries") OR "under served country") OR Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 202 "under served nation") OR "under served nations") OR "under served population") OR "under served populations") OR "under served world") OR "underdeveloped countries") OR "underdeveloped country") OR "underdeveloped economies") OR "underdeveloped economy") OR "underdeveloped nation") OR "underdeveloped nations") OR "underdeveloped population") OR "underdeveloped populations") OR "underdeveloped world") OR "underserved countries") OR "underserved country") OR "underserved nation") OR "underserved nations") OR "underserved population") OR "underserved populations") OR "underserved world") List of All Middle/Lo w Income Countries ((Afghanistan OR Albania OR Algeria OR ñAmerican Samoaî OR Angola OR Armenia OR Azerbaijan OR Bangladesh OR Belarus OR Byelarus OR Belorussia OR Belize OR Benin OR Bhutan OR Bolivia OR Bosnia OR Botswana OR Brazil OR Bulgaria OR Burma OR ñBurkina Fasoî OR Burundi OR ñCabo Verdeî OR ñCape verdeî OR Cambodia OR Cameroon OR ñCentral African Republicî OR Chad OR China OR Colombia OR Comoros OR Comores OR Comoro OR Congo OR ñCosta Ricaî OR ñCªte d'Ivoireî OR Cuba OR Djibouti OR Dominica OR ñDominican Republicî OR Ecuador OR Egypt OR ñEl Salvadorî OR Eritrea OR Ethiopia OR Fiji OR Gabon OR Gambia OR Gaza OR ñGeorgia Republicî OR Georgian OR Ghana OR Grenada OR Grenadines OR Guatemala OR Guinea OR ñGuinea Bisauî OR Guyana OR Haiti OR Herzegovina OR Hercegovina OR Honduras OR India OR Indonesia OR Iran OR Iraq OR Jamaica OR Jordan OR Kazakhstan OR Kenya OR Kiribati OR Korea OR Kosovo OR Kyrgyz OR Kirghizia OR Kirghiz OR Kirgizstan OR Kyrgyzstan OR ñLao PDRî OR Laos OR Lebanon OR Lesotho OR Liberia OR Libya OR Macedonia OR Madagascar OR Malawi OR Malay OR Malaya OR Malaysia OR Maldives OR Mali OR ñMarshall Islandsî OR Mauritania OR Mauritius OR Mexico OR Micronesia OR Moldova OR Mongolia OR Montenegro OR Morocco OR Mozambique OR Myanmar OR Namibia OR Nepal OR Nicaragua OR Niger OR Nigeria OR Pakistan OR Palau OR Panama OR ñPapua New Guineaî OR Paraguay OR Peru OR Philippines OR Phillippines OR Philipines OR Phillipines OR Principe OR Romania OR Rwanda OR Ruanda OR Samoa OR ñSao Tomeî OR Senegal OR Serbia OR ñSierra Leoneî OR ñSolomon Islandsî OR Somalia OR ñSouth Africaî OR ñSouth Sudanî OR ñSri Lankaî OR ñSt Luciaî OR ñSt Vincentî OR Sudan OR Suriham OR Suriname OR Swaziland OR Syria OR ñSyrian Arab Republicî OR Tajikistan OR Tadzhikistan OR Tadjikistan OR Tadzhik OR Tanzania OR Thailand OR Timor OR Togo OR Tonga OR Tunisia OR Turkey OR Turkmen OR Turkmenistan OR Tuvalu OR Uganda OR Ukraine OR Uzbek OR Uzbekistan OR Vanuatu OR Vietnam OR ñWest Bankî OR Yemen OR Zambia OR Zimbabwe)) Distributio n TITLE: ((Distribution OR balance OR imbalance OR shortage* OR inequality OR disparit* OR supply*)) Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 203 Health Care Worker (("Health workers" OR "health workforce" OR "human resources" OR "labor supply" OR "healthcare delivery" OR "healthcare professionals" OR "physician supply" OR "labor market")) *modified appropriately for other databases Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 204 Appendix B Quality Assessment of Included Studies by the Mixed Methods Appraisal Tool (Hong et al., 2018) Screening Questions Quantitative non-randomized Citation Are there clear researc h questio ns? Do the collecte d data allow to address the researc h questio ns? Are the participant s represent ative of the target population ? Are the measurem ents appropriat e regarding both the outcome and the interventio n (or exposure) ? Are there compl ete outco me data? Are the confoun ders accounte d for in the design and analysis ? During the study period, is the interventi on administ ered (or exposure occurred) as intended ? Figueire do et al (Figueir edo et al., 2019) yes yes yes yes yes yes yes Hesketh et al (Hesket h et al., 2017a) yes yes yes yes yes yes yes Okeke et al (Okeke et al., 2016) yes yes yes yes yes yes yes Quantitative Descriptive Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 205 Citation Are there clear research question s? Do the collected data allow to address the research questions ? Is the sample strategy relevant to address the research question? Is the sample represen tative of the target populati on? Are the measu rement s approp riate? Is the risk of nonrespon se bias low? Is the statistical analysis appropriate to answer the research question? Gross et al (Gross et al., 2010) yes yes yes yes yes yes yes Lagard e et al (Lagard e et al., 2012) yes yes yes yes yes yes yes Leon et al (Leon & Kolstad , 2010) yes yes yes yes yes yes yes Meliala et al (Meliala et al., 2013) yes yes yes yes yes yes yes Robyn et al (Robyn et al., 2015) yes yes yes yes yes yes yes Serneel s et al (Sernee ls et al., 2007) yes yes yes yes yes yes yes Song et al (Song et al., 2018) Unclear intervent ion Unclear yes yes yes yes yes Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 206 Thores en et al (Thores en & Fielding , 2010) yes yes yes yes yes yes yes Vujicic et al (Vujicic et al., 2011) yes yes yes yes yes yes yes Qualitative Citation Are there clear researc h questio ns? Do the collecte d data allow to address the researc h questio ns? Is the qualitati ve approac h appropri ate to answer the researc h question ? Are the qualitat ive data collecti on method s adequa te to addres s the researc h questio n? Are the findings adequat ely derived from the data? Is the interpreta tion of results sufficientl y sustained by the data? Is there coherence between qualitative data sources, collection, analysis and interpretati on? Meliala et al (Meliala et al., 2013) yes yes yes yes yes yes yes Munga et al (Munga et al., 2009) yes yes yes yes yes yes yes Kiwanuk a et al (Kiwanuk a et al., 2017) yes yes yes yes yes yes yes Online Journal of Rural Nursing and Health Care, 21(2) https://doi.org/10.14574/ojrnhc.v21i2.666 207 Nkomaza na et al (Nkomaz ana et al., 2015) yes yes yes yes yes yes yes Sirili et al (Sirili et al., 2019) yes yes yes yes yes yes yes Thoresen et al (Thorese n & Fielding, 2010) yes yes yes yes yes yes yes