IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 1 Engaging patients in the HIV care continuum through referral-making behaviours and patterns: A descriptive cross-sectional study Susan S. Witte1, Rogério M. Pinto2, Prema Filippone1, C. Jean Choi3, Melanie Wall3 1School of Social Work, Columbia University, New York, United States of America, 2School of Social Work, University of Michigan, Detroit, United States of America, 3Department of Psychiatry, Columbia University, New York, United States of America Corresponding author: R. M. Pinto (ropinto@umich.edu) ABSTRACT Introduction: HIV continuum of care consists of five steps needed to effectively treat and prevent the spread of HIV. Linkage to and retention of patients to this Continuum of Care is a global priority. However, the COVID-19 pandemic has impacted the quality of this Continuum, as people living with HIV, have had to shelter reducing their access to services. As well, HIV agencies have had to close, reduce hours, and shift personnel. Purpose and Methods: The purpose of this descriptive cross-sectional study was to examine the person-centered referral-making behaviors and patterns used by providers to engage patients in the care continuum. Three classes of linkage behaviors among 285 providers in 34 community agencies in New York City were identified using latent class analysis. Results: These linkage behaviors include High (48%); Moderate (34%); and Low (18%). Both High and Moderate consisted of a blend of active and passive strategies and tracking systems. The High included more active strategies such as escorting patients to appointments. Linkage class membership was significantly associated with frequency of linkages to primary care (p=.020). COVID-19 disruptions demonstrate how the Care Continuum has been undermined by insufficient organizational resources. Conclusion: Findings suggest, addresses gaps in linkages should enhance the overall Continuum of Care provided to individuals diagnosed and living with HIV. KEYWORDS HIV care continuum; HIV testing; HIV primary care; Pre-exposure prophylaxis, Person-centered linkage to care FUNDING SOURCE This work was supported by the United States National Institute of Mental Health (“Project ICI”: R01MH095676) INTRODUCTION The HIV continuum of care (“care continuum”) consists of five steps needed to effectively treat and prevent the spread of HIV: (1) HIV testing; (2) linkage to HIV primary care; (3) engagement and retention in HIV care; (4) treatment with antiretroviral therapy (ART); leading to (5) HIV viral suppression (United States Health Resources & Service Administration., October 2016). The continuum includes patients who have been (1) diagnosed with HIV infection and linked to care (visited a care provider 30-90 days after HIV positive diagnosis); (2) engaged or retained in care (received medical care for HIV infection); and (3) virally suppressed (“viral load” is at a very low level) (Centers for Disease Control and Prevention, 2018). Epidemiologically, the continuum represents a constant movement of patients – entering, re- entering, and exiting care (Gill & Krentz, 2009) – and different patterns of retention among specific populations over time (Rebeiro et al., 2013). In 2010 the National HIV/AIDS Strategy (NHAS) set a linkage to care (LTC) rate goal of 85% within 90 days of HIV testing (Centers for Disease Control and Prevention, 2015). Success of the continuum to achieve ultimate 90-90-90 goals by 2030 (Center for Disease Control and Prevention, 2019), requires achieving viral suppression is thought to be IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 2 influenced by the timing of engaging in HIV-related services, where “late enrollment” or delays in linking individuals to post-testing services, may lead to poorer outcomes over time (MacCarthy et al., 2015). Hence providers need to implement successful linkages – led by referral behaviors – to help patients access HIV testing, followed by referral to primary care services. The purpose of this descriptive cross-sectional study was to examine the person-centered referral- making behaviors and patterns (active, passive, follow-up, and tracking) used by providers to engage patients in the care continuum. CONCEPTUAL UNDERPINNING Person-centered referral-making The movement of patients across continuum stages is influenced by a diverse workforce of providers of social and public health services – social workers, health educators, care navigators, and others – providing psychosocial services in primary care, outpatient, and prevention settings. In their day-to- day practices, these workers make referrals and then establish and track linkages to care and help to retain patients in each of the five steps described above – e.g., by providing snacks during visits, transportation, and others. “Linkage to care continuum services” is a widespread practice best characterized as providers making referrals to services, offering psychoeducation information about Pre-Exposure Prophylaxis (PrEP), and emotional and cognitive support to reduce delays in engagement and retention in care (Cook, Lutz, Young, Hall, & Stacciarini, 2015; Philbin et al., 2016; Sullivan et al., 2015). The literature does not have an agreed upon empirical definition of “referral” or “linkage” nor do organizations and practitioners abide by a common definition. We used the term “referral making” – phoning, emailing, or walking the patient to meet another provider who can provide that service – as it is used in practice by social and public health service providers in their attempts at linking patients to services (e.g., HIV testing) (Rahman, R., Pinto, R. M., & Troost, J., 2021). The term “referral-making” resonates and reflects our experiences as practitioners and the expertise of our Interagency Community Collaborative Board (ICCB), described below, whose members make referrals in their day-to-day work (or who supervise those who do) (Pinto, Spector, Rahman, & Gastolomendo, 2015; Pinto, Spector, & Valera, 2011). Referrals include various behavioral patterns, including tracking and follow- up. The literature on person-centered care suggests that a person-centered referral focuses on the specific preferences, needs, and values of each individual patient (Oates, Weston, & Jordan, 2000; Park, 2020; Plsek, 2001; Queen, Crone, & Parker, 2015). Emerging evidence suggests that person-centered care may facilitate access to PrEP, HIV testing, and primary care (Campbell, Lippman, Moss, & Lightfoot, 2018; Fuster & Gelberg, 2019; Garland et al., 2011; Labhardt et al., 2018; R. M. Pinto, S. S. Witte, P. Filippone, C. J. Choi, & M. Wall, 2018a; Seth, Figueroa, Wang, Reid, & Belcher, 2015). In active person-centered referral-making, providers would connect patients directly with another service provider (e.g., primary HIV care physician) and subsequently follow up with that provider and/or the patient. In passive referrals, arguably less person- centered, providers typically give all patients the responsibility of following up with referrals, for example, by offering information only, and then expecting the patient to make the call for an appointment. Research suggests passive referral- making behaviors and linkages to services are customary, despite research showing that more active and patient-centered behaviors in the part of the practitioner may be more likely to help patients engage and remain in care (Garland et al., 2011). Likewise, comprehensive case management (including psychoeducation) and direct outreach to care providers can improve engagement and retention (Aziz & Smith, 2011; Bauman et al., 2013). Referrals are meant to connect (in the shortest time possible) newly diagnosed individuals to HIV primary care. However, significant barriers exist that include individuals experiencing stigma, distrust of providers, and lack of information about linkages at the time of diagnosis are more likely to delay or forgo engagement (Cook et al., 2015; Philbin et al., 2016). Once connected to care, patients may encounter additional barriers, such as inadequate health insurance, difficulty accessing medical providers, transportation to care settings, and immigration status (Aziz & Smith, 2011; Bauman et al., 2013; Dombrowski, Simoni, Katz, & Golden, 2015; Krakower, Ware, Mitty, Maloney, & Mayer, 2014; Remien et al., 2015). Referrals can be further delayed for individuals facing psychosocial issues, including depression, substance use disorders, and poverty (Bhatia, Hartman, Kallen, Graham, & Giordano, 2011; del Rio & Mayer, 2013; Moore, 2011; Remien et al., 2015). Given these many concerns, frontline workers are not always able to consistently offer substantive psychoeducation or make linkages to care and follow IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 3 up adequately (Pinto et al., 2015; Pinto, Spector, et al., 2018; Remien et al., 2015). This gave rise to the concept of person-centered care, ensuring more attention to a patient’s unique circumstances, and emphasizing the need for flexibility in cognitive and skill-based responses in referral and linkage making, including a larger set of referral behaviors to respond more adequately to each patient. Whether or not referrals achieve completion can make a difference on the impact of referral efforts (Brodkin, 2011; Lipsky, 2010) Providers’ active referral-making (including coordination and tracking efforts) can facilitate service users’ timely access to needed services and reduce waste of organizational resources (e.g., staff hours, social capital, backlogs) (Mehta et al., 2006). But despite the growing emphasis on person-centered care, few empirical studies have investigated the implications of person- centered orientation in organizations offering HIV services (Beach & Inui, 2006; Plsek, 2001). Without organizational supports for incorporating service users’ perspectives into care processes can easily become nominal and tokenized routines with little influence on service user outcomes (Park et al., 2020). Facilitators and barriers to care continuum linkages Most individuals go through a crisis period following their HIV diagnosis. To help newly diagnosed persons to cope and to engage and stay in care, more active linkages, including increased interprofessional collaboration (e.g., between providers of social and public health services and primary care physicians) is recommended. But active referral making leading to linkages requires a specific set of behaviors not always undertaken by providers. For example, the literature shows that successful linkages, those leading patients to accessing and staying in care, are often performed by frontline workers who perceive team work positively (Bauman et al., 2013; Kim et al., 2014) and who work at agencies where they have had exposure to evidence based HIV prevention (R. M. Pinto, S. S. Witte, P. L. Filippone, C. J. Choi, & M. Wall, 2018b). Our team found that interprofessional collaboration (IPC) and recent referral training were associated with higher care continuum engagement – higher rates of HIV testing and HIV primary care linkages and more frequent PrEP psychoeducation (Pinto, Witte, Filippone, et al., 2018a). A recent systematic review of best practices for increasing patient linkage to, retention and reengagement in HIV medical care found that only 3 out of 10 identified evidence informed best practices for engaging HIV positive individuals in care, focused on provider referral or linkage behaviors (Higa, Crepaz, & Mullins, 2016). Anti-Retroviral Treatment and Access to Services (ARTAS), for example, is one widely implemented in the United States. To examine HIV provider behavior and inform improved approaches to more person-centered referral interventions, we empirically examined specific combinations of behaviors and patterns on referrals to the care continuum services. Are there combinations of provider behaviors associated with a higher frequency of referral? METHODS Data for the current study emerged from a longitudinal project titled: Implementation Collaboration for Implementation (“Project ICI”: R01MH095676). Project ICI examined providers’ implementation of HIV services in primary care, outpatient treatment, and prevention programs in NYC. Project ICI was conceived and conducted in partnership with stakeholders and guided by an Interagency Collaborative Community Board (ICCB), from establishing study aims to developing and piloting survey questions to collecting and analyzing data (Pinto et al., 2015; Pinto et al., 2011). Project ICI consisted of a cross-sectional design in which survey data was collected from 379 providers across 36 agencies in 2013-2014. Twelve months later, 293 providers (77% retention rate) completed a similar survey with expanded questions, used in the current study, about provider services (e.g., PrEP psychoeducation). Two pairs of agencies merged between baseline and 12-months follow-up resulting in 34 agencies. Most loss-to-follow-up was due to high job loss. Eight participants were excluded because they were no longer a service provider, resulting in n=285 providers with survey data reflecting 34 agencies for the current study. ICI was approved by the appropriate Institutional Review Boards. Procedures Sample selection and methods are described here in brief. For full details on all ICI methods and procedures, please see Pinto et al. (Pinto, Witte, Wall, & Filippone, 2018). Sample size was determined by means of power analysis concerning the longitudinal study, explained above. Agency recruitment IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 4 All agencies were funded by the NYC Department of Health and/or the CDC to provide HIV-related services. We recruited from a list provided by the NYC Department of Health and the CDC of over 100 agencies. Study staff contacted agency representatives by phone and outlined study procedures and staff inclusion criteria. Nine agencies were in Manhattan, eight in Brooklyn, four in Queens, three in the Bronx, and nine had sites in two or more boroughs. Agencies received a computer (valued at $1,000) as an incentive to participate. Provider recruitment To be included, a provider was required to offer HIV services and/or make linkages to HIV services. There were no exclusion criteria. The average number of providers per agency was 10 (ranging from 2 to 25), representing from 100% (small agencies) to 10% (large agencies) of those providers eligible to participate. Providers received $30 gift cards upon completion of the survey we used for this study. Data collection Project staff implemented computer-assisted face- to-face interviews. Notebook computers that contained password-protected survey software powered by DATSTAT Illume 6.0 were used. All data were stored in computers to which only relevant personnel had access. Provider interviews lasted 45- 60 minutes. Meanwhile, agency leaders took a short Organizational Survey about their agencies (15-20 minutes). Participants read and signed informed consent prior to interviews. Measures Engagement in the HIV Continuum of Care Providers were encouraged to link patients to HIV testing to find out their HIV status. Those who test positive were then linked to primary care. Those who test negative, but who were likely to be exposed to HIV, were provided psychosocial education about PrEP. These variables were operationalized in collaboration with ICCB providers, based on how providers typically talk about and record referrals, assuming that, in their day-to-day practice, providers use myriad strategies for linking patients. Linkage to HIV testing and Linkage to primary care. The following question was asked: How many patients did you link to primary care/HIV testing within the past 6 months? (More than 20 patients; 16- 20; 11-15; 5-10; fewer than five patients). Pre-exposure prophylaxis. PrEP use was assessed by asking the following: “In the past 6 months, how often have you given information or educated patients about PrEP? (Several times per week; about once per week; about once per month; less than once per month; have not educated in past six months).” These questions were followed by abbreviated definitions of PrEP from the CDC website. Referral patterns Participants were asked to “... share with us how you make referrals to patients.” A list of both passive and active types of linkage-making behaviors was provided to participants, who were asked to check all responses that applied. Examples of active behaviors identified on the list included escorting patients to services; calling or emailing other providers while the patient was in the office; asking the patient to call or email provider. Passive behaviors identified included providing contact card; calling or emailing the other provider after the patient left the provider’s office; offering patient information about the services and/or the provider (brochures/pamphlets/Website). The type and extent to which providers followed up with and/or tracked referrals were also measured. For follow up, two questions were asked and included: “I follow up by contacting the provider to whom I made the referral” and “I follow up by asking the patient the outcome of the referral” (always, usually, sometimes, rarely, and never. To assess tracking, the following question was asked: “Do you have a system to track the referrals you make to other agencies?” – yes/no. Caseload The following question was asked, “Please tell us, on average, how many patients you provide services to each week (individually or in groups)? Responses included, <30, 31-50, and >50. Demographics Age was measured in years. Ethnicity included Latino/Hispanic or non- Latino/Hispanic. Race included White, African American, “more than one race identified by the participant,” and a grouping of Asian, Native Hawaiian, Alaskan Native, and American Indian or Native American. Gender was IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 5 categorized as male or female. Education included high school, associate degree, bachelor’s degree(s), master’s degree(s), and PhD(s). Work positions included supervisor, counselor, case manager, navigator, educator/outreach, program administrator, and other. Licensures/certifications Licensure was measured by providing a list (alcohol/drug counselor, nurse, physician, psychologist, mental health counselor, social worker, other) where participants could check all that applied. This variable was dichotomized into participants who held at least one license compared to no licenses. Data Analysis Latent class analysis (LCA) was used to identify distinct referral patterns in service providers based on the nine items related to person-centered referral- making, including follow-up behaviors and whether providers used a tracking system. This did not include provider's own personal tracking system. The best fitting number of classes was determined based on Nylund et al. (Nylund, Asparouhov, & Muthén, 2008). The highest posterior probability of belonging to each class was used to assign providers to a mostly likely class. Proportions of each item conditional on the most likely class membership were calculated and plotted to facilitate interpretation and naming of the classes. Comparisons between class membership and demographic characteristics and HIV Continuum outcomes were performed using chi-square tests for categorical measures and analyses of variance for continuous measures. Further pairwise comparisons were performed for significant associations. LCA analyses were fit using MPlus (Version 7.4) and all other analyses were performed using SAS (Version 9.4). RESULTS Agency and provider samples All agencies were non-profit organizations providing medical services (e.g., HIV testing and care) and/or social services (e.g., HIV counseling, workshops, homeless shelter). Of the 34 agencies 20 (59%) had budgets below $10 million. Eight (24%) employed more than 100 providers. Table 1 summarizes the demographic characteristics of the final sample of providers used in the analyses. The average age was 43 (standard deviation (SD) = 12). Most participants were female (63%), were Black or African American (54%), and held a Bachelor’s degree or higher (66%). Providers identified their work roles as: case managers (19%), counselors (e.g., social workers, 17%), education/outreach (15%), supervisors (e.g., of counselors, case managers, etc.; 24%); program administrators (14%), health navigators (5%), and other (7%). More than half (57%) did not hold a professional license of any kind. Fifty-eight percent of providers reported serving fewer than 30 patients each week. Sixty-one percent of providers reported receiving formal training (curriculum-based training) in HIV prevention. Referral Patterns Latent class models were fit for 2 to 4 classes based on the nine items related to referral-making, including follow-up behaviors and whether providers reported using a tracking system. The final best class model was determined to be three classes (BIC = 4019.77, Entropy = 0.730). Fit indices worsened when the 4-class model was fit (Nylund, Asparouhov, & Muthén, 2008). Figure 1 shows the observed frequencies and proportions of the nine linkage items used to form the three referral pattern classes. Providers in the first latent class (“High”, 48%) are high in both active and passive referral styles (e.g., calling the service provider while the patient is still in the office, offering the patient information and handing the patient a card with the referral information on it), as well as following up by asking the patient the outcome of the referral and using a tracking system. Providers in the second latent class (“Moderate”, 34%) are characterized with a moderate passive referral style; approximately 40 to 50% performed passive referrals and follow up procedures. Over half of providers in both the High and Moderate latent classes use tracking systems. The majority of providers in the third latent class (“Low”, 18%) do not perform active or passive referral behaviors and do not follow-up with their patients. Approximately half of the providers in the Low group do not use a tracking system. Forty-seven percent of these providers only offer information to patients, e.g., brochure, pamphlet, informational website. Comparisons between the referral pattern classes and demographic characteristics and HIV continuum outcomes are also displayed in Tables 1 and 2. In Table 1, provider age (p=.004) and work position (p=.006) were significantly associated with the IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 6 referral pattern classes. Further pairwise comparisons show that providers in the Moderate pattern class were significantly older than providers in the High (p=.01) and Low (p=.002) pattern classes, with no significant difference (p=.26) between High and Moderate pattern classes. Compared to the Low pattern class (2%), providers were significantly more likely to be case managers if they were in the Moderate (26%; p<.001) or High (20%; p=.002) pattern classes. Providers in the Low pattern class were more likely to be: educators/outreach (24%; p=.032) compared to the Moderate pattern class (10%); program administrators (24%; p=.029) compared to the High pattern class (11%); and identify as “Other” (16%) when compared to either the Moderate (5%; p=.031) or High pattern class (5%; p=.017). There was no significant difference between referral pattern class and identifying as a supervisor, counselor or navigator. In Table 2, referral pattern class membership was significantly associated with referral frequency to primary care services (p=.02). Thirty-seven percent of providers in the High pattern class linked more than 10 patients to primary care in the past six months compared to only 24% of providers in the Moderate (p=.038) pattern class and 16% in the Low (p=.006) pattern class. DISCUSSION We identified three distinct classes or groups of person-centered referral-making behaviors among a large cohort of social and public health service providers in New York City, an epicenter of the HIV epidemic (Auerbach, C., & Beckerman, N. L., 2010). We found that these providers use combinations of active and passive behaviors, essentially reflecting what they already know: “more is better,” but also that more active behaviors reflect a person-centered approach to referral making. In other words, more combinations of both active and passive behaviors yield more reported referrals and improved association with referrals to care continuum services. Providers in the High referral pattern class were found to make more referrals to primary care, the most important first step to continuum engagement. This class of High referral makers reflects high levels of combined active and passive behaviors and reflect having access to tracking and follow up systems and behaviors. Because we know that linkage to primary care is the critical first step to the continuum, we recommend that High group behaviors be incorporated into trainings for all providers and examined for their efficiency to ensure receipt of antiretroviral medications (ARVs) for HIV positive individuals. Figure 1 shows that Low and High referral makers shared many of the same characteristics, suggesting a potential standardization of practice behaviors that might be modified by training providers in the science and the art of more person-centered referral-making. On the other hand, Moderate referral makers appeared to be more selective in their efforts, indicated by the way they diverge from the other two—combining more passive behaviors with tracking and the highest follow up compared to the High or Low referral makers. This may reflect the fact that more of those providers were case managers, and may already engage in a person-centered approach, as discussed further below. Age and work position were characteristics significantly associated with Low, Moderate or High referral maker class membership. Age may confer maturity, comfort engaging in referral-making, increased training opportunities, time in the field or practice wisdom, all of which would be expected to increase incorporation of a variety of referral-making behaviors. However, providers in the Moderate class were the oldest. Findings related to work position may simply reflect what we know about provider roles: case managers are those most closely associated with making linkages to care (Gilman, Hidalgo, Thomas, Au, & Hargreaves, 2012) while administrators and educators spend a much smaller proportion of time, if any, engaged in active or passive referral making. Further, it may reflect the state of the field of referral and linkage interventions, as follows. In one of three available intervention studies, Gardner et al (Gardner et al., 2005) found that individuals in a strengths-based case-management intervention were significantly more likely to have attended care compared to those receiving a standard referral approach. This suggests that case managers, who often work from a systematic checklist of anticipated referral behaviors, may be more successful at linking HIV positive individuals to primary care. Despite lack of data on actual patient engagement in primary care, our findings are consistent with case management outcome studies reinforcing that a case- management approach may help patients identify their own internal strengths and assets, which in turn should empower them to find the services they need (Brennan, Browne, & Horgan, 2014; Gardner et al., 2005; Higa et al., 2016). Our team found that interprofessional collaboration (IPC) and recent linkage training among were associated with higher rates of linkages to HIV testing and HIV primary care and more IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 7 frequent PrEP psychoeducation (Pinto, Witte, Filippone, et al., 2018a). We also found that providers exposed to more evidence-based HIV prevention programs (e.g. DEBIS) were more likely to refer patients to primary care, which may explain why non- case managers may also have moderate or high referral behaviors (Pinto, Witte, Filippone, et al., 2018b). Our findings are reflective of three of the four core intervention components identified and described by Gilman et al (Gilman et al., 2012): directly employed referral workers, active referral to medical care, person-centered referral case management and cultural and linguistic concordance. Consistent with Gilman, we recommend that agencies consider identifying workers and tasking them to specifically engage in person-centered referral- making and emphasizing training in active referral behaviors and person-centered case management. We propose more training for case managers and other providers, with attention to role flexibility and clarification of service outcomes as they relate to linkage making. HIV services agencies should examine who on staff is best suited to be tasked with specific referral- making outcomes (Gilman et al., 2012). This study’s findings do not tell us about individual patient level factors that may influence retention in care; and there may be barriers to referral-making within each of the classes we identified. For instance, providers may be aware of a particular patient’s inability to pay for services, lack of transportation, or other concern related to interpersonal access or ability to follow through. Some of these factors might influence choice of active or passive referral behaviors. Future research should incorporate qualitative case studies that examine some of the individual behavior level barriers and how they influence successful engagement. Findings raise questions about provider role flexibility and systems access. Are some providers unable to access a tracking system, which might strengthen their ability to make linkages? Tracking systems are more typically built into organizational structure and data collection mechanisms, often mandatory, whereas “follow up” may be differently defined from provider to provider, or agency to agency, and considered a softer expectation. Finally, our work focuses here on only initial engagement, but re-engagement is becoming increasingly as compelling and necessary given the number of individuals who are not retained in the continuum after their first engagement (Higa et al., 2016; MacCarthy et al., 2015). Interventions to improve linkage and retention in HIV care underscore the importance of strengths-based case management in conjunction with other evidence- informed strategies such as peer navigation, experiencing higher level of engagement from medical providers, and clinics having closer contact with patients (Higa et al., 2016; Thompson et al., 2012). Operational strategies that take into account provider and staff training to improve interactions with patients and reduce role confusion, integrative and collocated services particularly medical care, mental health services and substance use programs, agency-wide buy-in, and the employment of community-based staff all help to increase engagement into HIV care and make an more long- term positive impact on HIV care (Krakower et al., 2014; Ma, Chambers, Jenkins Hall, Tanner, & Piper, 2017). These findings may be useful in testing future approaches for re-engagement in the continuum as well. LIMITATIONS This study is limited by self-reported data and its cross-sectional design. Future research should use longitudinal data to highlight the long-term impact of linkage training and interprofessional collaboration in provider engagement in the care continuum. We were unable to use longitudinal data because we did not collect PrEP-related questions until after the baseline had been completed. Future research should include qualitative methods to examine specific cases of particularly successful linkage strategies to clarify how best to adapt and replicate for other settings. Our data are limited in terms of how we measured frequency of linkages. The questions used to measure frequency of linkages assumes patient access to HIV testing and/or HIV primary care, but we do not have data to confirm that linkages occurred, which would offer quality assurance and allow for a better understanding of if/whether actions of High linkage making yielded better outcomes in terms of care. Self-reported main outcomes, based on six-month recall, may influence accuracy. Nonetheless, both recall and accuracy were helped by the fact that most providers (nearly 90%) track these linkages through tracking systems. IMPLICATIONS To achieve the Joint United Nations Programme on HIV/AIDS 90-90-90- goals, 90% of all individuals with HIV will be aware of their status; 90% with HIV infection will receive antiretroviral therapy; and 90% will receive ART and achieve viral suppression by 2030. To achieve these goals, the field must ensure success of the Continuum and importantly, early, and immediate linkage to primary care among newly IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 8 diagnosed individuals. This imperative is now exacerbated by the impact of the COVID-19 pandemic, as people living with HIV have had to shelter in place and distance themselves from others while during many months in 2020 HIV services agencies have had to close, reduce hours, and task shift personnel to primary care in COVID-19 hospital units (Pinto & Park, 2020). In terms of best practices, these COVID-19 disruptions allow us to see how the HIV care continuum has been undermined routinely by insufficient concrete and human organizational resources, and by failures to follow up and track provider referrals to HIV services. Study findings provide evidence of identifiable differences in approaches to referrals and linkage that may be improved upon to ensure and increase linkages once agencies “open back up.” Strategies used by High and Moderate linkage makers should be integrated into ongoing training across providers at HIV services agencies. Agencies should increase training to improve these specific skills and provide supportive supervision to reinforce their use and to sustain their implementation. Improved linkage to continuum services may ultimately ensure viral suppression and ability to achieve goals towards ending the epidemic. In terms of research, this moment points to the need to identify the gaps and limitations of past research (e.g., lack of large-scale qualitative evidence and limited involvement of clients and providers) so that researchers may address this problem of limited evidence on referral-making and linkage practices that could help clients access the HIV services to which they are referred (“referral completion”), and, ultimately, end the HIV pandemic within this decade. REFERENCES Auerbach, C., & Beckerman, N. L. (2010). HIV/AIDS prevention in New York City: identifying socio cultural needs of the community. Social work in health care, 49(2), 109–133. Aziz, M., & Smith, K. Y. (2011). Challenges and successes in linking HIV-infected women to care in the United States. Clinical infectious diseases, 52(suppl_2), S231-S237. Bauman, L. J., Braunstein, S., Calderon, Y., Chhabra, R., Cutler, B., Leider, J., . . . Watnick, D. (2013). Barriers and facilitators of linkage to HIV primary care in New York City. Journal of acquired immune deficiency syndromes (1999), 64(0 1), S20. Beach, M. C., & Inui, T. (2006). Relationship-centered care. Journal of general internal medicine, 21(1), 3-8. Bhatia, R., Hartman, C., Kallen, M. A., Graham, J., & Giordano, T. P. (2011). Persons newly diagnosed with HIV infection are at high risk for depression and poor linkage to care: results from the Steps Study. AIDS and Behavior, 15(6), 1161-1170. Brennan, A., Browne, J. P., & Horgan, M. (2014). A systematic review of health service interventions to improve linkage with or retention in HIV care. AIDS care, 26(7), 804- 812. Brodkin, E. Z. (2011). Policy work: Street-level organizations under new managerialism. Journal of Public Administration Research and Theory, 21(suppl_2), i253-i277. Campbell, C. K., Lippman, S. A., Moss, N., & Lightfoot, M. (2018). Strategies to increase HIV testing among MSM: a synthesis of the literature. AIDS and Behavior, 22(8), 2387- 2412. Center for Disease Control and Prevention. (2019). Health Equity. https://www-cdc- gov.ezproxy.cul.columbia.edu/chronicdiseas e/healthequity/index.htm. Centers for Disease Control and Prevention. (2015). The National HIV/AIDS Strategy. https://www.cdc.gov/hiv/policies/hip/stra tegy.html. Centers for Disease Control and Prevention. (2018). Understanding the HIV Care Continuum.; 2018:4. https://www.cdc.gov/hiv/pdf/library/facts heets/cdc-hiv-care-continuum.pdf. Cook, C. L., Lutz, B. J., Young, M.-E., Hall, A., & Stacciarini, J.-M. (2015). Perspectives of linkage to care among people diagnosed with HIV. Journal of the Association of Nurses in AIDS Care, 26(2), 110-126. del Rio, C., & Mayer, K. (2013). Editorial commentary: A tale of 2 realities: What are the challenges and solutions to improving engagement in HIV care? In: Oxford University Press. Dombrowski, J. C., Simoni, J. M., Katz, D. A., & Golden, M. R. (2015). Barriers to HIV care and treatment among participants in a public health HIV care relinkage program. AIDS patient care and STDs, 29(5), 279-287. Fuster, D., & Gelberg, L. (2019). Community screening, identification, and referral to primary care, for hepatitis C, B, and HIV among homeless persons in Los Angeles. https://www-cdc-gov.ezproxy.cul.columbia.edu/chronicdisease/healthequity/index.htm https://www-cdc-gov.ezproxy.cul.columbia.edu/chronicdisease/healthequity/index.htm https://www-cdc-gov.ezproxy.cul.columbia.edu/chronicdisease/healthequity/index.htm https://www.cdc.gov/hiv/policies/hip/strategy.html https://www.cdc.gov/hiv/policies/hip/strategy.html https://www.cdc.gov/hiv/pdf/library/factsheets/cdc-hiv-care-continuum.pdf https://www.cdc.gov/hiv/pdf/library/factsheets/cdc-hiv-care-continuum.pdf IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 9 Journal of Community Health, 44(6), 1044- 1054. Gardner, L. I., Metsch, L. R., Anderson-Mahoney, P., Loughlin, A. M., Del Rio, C., Strathdee, S., . . . Holmberg, S. D. (2005). Efficacy of a brief case management intervention to link recently diagnosed HIV-infected persons to care. Aids, 19(4), 423-431. Garland, P. M., Valverde, E. E., Fagan, J., Beer, L., Sanders, C., Hillman, D., . . . Bertolli, J. (2011). HIV counseling, testing and referral experiences of persons diagnosed with HIV who have never entered HIV medical care. AIDS Education and Prevention, 23(3_supplement), 117-127. Gill, M., & Krentz, H. (2009). Unappreciated epidemiology: the churn effect in a regional HIV care programme. International journal of STD & AIDS, 20(8), 540-544. Gilman, B., Hidalgo, J., Thomas, C., Au, M., & Hargreaves, M. (2012). Linkages to care for newly diagnosed individuals who test HIV positive in nonprimary care settings. AIDS patient care and STDs, 26(3), 132-140. Higa, D. H., Crepaz, N., & Mullins, M. M. (2016). Identifying best practices for increasing linkage to, retention, and re-engagement in HIV medical care: findings from a systematic review, 1996–2014. AIDS and Behavior, 20(5), 951-966. Horn, T., Sherwood, J., Remien, R. H., Nash, D., Auerbach, J. D., Group, T. A., & Group, F. f. A. R. H. P. C. W. (2016). Towards an integrated primary and secondary HIV prevention continuum for the United States: a cyclical process model. Journal of the International AIDS Society, 19(1), 21263. Kim, J. J., Maulsby, C., Kinsky, S., Riordan, M., Charles, V., Jain, K., & Holtgrave, D. R. (2014). The development and implementation of the national evaluation strategy of access to care, a multi-site linkage to care initiative in the United States. AIDS Education and Prevention, 26(5), 429-444. Krakower, D., Ware, N., Mitty, J. A., Maloney, K., & Mayer, K. H. (2014). HIV providers’ perceived barriers and facilitators to implementing pre-exposure prophylaxis in care settings: a qualitative study. AIDS and Behavior, 18(9), 1712-1721. Labhardt, N. D., Ringera, I., Lejone, T. I., Klimkait, T., Muhairwe, J., Amstutz, A., & Glass, T. R. (2018). Effect of offering same-day ART vs usual health facility referral during home- based HIV testing on linkage to care and viral suppression among adults with HIV in Lesotho: the CASCADE randomized clinical trial. Jama, 319(11), 1103-1112. Lipsky, M. (2010). Street-level bureaucracy: Dilemmas of the individual in public service: Russell Sage Foundation. Ma, A., Chambers, B. D., Jenkins Hall, W., Tanner, A. E., & Piper, C. N. (2017). Individual and structural factors influencing HIV care linkage and engagement: Perceived barriers and solutions among HIV-positive persons. Journal of HIV/AIDS & Social Services, 16(1), 34-42. MacCarthy, S., Hoffmann, M., Ferguson, L., Nunn, A., Irvin, R., Bangsberg, D., . . . Dourado, I. (2015). The HIV care cascade: models, measures and moving forward. Journal of the International AIDS Society, 18(1), 19395. Mehta, S. H., Lucas, G. M., Mirel, L. B., Torbenson, M., Higgins, Y., Moore, R. D., . . . Sulkowski, M. S. (2006). Limited effectiveness of antiviral treatment for hepatitis C in an urban HIV clinic. Aids, 20(18), 2361-2369. Moore, R. D. (2011). Epidemiology of HIV infection in the United States: implications for linkage to care. Clinical infectious diseases, 52(suppl_2), S208-S213. Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2008). " Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study": Erratum. Oates, J., Weston, W. W., & Jordan, J. (2000). The impact of patient-centered care on outcomes. Fam Pract, 49(9), 796-804. Park, S. (2020). Beyond patient-centred care: a conceptual framework of co-production mechanisms with vulnerable groups in health and social service settings. Public Management Review, 22(3), 452-474. Park, S., Grogan, C. M., Mosley, J. E., Humphreys, K., Pollack, H. A., & Friedmann, P. D. (2020). Correlates of patient-centered care practices at US substance use disorder clinics. Psychiatric services, 71(1), 35-42. Philbin, M. M., Tanner, A. E., DuVal, A., Ellen, J. M., Xu, J., Kapogiannis, B., . . . Interventions, A. T. N. f. H. A. (2016). HIV testing, care referral and linkage to care intervals affect time to engagement in care for newly diagnosed HIV-infected adolescents in fifteen adolescent medicine clinics in the United States. Journal of acquired immune deficiency syndromes (1999), 72(2), 222. Pinto, R. M., & Park, S. (2020). COVID-19 Pandemic Disrupts HIV Continuum of Care and IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 10 Prevention: Implications for Research and Practice Concerning Community-Based Organizations and Frontline Providers. AIDS and Behavior, 1. Pinto, R. M., Spector, A. Y., Rahman, R., & Gastolomendo, J. D. (2015). Research advisory board members' contributions and expectations in the USA. Health promotion international, 30(2), 328-338. Pinto, R. M., Spector, A. Y., & Valera, P. A. (2011). Exploring group dynamics for integrating scientific and experiential knowledge in Community Advisory Boards for HIV research. AIDS care, 23(8), 1006-1013. Pinto, R. M., Spector, A. Y., Witte, S. S., Filippone, P., Choi, C. J., & Wall, M. (2018). Training in Evidence-Based Practices Increases Likelihood to Integrate Different HIV Prevention Services with Substance-Using Clients. Social work in public health, 33(3), 202-214. Pinto, R. M., Witte, S. S., Filippone, P., Choi, C. J., & Wall, M. (2018a). Interprofessional collaboration and on-the-job training improve access to HIV testing, HIV primary care, and pre-exposure prophylaxis (PrEP). AIDS Education and Prevention, 30(6), 474- 489. Pinto, R. M., Witte, S. S., Filippone, P. L., Choi, C. J., & Wall, M. (2018b). Policy interventions shaping HIV prevention: Providers’ active role in the HIV continuum of care. Health Education & Behavior, 45(5), 714-722. Pinto, R. M., Witte, S. S., Wall, M. M., & Filippone, P. L. (2018). Recruiting and retaining service agencies and public health providers in longitudinal studies: Implications for community-engaged implementation research. Methodological innovations, 11(1), 2059799118770996. Plsek, P. (2001). Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC National Academies Pr. Queen, M., Crone, D., & Parker, A. (2015). Long-term engagement with a practice-based exercise referral scheme: Patients' perceptions of effectiveness. European Journal for Person Centered Healthcare, 3(3), 369-376. Rahman, R., Pinto, R. M., & Troost, J. (2021). Examining Interprofessional Collaboration Across Case Managers, Peer Educators, and Counselors in New York City. Social Work in Public Health, 448-459. Rebeiro, P., Althoff, K. N., Buchacz, K., Gill, M. J., Horberg, M., Krentz, H., . . . Gebo, K. A. (2013). Retention among North American HIV–infected persons in clinical care, 2000– 2008. Journal of acquired immune deficiency syndromes (1999), 62(3), 356. Remien, R. H., Bauman, L. J., Mantell, J., Tsoi, B., Lopez-Rios, J., Chhabra, R., . . . Teitelman, N. (2015). Barriers and facilitators to engagement of vulnerable populations in HIV primary care in New York City. Journal of acquired immune deficiency syndromes (1999), 69(0 1), S16. Seth, P., Figueroa, A., Wang, G., Reid, L., & Belcher, L. (2015). HIV testing, HIV positivity, and linkage and referral services in correctional facilities in the United States, 2009–2013. Sexually transmitted diseases, 42(11), 643. Sullivan, K. A., Schultz, K., Ramaiya, M., Berger, M., Parnell, H., & Quinlivan, E. B. (2015). Experiences of women of color with a nurse patient navigation program for linkage and engagement in HIV care. AIDS patient care and STDs, 29(S1), S49-S54. Thompson, M. A., Mugavero, M. J., Amico, K. R., Cargill, V. A., Chang, L. W., Gross, R., . . . Bartlett, J. G. (2012). Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an International Association of Physicians in AIDS Care panel. Annals of internal medicine, 156(11), 817-833. United States Health Resources & Service Administration. (October 2016). About the Ryan White HIV/AIDS Program. https://hab.hrsa.gov/about-ryan-white- hivaids-program/hiv-care-continuum. https://hab.hrsa.gov/about-ryan-white-hivaids-program/hiv-care-continuum https://hab.hrsa.gov/about-ryan-white-hivaids-program/hiv-care-continuum IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 11 Table 1. Sample Demographic Characteristics (N = 285) Linkage Pattern Classes Overall (N=285) High (n=137) Moderate (n=97) Low (n=51) Diff between groups Characteristic n % n % n % n % p-valuea Age (Mean, SD) 285 42.5 (11.8) 137 41.6 (11.8) 97 45.6 (11.5) 51 39.4 (11.4) 0.004 Gender 0.112 Male 105 36.8% 52 38.0% 29 29.9% 24 47.1% Female 180 63.2% 85 62.0% 68 70.1% 27 52.9% Ethnicity 0.449 Not Hispanic or Latino 185 64.9% 87 63.5% 61 62.9% 37 72.5% Hispanic or Latino 100 35.1% 50 36.5% 36 37.1% 14 27.5% Race 0.347 More than one race 44 15.4% 26 19.0% 12 12.4% 6 11.8% White 73 25.6% 32 23.4% 23 23.7% 18 35.3% Black or African American 153 53.7% 72 52.6% 58 59.8% 23 45.1% Native Hawaiian, Asian, American Indian, Alaskan Nat. 15 5.3% 7 5.1% 4 4.1% 4 7.8% Highest Level of education 0.077 Less than high school 3 1.1% 0 0.0% 3 3.1% 0 0.0% High school diploma/GED 66 23.2% 36 26.3% 16 16.5% 14 27.5% Associate's Degree 28 9.8% 14 10.2% 9 9.3% 5 9.8% Bachelor's degree 90 31.6% 36 26.3% 41 42.3% 13 25.5% Master's Degree 96 33.7% 50 36.5% 28 28.9% 18 35.3% Doctoral Degree 2 0.7% 1 0.7% 0 0.0% 1 2.0% Professional Licensure 0.207 None 163 57.2% 79 57.7% 50 51.5% 34 66.7% At least one 122 42.8% 58 42.3% 47 48.5% 17 33.3% Work Position 0.006 Supervisor 69 24.2% 36 26.3% 23 23.7% 10 19.6% Counselor 48 16.8% 25 18.2% 16 16.5% 7 13.7% Case manager 53 18.6% 27 19.7% 25 25.8% 1 2.0% Navigator 14 4.9% 7 5.1% 6 6.2% 1 2.0% IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 12 Linkage Pattern Classes Overall (N=285) High (n=137) Moderate (n=97) Low (n=51) Diff between groups Characteristic n % n % n % n % p-valuea Educator/Outreach 42 14.7% 20 14.6% 10 10.3% 12 23.5% Program Administrator 39 13.7% 15 10.9% 12 12.4% 12 23.5% Other 20 7.0% 7 5.1% 5 5.2% 8 15.7% Curriculum-based HIV Knowledge 0.142 No 110 38.6% 46 33.6% 39 40.2% 25 49.0% Yes 175 61.4% 91 66.4% 58 59.8% 26 51.0% Caseload 0.833 Fewer than 30 patients 165 57.9% 75 54.7% 59 60.8% 31 60.8% 31-50 patients 74 26.0% 37 27.0% 25 25.8% 12 23.5% More than 50 patients 46 16.1% 25 18.2% 13 13.4% 8 15.7% aDifferences are assessed using analyses of variance for continuous measures and chi-square test for categorical measures IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 13 Table 2. Descriptive Statistics of HIV Continuum of Care (in the past 6 months) Linkage Pattern Classes Overall (N=285) High (n=137) Moderate (n=97)* Low (n=51) Diff between groups Characteristic n % n % n % n % p-valuea HIV Testing 0.224 0-10 187 65.6% 84 61.3% 65 67.0% 38 74.5% 11+ 98 34.4% 53 38.7% 32 33.0% 13 25.5% Primary Care 0.020 None 55 19.3% 24 17.5% 17 17.5% 14 27.5% 1-4 82 28.8% 35 25.5% 27 27.8% 20 39.2% 5-10 67 23.5% 28 20.4% 30 30.9% 9 17.6% 11+ 81 28.4% 50 36.5% 23 23.7% 8 15.7% PEP 0.084 Have not educated 135 47.5% 54 39.4% 54 56.3% 27 52.9% Once a month or less 70 24.6% 36 26.3% 21 21.9% 13 25.5% At least once a week 79 27.8% 47 34.3% 21 21.9% 11 21.6% PrEP 0.486 Have not educated 146 51.4% 63 46.0% 53 55.2% 30 58.8% Once a month or less 80 28.2% 44 32.1% 24 25.0% 12 23.5% At least once a week 58 20.4% 30 21.9% 19 19.8% 9 17.6% a Differences are assessed using chi-square test for categorical measures * Moderate (n=96) for the PEP and PrEP outcomes due to missing data. IHTP, 2(1), 1-14, 2022 CC BY-NC-ND 4.0 ISSN 2563-9269 14 Figure 1. Observed Proportions of Linkage Items by Pattern Classes