Gateways: International Journal of Community Research and Engagement Vol. 12, No. 1 January 2019 PRACTICE-BASED ARTICLE Development and evaluation of a community- engaged research training program: Building capacity of Marshallese stakeholders and academic researchers to conduct health research Pearl A McElfish1*, Brett Rowland2, Britni L Ayers1, Gail E O’Connor3, Rachel S Purvis2, Nia Aitaoto4, Lucy Capelle5, Melisa Laelan5, Holly C Felix6, M Kathryn Stewart6 and Karen HK Yeary7 1 College of Medicine, University of Arkansas for Medical Sciences Northwest, 1125 North College Avenue, Fayetteville, AR, USA 2 Office of Community Health and Research, University of Arkansas for Medical Sciences Northwest, 1125 North College Avenue, Fayetteville, AR, USA 3 Association of Asian Pacific Community Health Organizations, 101 Callan Avenue, Suite 400, San Leandro, CA, USA 4 Department of Nutrition and Integrative Physiology, College of Health, University of Utah, 250 South 1850 East, Salt Lake City, UT, USA 5 Arkansas Coalition of Marshallese, 614 East Emma Avenue, Suite 210, Springdale, AR, USA 6 Fay W Boozman College of Public Health, University of Arkansas for Medical Sciences, 4301 West Markham Street, Little Rock, AR, USA 7 Department of Health Behavior and Health Education, University of Arkansas for Medical Sciences, 4301 West Markham Street, Little Rock, AR, USA *Corresponding author: Pearl A McElfish; pamcelfish@uams.edu DOI: http:dx.doi.org/10.5130/ijcre.v12i1.6198 Article history: Received 26/05/2018; Revised 14/07/2018; Accepted 01/08/2018; Published 15/01/2019 © 2019 by the author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) License (https:// creativecommons.org/licenses/ by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Citation: McElfish, P.A., Rowland, B., Ayers, B.L., O’Connor, G.E., Purvis, R.S., Aitaoto, N., Capelle, L., Laelan, M., Felix, H.C., Stewart, M.K. and Yeary, K.H.K. 2019. Development and evaluation of a community-engaged research training program: Building capacity of Marshallese stakeholders and academic researchers to conduct health research. Gateways: International Journal of Community Research and Engagement, 12:1, Article ID 6198. http:dx.doi.org/10.5130/ ijcre.v12i1.6198 ISSN 1836-3393 | Published by UTS ePRESS | http://ijcre.epress. lib.uts.edu.au DECLARATION OF CONFLICTING INTEREST The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. FUNDING The Translational Research Institute at UAMS supported the community engagement that allowed us to develop this project through the National Institutes of Health (NIH) National Center for Research Resources and the National Center for Advancing Translational Sciences (grant UL1TR000039). In addition, the CERT program was funded by Patient Centered Outcomes Research Institute (PCORI) (grant #1513-UAMS). One of the research projects was funded by a UAMS intramural grant from the Sturgis Foundation Endowment and a second project was funded by the University of Arkansas Pi Theta Nursing Honor Society (grant #0402-82275-21-0000). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders. 1 PAGE NUMBER NOT FOR CITATION PURPOSES mailto:pamcelfish@uams.edu http:dx.doi.org/10.5130/ijcre.v12i1.6198 https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ http://ijcre.epress.lib.uts.edu.au http://ijcre.epress.lib.uts.edu.au Abstract Pacific Islanders are a growing, yet understudied population who suffer from high rates of chronic diseases such as obesity and diabetes. Given the historical trauma experienced by Pacific Islanders, community-based participatory research (CBPR) is an appropriate way to conduct research focused on reducing the health disparities observed in this population. This article presents the process of engaging the Marshallese community to design, conduct and evaluate a community-engaged research training program. The goal of the program was to build the capacity of both academic researchers and community stakeholders to conduct CBPR for the purpose of addressing health disparities identified and prioritised by the Marshallese community. The program included both didactic training and experiential mentored research components delivered over a period of two years. Eleven Marshallese community stakeholders and eleven academic researchers participated in the program. Results indicated that the program successfully increased participants’ knowledge regarding the CBPR process. Groups of participants have completed exploratory research projects based on the topics identified by the community. The evaluation adds important insights to the current CBPR training literature and can inform future CBPR trainings. Keywords Community-based participatory research, community-engaged research, Pacific Islanders, minority health, health disparities, interprofessional training Introduction The Pacific Islander population is growing rapidly in the United States (US). Between 2000 and 2010, the population of Pacific Islanders in the US grew by 40 per cent, with the fastest growth in the southern US (66 per cent) (Hixson, Hepler & Kim 2012). For example, Arkansas had a 252 per cent growth in the Pacific Islander population, the vast majority of which are Marshallese migrants from the Republic of the Marshall Islands (RMI). Arkansas now has the largest population of Marshallese living in the continental US (Hixson, Hepler & Kim 2012). Pacific Islanders residing in the US, and Marshallese specifically, have significant health disparities, including high rates of chronic and infectious diseases (Cardenas et al. 2016; Fischer et al. 2009; McElfish, Rowland et al. 2017; Rothfeldt et al. 2016; Woodall, Scollard & Rajan 2011). Despite significant health disparities, Pacific Islanders are under-represented in health research, resulting in limited scientific knowledge, masked health disparities and insufficient resource allocation for culturally appropriate research and programs (Park et al. 2009; Ro & Yee 2010; Roehr 2010; Srinivasan & Guillermo 2000). To help fill the gap in research on health disparities among Pacific Islanders, the University of Arkansas for Medical Sciences (UAMS) developed a community-engaged research training (CERT) program. The program brought together Marshallese community stakeholders and academic researchers to address the health disparities of the Pacific Islander/Marshallese community in Arkansas. This article presents: 1) the process of engaging community stakeholders and developing the unique CERT curriculum; 2) implementation of the CERT program; and 3) results of the community-engaged, mixed-methods program evaluation. Results from this article can serve as a useful model for other community-academic partnerships addressing health disparities in under-served populations. McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 2 PAGE NUMBER NOT FOR CITATION PURPOSES UNDERSTANDING HISTORICAL TRAUMA Any research training program targeted at reducing health disparities within the Pacific Islander/Marshallese community must consider the historical relationship between the US and the RMI. The US military tested more than 60 atomic and thermonuclear weapons in the RMI from 1946 to 1958 (Barker 2012; Simon 1997). Marshallese who inhabited the islands and atolls where the testing took place were relocated prior to the tests; however, those living on nearby islands and atolls were not evacuated (Barker 2012; Simon 1997). As a result, many Marshallese were exposed to radioactive fallout (Barker 2012; Cronkite et al. 1955; Lessard et al. 1984; Robison et al. 1997; Simon 1997; Takahashi et al. 2003). US researchers studied the effects of the nuclear radiation on exposed islanders in a study termed Project 4.1 (Barker 2012; Simon 1997). Informed consent of Marshallese community members was not provided and the study was conducted without translation of information into the native Marshallese language (Barker 2012). US military reports stated that exposure to nuclear radiation was accidental (Cronkite et al. 1955); however, many Marshallese believed that the exposure was not an accident (Barker 2012). Consistent with other populations that have experienced historical trauma, the Marshallese community often exhibits distrust towards health professionals and academic researchers (Barker 2012). One way to address this historical trauma in the Marshallese, as well as in other Pacific Islander communities, is through community-engaged research focused on topics prioritised by the community, with full participation from community members. Community- engaged research shares power and builds trust between academic researchers and community stakeholders (Israel et al. 1998; Minkler & Wallerstein 2008). Community-engaged research encompasses a wide range of community participation in research and has been referred to by several terms, including action research and patient-centred outcomes research. Included in community engagement is community-based participatory research (CBPR), whereby community and academic partners collaborate completely on every aspect of the research project. Given that the partnership described in this article used a CBPR approach, the authors will use the term CBPR throughout the article. Methods ENGAGEMENT AND PROGRAM DEVELOPMENT Engagement process. In 2013, the lead author began a concentrated effort to better understand the health disparities of the Pacific Islander/Marshallese community in northwest Arkansas and formed a community advisory board composed of Marshallese community stakeholders to partner in the process (McElfish et al. 2015). The process began with UAMS researchers asking Marshallese community members to share their history, stories and perspectives on health and research. Additionally, UAMS researchers and community stakeholders compiled and reviewed secondary data about the Arkansas Pacific Islander/Marshallese population from the census, local school districts, the Adult and Youth Behavior Risk Factor Surveillance System (BRFSS), Arkansas Department of Health Vital Records and needs assessments conducted in 2004 and 2010 by the local Home-Town Health Coalition and Community Foundation. The lead investigator conducted qualitative interviews with participants to better understand the most pressing needs of the community (Hallgren, McElfish & Rubon-Chutaro 2015; McElfish, Hallgren et al. 2016; McElfish, Purvis et al. 2016). Additionally, the CBPR team surveyed almost 300 Marshallese participants regarding self-reported health status (from Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 3 PAGE NUMBER NOT FOR CITATION PURPOSES the BRFSS), health research priorities, and perceptions regarding participation in research (McElfish, Long et al. 2018). Defining assets, needs and priorities. After the asset and need assessments, interviews and survey responses were compiled, focus groups were held with community stakeholders to document their interpretation of the data and prioritise the community’s most pressing issues. Community stakeholders identified diabetes, chronic disease management, obesity, physical activity and culturally appropriate health care as top priorities (McElfish et al. 2015). Stakeholders also identified their collectivist culture and faith-based community as assets in addressing health disparities. In addition, community stakeholders voiced the need to build the capacity of community members to partner with academic researchers in order to address the health disparities of the Marshallese community. After establishing the research priorities with community stakeholders, the lead author and team members met with academic researchers with expertise in the areas identified by community stakeholders to discuss their willingness to partner with the community to address the stated needs. In total, we met with 28 researchers. While there was significant interest in the research that could be done, none of these researchers felt fully equipped to conduct CBPR with the Marshallese community. Although the lead author had successfully conducted CBPR with the Marshallese community in northwest Arkansas, additional academic researchers and community stakeholders needed to be trained to collaboratively address the health disparities identified by the community. To close the gaps in training identified by both academic researchers and community stakeholders (described below), the CBPR team developed a collaborative research training program: the Community-Engaged Research Training (CERT) Program. Developing the training program. The CERT program was collaboratively developed to meet the needs identified by both community members and academic researchers. The lead researcher met with community stakeholders who had been conducting CBPR with UAMS researchers and asked them to describe the areas of research training they wanted to receive, and what areas they felt researchers needed to better understand in order to conduct research with the Marshallese community. Community stakeholders requested assistance in understanding: 1) the definition of research and how it helps the community; 2) how the community can be true partners in the research; and 3) resource sharing and budgeting. Furthermore, community stakeholders suggested that academic researchers needed assistance with: 1) respecting community knowledge; 2) partnering with the community; 3) respecting community partners; and 4) Pacific Islander/Marshallese history and culture. The lead researcher also had discussions with local academic researchers in order to understand their research training needs. Researchers requested assistance with: 1) understanding CBPR; 2) how to conduct CBPR; 3) how to recruit and retain hard to reach participants; 4) how to fund CBPR; 5) Institutional Review Board (IRB) issues with CBPR; and 6) how to partner with communities. In addition, researchers identified gaps in the community’s knowledge from prior attempts to conduct research with the Marshallese community. Specifically, researchers recommended community stakeholders have training to understand: 1) research design; 2) random assignment; 3) informed consent; and 4) quantitative and qualitative research methods. The goal of the CERT program was to build the capacity of both academic researchers and community stakeholders to conduct CBPR for the purpose of addressing health disparities identified by the Marshallese community. The CERT program includes both didactic training McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 4 PAGE NUMBER NOT FOR CITATION PURPOSES and experiential mentored research components delivered over two years. The CERT program views community stakeholders and academic researchers as equals who both have knowledge and wisdom to contribute. PROGRAM IMPLEMENTATION Recruitment and selection. The CERT program was advertised to community stakeholders through community non-profit partners, social media and community health workers, and was advertised to academic researchers at three local universities through those universities’ email lists. The inclusion criteria for researchers to engage in the CERT program were: 1) have expertise in one of the priority areas identified by community stakeholders; 2) be committed to working in the community on CBPR projects for at least five years after the program; and 3) commit to attending all training sessions and spending at least six hours per month engaged in research planning and execution with their community partners during the two-year training period. The inclusion criteria for community stakeholders were: 1) have an interest in learning about research; 2) be open to working with academic researchers for at least five years after the program; and 3) commit to attending all training sessions and spending at least six hours per month engaged in research planning and execution with their academic research partners during the two-year training period. Both community stakeholders and academic researchers completed an application that described why they wanted to participate and confirmed their time commitment to the program. Participants were selected with input from a community co-investigator. Eleven researchers and eleven community members met the inclusion criteria and were selected to participate in the program, which meant no-one who met the criteria was excluded. Each participant signed a memorandum of understanding that outlined the expectations and the amount of time that would be dedicated to the training. This evaluation was reviewed by the UAMS IRB and received an exemption (IRB #207156). CERT modules and implementation. The CERT program was a two-year training program, implemented from August 2015 to July 2017. The first year focused on interactive training, and the second year focused on implementing mentored group research projects. Eight training modules were presented monthly during the first year. Meetings were not held during June, July, November and December. Each session lasted approximately three hours. The interactive training modules covered the most important topics identified by the community stakeholders and academic researchers (see Table 1). Based on community stakeholder input, the CERT program was designed so that training was delivered to both community stakeholders and academic researchers together in the same sessions. Community stakeholders were compensated $50 per training; academic researchers were not compensated for participation. Table 1 CERT Curriculum Overview Module Focus Area Module 1 Pacific Islander/Marshallese Culture and History Module 2 Introduction to CBPR and PCOR Module 3 Overview of Research Design and Analysis Module 4 Community Perspectives/Wisdom: How They Improve Research Module 5 PCORI Methodology Standards Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 5 PAGE NUMBER NOT FOR CITATION PURPOSES https://clara.uams.edu/clara-webapp/protocols/207156/dashboard Module Focus Area Module 6 Ethics in Research Module 7 Disseminating Results Module 8 Facilitating Qualitative Interviews Note: CBPR = community-based participatory research; PCOR = patient-centered outcomes research; PCORI = Patient Centered Outcomes Research Institute Formation of groups and research topics for group projects. As part of the third module, community stakeholders and academic researchers began working together to develop research topics of interest. Three groups were formed and continued to work together throughout the rest of the program. Each research group consisted of at least three community stakeholders and at least three academic researchers. Each group identified a broad research area to work on, and collaboratively presented their research idea(s) and received feedback from the other participants. During the remaining five sessions, groups spent at least an hour each session refining their research topic. (See Table 2 for a description of the groups’ projects.) Table 2 Overview of Group Projects Group Overview Group 1 Title Exploratory study of barriers to and facilitators of medication adherence in Marshallese patients Study Aims Aim 1: To understand Marshallese patients’ perceptions of and experiences with barriers to and facilitators of medication adherence Aim 2: To understand healthcare provider identified barriers to and facilitators of medication adherence by the Marshallese in northwest Arkansas Aim 3: To understand Marshallese community health worker identified barriers to and facilitators of medication adherence for the Marshallese in northwest Arkansas Participants 65 participants: 40 Marshallese patients, 15 health care providers and 10 Marshallese community health workers Method Qualitative design with both focus groups and individual interviews; surveys to document participant demographic information Funding $25,000 intramural grant Group 2 Title Exploratory study of infant and child feeding practices and customs in Marshallese community Table 1 continued McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 6 PAGE NUMBER NOT FOR CITATION PURPOSES Group Overview Study Aims Aim 1: To understand infant and child feeding practices in the Marshallese community Aim 2: To understand the factors that may influence infant and child feeding practices Participants 29 participants: Marshallese women with children 1–3 years of age and caregivers of those children Method Qualitative study design with four focus groups; survey to document participant demographic information Funding $5000 intramural grant Group 3 Title Exploratory study of food practices in the Marshallese community Study Aims Aim 1: To understand Marshallese knowledge, beliefs, cultural norms and definitions of healthy eating Aim 2: To understand Marshallese experiences with purchasing, preparing and eating food Participants 40 participants: Marshallese adults aged 25 or older Method Mixed methods with food practices survey and qualitative interviews Funding No funding was received Mentored research phase. The mentored research phase was designed to take participants through the research preparation and proposal process over a one-year time period (see Table 3). Groups met monthly to collaboratively develop: 1) research ideas; 2) protocols for IRB approval; 3) data collection instruments; and 4) recruitment and consent documents. Groups progressed at different speeds, with one group collecting and analysing data during the second year of mentored research, another group completing their protocol and collecting about half of their data during the second year, and the third group completing the planning and IRB approval process by the end of the second year. Table 3 Timeline of Year 2 – Mentored Research Phase Month Objective 1 • Confirm research topics • Discuss and identify who else needs to participate in study 2 • Confirm your research questions and aims • Identify whether funding is needed for the projects Table 2 continued Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 7 PAGE NUMBER NOT FOR CITATION PURPOSES Month Objective 3, 4, 5 • Develop protocols, including: ⚬ Participant inclusion criteria ⚬ Sampling ⚬ Recruitment methods ⚬ Participant incentives ⚬ Data to be collected ⚬ Data collection instruments ⚬ Data analysis plans 6, 7 • Collect data 8, 9, 10 • Analyse data 11, 12 • Write up results • Disseminate results to the community The CBPR investigators/trainers. The CERT program was developed and delivered by two CBPR researchers, both of whom spent the majority of their careers outside academia implementing programs and conducting research in community settings. In addition, community co-investigators provided significant input into training in the areas of cultural competency, research topics important to the community, and culturally appropriate processes for engaging with the community and implementing research. Marshallese community health workers who had facilitated health research projects co-taught four of the modules. Approximately half of all presentations were presented by a Pacific Islander researcher, Pacific Islander community health worker, or Pacific Islander community co-investigator. Once the CERT program entered into the mentored research phase, each of the three groups was supported by a UAMS research associate who facilitated meetings, assisted in research design, and helped draft the protocol, consent document and study instruments. EVALUATION OF THE CERT PROGRAM Evaluation methods. A multi-component, mixed-methods program evaluation was designed and implemented (Tashakkori & Teddlie 2010). The mixed-methods evaluation had three components: 1) a pre/post-test survey; 2) implementation of a continuous process improvement model; and 3) qualitative focus group interviews with academic and community investigators. The pre/post-test survey assessed participants’ knowledge, experiences and opinions related to: 1) health research; 2) the CBPR process; 3) the benefit of medical research; 4) importance of community partners; 5) Pacific Islanders’ knowledge of CBPR; 6) academic researchers’ knowledge of Pacific Islanders/Marshallese; and 7) future participation in CBPR. The survey was conducted at the beginning of the first training session and again at the end of the last training session held during the first year. Descriptive statistics were computed for the community stakeholders and academic researchers. Pre/post-test survey results were analysed using paired sample t-tests. Second, a continuous process improvement model, based on the Kaizen approach, was used to make real-time adjustments to the training based upon participants’ feedback (Imai 1986, 1997). During the last 20 minutes of each training session, feedback from participants Table 3 continued McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 8 PAGE NUMBER NOT FOR CITATION PURPOSES was gathered on which elements of the training were working well and which elements were not working well. Participant feedback and the resulting changes made to the program were systematically documented. Third, qualitative focus group interviews with community stakeholders and academic researchers were conducted at the end of the interactive training phase (first year) and mentored research phase (second year) of the program. A semi-structured interview guide was used to gather the CERT program participants’ thoughts on what worked well and did not work well. The broad questions were asked in several different ways to elicit feedback on the program. These interviews were conducted by the lead author with two research associates taking field notes. The interviews were audio recorded and transcribed verbatim. Text was coded using emergent thematic analysis by a primary coder and a confirmation coder who had extensive training in qualitative methods. Summary analysis and quotes that best represented participants’ feedback are presented below. Results QUANTITATIVE RESULTS Participant characteristics. As shown in Table 4, a total of 11 community and 11 academic investigators participated in the training. The mean age of all participants was 39 years (SD = 9.9). The majority of participants were female (77.3 per cent). Ten of the 11 academic researchers self-reported their race as white, and one self-reported as multi-racial. All community stakeholders self-reported their race as Pacific Islander/Marshallese. More than half of the participants had completed college (54.5 per cent), and half had prior experience with health research. Among the academic researchers, there were five pharmacists, four nurses, one dietitian and one cultural anthropologist. The vast majority of participants indicated a positive perception of research, as well as the importance of community involvement in research (see Table 5). Table 4 Participant Demographics Characteristic N % Age (M ± SD) 38.8 ± 9.9 Gender Male 5 22.7 Female 17 77.3 Race White 11 50.0 Pacific Islander 10 45.5 Other/Multiracial 1 4.5 Education Some high school 1 4.5 High school graduate 2 9.1 Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 9 PAGE NUMBER NOT FOR CITATION PURPOSES Characteristic N % Some college 7 31.8 College graduate 12 54.5 Type of Researcher Academic 11 50.0 Community 11 50.0 Note: Only valid percentages shown. Percentages may not total 100 due to rounding. N = number; M = mean; SD = standard deviation. Table 5 Participants’ Attitudes and Experiences Related to Health Research Response Category N % Ever participated in a health research study? No 11 50.0 Yes 11 50.0 How was your experience with health research studies? All positive 4 36.4 Some positive/some negative 7 63.6 Medical research is done for the benefit of humankind Yes 22 100 Medical research is done for the benefit of my community Yes 22 100 Ever been involved in CBPR? No 12 54.5 Yes 10 45.5 How important are community partners to health research? Very important 21 100 How important is it for the community to understand research in order to be partners in research? Very important 22 100 Is the input of community partners valued as much as academic partners? No 1 5.6 Yes, by some academics 8 44.4 Yes, by all academics 9 50.0 Note: Only valid percentages shown. Percentages may not total 100 due to rounding. N = number. McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 10 PAGE NUMBER NOT FOR CITATION PURPOSES Pre/post-test results. Results of the pre/post evaluation questions indicated a perceived gain in participants’ knowledge regarding the CBPR process. Participants’ post-program scores (M = 2.57, SD = 0.51) were significantly higher than their pre-program scores (M = 1.79, SD = 0.43), t(13) = 5.08, p < .001, d = 1.36. Likewise, survey results showed an increase regarding how knowledgeable participants felt Pacific Islanders/Marshallese were about CBPR. Participants’ post-program scores (M = 2.08, SD = 0.64) were significantly higher than their pre-program scores (M = 1.69, SD = 0.63), t(12) = 2.74, p = .018, d = 0.76. Though the scores increased, no significant difference was observed between the participants’ pre- and post-program scores for the item assessing academic researchers’ knowledge of the needs and values of Pacific Islanders/Marshallese (p = .189). (See Table 6.) Table 6: Comparison of Pre/Post-Survey Responses Measures Pre- Program Score (SD) Post- Program Score (SD) n t df p How knowledgeable do you feel about the CBPR process? 1.79 (0.43) 2.57 (0.51) 14 5.08 13 <.001 How knowledgeable do you feel Pacific Islanders/ Marshallese are about the CBPR process? 1.69 (0.63) 2.08 (0.64) 13 2.74 12 .018 How knowledgeable do you feel academic researchers are about the needs and values of Pacific Islanders/Marshallese? 1.43 (0.51) 1.64 (0.63) 14 1.39 13 .189 Note: Of the total 22 program participants, analyses are based on the number who fully completed each pre- and post-measure. Bolded p-values are statistically significant. SD = standard deviation. CONTINUOUS PROCESS IMPROVEMENT The continuous process improvement approach led to three major changes to the training procedures. Table 7 documents the initial plan, participant feedback, and how that feedback was implemented as a process change in the training program. Table 7 Process Improvement Changes Initial Plan Participant Feedback Change Every training session will be 70% didactic and 30% interactive/ group projects After modules 1–4, participants wanted to reduce the amount of didactic training and increase the amount of interactive/group work. Modules 6 and 7 adjusted to 50% didactic and 50% interactive/ group work; Modules 7 and 8 adjusted to 20% didactic and 80% interactive/group work. Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 11 PAGE NUMBER NOT FOR CITATION PURPOSES Initial Plan Participant Feedback Change Form 11 groups with one community stakeholder and one academic researcher per group Participants said they would feel more comfortable and be able to accomplish more work in larger groups. Three groups were formed with three to four community stakeholders and three to four academic researchers per group. Form groups and choose topics during the last month of the interactive training Participants wanted to form their groups earlier so they could discuss and apply what they had learned with their topic in mind. Groups were formed during month 3, and after each training module, the groups discussed how the information would be integrated into their project. QUALITATIVE RESULTS When asked questions regarding what worked well in the CERT program, what was most beneficial and what should be repeated if the CERT program were implemented again, community stakeholders and academic researchers identified three broad areas: 1) group interaction, collaboration and co-learning between community stakeholders and academic researchers; 2) the interprofessional nature of the academic researchers; and 3) presentations from Pacific Islander researchers, community health workers and community stakeholders. Group interaction, collaboration and co-learning between community stakeholders and academic researchers. The interaction and opportunity for teamwork between community members and academic researchers was consistently mentioned as the most successful part of the program. A community stakeholder noted, ‘I really like the way that we are all communicating and trying to find the solutions to our problems, because that is the reason why we are still here, we want to help our community.’ Participants discussed their appreciation of the interactions between academic researchers and community members as they allowed for a more productive learning environment and also increased their humility about their level of knowledge. As we moved into the mentored research phase, participants appreciated the continued interaction and also cited an increased level of trust that allowed community stakeholders to be more candid with researchers regarding research design. As one academic researcher explained, ‘I appreciate the fact that they [community members] were so honest. The fact that they said, “no, that is not going to work”, I very much appreciated that.’ A community stakeholder stated ‘I enjoyed that we had the sessions in the groups, because we had community [members], but also researchers, and we were able to discuss and start coming up with a framework for a project.’ One community stakeholder reflected on prior research they had participated in and how this process was different: They [researchers] all came and were trying to do research on us and the [Marshallese] community. So when they were working with our people it was always, ‘No. No, it’s not like that.’ They would try to bring up their ways to research on our lives and what I like about this research, because you guys are willing to learn our ways. If it’s not our ways, then it’s not going to work and we don’t need it, because that’s the truth. We work together and that’s what makes us strong. So it’s not just easy to just come in and say ‘Oh, this is the way to do it, because we’re researchers, we’re doctors.’ I really like this McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 12 PAGE NUMBER NOT FOR CITATION PURPOSES [training program] because people are willing to respect our ways. I like to do research with you guys. Interprofessional nature of the academic participants. In addition to the interaction between community stakeholders and academic researchers, participants voiced an appreciation of the broad academic and health care professions represented in the CERT program and the opportunity to conduct interprofessional research with the community. One academic participant explained, ‘For me, I think the interprofessional [faculty] and also the community partners that were here to be able to work in a team. Even though sometimes it is difficult because we all came with different backgrounds and levels of knowledge, I liked working with everybody, a variety of people.’ Another academic researcher agreed stating, ‘Working with people outside of your profession . . . I get to work with an awesome group of nurses and I am the only pharmacist on their team so I would not have had that opportunity if we were just working in the pharmacy department. And working with different universities . . . this is with a different institution and perspective as well.’ A community stakeholder added, ‘We got to see the professionals of each individual that is working for the health systems here like the pharmacists and nurses. It was cool.’ Presentations from Pacific Islander researchers, community health workers and community stakeholders. The participants, especially the academic researchers, appreciated learning from Pacific Islander researchers, community health workers and community stakeholders. One researcher explained, ‘I learned so much from community health workers that I would not have even thought of.’ Another academic researcher summarised, ‘I loved hearing all of the [community stakeholders] stories.’ Specifically, academic participants stated that the sessions from Pacific Islander presenters helped them broaden their research perspective: ‘I learned so much. Listening to the stories and hearing how to do things in the Pacific Islander culture helps me. . . really broadened my knowledge, and helps me to identify new areas of research.’ Hearing directly from Pacific Islanders/Marshallese about their culture seemed to create a greater sense of cultural understanding and cultural humility among many academic participants, who reflected that they thought they knew more about the Pacific Islander/Marshallese culture than they actually did and credited the training for helping them understand their lack of knowledge. One academic participant explained, ‘I think I knew the culture as a medical provider but I didn’t really know the culture. So I think that was huge for me.’ Another academic researcher stated, ‘I have learned a lot about the Marshallese and especially in relation to research. I’ve learned a lot of new things. I thought I knew more than I knew, I now know more than I did know, but I still have a whole lot left to learn.’ Pacific Islander community members also appreciated that academic researchers wanted to hear directly from them. A community participant said, ‘I really liked it that more people are aware and are very interested in learning more about us and trying to find ways to find a better life for us.’ When asked questions regarding what did not work well and what should be done differently if the CERT program were implemented again, participants identified three areas: 1) deepen the relationships between academic partners and community members; 2) parts of the training were elementary; and 3) the location of the training. Deepen the relationships. Both community stakeholders and academic researchers stated that they appreciated the opportunity to work collaboratively, but thought there was more that could be done to deepen the relationships between community stakeholders and academic researchers. Participants’ primary recommendation was to incorporate a ‘buddy system’ that Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 13 PAGE NUMBER NOT FOR CITATION PURPOSES paired an academic researcher with a community stakeholder, so they could spend time together getting to know the Pacific Islander community without the difficulty of scheduling group meetings. Both community stakeholders and academic researchers voiced that this would have the dual benefit of deepening relationships while also keeping community stakeholders involved and engaged throughout the two-year CERT program. A community stakeholder participant suggested, ‘In the beginning at the initial gathering, maybe assign a community member to each [researcher].’ An academic researcher echoed, ‘Pairing up some mentorships where a Marshallese [community member] could mentor you and then you could mentor them on the research side. It would be very mutually beneficial. Then you would have that contact and that friendship.’ Another academic researcher stated they needed ‘more time to build those relationships with the community, because that is what it really is all about. Without them there is no trust. . .with a relationship they will share more and we will share more.’ Parts of the training were elementary. Although there was a conscious decision to include all participants in all modules, the academic researchers found the modules related to research ethics and the IRB redundant. An academic researcher explained, ‘The module where we went over what is IRB. . .I felt like I was zoning out, so maybe having that for people who were not familiar in a separate group.’ Another academic participant described the research ethics module as her least favourite and stated that it seemed ‘elementary’ since she had completed research ethics training multiple times. The location of the training. The trainings were held on the UAMS campus, approximately 16 kilometres from the primary residences of the community stakeholders, and concern was expressed about the location where the trainings were held. Participants noted that the location required some community stakeholders to find transportation to each session. Furthermore, it was expressed that having the trainings in a non-academic setting could make community stakeholders feel more comfortable. Both community stakeholders and academic researchers recommended that some or all of the training sessions be held in the community. A community stakeholder summarised, ‘I think that the biggest setback was that the trainings was here [at UAMS] and you have the community in Springdale.’ Another community stakeholder noted that data collection events held in the community are better, ‘I really like that we did our focus groups in the community at the Arkansas Coalition of Marshallese office, because I think that really worked out well.’ Academic researchers also saw value in making sure some of the training sessions were held in the community, ‘Maybe one of the classes could be somewhere in the community? Maybe early on, to make it feel like this is more of an equal partnership.’ Another academic researcher agreed and voiced concern that ‘we learned about their community but we never went into the community as a group.’ Discussion National funding agencies, including the National Institutes of Health (NIH) and the Patient Centered Outcome Research Institute (PCORI), have encouraged CBPR as a way to improve translational research and address health disparities in disenfranchised communities (Michener et al. 2012; National Institutes of Health 2017; Patient-Centered Outcomes Research Institute 2014). CBPR is increasingly cited as an important tool in addressing health disparities in communities who have experienced historical trauma (LaVeaux & Christopher 2009; Minkler & Wallerstein 2008; Wallerstein & Duran 2006). However, conducting CBPR requires building the capacity of both community stakeholders and academic researchers (Crosby et al. 2013; Goytia et al. 2013; Kwon et al. 2012). Though several CBPR engagement McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 14 PAGE NUMBER NOT FOR CITATION PURPOSES and training models have been published (Allen et al. 2013; Baquet et al. 2013; Cole et al. 2013; Crosby et al. 2013; Goytia et al. 2013; Healey, Reed & Huber 2013; Martin del Campo et al. 2013; Rideout et al. 2013; Smith, Kaufman & Dearlove 2013; Young-Lorion et al. 2013), few have focused on work with Pacific Islander communities (DiStefano et al. 2013; Kaholokula et al. 2014; Kwan et al. 2012, 2014; Palmer 2014). The CERT program is one of the first published CBPR training programs with Pacific Islander communities in the southern US. The mixed-method evaluation showed that the program was successful in improving CBPR knowledge among participants. Furthermore, the evaluation documented participants’ assessments of what they liked about the program and how they felt it could be improved. This evaluation of the CERT program expands the literature through several key insights that can be integrated into other CBPR trainings. The CERT program is unique, in that it was designed to have community stakeholders and academic researchers attend all training sessions together. Most other CBPR training programs instruct community stakeholders and academic researchers separately and then pair them together for research projects after the training sessions. The evaluation shows that the collaborative learning model has both benefits and challenges. Both community stakeholders and academic researchers voiced their appreciation for the time spent together and cited this as a primary benefit of the CERT program. However, academic researchers found some of the sessions, such as the research ethics and IRB training, to be at an elementary level, and noted it was an area that could be improved. A hybrid model may be needed to keep the benefits of joint training, but reduce the unproductive time for academic researchers. The CERT program appeared to be effective in helping academic participants increase their cultural humility (Tervalon & Murray-Garcia 1998). Academic researchers showed evidence of self-evaluation and self-critique of their knowledge of Pacific Islander/Marshallese culture. When asked related questions in the focus group, academic researchers reflected that they thought they knew more than they actually did when they began the program. This finding could indicate some level of response shift bias in the quantitative findings, due to an overestimation of knowledge at the outset of the program. Furthermore, the academic researchers credited the co-learning environment and presentations directly from Pacific Islanders for helping them identify their gaps in knowledge and strengthen their commitment to continuous learning and partnership. This process of self-evaluation and self-critique of one’s own knowledge and the cultural humility that comes from such reflection is a critical component of conducting CBPR (LaVeaux & Christopher 2009; Minkler & Wallerstein 2008; Tervalon & Murray-Garcia 1998; Wallerstein & Duran 2006). The most common positive feedback was the opportunity to learn about, with and from each other. The co-learning between community stakeholders and academic researchers and between researchers from multiple universities and multiple professions was cited as extremely valuable. In addition to the co-learning environment, academic researchers voiced appreciation that more than 50 per cent of the presentations were delivered by Pacific Islander/Marshallese community members. Having members of the Pacific Islander/Marshallese community present about their culture was key to the success of the CERT program. The CERT program is one of the first trainings to incorporate both an interactive training component and a year-long mentored research component. While the mentoring phase was successful, the program evaluation showed areas that should be considered if the training were implemented again. While all the projects were similar in scope and design, the projects progressed at different rates, and only two of the three projects received outside funding. Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 15 PAGE NUMBER NOT FOR CITATION PURPOSES In future trainings, it would be best to secure funding for all projects prior to beginning the mentored research phase of the training. Furthermore, based upon input regarding process improvement, we moved from a model that paired one academic researcher with one community stakeholder to a group model with six to eight members (half academic researchers and half community stakeholders). While this change was based upon participant input, qualitative feedback suggests that a more balanced approach is needed. Future implementations should consider combining the one-on-one and group models so that each community stakeholder is paired with an academic researcher, and then facilitate a move to three pairs working together in a focused research group. The CERT program sought to model CBPR principles in its implementation process. This goal was achieved in some ways, but failed in others. The primary way that the CERT program modelled CBPR principles was through the continuous process improvement that sought input from participants, discussed possible solutions and made real-time changes to the program. The collaborative process of discussion, debate, and willingness to make changes to protocols is an essential component of CBPR (LaVeaux & Christopher 2009; Wallerstein & Duran 2006). Participants were able to engage in this collaborative decision making as part of the training, which should help them feel more comfortable integrating it into their future CBPR projects. In addition, the CERT program modelled the importance of community leadership through a majority of the presentations being delivered by members of the Pacific Islander/Marshallese community. However, it is important to note that the CERT program failed to model CBPR principles in choosing the location of the training. The training was held approximately 16 kilometres from the primary residences of the Pacific Islander/ Marshallese community. Holding the trainings on the university campus saved money and was convenient for academic researchers, but community stakeholders voiced their concern that the location was more difficult to get to and required transportation for community stakeholders. Furthermore, it may have created a shift in the balance of power because training was in an academic setting rather than a community setting. Future trainings should consider holding all or part of the sessions in community settings. This could help facilitate community stakeholder participation and would also model principles of CBPR. Limitations This mixed-methods evaluation of the CERT program is not without limitations. First, the pre/post-test data did not analyse community stakeholder and academic researcher data separately. The choice to keep the pre/post-surveys anonymous and not have participants disclose if they represented academia or the community was intentional; however, it did reduce the analytical value of the data. Furthermore, several participants were unable to attend the final training session where post-surveys were completed; thus, the results of the inferential tests should be interpreted with caution, due to the restricted sample size. As mentioned previously, the qualitative results indicated that using a traditional pre/post-survey design may have led to inaccurate pre-program ratings. Future studies should employ a retrospective pre/post-survey design to reduce the potential for such bias, or conduct both traditional and retrospective pre/post-tests simultaneously in order to observe whether a response shift exists, and to what degree (Drennan & Hyde 2008; Howard 1980; Pratt, McGuigan, & Katzev 2000; Rohs 1999). Participants were provided the opportunity to do an individual interview or participate in a focus group interview. All participants chose a focus group. While this may point to the trust developed among members, it may have also constrained responses. Finally, the training program only included 22 participants: 11 community stakeholders and 11 McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 16 PAGE NUMBER NOT FOR CITATION PURPOSES academic researchers. The small sample size provides limited information about the ability to scale the CERT program to larger groups or different populations. Conclusions This is one of the first CBPR training evaluations with Pacific Islanders to be published. Although there were issues encountered in the inaugural CERT program, it was successful in accomplishing its goal of building community-engaged research capacity among Pacific Islander/Marshallese community stakeholders and academic researchers in northwest Arkansas. Overall, participants perceived an increase in their understanding of CBPR and expressed satisfaction with their experience in the program. The process of conducting this evaluation and group reflection on what worked and what did not work incorporated elements of cultural humility, shared power and shared decision making, which are foundational principles of CBPR. The evaluation adds important insights to the current CBPR training literature and can inform future CBPR trainings. References Allen, M, Svetaz, A, Hurtado, G, Linares, R, Garcia-Huidobro, D & Hurtado, M 2013, ‘The developmental stages of a community-university partnership: The experience of Padres Informados/ Jovenes Preparados’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 271–79. doi:10.1353/ cpr.2013.0029 Baquet, C, Bromwell, J, Hall, M & Frego, J 2013, ‘Rural community-academic partnership model for community engagement and partnered research’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 281–90. doi:10.1353/cpr.2013.0028 Barker, H 2012, Bravo for the Marshallese: Regaining control in a post-nuclear, post-colonial world, Cengage Learning, Independence, KY. Cardenas, V, Orloff, M, Kaminaga, J, Cardenas, I, Brown, J, Hainline-Williams, S & Bates, J 2016, ‘Tuberculosis and leprosy infections in the Marshallese population of Arkansas, USA’, Leprosy Review, vol. 87, no. 1, pp. 109–12. Cole, C, Edelman, E, Boshnack, N, Jenkins, H, Richardson, W & Rosenthal, M 2013, ‘Time, dual roles, and departments of public health: Lessons learned in CBPR by an AIDS service organization’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 323–30. doi:10.1353/cpr.2013.0034 Cronkite, E, Bond, V, Conard, R, Shulman, N, Farr, R, Cohn, S & Browning, L 1955, ‘Response of human beings accidentally exposed to significant fall-out radiation’, The Journal of the American Medical Association, vol. 159, no. 5, pp. 430–34. https://doi.org/10.1001/jama.1955.02960220020007 Crosby, L, Parr, W, Smith, T & Mitchell, M 2013, ‘The community leaders institute: An innovative program to train community leaders in health research’, Academic Medicine, vol. 88, no. 3, pp. 335–42. doi:10.1097/ACM.0b013e318280d8de DiStefano, A, Peters, R, Tanjasiri, S, Quitugua, L, Dimaculangan, J, Hui, B & Takahashi, L 2013, ‘A community-based participatory research study of HIV and HPV vulnerabilities and prevention in two Pacific Islander communities: Ethical challenges and solutions’, Journal of Empirical Research on Human Research Ethics, vol. 8, no. 1, pp. 68–78. doi:10.1525/jer.2013.8.1.68 Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 17 PAGE NUMBER NOT FOR CITATION PURPOSES http://muse.jhu.edu/article/521217 http://muse.jhu.edu/article/521217 http://muse.jhu.edu/article/521223 http://muse.jhu.edu/article/521226 https://doi.org/10.1001/jama.1955.02960220020007 https://insights.ovid.com/crossref?an=00001888-201303000-00018 https://journals.sagepub.com/doi/10.1525/jer.2013.8.1.68 Drennan, J & Hyde, A 2008, ‘Controlling response shift bias: The use of the retrospective pre-test design in the evaluation of a master’s programme’, Assessment and Evaluation in Higher Education, vol. 33, no. 6, pp. 699–709. https://doi.org/10.1080/02602930701773026 Fischer, G, Wang, S, Ahring, S, Fowler, K, Hainline, S, Chinglong, M & Williams, I 2009, ‘An investigation of perinatal hepatitis B virus infections among a high risk population: The delivery hospital as a safety net’, The Pediatric Infectious Disease Journal, vol. 28, no. 7, pp. 593–97. doi:10.1097/ INF.0b013e318196bf5c Goytia, C, Todaro-Rivera, L, Brenner, B, Shepard, P, Piedras, V & Horowitz, C 2013, ‘Community capacity building: A collaborative approach to designing a training and education model’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 291–99. doi:10.1353/cpr.2013.0031 Hallgren, E, McElfish, P & Rubon-Chutaro, J 2015, ‘Barriers and opportunities: A community-based participatory research study of health beliefs related to diabetes in a US Marshallese community’, The Diabetes Educator, vol. 41, no. 1, pp. 86–94. doi:10.1177/0145721714559131 Healey, W, Reed, M & Huber, G 2013, ‘Creating a community-physical therapy partnership to increase physical activity in urban African-American adults’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 255–62. doi:10.1353/cpr.2013.0038 Hixson, L, Hepler, B & Kim, M 2012, ‘The Native Hawaiian and other Pacific Islander population: 2010 census briefs’, www.census.gov/prod/cen2010/briefs/c2010br-12.pdf Howard, G 1980, ‘Response shift bias: A problem in evaluating interventions with pre/post self-reports’, Evaluation Review, vol. 4, no. 1, pp. 93–106. https://doi.org/10.1177/0193841x8000400105 Imai, M 1986, Kaizen: The key to Japan’s competitive success, McGraw-Hill, New York. Imai, M 1997, Gemba Kaizen: A commonsense, low-cost approach to management, McGraw-Hill, New York. Israel, B, Schulz, A, Parker, E & Becker, A 1998, ‘Review of community-based research: Assessing partnership approaches to improve public health’, American Journal of Public Health, vol. 19, no. 1, pp. 173–202. https://doi.org/10.1146/annurev.publhealth.19.1.173 Kaholokula, J, Kekauoha, P, Dillard, A, Yoshimura, S, Palakiko, D, Hughes, C & Townsend, C 2014, ‘The PILI Ohana project: A community-academic partnership to achieve metabolic health equity in Hawai’i’, Hawai’i Journal of Medicine & Public Health, vol. 73, pp. 29–33. Kwan, P, Briand, G, Lee, C, Lepule, J, Pang, J, Sabado, M & Palmer, P 2014, ‘Use of a community- based participatory research approach to assess knowledge, attitudes, and beliefs on biospecimen research among Pacific Islanders’, Health Promotion Practice, vol. 15, no. 3, pp. 422–30. doi:10.1177/1524839913516464 Kwon, S, Rideout, C, Tseng, W, Islam, N, Cook, W, Ro, M & Trinh-Shevrin, C 2012, ‘Developing the community empowered research training program: Building research capacity for community-initiated and community-driven research’, Progress in Community Health Partnerships, vol. 6, no. 1, pp. 43–52. doi:10.1353/cpr.2012.0010 LaVeaux, D & Christopher, S 2009, ‘Contextualizing CBPR: Key principles of CBPR meet the Indigenous research context’, Pimatisiwin, vol. 7, no. 1, pp. 1–25. Lessard, E, Miltenberger, R, Cohn, S, Musolino, S & Conard, R 1984, ‘Protracted exposure to fallout: The Rongelap and Utirik experience’, Health Physics, vol. 46, no. 3, pp. 511–27. https://doi. org/10.1097/00004032-198403000-00002 McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 18 PAGE NUMBER NOT FOR CITATION PURPOSES https://doi.org/10.1080/02602930701773026 https://insights.ovid.com/crossref?an=00006454-200907000-00006 https://insights.ovid.com/crossref?an=00006454-200907000-00006 http://muse.jhu.edu/article/521218 https://journals.sagepub.com/doi/10.1177/0145721714559131 http://muse.jhu.edu/article/521221 http://www.census.gov/prod/cen2010/briefs/c2010br-12.pdf https://doi.org/10.1177/0193841x8000400105 https://doi.org/10.1146/annurev.publhealth.19.1.173 https://journals.sagepub.com/doi/10.1177/1524839913516464 http://muse.jhu.edu/article/468609 https://doi.org/10.1097/00004032-198403000-00002 https://doi.org/10.1097/00004032-198403000-00002 Martin del Campo, F, Casado, J, Spencer, P & Strelnick, H 2013, ‘The development of the Bronx Community Research Review Board: A pilot feasibility project for a model of community consultation’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 341–52. doi:10.1353/cpr.2013.0037 McElfish, P, Hallgren, E, Henry, L, Ritok, M, Rubon-Chutaro, J & Kohler, P 2016, ‘Health beliefs of Marshallese regarding type 2 diabetes’, American Journal of Health Behavior, vol. 40, no. 1, pp. 24–57. McElfish, P, Kohler, P, Smith, C, Warmack, S, Buron, B, Hudson, J & Rubon-Chutaro, J 2015, ‘Community-driven research agenda to reduce health disparities’, Clinical and Translational Science, vol. 8, no. 6, pp. 690–95. doi:10.1111/cts.12350 McElfish, P, Long, C, Stephens, R, Spencer, N, Rowland, B, Spencer, H & Stewart, M 2018, ‘Assessing community health priorities and perceptions about health research: A foundation for a community- engaged research program’, Journal of Higher Education Outreach and Engagement, vol. 22, no. 1, pp. 107-128. McElfish, P, Purvis, R, Maskarinec, G, Bing, W, Jacob, C, Ritok, M & Riklon, S 2016, ‘Interpretive policy analysis: Marshallese COFA migrants and the Affordable Care Act’, International Journal for Equity in Health, vol. 15, no. 1, p. 91. doi:10.1186/s12939-016-0381-1 McElfish, P, Rowland, B, Long, C, Hudson, J, Piel, M, Buron, B & Warmack, T 2017, ‘Diabetes and hypertension in Marshallese adults: Results from faith-based health screenings’, Journal of Racial and Ethnic Health Disparities, vol. 4, no. 6, pp. 1042-1050. doi: 10.1007/s40615-016-0308-y Michener, L, Cook, J, Ahmed, S, Yonas, M, Coyne-Beasley, T & Aguilar-Gaxiola, S 2012, ‘Aligning the goals of community-engaged research: Why and how academic health centers can successfully engage with communities to improve health’, Academic Medicine, vol. 87, no. 3, pp. 285–91. doi:10.1097/ ACM.0b013e3182441680 Minkler, M & Wallerstein, N (eds) 2008, Community-based participartory research for health: From process to outcomes, Jossey-Bass Publishers, San Francisco, CA. National Institutes of Health 2017, ‘Community-Based Participatory Research Program’ (CBPR), viewed 14 September 2017, www.nimhd.nih.gov/programs/extramural/community-based-participatory.html. Palmer, P 2014, ‘Improving Pacific Islander health through community participation: A case study’, in R Huff, M Kline & D Peterson (eds), Health promotion in multicultural populations, SAGE Publications, Thousand Oaks, CA, pp. 425–434. Park, C, Bruan, K, Horiuchi, B, Tottori, C & Onaka, A 2009, ‘Longevity disparities in multiethnic Hawai’i: An analysis of 2000 life tables’, Public Health Reports, vol. 124, no. 4, pp. 579–84. Patient-Centered Outcomes Research Institute 2014, ‘PCORI Engagement Rubric’, www.pcori.org/ sites/default/files/Engagement-Rubric.pdf. Pratt, C, McGuigan, W & Katzev, A 2000, ‘Measuring program outcomes: Using retrospective pretest methodology’, American Journal of Evaluation, vol. 21, no. 3, pp. 341–49. https://doi. org/10.1177/109821400002100305 Rideout, C, Gil, R, Browne, R, Calhoon, C, Rey, M, Gourevitch, M & Trinh-Shevrin, C 2013, ‘Using the Delphi and snow card techniques to build consensus among diverse community and academic stakeholders’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 331–39. doi:10.1353/ cpr.2013.0033 Ro, M & Yee, A 2010, ‘Out of the shadows: Asian Americans, Native Hawaiians, and Pacific Islanders’, American Journal of Public Health, vol. 100, no. 5, pp. 776–78. doi:10.2105/AJPH.2010.192229 Development and evaluation of a community-engaged research training program Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 19 PAGE NUMBER NOT FOR CITATION PURPOSES http://muse.jhu.edu/article/521227 https://ascpt.onlinelibrary.wiley.com/doi/full/10.1111/cts.12350 doi:10.1186/s12939-016-0381-1 https://link.springer.com/article/10.1007%2Fs40615-016-0308-y https://insights.ovid.com/crossref?an=00001888-201203010-00014 https://insights.ovid.com/crossref?an=00001888-201203010-00014 http://www.nimhd.nih.gov/programs/extramural/community-based-participatory.html http://www.pcori.org/sites/default/files/Engagement-Rubric.pdf http://www.pcori.org/sites/default/files/Engagement-Rubric.pdf https://doi.org/10.1177/109821400002100305 https://doi.org/10.1177/109821400002100305 http://muse.jhu.edu/article/521219 http://muse.jhu.edu/article/521219 https://ajph.aphapublications.org/doi/10.2105/AJPH.2010.192229 Robison, W, Noshkin, V, Conrado, C, Eagle, R, Brunk, J, Jokela, T & Wong, K 1997, ‘The Northern Marshall Islands Radiological Survey: Data and dose assessments’, Health Physics, vol. 73, no. 1, pp. 37–48. https://doi.org/10.1097/00004032-199707000-00004 Roehr, B 2010, ‘Asians and Pacific Islanders in US need greater prominence in research’, BMJ, vol. 340. doi:10.1136/bmj.c2495 Rohs, F 1999, ‘Response shift bias: A problem in evaluating leadership development with self-report pre-test–post test measures’, Journal of Agricultural Education, vol. 40, pp. 28–37. https://doi.org/10.5032/ jae.1999.04028 Rothfeldt, L, Patil, N, Haselow, D, Williams, S, Wheeler, J & Mukasa, L 2016, ‘Notes from the field: Cluster of tuberculosis cases among Marshallese persons residing in Arkansas – 2014–2015’, Morbity and Mortality Weekly Report, vol. 65, no. 33, pp. 882–83. doi:10.15585/mmwr.mm6533a7 Simon, S 1997, ‘A brief history of people and events related to atomic weapons testing in the Marshall Islands’, Health Physics, vol. 73, no. 1, pp. 5–20. https://doi.org/10.1097/00004032-199707000-00001 Smith, M, Kaufman, N & Dearlove, A 2013, ‘External community review committee: A new strategy for engaging community stakeholders in research funding decisions’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 301–12. doi:10.1353/cpr.2013.0030 Srinivasan, S & Guillermo, T 2000, ‘Toward improved health: Disaggregating Asian American and Native Hawaiian Pacific Islander data’, American Journal of Public Health, vol. 90, no. 11, pp. 1731–34. https://doi.org/10.2105/ajph.90.11.1731 Takahashi, T, Schoemaker, M, Trott, K, Simon, S, Fujimori, K, Nakashima, N & Saito, H 2003, ‘The relationship of thyroid cancer with radiation exposure from nuclear weapon testing in the Marshall Islands’, Journal of Epidemiology, vol. 13, no. 2, pp. 99–107. https://doi.org/10.2188/jea.13.99 Tashakkori, A & Teddlie, C 2010, SAGE Handbook of mixed methods in social & behavioral research, 2nd edn, SAGE Publications, Thousand Oaks, CA. Tervalon, M & Murray-Garcia, J 1998, ‘Cultural humility versus cultural competence: A critical distinction in defining physician training outcomes in multicultural education’, Journal of Health Care for the Poor and Underserved, vol. 9, no. 2, pp. 117–25. https://doi.org/10.1353/hpu.2010.0233 Wallerstein, N & Duran, B 2006, ‘Using community-based participatory research to address health disparities’, Health Promotion Practice, vol. 7, no. 3, pp. 312–23. doi:10.1177/1524839906289376 Woodall, P, Scollard, D & Rajan, L 2011, ‘Hansen Disease among Micronesian and Marshallese persons living in the United States’, Emerging Infectious Diseases, vol. 17, no. 7, pp. 1202–08. doi:10.3201/ eid1707.102036 Young-Lorion, J, Davis, M, Kirks, N, Hsu, A, Slater, J, Rollins, N & McGinnis, P 2013, ‘Rural Oregon community perspectives: Introducing community-based participatory research into a community health coalition’, Progress in Community Health Partnerships, vol. 7, no. 3, pp. 313–22. doi:10.1353/cpr.2013.0032 McElfish, Rowland, Ayers, O'Connor, Purvis, Aitaoto, Capelle, Laelan, Felix, Stewart and Yeary Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019 20 PAGE NUMBER NOT FOR CITATION PURPOSES https://doi.org/10.1097/00004032-199707000-00004 https://www.bmj.com/content/340/bmj.c2495 https://doi.org/10.5032/jae.1999.04028 https://doi.org/10.5032/jae.1999.04028 https://www.cdc.gov/mmwr/volumes/65/wr/mm6533a7.htm https://doi.org/10.1097/00004032-199707000-00001 http://muse.jhu.edu/article/521224 https://doi.org/10.2105/ajph.90.11.1731 https://doi.org/10.2188/jea.13.99 https://doi.org/10.1353/hpu.2010.0233 https://journals.sagepub.com/doi/10.1177/1524839906289376 https://wwwnc.cdc.gov/eid/article/17/7/10-2036_article https://wwwnc.cdc.gov/eid/article/17/7/10-2036_article http://muse.jhu.edu/article/521225