Layout 1 [Qualitative Research in Medicine & Healthcare 2019; 3:7943] [page 15] Introduction Training in translational science aims to help partici- pants transfer fundamental research discoveries from lab- oratory into clinical practice. Decades ago, the National Institute of Health issued funding mechanisms, referred to as Clinical and Translational Science Awards (CTSAs) to support university development of a clinical translational science institutes (CTSI) and a cadre of physician-scientists whose research aims are directed at achieving translational science goals. The processes directed towards helping stu- dents develop an understanding and acquiring related prac- tices necessary to conducting translational science are not well documented.1-4 Translational science institutes across the US support students through TL1 (predoctoral training) and T32 (pre- and postdoctoral training) funding mecha- nisms. Educational experiences in these training programs have generally focused on cultivating collaboration with seasoned translational scientists while students also take coursework related to team science, translational science models and research ethics. However, studies on the devel- opment of translational science educational environments and on the teaching of research ethics are scarce.5 The pur- pose of this study was to analyze clinical translational sci- ence students’ perceptions of their research ethics coursework, through interviews6 and focused groups. A sec- ond purpose was to describe how well the course content and instructional strategies aligned with the principles of ethical training as described in the literature. Teaching research ethics in translational science DuBois et al.7 identified what content should comprise coursework in responsible conduct of research. Kon et al.8 also provided a summary of current materials used across clinical translational science hubs. Similar to the Office Clinical translational students’ perceptions of research ethics coursework: a case study Linda S. Behar-Horenstein, Huibin Zhang School of Human Development and Organizational Studies in Education, University of Florida, Gainesville, FL, USA ABSTRACT Relatively unknown is whether coursework in responsible conduct of research actually achieve the purposes for which it is designed. In this study, the authors report clinical translational students’ perceptions of their research ethics coursework and the alignment between course content as recommended in the literature. We used grounded theory to portray emergent findings across focus groups and semi- structured interviews among 31 participants at one clinical translational science hub. We also used thematic analysis to analyze course syllabi. Two themes emerged: Averting scientific misconduct and Responding to ethical dilemmas. Students reported that they did not acquire requisite strategies to address research ethical dilemmas. One of the course syllabi indicated the provision of active learning opportunities. However, the findings did not offer support. Developing experiential learning activities and ensuring that course content is aligned with the contemporary ethical practices, such as case study and portfolio development, is recommended. Correspondence: Linda S. Behar-Horenstein, School of Human Development and Organizational Studies in Education, University of Florida, Gainesville, FL, USA. E-mail: Lsbhoren@ufl.edu Key words: Responsible conduct of research; Clinical translational science; Student perceptions; Grounded theory; Thematic analysis. Contributions: LB-H made substantial contributions to the concep- tion or design of the work; the acquisition, analysis, or interpreta- tion of data for the work; drafting the work or revising it critically for important intellectual content; final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. HZ made substantial contributions to the analysis, or interpretation of data for the work; drafting the work or revising it critically for important intellectual content; final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Conflict of interest: the authors declare no potential conflict of in- terest. Funding: research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under University of Florida Clinical and Translational Science Awards TL1TR001428, KL2TR001429, and UL1TR001427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Received for publication: 17 November 2018. Revision received: 31 January 2019. Accepted for publication: 31 January 2019. This work is licensed under a Creative Commons Attribution Non- Commercial 4.0 License (CC BY-NC 4.0). ©Copyright L.S. Behar-Horenstein and H. Zhang, 2019 Licensee PAGEPress, Italy Qualitative Research in Medicine & Healthcare 2019; 3:15-24 doi:10.4081/qrmh.2019.7943 Qualitative Research in Medicine & Healthcare 2019; volume 3:15-24 No n- co mm er cia l u se on ly of Research Integrity9 Kon et al. recommended content domains i) data acquisition, management, sharing, and ownership; ii) mentor/trainee responsibilities; iii) publi- cation practices and responsible authorship; iv) peer re- view; v) collaborative science; vi) human subjects; vii) research involving animals; viii) research misconduct; and ix) conflict of interest and commitment). We adopted Helton-Fauth et al.10 four dimensions of research ethics: data management, study conduct, profes- sional practices and business practices for this study be- cause it was the most contemporary framework. Data management refers to the appropriate methods for han- dling and reporting data. Study conduct refers to informed consent and debriefing, confidentiality, and the treatment of human and animal participants according to board guidelines. Professional practices refer to protection of in- tellectual property, protection of public welfare and the environment, adhering to professional commitments, mentoring, and the treatment of collaborators. Business practices refer to contract and grant bidding, the use of physical resources, conflict of interest, and laboratory management. Gaining insight into how well these dimen- sions are enacted in research ethics education needs to be examined. Researchers have shown that approaches to teaching the responsible conduct of research (RCR) tend to negate the importance of developing tolerance and responding ap- propriately to complex ethical dilemmas. Antes et al.11 ex- amined the impact of RCR instruction on research ethical decision-making with 173 participants. Results showed that the ethicality of decisions made in relationship to data man- agement, study conduct, and professional practices did not improve or decline following instruction. Also, following the completion of the RCR course, the ethicality of partic- ipant’s business related decisions (i.e., contract bidding) de- creased. Individual awareness of situational elements and their ability to weigh personal motivations improved. How- ever, with regard to the social elements of ethical problems, they were more internally focused and closed off when making decisions. Antes et al.11 concluded that when in- structors exaggerate evading ethical dilemmas, students de- velop unrealistic expectations about their capabilities to handle ethical problems. These researchers showed that the RCR research ethics courses may not be effective or may even be harmful.7 They recommended exploring possible obstacles and characteristics of effective research ethics ed- ucation in future studies. Yarborough and Hunter12 argued that the role of re- search ethics is not sufficiently emphasized in science at the graduate and postgraduate levels. They emphasized the Presidential Commission for the Study of Bioethical Issues Moral Science: Protecting Participants in Human Subjects Research13 report recommendations, which sug- gested creating a culture of responsibility, as well as uti- lizing creative and innovative approaches in teaching research ethics. Moreover, they advised that research ethics instructors focus on teaching researchers how to avoid misconduct and counsel them about what they must and must not do. Experiential learning theory This study is grounded in the well-established Experi- ential Learning Theory (ELT).14 This model posits that ac- quisition of new knowledge and skills depends on a cycle of successive learning experiences. The premise underlying ETL is that learning evolves via four types of engagement: i) experiential (i.e., concrete experience) whereby re- searchers gather information from the world (e.g., obser- vations of inappropriate author attribution); ii) reflective (i.e., reflective observation) whereby researchers take time to think, process, organize and relate inputs to other known factors that surround that experience (e.g., why PIs falsify data); iii) abstract (i.e., abstract conceptualization) whereby researchers create new meanings and develop new ways of looking at existing information (e.g., how data falsification differs among senior and junior researchers); and iv) action (i.e., active experimentation) whereby researchers actively test a hypothesis (e.g., using an existing dataset, the re- searcher tests her emergent hypothesis that senior faculty tend to falsify data more frequently than junior faculty). This new knowledge is then re-introduced back into the learning cycle. However, the cycle is in continuous flux as researchers engage in experiences inside and outside the learning environment. The ELT promotes reflective con- versation that enables researchers to shape their responses to project goals (e.g., creating a conversational space for members to reflect on their experiences with research eth- ical dilemmas) while applying and assessing the utility of new knowledge15,16 ELT stimulates sharing the functional role of the leadership17 whereby personal needs are replaced by shared roles necessary for meeting project or team goals. Kolb18 showed that the training of researchers or teams are cultivated by sharing experiences and reflecting on the meaning of those experiences together. Materials and Methods Participants Prior to beginning the study, IRB approval was granted (IRB #2015-U-0988). The study sample (n=31 out of 53 or 58%) included 5 predoctoral and 10 postdoc- toral fellows funded by a T32 award and 16 students funded by a TL1 award. Participants were drawn from all students (15 predoctoral, 17 postdoctoral, and 21 TL1 stu- dents) enrolled in one or more of the RCR courses at a single CTSI hub over a two-year period. Postdoctoral fel- lows were funded by the National Research Service Awards postdoctoral training program (T32), which is de- signed to ensure that a diverse and highly trained work- force is available to meet the nation’s biomedical, behavioral, and clinical research needs. TL1 graduate stu- [page 16] [Qualitative Research in Medicine & Healthcare 2019; 3:7943] Article No n- co mm er cia l u se on ly dents funded by a CTSA award were expected to acquire the skills needed to develop a career in multidisciplinary clinical and translational research. Mentoring and didactic training was provided for all participants. Also, while en- rolled in the program, participants performed clinical and/or translational research in health-related fields. Per grant requirements, all T32 and TL1 trainees were also required to enroll in one of the two academic health sci- ence center’s research ethics courses: RCR or the Ethical Policy Issues in Clinical Research. Procedures We conducted 15 semi-structured interviews with T32 pre- and post-doctoral students. Interviews ranged from 17-55 minutes. Five focus groups, ranging from 60-90 minutes, were conducted with 4-6 pre-doctoral TL1s. We conducted interviews with sets of T32 trainees because they worked in different laboratories. This ap- proach was the easiest way to encourage their participa- tion in the study. Since TL1 trainees were cohort groups, we used focus groups to access their perceptions. The purpose of the interview and focus groups protocols was to evaluate the program’s effectiveness in providing rel- evant training experiences. We asked students to de- scribe to what extent they believed that coursework prepared them to avoid misconduct and how to deal with scientific ethical dilemmas. We also asked them to pro- vide specific examples of learning activities that they considered instrumental in increasing their knowledge of RCR and skills. The same questions were used for both data collection types. Data analysis The researchers, a senior professor in education, and a 2nd year doctoral student in school psychology independ- ently analyzed the compiled data by applying a construc- tivist, grounded-theory approach as described by Charmaz.19 Prior to the analysis, we assigned pseudonyms to each participant. Both Behar-Horenstein and Zhang read each of the focus group and interview transcripts line-by-line as a separate entity. Each of us developed a list to denote emergent themes and described each theme in 4-6 words. We met to discuss our independent notions of the emer- gent themes. Then, we used open coding while reading line by line. We compared open coding to ensure that a systematic approach to analysis was occurring before moving on to develop categories and emergent themes. The authors reached consensus on the identification of the categories as well as on the two themes that emerged in this study, awareness of ethical violations and responding to ethical dilemmas. Consistent with Charmaz’s19 guid- ance, we used gerunds to connote observable and concep- tual action in the data. Each researcher tagged text and we audited each other's selected text to ensure agreement. We illustrate the progression from open codes to cat- egories/axial codes and themes in the following exam- ples. For a portion of the focus group text that read: I think that’s more of the benefits of the course, it just gives you the vocabulary to say this is how I identify this is wrong, this is the kind of…it was open coded as de- veloping vocabulary. Next, it was conceptualized as be- longing to the category/axial code ethical violation belonging to the theme, Awareness of ethical violations. How interview data was coded can be seen in the fol- lowing example. The text that read: Depends on the dy- namics of the relationship with whoever is scientific misconduct is occurring with, and there is no clear-cut answer was open coded as Responding depends on dy- namics. This text was conceptualized as belonging to the category/axial code, Ethical dilemma response, belong- ing to the theme, Responding to ethical dilemmas. There were two axial codes, Ethical violation corresponding to theme, Awareness of ethical violations, and Ethical dilemma response corresponding to theme, Responding to ethical dilemmas. An audit trail shows progression from selected interview and focus group data to open coding, categorization, themes, and their related concep- tual definitions (Table 1). The use of two independent coders aided us in re- ducing the potential for bias. Reading line-by-line and coding segments assisted in making supporting quota- tions more accessible. Data were analyzed by coding and identifying participants’ spoken words to support the categories that were identified. This rigorous and systematic approach allowed us to feel confident that what we report is representative of participants’ per- spectives. Attention to credibility, and confirmability facilitated establishing trustworthiness. Credibility, confidence in the truth of the findings, was achieved through triangulation and peer debriefing. Triangulation was accomplished by: i) using two analysts; ii) reviewing multiple interview transcripts; iii) using qualitative line by line coding and peer debriefing to ensure the accuracy of interpretations. Confirmability was achieved by engaging more than one person in analyzing the data. Analysis of course syllabi We used thematic analysis20 to analyze the two ethics course syllabi. First, we applied Helton-Fauth’s et al.10 four dimensions of research ethics to search for what we could argue was evidence of content, aligned with topics in data management, study conduct, professional prac- tices, and business practices. Next, we reviewed the syl- labi for the presence or absence of described learning experiences and examined the syllabi for a description of instructional methods used (e.g., lecture, small group dis- cussions) and evidence of experiential learning activities (e.g., student led-discussions or presentations, exploration of research ethics case studies). [Qualitative Research in Medicine & Healthcare 2019; 3:7943] [page 17] Article No n- co mm er cia l u se on ly Results Awareness of ethical violations This theme was defined conceptually as, having a clear idea of how to recognize what constitutes unethical con- duct (e.g., falsifying data, inappropriately handling, storing sharing, and reporting data, not treating human and animal subjects according to board guidelines, accepting author- ship without meeting acceptable guidelines for intellectual contributions). Representative examples showed that par- ticipants described if and how research ethics coursework offered adequate preparation to help them avoid unethical misconduct. Their views on whether the research ethics coursework would help them avert scientific misconduct and if they received adequate preparation to avoid miscon- duct were mixed. Five (16%) participants believed that they already knew how to prevent scientific misconduct from past experiences or from interaction with mentors. Coursework did not change their perspective on ethical misconduct. Beverly shared that, I don’t think T32 specif- ically will help me to avert scientific misconduct. I think that’s just something you learn from your mentors. Diane said, That’s one where I see a little bit less impact on what I’ve been doing just because I used to work as the project manager. All of my background is very focused on ethics training. Similarly, John asserted that, My academic pro- gram and my department already have a lot of emphasis on research conduct, and so I don’t think that the T32 has really added to that. Both Diane and John asserted that they obtained requisite knowledge on averting unethical actions prior to entering the program. 10 (32%) participants believed that the course would help them avert scientific misconduct. Several participants noted feeling confident about this specific issue. Andrew said, I think being forced [to] interact with people who are medically oriented…pushes me further away from sci- entific misconduct…Susan felt that learning about the im- portance of interpreting data in a less biased fashion in the research ethics course would assist her in averting scien- tific misconduct in the future. The responses suggested the courses helped some trainees recognize science mis- conduct more easily and would likely ensure that they avoided it in future interactions. Ten (32%) participants opined that the research ethics course would not help them avert scientific misconduct. For example, Robbie reported having no ethical training within the program. Natasha did not believe the ethical [page 18] [Qualitative Research in Medicine & Healthcare 2019; 3:7943] Article Table 1. Audit trail progression: selected interview (I) or focus group (FG) data, open coding, categories/axial codes, themes, and conceptual definitions. I and FG data Open codes Categories/Axial codes Themes Conceptual definition I think that’s more of the Developing vocabulary Ethical violation Awareness of ethical Having a clear idea of how benefits of the course, it just violations to recognize what gives you the vocabulary to constitutes unethical say this is how I identify conduct (e.g., falsifying this is wrong, this is the data, inappropriately kind of… (FG) handling, storing sharing, and reporting For me, it just creates Developing awareness Ethical violation data, not treating human awareness for us being and animal subjects in the scientific community according to board because actually, I didn’t guidelines, accepting realize, learning through authorship without all these examples I’ve meeting acceptable learned in class so far, guidelines for intellectual how common it is. (FG) contributions) Depends on the dynamics Responding depends Ethical dilemma response Responding to ethical Acquiring strategies or of the relationship with on dynamics dilemmas principles to effectively whoever is scientific address situations, in misconduct is occurring which scientific with, and there is no misconduct occurs, clear-cut answer. (I) especially with people who they had have It would depend on what Overlooking vs Ethical dilemma response special relationships they were doing if that’s deliberately doing with you, such as something that could be mentors or peers addressed directly with them…maybe just something that they overlooked and didn’t realize they were doing vs something that was deliberately depends. (I) No n- co mm er cia l u se on ly class helped her to avert scientific misconduct. Cindy rea- soned that, if you are going to be an ethical person, you are going to be an ethical person. She asserted that taking a research ethics class was inconsequential. Julie shared that she did not, …think that the CTSI has a huge empha- sis on that scientific misconduct. Maybe just because I just took an ethics class like a little while ago you know… Pro- portionally, the number of participants who did and did not believe that research ethic courses changed their per- ception of science misconduct was the same. This finding suggested that perhaps the courses were not instrumental in augmenting participant knowledge of research ethics or in promoting their understanding of how to avert and address it. The remaining six (19%) participants did not share opinions regarding whether the course was helpful in their prospective efforts to avert scientific misconduct. Among those 10 (32%) participants who believed that the course would help them avert scientific misconduct, five (16%) participants reported that the coursework pro- vided them with a clear idea of how to recognize what constitutes unethical conduct (e.g., managing data, inap- propriately, not treating human and animal subjects ac- cording to board guidelines, accepting authorship without meeting acceptable guidelines for intellectual contribu- tions). Eight (26%) participants talked about their aware- ness of ethical violations and remarked that the research ethic courses had improved their awareness. For example, Sue reported that, It’s easy to not realize that it might not be ethical, but at least it points it out and makes it – now to me I think it’s more obvious, now that I’ve been exposed to it and I can tell. Six (19%) participants shared that their awareness of ethical violations had improved by talking about unethical examples. Violetta acknowledged that … you cannot change the trend of the data because then you would be falsifying the data. Daniel felt that one benefit of the course was acquiring the vocabulary to say this is how I identify [that] this is wrong. Sue pointed out how class content and interactions helped her appreciate that, It’s easy to not realize that it might not be ethical, but at least it points it out and makes it…more obvious, now that I’ve been exposed to it. Evie pointed out that coursework ex- amples made her realize just how common it is. Learning about potential ethical violations raised participant cog- nizance of what constitutes misconduct and helped them recognize that it was more prevalent than they thought. Responding to ethical dilemmas This theme was defined conceptually as, acquiring strategies or principles to effectively address situations, in which scientific misconduct occurs, especially with people who they had have special relationships with you, such as mentors or peers. Exemplification of this theme was evidenced by participants describing if their course- work offered strategies to help them address scientific misconduct. Sue said: In that course they talk about how if you experi- ence it, how to approach the correct people, like the Research Integrity Office, to think of your spe- cific university, if it were to get to that point, but also how to try to recognize the problem if you see it in your lab setting [such as] how it can affect your research [or] how to talk to your PI maybe and/or if you think that that’s what’s happening. I haven’t been exposed to that thankfully, but I feel like if it were to happen, [now I have a better sense of] what actions to take. Anthony said the course content and interactions, raise[d] awareness of how things aren’t black and white, but the principals involved in it, staying briefed on it and honestly reinforcing that the end never justify the means with any research endeavor… Only one person talked about particular instructional strategies used in the courses such as role play or the types of learning activities (i.e., developing their own research ethics case and potential responses, debating ethical responses to cases involving transplantation, genetic testing, or cloning). Cindy re- ported that participants never really learned exactly how to confront someone committing ethical misconduct be- cause it mainly focused on cases that had happened in the past. Thus, students received little to no strategies or ap- proaches for how to respond to observations of unethical behavior. They did not report who to go to if they saw a problem. This finding points to a need teaching partici- pants about potential on the approaches that could be used while responding to unethical behaviors. When talking about how to respond to ethical dilemmas, three (10%) participants believed that they had received clear direc- tions or guidance. Participants acknowledged that that their response to a violation would depend on the interactional context and on the type of ethical issues they observed. Cory stated that his reaction to an ethical dilemma would depend on the dynamics of the relationship with whoever the scien- tific misconduct is occurring with, and there is no clear- cut answer. His comment points out the imminent, but multi-faceted conflict that ensues when misconduct is caused by individuals who mentor and fund student re- search. Jordan said that his course helped him realize that things are not black and white, and acknowledged that there were ethical principles he could use as references. Larry appreciated learning about the ambiguity of ethical dilemmas. He claimed that grey areas are mine every day and now understood that individuals could make, just bad decisions or careless ones. Summary sentence here. Three (10%) participants felt confident that they could handle challenges related to ethical misconduct because they learned related strategies from previous classes, or they believed they could find people with whom they could seek their counsel. Notably, these participants ref- [Qualitative Research in Medicine & Healthcare 2019; 3:7943] [page 19] Article No n- co mm er cia l u se on ly erenced courses that were not included in this study. Kate who had taking another research ethics class that was not part of this study, reported that she acquired a solution- focused perspective, an ability to recognize problems, and would seek help from others. Only three (10%) participants said that when faced with an ethical dilemma they would contemplate its nature before taking any action. This response signifies their un- certainty. In contrast, Diane reported that if faced with un- ethical mentor behavior that initially she would try to determine the feasibility of discussing the situation di- rectly with person directly involved and assess if it was a conscious action or an overlooked mistake. Only three (10%) participants reported that they ob- tained new insight about how to handle ethical dilemmas although they did not offer more specific details. Suzi said, It’s been helpful and then it also brings up a lot of dilemmas that I didn’t really know still existed when con- sidering placebo-controlled trials. While claiming that she acquired insight into how to resolve dilemmas, she did not describe specific strategies that would guide her reactions. Analysis of ethics courses Using thematic analysis20 of the RCR course syllabus, we found that while it covered Helton-Fauth’s et al.10 four dimensions of research ethics. However, only one element within the business practices – conflict of interest – was referenced in the syllabi. The findings showed that course content did not include consent and debriefing, confiden- tiality, mentoring, protection of public welfare and envi- ronment, contract and grant bidding, the use of physical resources, and laboratory management. The Ethical Policy Issues in Clinical Research course covered four dimensions of research ethics. However, it did not include coverage of mentoring, treatment of col- laborators, contract and grant bidding, the use of physical resources, and laboratory management (Table 2). Neither of the research ethics course syllabi included a list of course goals and course objectives or a description of instructional delivery methods, assignments, or evalu- ation methods. Although the RCR course listed a numer- ical scale for assigning letter grades (unlike the Ethical Policy Issues in Clinical Research course), the linkage be- tween student work and how assignments were graded was not described. From a pedagogical perspective, these syllabi would leave most students asking what tangible learning outcomes they would be able to demonstrate by the conclusion of these courses. Without a list of learning objectives, it is difficult to appreciate where the instructor is headed and what students are expected to be able to do.21 A lack of clarity regarding course objectives and stu- dent work products likely influenced the apparent lack of experiential teaching and ultimately the quality of student experiences.21 Other than the mention of team-based learning for some RCR course sessions, there was no de- scription of how learning activities fostered student dia- logue or hypothetical actions to promote effective re- sponses to real world ethical challenges.22,23 Also, we found no other information in the RCR course syllabi to demonstrate that active instruction was occurring. In the Ethical Policy Issues in Clinical Research syllabus, the inclusion of interactive analysis of human subject scan- dals using data and video, discussion of informed consent procedures applied to cases, review of recruiting ads, and critique of informed consent suggested the use of active learning experiences. However, a failure to describe re- lated student work products and processes, calls into ques- tion whether such activities were actually provided. Discussion From a theoretical perspective the study findings show that students found the ethics courses to be lacking an em- phasis in promoting three of the four types of learning, re- flection, abstraction, and action. Students reported little to no opportunity create new meanings, to develop new ways of looking at information, or to actively test hy- potheses.14 Despite the mentioning of active learning ex- periences documented in the Ethical Policy Issues in Clinical Research syllabus, students’ perceptions did not align. However, consistent with the first ELT level, expe- riential, students described receiving conceptual informa- tion from both courses. The findings showed that the research ethics courses were effective in raising participant awareness of predom- inant forms of ethical violations such as, inappropriate data management, failure to treat human or animal sub- jects appropriately, and violating others’ intellectual prop- erty. About a third of the participants believed that the courses would help them avert prospective unethical mis- conduct while another third disagreed. The remainder ex- pressed no opinion one way or another or stated that they already knew how to prevent scientific misconduct. Many participants found the research ethics courses peripheral to their interests, or even a distraction from what they con- sider their real work. None of the participants reported acquiring specific strategies regarding how, if presented with an ethical dilemma in the future, they would address those dilemmas. Overall, participants were not being well enough equipped to address ethical dilemmas. This raises questions regard- ing the potential impact for them going forward as profes- sional/academic researchers and, in turn, on their students, and with those with whom projects are conducted. These findings suggest that greater effort must be devoted to identifying specific guidelines to help students handle dilemmas.24 Several participants reported that they had no idea how they would actually respond if presented with an ethical dilemma. Consistent with Domen’s25 suggestion, research ethics educational experiences ought to focus not only on conveying relevant knowledge but also on foster- [page 20] [Qualitative Research in Medicine & Healthcare 2019; 3:7943] Article No n- co mm er cia l u se on ly ing ethical problem-solving skills. Bærøe22 proposed adopting a medical framework to use when making ethical decision. He recommended a five-step process that includes: i) identification of the eth- ical challenge; ii) development of a normatively justified approach; iii) testing of the feasibility of the proposed ap- proach in a real-world setting; iv) implementation of the adjusted result; and v) evaluation of the resulting practice. Unsurprisingly, none of the participants talked about how to make ethical decisions based on this model. [Qualitative Research in Medicine & Healthcare 2019; 3:7943] [page 21] Article Table 2. Thematic analysis of ethics topic coverage and learning activities in the responsible conduct of biomedical research and ethical policy issues in clinical research courses. Responsible conduct Evidence of learning Ethical policy issues Evidence of learning of biomedical research activities in clinical research activities Topics* Data management Appropriate methods of Yes/Data Management None Yes/Issues in genetics Regulation cases handling, and reporting data Practices and human embryonic stem cells, Fraud, fabrication and falsification of data Study conduct Yes Yes Consent & debriefing No None Yes/Informed Consent Critique of informed in Research consents Translation of protocol language comprehensible to lay audience Confidentiality No None Yes/Maintaining the Case studies Confidentiality of Research Data Treating human participants Yes/Protection of Human None Yes/Clinical Research with Case studies, according to board guidelines Subjects Special Populations: Critiques of recruiting Children and Adolescents/ ads, critique cases Clinical Research with with minors, vulnerable Special Populations: populations and prisoners Cognitively Impaired People, Prisoners, Soldiers and Students Treating animal Yes/Welfare of None Yes/Ethical Use of Animals None participants according Laboratory Animals in Research to board guidelines Professional practices Yes Yes Protection of intellectual Yes/Authorship & None Yes/Ethical issues in light Outline criteria for property Publication of intellectual property determination of authorship. issues/Rules of Authorship Application of misconduct procedures Protection of public welfare No None Yes/Ethical issues in Case studies and environment community engagement research Adhering to professional Yes/Avoiding Research None Yes Explain the procedures for commitments Misconduct reporting and investigating misconduct in research Mentoring Yes/Mentor & Trainee None No None Responsibilities Treatment of collaborators Yes/Collaborative Research None No None Business practices Yes Yes Contract and grant bidding No None No None Use of physical resources No None No None Conflict of interest Yes/Conflicts of Interest None Yes/Conflicts of Interest Critique of cases & Commitment Laboratory management No None No None *Denotes yes to indicate if topic was described in syllabus or no if it was not taught, and if appropriate indicates what content domain. No n- co mm er cia l u se on ly Also absent from student discussion was an explicit conversation as recommended by Bærøe regarding how to counteract identified barriers to ideal decision-making, regarding the context (e.g., resource scarcity, geographic and demographic challenges), biased distribution of op- portunities among individuals (e.g., voicing concern, de- liberating a processing information, reaching a conclusion and acting on it), and pragmatic barriers that undermine feasibility (e.g., the organization of information flow in institutions).22 Students made only a passing reference to the interplay between the importance of gaining theoreti- cal and practical ethical knowledge when they hypothe- sized how they might respond to misconduct committed by a mentor. Otherwise, participants did not describe chal- lenges related to the real world of practice and ethical de- cision-making22 Moreover, none of the participants discussed how organizational systems or cultural norms might ultimately contribute to impeding human research subject protection.26 None of the participants discussed the inherent chal- lenges related to providing informed consent when it em- braced clinical situations new to both patient and clinician, or how the informed consent process might be impacted by issues involving transplantation, genetic testing, or cloning.27Also, no participant referenced how they might respond to ethical and scientific issues related to the use of health technologies, the ethics of randomization, clinical trials28 or non-medical tasks, such as information technol- ogy, engineering, nanotechnology, and economy.23 Although Hostiuc et al.23 argued that the practice of research ethics should be concordant with the elements of the translational process: phases, gaps, and data transfer, not a single participant referred to these matters. Along the same lines there was no evidence showing that partic- ipants discussed critical ethical questions coming from in- teractions between investigators, IRBs and regulators. Along the same lines, participants did not describe the kind of attention they should demonstrate towards trial safety dimensions such as disease gene transfer trial.29 Analysis of the courses showed that both covered most of the recommended content domains of research ethics. However, there was little evidence to illustrate the use of learning activities in the RCR course that could have oth- erwise promoted deep student understanding of the prac- tical application of research ethics information. Although some of active learning experiences were listed in the Eth- ical Policy Issues in Clinical Research syllabus, without description of related student work products or other out- comes, we cannot discern if such activities took place. These findings suggest that neither course may not have been adequately equipping students to address ethical dilemmas in the profession. From our analysis, it seems likely that both instructors relied heavily on a passive teaching approach characterized by dissemination of in- formation. Antes et al.7 used scenarios to measure ethical deci- sion-making. However, as shown in this study neither course employed role play or scenarios that might foster practice and application of what students were taught in class. With respect to topics pertaining to: i) treating human participants according to board guidelines, ii) pro- tection of intellectual property, iii) adhering to profes- sional commitments and iv) coverage of conflicts of interest, there was no information regarding whether this was an individual or group activity. Additionally, the as- sessment of student work products was not described. Kon et al.8 and Dubois et al.7 focused on RCR course content and materials. They recommended assessing the effectiveness of courses to determine whether existing RCR training programs are effective in achieving their in- tended purposes, while noting that it is a limitation of most studies.30-32 Although this study represents a single clinical translational hub, it represents a sizeable number of participants. We suggest that scholars consider methodological changes in future studies. First, we advise researchers to interview course directors, make direct observation of teaching or administer surveys to measure participant eth- ical knowledge before and after course completion. Sec- ond, we suggest differentiating the participants, by their field of study and their level of training, and ethics courses participation to determine if there are subgroup differ- ences. Third, recommend asking participants for their sug- gestions and recommendations regarding how courses can be improved. Educational implications and best practices In this study, the researchers examined students’ per- ceptions of how well research ethics coursework prepared them to respond to ethical research dilemmas in the clin- ical translational research workplace. We examined the fit of course content with best practices as described in the literature. As shown, outcomes were not uniformly posi- tive. Limitations associated with course content and in- structional practices were also described. We draw upon Klimmelman29 and Domen25 and ELT to offer recommen- dations regarding how to improve course effectiveness and achieve desired student outcomes. First, while teach- ing research ethics, we propose that instructors distribute attention equally between the known and unknown risks of the translational research for all actors who participate in it.7 This can be addressed in part by utilizing currently available resources, updating, and revising the course con- tent. Second, we advise adding case-based scenarios that cause students to hypothesize and discuss how they would respond to real world dilemmas in research ethics practice such as: i) counteracting identified barriers; ii) embracing clinical situations new to both patient and clinician; iii) pondering the advances of science and ethical limitations related to transplantation and health technologies; iv) weighing the ethics of randomization, clinical trials, and non-medical tasks, such as information technology, engi- [page 22] [Qualitative Research in Medicine & Healthcare 2019; 3:7943] Article No n- co mm er cia l u se on ly neering, nanotechnology, and economy; and v) respond- ing to the phases, gaps, and data transfer inherent to trans- lational research.22 Implementing this approach along with experiential learning activities would strengthen student capacity to discuss, demonstrate and write about their rea- soning.22-24 and bolster their capacity to judiciously weigh competing viewpoints.25 Third, we suggest using case-based scenarios and ask- ing students to respond in writing how they would address particular ethical research problems.33-35 Further, we suggest asking students to provide evidence that supports their per- spective. This activity would move students beyond the stages of concrete experience, reflective observation and abstract conceptualization towards active experimentation and hypothesis testing. Along the same lines, classmates could serve as peer evaluators and provide feedback.34 Such an approach would encourage students to introspect, doc- ument and provide evidence to support their reasoning while bolstering their skills in written communication. Ad- ditionally, it would provide a record of changes for both in- structor and student consideration as students gain maturity in their ethical reasoning skills. Fourth, we endorse fostering student capacity to weigh complex interactional information. We suggest requiring students to develop their own case studies around research ethical dilemmas and include a set of questions for readers to use as they contemplate responding.36 This activity would provide additional opportunities to work through and solve research ethical dilemmas as a team. Also, we commend the use of real-time role play ac- tivities during class whereby students would respond in real time to ethical research dilemmas is suggested. Fol- lowing each role play, the instructor could review student actor responses, engage the class audience in debriefing and ask the students to offer recommendations for alter- native responses.33-34 This activity would encourage think- ing in action and practice in providing prompt responses. To further strengthen student ability to judiciously weigh interactional information in the face of ambiguity, we suggest assigning a group project that requires students to find a real case and describe how the researchers re- sponded. Similar to the suggestions above, this type of ac- tivity would promote the application of newly learned information and provide opportunities for students to spec- ulate about its utility. Consistent with the premise of ELT,14- 18 each of the aforementioned activities successively builds upon initial knowledge acquisition and moves towards ac- tive experimentation. Teaching in this manner is likely to ensure that students develop facility in responding inde- pendently and effectively to research ethical dilemmas. The National Institute of Health requires that all fed- erally funded institutions offer RCR. However, requiring enrollment in research ethics courses is insufficient with- out a concomitant emphasis on evaluating instruction and student outcomes, suggesting that they are in effect a stamp of approval. To ensure that research ethics courses adequately prepare researchers to handle the complexities of ethical dilemmas in translational science research, we offer several ideas to further this research agenda. First, we recommend evaluating course activities to demon- strate how well teaching research ethics impacts student knowledge, attitudes and skills. Second, we encourage researchers to simultaneously interview course directors as well as students who com- pleted the courses and compare datasets. Third, we rec- ommend using teaching observations to document the nature of student-faculty interactions. Analysis of this dataset could shed light on how instructional practices im- pact potential changes in student knowledge, attitudes, and skills. Fourth, we suggest acquiring bigger sample sizes across the nation’s CTSIs to amass a large database of student outcomes and insight about the similarities and differences in course content and learning activities. Using a standardized approach to evaluation, comparative findings from such studies might advance the common metrics movement by permitting institutional and intra- institutional perspectives that showcase outcomes from research ethics coursework.37 Fifth, to ensure the contem- porariness of educational delivery, we recommend mov- ing these courses to an online format while examining the effect of online teaching and learning outcomes.38 Finally, we urge undertaking an analysis of student portfolio doc- umentation of ethical issues and their responses over suc- cessive years is recommended to illuminate the manner in which students make meaning and apply their emergent knowledge of research ethics education. Conclusions Using grounded theory and thematic analysis, the au- thors report clinical translational students’ perceptions of their research ethics coursework and the alignment between course content and what is recommended in the literature. While course content focused on most of the recommended dimensions of research ethics, students had little opportu- nity to actively apply what they were learning and received few strategies to assist them in addressing research ethical dilemmas. We suggest course revisions, supported by ex- periential learning activities, that aim to ensure that student acquire and utilize requisite tools to address ethical dilem- mas in clinical translational research. References 1. Allen D, Ripley E, Coe A, Clore J. Reorganizing the general clinical research center to improve the clinical and transla- tional research enterprise. Eval Health Professions 2013;36:492-504. 2. Clinical and Translational Science Awards. Available from: https://ncats.nih.gov/ctsa. Accessed: January 24, 2019. 3. Institute of Medicine. The CTSA Program at NIH: opportu- nities for advancing clinical and translational research. [Qualitative Research in Medicine & Healthcare 2019; 3:7943] [page 23] Article No n- co mm er cia l u se on ly Washington, DC: The National Academies Press; 2013. 4. Pincus HA, Abedin Z, Blank AE, Mazmanian PE. Evalua- tion and the NIH Clinical and Translational Science Awards: a “top ten” list. Eval Health Professions 2013;36:411-31. 5. Estape E, de Laurido LES, Shaheen M, et al. A multi-insti- tutional, multidisciplinary model for developing and teach- ing translational research in health disparities. Clin Trans Sci 2011;4:434-8. 6. Loue S. The impact of a research ethics training program. JERHRE 2014;9:46-55. 7. DuBios JM, Schilling DA, Heitman E, et al. Instruction in the responsible conduct of research: an inventory of pro- grams and materials. Clin Transl Sci 2010;3:109-11. 8. Kon AA, Schilling DA, Heitman E, et al. Content analysis of major textbooks and online resources used in responsible conduct of research instruction. AJOB Pri Res 2011;2:42-6. 9. Steneck NH. ORI Introduction to the responsible conduct of research. Washington, DC: U.S. Government Printing Of- fice; 2004. 10. Helton-Fauth W, Gaddis B, Scott G, et al. A new approach to assessing ethical conduct in scientific work. Acct Res: Pol & Qual Assr 2003;10:205-28. 11. Antes AL, Wang X, Mumford MD, et al. Evaluating the ef- fects that existing instruction on responsible conduct of re- search has on ethical decision making. Acad Med 2010;85:519-26. 12. Yarborough M, Hunter L. Teaching research ethics better: focus on excellent science, not bad scientists. Clin Trans Sci 2013;6:201-3. 13. Presidential Commission for the Study of Bioethical Issues. Moral science: protecting participants in human subjects re- search. Available from: https://bioethicsarchive.george- town.edu/pcsbi/node/558.html Accessed: August 6, 2018. 14. Kolb AY, Kolb DA. Learning styles and learning spaces: En- hancing experiential learning in higher education. Acad Man Learn Educ 2005;4:193-212. 15. Baker AC, Jensen PJ, Kolb DA. Conversational learning: An experiential approach to knowledge creation. Westport, Connecticut: Greenwood Publishing Group; 2002. 16. Dewey J. My pedagogic creed. Curriculum Studies Reader E2. Washington: Routledge; 2013. pp 29-35. 17. Mills M. The sociology of small groups. Upper Saddle River, NJ: Prentice Hall; 1984. 18. Kolb DA. Experiential learning: Experience as the source of learning and development. New Jersey: FT Press; 2014. 19. Charmaz K. Constructing grounded theory: a practical guide through qualitative analysis. London, Thousand Oaks, Calif: Sage Publications; 2006. 20. Aguinaldo JP. Qualitative analysis in gay men’s health re- search: comparing thematic, critical discourse, and conver- sation analysis. J Homosexual 2012;59:765-87. 21. Nemec PB, Bussema E. Learning objectives. Psychiatr Re- habil J 2010;33:328-30. 22. Bærøe K. Translational ethics: an analytical framework of translational movements between theory and practice and a sketch of a comprehensive approach. BMC Med Ethics 2014;15. 23. Hostiuc S, Moldoveanu A, Dascălu M-I, et al. Translational research – the need of a new bioethics approach. J Trans Med 2016;14:16. 24. Colnerud G. Brief Report: ethical problems in research prac- tice. JERHRE 2013;8:37-41. 25. Domen RE. The ethics of ambiguity. Academic Path 2016;3. 26. Strosberg MA, Genefas E, Famenka A. Research ethics re- view: identifying public policy and program gaps. JERHRE 2014;9:3-11. 27. Kagarise MJ, Sheldon MD. Translational ethics: A perspec- tive for new millennium. JAMA Surg 2000;135:39-45. 28. STREAM research. Available from: http://www.transla- tionalethics.com/ Accessed: August 6, 2018. 29. KIimmelman J. Ethics, ambiguity aversion, and the review of complex translational clinical trials. Bioethics 2011;26:242-50. 30. Kligyte V, Marcy RT, Sevier ST, et al. A qualitative approach to responsible conduct of research (RCR) trining develop- ment: identification of metacognitive strategies. Sci Eng Ethics 2008;14:3-31. 31. Anderson MS, Horn AS, Risbey KR, et al. What do mentor- ing and training in the responsible conduct of research have to do with scientists’ misbehavior? Findings from a national survey of NIH-funded scientists. Acad Med 2007;82:853-60. 32. Heitman E, Olson CH, Anestidou L, Bulger RE. New grad- uate students’ baseline knowledge of the responsible conduct of research. Acad Med 2007;82:838-45. 33. Behar-Horenstein LS, Feng X, Isaac C, Lee B. Dental stu- dents’ expression of cultural competence. J Ethnograph Qual Res 2017;11:171-87. 34. Behar-Horenstein LS, Feng X. Enhancing cultural compe- tence among dental students through active teaching and ex- periential learning. Qual Rep 2017;22:1169-85. 35. Behar-Horenstein LS, Catalanotto FA, Garvan CW, Hudson- Vassell CN. An assessment of faculty and dental student de- cision-making in ethics. J Am Coll Dent 2014;81:44-50. 36. Johnson J, Bagdasarov Z, Connelly S, et al. Case-based ethics education: the impact of cause complexity and out- come favorability on ethicality. JERHRE 2012;7:63-77. 37. Rubio DG. Common metrics to assess the efficiency of clin- ical research. Eval Health Prof 2013;36:432-46. 38. Silverman H. Strosberg M, Luna F, et al. An analysis of on- line courses in research ethics in the Fogarty-sponsored bioethics training programs. JERHRE 2013;8:59-74. [page 24] [Qualitative Research in Medicine & Healthcare 2019; 3:7943] Article No n- co mm er cia l u se on ly