Layout 1 Introduction This qualitative research study involves the analysis of interviews from a small pool of physicians, in which we asked them about their use of clinical practice guide- lines (CPGs), evolution, gaps in adherence of CPGs, and how patients may be affected by these practices. CPGs are evidence-based recommendations that are intended to optimize and standardize patient care based on the best evidence available.1 They are developed by reviewing and assessing clinical evidence in a specific therapeutic area. Physician experts are heavily involved in developing guidelines and offering recommendations based on the strength of evidence currently available. Research shows that adherence to CPGs varies, in differing levels, in all fields of medicine and public health.2 Furthermore, studies looking at sustainability of adherence, albeit a rather new term studied in healthcare, after an implementation plan show partial sustainability of healthcare innovations rather than a complete sustainability.2,3 How guideline adherence affects patients is still under investigation. This qualitative research study was de- signed to explore one small and rather introductory seg- ment of adherence; and more precisely, how physicians from different medical specialties currently view and adopt CPGs as opposed to the time when they were med- ical residents. We aimed to understand how the evolution of CPGs from the 1990s to 2016 or earlier, had impacted physicians’ views and adherence to practice guidelines. We also explored the barriers of CPG non-adherence. The goals of the study were twofold. First goal was to help with future studies in studying the most important barriers to adherence, quality of care to patients, and sustainable implementations plans of adherence to CPGs. Secondly, the goal was to provide insights into the social space of action-oriented recommendations that may improve the A view into clinical practice guidelines: who uses them, who doesn’t and possibly, why Mihaela C. Munteanu,1,2 Julie Choi Jordan3 1University of Drexel, Philadelphia, PA; 2Incyte Corporation, Wilmington, DE; 3Teva Pharmaceuticals, Philadelphia, PA, USA ABSTRACT Medical professional societies each develop specific clinical practice guidelines (CPGs). Based on the best available evidence, CPGs are intended to control variability and optimize quality of care in clinical practice. Yet, healthcare providers often do not accept or adhere to guidelines, but their reasons are not fully understood. When providers opt to choose not to follow CPGs, unfavorable patient outcomes including unequal access to treatment become negative consequences. In this small qualitative study, we will explore what causes non-adherence to CPGs and what changes have been made to CPGs from when physicians completed their medical residencies to the present. We interviewed physicians from a variety of medical specialties to assess how these changes may influence guideline adherence as well as the consequences of not following them. We found that guidelines may not be followed in cases where patients have comorbidities that are not described in the guidelines or when physicians do not incorporate new evidence and technology advances into their practice. In some specialties, physicians can develop a poor reputation if they do not adhere to the CPGs, and managed care agencies may deny reimbursement for care they provided. To best serve the physician and the patient, we need to find ways to improve CPG adherence. Tactics such as improving the methodology of CPG formation, using information technology, and creating ways to change physician attitudes and behavior are all viable options. Correspondence: Mihaela C. Munteanu, Doctorate Candidate, Uni- versity of Drexel, Philadelphia, USA. Tel: +1.484.325-4103. Email: tantinica@gmail.com Acknowledgements: We would like to acknowledge Dr. Edward Bureau from the University of Drexel for his advice and guidance on qualitative research. Key words: adherence, clinical practice guidelines (CPGs), physi- cian, patients’ comorbidities, medical residency Contributions: MCM, concept, design, review and format of the data, analysis and interpretation of data, draft and final revision of the manuscript; JCJ, interpretation of data, draft and final revision of the manuscript. Funding: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Received for publication: 15 January 2017. Revision received: 23 April 2017. Accepted for publication: 24 April 2017. This work is licensed under a Creative Commons Attribution Non- Commercial 4.0 License (CC BY-NC 4.0). ©Copyright M.C. Munteanu and J. Choi Jordan, 2017 Licensee PAGEPress, Italy Qualitative Research in Medicine & Healthcare 2017; 1:44-50 doi:10.4081/qrmh.2017.6544 [page 44] [Qualitative Research in Medicine & Healthcare 2017; 1:6544] Qualitative Research in Medicine & Healthcare 2017; volume 1:44-50 No n- co mm er cia l u se on ly utilization of and adherence to CPGs by physicians across different specialties. Problem statement and purpose Because we lack understanding of how current CPGs are meaningful to doctors in their professional workplace compared to their residency program, three research ques- tions are as follows: (a) What do doctors want to have changed in the development of CPGs?, (b) Do doctors be- lieve that current CPGs are useful in their practice or today’s clinical centers from the time they were resi- dents?, and (c) How do doctors find current CPGs helpful in providing equal treatment to patients (i.e., quality of care to patients)? We argue that what can be learned from interviewees’ situated accounts that are generated in the context of a re- searcher-interviewee relationship is only possible from qualitative research.4-6 This type of meaningful knowledge from participants’ direct social world as compared to quantitative research knowledge is both unique and criti- cal in studying complex problem statements with impli- cations in healthcare. Likewise, these implications can be further studied in all their complexity if the main problem is understood at a fine-grained level first. Sample, framework We interviewed ten physician participants. They all trained and were licensed in the United States (US). The participants involved were not restricted to one medical specialty; both former and currently practicing physicians participated. The periods in which they conducted their residencies were 10 to 20 years prior to these interviews (January 2016). The purposive sample and type of medi- cine were also based on availability to participate in the interviews irrespective of specialty.5-7 Currently, all par- ticipants are either still practicing medicine, and/or now work in the drug research and development industry after practicing medicine full time medicine first. All voluntary participants signed and dated an informed consent. The approach of the study for the data collection and design is a modified phenomenological approach, follow- ing Moustakas.8 The participants were from different spe- cialties of medicine. Five individual interviews and one collective focus-group interview of five different partici- pants were conducted. The focus group interview and two individual interviews were conducted in person; the three other individual interviews were conducted by phone. As in Table 1, the medical specialties of the participants (n) were oncology (3), neurology (2), cardiology (1), infec- tious disease (1), pediatric rehabilitation (1), and internal medicine (2). Most participants resided in the East Coast region of the US, while the West Coast, Mid-West, and Southern regions of the US were represented by 1 partic- ipant each. For all 10 participants in this study, the dura- tion of medical practice, including those not currently in practice, was at least 10 years, exclusive of the duration of their residency programs. To protect confidentiality, all participants are referenced by numbers. Three (1, 6, and 9) of the 10 participants are women; seven (2, 3, 4, 5, 7, 8, and 10) are men. The individual and focus group interviews lasted ap- proximately 30 to 45 minutes, with follow ups where needed, and 1 hour, respectively. Physicians were asked semi-structured questions designed in accordance to the purpose of the study.5,6,9 Types of questions followed the direction given by Merriam.9 We recorded and transcribed the interviews according to Merriam.9 We followed the guidelines for developing interview questions and writing transcripts as per Merriam.9 We also developed some of the interview questions in line with the theoretical framework drawn from literature on attitudes and potential barriers to the use of standard- ized treatment guidelines such as CPGs.10 We considered [Qualitative Research in Medicine & Healthcare 2017; 1:6544] [page 45] Article Table 1. Medical specialty, current role, and gender of all the participants. Participants* Medical specialty Current role° Gender 1 Oncology/internal medicine Pharmaceutical industry and In practice F 2 Neurology Pharmaceutical industry (for approximately last 4 years) M 3 Oncology (nuclear medicine)/internal medicine In practice M 4 Internal medicine Pharmaceutical industry (for approximately last 7 years) M 5 Oncology/internal medicine In practice M 6 Internal medicine Pharmaceutical industry (for approximately 4 years) F 7 Cardiology In practice M 8 Infectious disease Pharmaceutical industry and In practice M 9 Orthopedic pediatric medicine In practice F 10 Neurology Pharmaceutical industry (for approximately 5 years) M *Participants are referenced by numbers. °All participants completed a medical residency program in the US, and have had experience in practicing medicine full time before the current role assessed in 2016. No n- co mm er cia l u se on ly the barriers stemming from the physician’s knowledge of CPGs, attitudes towards CPGs, resource limitations and their behavioral response to guidelines. Furthermore, to determine themes present in the participants’ behavior and attitudes we analyzed the field notes, gathered during the interviews, together with the transcripts of the focus group and individual interviews to determine themes.5,7,9,11 We used descriptive and in vivo manual coding via two recur- sive cycles as described by Saldana.11 Results Three themes emerged from the interview data set and field notes: a) Awareness of CPGs, b) Minor gaps in adher- ence of CPGs and consequences, and c) Moving forward: good adherence to and acceptance of CPGs (Figure 1). While not all will be discussed fully in this paper due to space limitations they are described next individually. Ex- cerpts from the interviews conducted will be presented as well to support these themes. Awareness of clinical practice guidelines theme In this theme, found through coding the data, the study participants shared that they were either not aware of CPGs as medical residents, or that CPGs did not exist dur- ing their residency; the latter was more prevalent. This theme had three subthemes: (a) CPGs evolved since the 1990s and early 2000s; (b) variability in practicing med- icine was more prevalent and more accepted in the 1990s and early 2000s than currently; and (c) CPGs have be- come a resource for physicians and provide education with the option to retain flexibility. These subthemes will not be discussed individually in this paper but referred to in the discussion. During the 10-20 years between our in- terviewees’ participation in residency programs and their current roles, there was an evolution in the development of CPGs. The participants indicated that there was much more variability in practicing medicine in the past and that they wished they had more standardization in the clinical practice during their residency. Clinical practice guidelines adherence and consequences theme Currently CPGs have high adherence and are recog- nized as evidence-based recommendations, but there re- mains a small group of physicians not adhering to them. The data gathered from this small sample support this no- tion. In this theme of minor gaps in the CPG adherence and consequences, participants number 1, 7, and 6 de- scribe it below as an ‘attitude’ in different interviews; this concept has been found in the literature.10 1: There are some physicians who, I don’t know that they run out of time, but they just wouldn’t make time to do that [follow CPGs]. They believe it the way they’ve learned to practice and it’s probably more the older physicians that are closer to retirement who’ve been out in private practice for a long time. They have their usual selection of what they use for first line, second line, third line, fourth line salvage, whatever, pallia- tion, and they will continue to use that no matter. Whether they attend meetings and learn about new things or not, they have their little like we used to call cookbook recipe kind of way of treating and that’s what they use. Interviewer: It’s not necessarily the time, it is more of an atti- tude? 1: It’s an attitude. That’s exactly, yeah. 7: .... I have a number of partners I practice with… the [page 46] [Qualitative Research in Medicine & Healthcare 2017; 1:6544] Article Figure 1. Themes and subthemes. Three themes emerged from the interview data set and field notes: awareness of clinical prac- tice guidelines (left); moving forward: good adherence to and acceptance of clinical practice guidelines (center); minor gaps in adherence of clinical practice guidelines and consequences (right). No n- co mm er cia l u se on ly older the guy is that I’m coming behind, the less likely he is ... 6: To even be aware. I was in a practice with a bunch of really senior people, like in their 60s, and they were not even aware of the guidelines in many cases. Interviewer: Why do you think that is? 6: Physicians pride themselves on the more years of practice, the more experience you have. You become a better doctor. I mean, just the more things you see. And that’s what a lot of physicians pride themselves on. And these young people coming up with guide- lines, ‘what do they know?’ So that is, or carries a lot of the attitude. Participant number four shares the sentiment and fur- ther goes on to say that as part of his daily job, he reviews the inferior patient outcomes that occur when doctors do not follow CPGs: 4: As I review safety reports on my current job, it be- comes very clear the physicians who are following es- tablished guidelines versus those who aren’t because you can see the gaps in treatment. Unfortunately, the patient outcomes show it. Participant number five shares the feeling that there are still gaps in adherence to CPGs. From his point of view as a practicing physician in oncology, there are a few reasons: 5: There is variation in how much individual physicians use practice guidelines. I think there are a variety of reasons for that. As you mentioned number 1, physi- cians may feel that they don’t want to change their ways. Number 2, physicians may be aware of guide- lines but disagree. Some physicians think more is bet- ter and want to add the next treatment, the latest treatment every time regardless of what the guidelines say or what the evidence supports. Number 3, I would say some physicians are not as savvy with using the internet to access guidelines as others. For example, we have some physicians in our practice who really struggle with even using an EMR [electronic medical record] and are much more comfortable with the old pen and paper documentation. Developing a poor reputation as a consequence of clinical practice guidelines non-adherence The consequences of not following CPGs adequately or comprehensively can include poor patient outcomes, worsened reputations for the doctors or their working pro- cedures, followed by, for example, by doctors not receiv- ing performance bonuses from their practices as a penalty. Interviewees stated these various consequences. Overall, the interviewees reported that the consequences vary de- pending on the medical specialty. Further, in large medical specialties such as cardiology and infectious disease, patient outcomes and adherence to treatment guidelines are tightly regulated as voiced by participant number seven below. This informal regulation is because in some of these specialties, good patient out- comes have been associated with use of specific drugs through large unequivocal trial data. Likewise, some spe- cialties have more data available to handle complex pa- tient cases. Participant number seven, who practices at six private medical offices, each composed of six or more car- diologists, and is associated with a large hospital in the Southern US expresses this point below: 7: Well, depending on your specialty, internal medicine is very complicated. You deal with the whole range of certain disease processes; disease coverages are very broad. You deal with a lot of very unusual diseases. Infectious disease is difficult. Neurology is the worst. But in cardiology, it’s very straightforward. The guide- lines become very important to us. I mean, you have to follow the guidelines with systolic heart failure, and if you don’t, then you’re really outside of the standard of care. We report quality statistics to CMS [Centers for Medicare & Medicaid Services] always and do in- dividual performance measurements on all practices. Interviewer: It is about reputation? 7: Yeah! Our specialty is very different that way. For acute MI [myocardial infarction] treatment, you have to get arteries open in XX minutes if you have an acute ST elevation line. So those data on your compliance with the XX-minute STEMI [ST-segment elevation myocardial infarction] ... data are publicly reported via the ACC [American College of Cardiology] for the ACC database, so everybody knows that. Internally, we monitor our quality statistics. If I don’t meet that more than a benchmark, I wouldn’t get payment. Financial consequences of not following clinical practice guidelines The participating doctors noted that if CPGs are not followed and the poor performance of a medical office is made public, third-party insurance payers who may mon- itor compliance may not send patient referrals. Addition- ally, bills for patients, or bills for the complications of the first treatment, may not consequently be paid. Our partic- ipants did not mention losing the accreditation of an office yet or of a hospital, although our sample was very small. But they voiced this behavior may affect patients nega- tively in the long run. A few from the focus group inter- view describe below: [Qualitative Research in Medicine & Healthcare 2017; 1:6544] [page 47] Article No n- co mm er cia l u se on ly 7: There are CMS [Centers for Medicare & Medicaid Services] changes in reimbursement that are really im- portant about the newer events. You’re not allowed to have a DVT [deep venous thrombosis] in the hospital ...So there are rules about DVT prophylaxis in our hos- pital. If you don’t write for DVT prophylaxis as per the guidelines that are recommended by the ACC… I mean, it’s probably the Orthopedic Society or the Internal Medicine Societies that talk about inpatient medicine. If you don’t follow those guidelines, the hospital ad- ministrators will call you and say you didn’t do DVT prophylaxis. We have prompts on our computer systems that say you didn’t do DVT prophylaxis. 10: They don’t pay for those complications [in my spe- cialty] and patients suffer... Moving forward: good adherence and acceptance of clinical practice guidelines theme The interviews revealed multiple examples of good adherence to CPGs in different medical specialties. The data describe an improvement in the content of and adherence to CPGs from the time the participants were residents over time suggesting a trend and desires in the field of medicine to good quality of care. The participants reiterated that following the CPGs felt like following the best choice as opposed to a mandatory policy. Also, there was agreement that they have seen an in- creased adherence to CPGs over the years, which coin- cided with overall better patient outcomes. Participants liked the methodology by which CPGs are developed. Participant number four describes the CPGs as an im- provement in raising the bar in providing equal high-qual- ity treatment for patients once doctors accept the CPGs: 4: There was a lot of resistance, but I think gradual ac- ceptance, especially when they’re proven to provide good outcomes and lower cost, I think that’s when physicians finally got it, and when they realized that they were stakeholders of the process. It wasn’t a sys- tem where someone is dictating to them/us what they’re going to do... Furthermore, both participants number four and three explain that the guidelines can streamline medicine in view of the many treatments available and many special- ists seeing the same patient. Having the guidelines avail- able directly in EMRs allows one to easily access what is best for the patient right away and to decide if changes to treatment are needed and when. 4: I think having everything computerized and in the electronic medical record provides all that information to all the patient’s treatment physicians….so that they can see exactly what the patient is going through, what they’ve had done. It streamlines care. 3: I think it’s a definite improvement, compared to how we did things 20 years ago. The easier it is to access these things, to collaborate, and get everyone on the same page working together to make the specialty shine, it’s important. I think more effort should be made to do this and I think that’s the case. Sound methodology for developing and updating clinical practice guidelines The advancement of information technology has fa- cilitated the use of CPGs. Most hospitals now have CPGs incorporated as part of the EMRs. In the past 15 to 20 years, our participants saw improvements in both the availability and accessibility of CPGs. With regards to the sound methodology of CPGs, a current practicing physi- cian who is also part of the development committee for CPGs, describes the benefits of the process: 3: The procedures…are orchestrated by national and in- ternational professional organizations... There are committees formed that go over these things and they enlist participation of recognized clinical experts in a given field. Those who have published on a given topic about a given procedure…they put together guidelines and algorithms that are based on peer-re- viewed research with comments on quality of existing research and, also grey areas that often require addi- tional clinical judgment. While there may be some opposition to adopting guidelines due to fear of losing the ability to use judge- ment and experience, proponents argue that having evi- dence-based guidelines will offer an authoritative source of information that can empower clinicians, particularly those who do not get the opportunity to discuss their cases with colleagues and experts.12 This current view was shared in those interviewed in this study. Exceptions to when not to follow clinical practice guidelines We found that doctors agree that there are exceptions when it is acceptable not to follow CPGs, especially when patients’ safety is concerned. Physicians are trained to make clinical judgements that will provide the best out- come for the patient. Clinical judgments that would be in opposition to the guidelines were a valid reason given by all interviewed, and perhaps described the best by partic- ipant number one: 1: These are guideline recommendations and that’s ex- actly what they are, guidelines to help us make a de- cision, but they may or may not necessarily be the right combination, for instance, chemotherapy for your particular tumor because, say, in colon cancer, we recommend oxaliplatin in … but oxaliplatin can make neuropathy from diabetes worse. We may, in that case, not want to give oxaliplatin depending on the patient, the status of their diabetes. [page 48] [Qualitative Research in Medicine & Healthcare 2017; 1:6544] Article No n- co mm er cia l u se on ly Participants described sometimes that the reason for not following CPGs is an attitude or behavior. CPGs are not followed sometimes because some doctors pride themselves in seniority and they do not believe they need to make changes, particularly if junior physicians are cre- ating the guidelines. 4: Physicians pride themselves on the more years of practice, the more experience you have. You become a better doctor. I mean, just the more things you see. Additionally, the participants believe that doctors should still have flexibility and be allowed to exercise their clinical judgment. Doctors do not like to be penalized when they cannot adhere to CPGs because of patients’ comor- bidities. This experience was described as follows: 5: Patients don’t always fit. Physicians can have patients that clearly will not be able to tolerate what is recom- mended in clinical practice guidelines and that’s an ex- ample when it’s not appropriate to really stick with a guideline…patient comorbidities are a common exam- ple of when it’s appropriate to deviate from guidelines. Discussion Based on participants’ accounts, we were able to un- derstand participants’ perspectives on CPGs from their time as residents through to their socialization into the medical community. While CPGs are evidence-based rec- ommendations with high adherence, some physicians still do not follow them. There are a few significant reasons to explain when and why CPGs are not followed. All the participants agreed that it is acceptable for a physician to deviate from the CPGs when patients have comorbidities that are not detailed in the guidelines. This finding of pa- tients’ care was one of many situations also described in the literature.13 Further, participants in this study indicated that the most significant reason for non-adherence was patient co- morbidities. Participants shared this explanation unani- mously. Other factors or barriers described by physicians in both this study and the literature included physician seniority and experience in the field.14,15A physician with many years of practice may not see a personal need to consult CPGs. In addition, some doctors do not know how to utilize certain technologies as mentioned by some par- ticipants. Many physicians were trained in a time where textbook and journals were the main source of education. The emergence of the internet and devices that pro- mote the spread of medical information has had a dra- matic impact on the medical community, and not all physicians are keeping up. These important findings can be further studied in research with larger sample sizes and by focusing on each medical specialty. By understanding what these barriers are, we may find ways to eliminate them. For example, future CPGs and updates of current guidelines may benefit from discussing how patients’ co- morbidities can influence adequate treatment for patients. CPGs can be designed to allow flexibility in practice when comorbidities are present. This way physicians do not have to worry about reimbursement and being labeled as non-adherent. CPGs are updated regularly with experts from the field who apply minimal bias and base their recommendations on the best research available.1,16,17 When the participants were asked what they would change with regards to the methodology of CPG development, they did not have rec- ommendations. Because doctors like the methodology by which CPGs are developed and updated, it has led to the acceptance of and adherence to CPGs. This satisfaction, unanimous among our participants, was not always re- flected in the literature.18-20 Our findings are from a small sample size, and our participants expressed that they knew colleagues who were dissatisfied by the methodology of CPG develop- ment. Some participants mentioned that they knew of col- leagues who did not follow CPGs as documented also in the literature.3 This was because they did not agree with the tools used in developing CPGs or did not agree with the guidelines themselves. Our sample of 10 doctors, while diverse in specialties, is smaller than those in other published data. This diversity allowed a widespread representation and fuller understand- ing across medical fields. Through this deep journey of the doctors interviewed in a qualitative research setting, under- standing can be derived to make sustained improvement to healthcare, and offer quality of care to patients. Conse- quently, unique insights from conversations as found in this qualitative research study can be examined later in larger studies, in what influences their decisions to adhere con- sistently, or not adhere, to CPGs. Conclusions We recommend that CPGs should not only be evalu- ated on sound methodology, but should also address the barriers that cause non-adherence and perhaps reimburse- ment. One recommendation for adherence and sustain- ability of CPGs is to include sections that deal with patients’ comorbidities and introduce parameters that can be used as a decision tree for all doctors. Medical mem- bership associations and hospital administrators can con- duct surveys on these items to ensure a consensus, in addition to reaching out to the experts, is achieved before implementing CPGs. Secondly, it is important that doctors have a universal system of documenting in the medical chart or the EMR when they deviate from the guidelines. In this way, data trends can be shared systematically with the CPG development committees. To increase doctors’ adherence to the guidelines, we suggest that training could be implemented to help [Qualitative Research in Medicine & Healthcare 2017; 1:6544] [page 49] Article No n- co mm er cia l u se on ly physicians use technology so that they can better access medical information. Tailored and structural organiza- tional solutions can be given to change physician attitude and behavior; for example, many institutions are man- dating EMR versus ledger-type patient records. With time and flexibility, we believe that practical solutions can improve adherence to CPGs so that patients being treated in any field can all be given a reliable standard of care. References 1. Graham, R, Mancher, M, Wolman, D, at al. Clinical practice guidelines we can trust. Washington, DC: The National Academies Press; 2011. 2. Ament SMC, de Groot JJA, Maessen JMC, et al. Sustain- ability of professionals’ adherence to clinical practice guide- lines in medical care: a systematic review. BMJ Open 2015;5:1-13. 3. Wiltsey S, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: a review of the empirical lit- erature and recommendations for future research. Implement Sci 2012;7:2-17. 4. Berkwits, M, Inui, ST. Making use of qualitative research techniques. J Gen Intern Med 1998;13:195-9. 5. Creswell, JW. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage Publications; 2013. 6. Creswell, JW. Research design: Qualitative, quantitative, and mixed methods approaches. 4th ed. Thousand Oaks, CA: Sage; 2014. 7. Malterud, K. Qualitative research: standards, challenges, and guidelines. Lancet 2001;358:483-8. 8. Moustakas, C. Phenomenological research methods. Thou- sand Oaks, CA: Sage Publications, Inc.; 1994. 9. Merriam SB. Qualitative research: A guide to design and im- plementation. San Francisco, CA: Jossey-Bass, a Wiley im- print; 2009. 10. Cabana, MD, Rand, CS, Powe, NR, et al. Why don’t physi- cians follow clinical practice guidelines? JAMA 1999;282:1458–64. 11. Saldana J. The coding manual for qualitative researchers. 2nd ed. Los Angeles, CA: Sage; 2015. 12. Raghavan D, Kim ES. Clinical pathways: Are we training cookbook oncologists? 2016 Jun 1 Available from: http:// am.asco.org/clinical-pathways-are-we-training-cookbook- oncologists?et_cid=37910128&et_rid=1614915405&linkid =Raghavan 13. Erickson, BK, Martin, JY, Shah, MM, et al. Reasons for fail- ure to deliver National Comprehensive Cancer Network (NCCN)-adherent care in the treatment of epithelial ovarian cancer at an NCCN cancer center. Gynecol Oncol 2014;133:142–6. 14. Falandry, C, Campone, M, Cartron, G, et al. Trends in G- CSF use in 990 patients after EORTC and ASCO guidelines. Eur J Cancer 2010;46:2389–98. 15. Link, H, Nietsch, J, Kerkmann, M, Ortner, P. Adherence to granulocyte-colony stimulating factor (G-CSF) guidelines to reduce the incidence of febrile neutropenia after chemotherapy: A representative sample survey in Germany. Support Care Cancer 2016;24:67–6. 16. Goldberg, SL, Akard, LP, Dugan, MJ, et al. Barriers to physi- cian adherence to evidence-based monitoring guidelines in chronic myelogenous leukemia. JOP 2015;11:398–404. 17. Gnant, M. Guidelines: usefulness and limitations. Breast Care (Basel) 2013;8:172-3. 18. Abdelsattar, ZM, Reames, BN, Regenbogen, SE, et al. Crit- ical evaluation of the scientific content in clinical practice guidelines. Cancer 2015;121:783–9. 19. Lenzer J, Hoffman J, Furberg C, Ioannidis J. Ensuring the integrity of clinical practice guidelines: A tool for protecting patients. BMJ 2013;347:1–10. 20. Reames, BN, Krell, RW, Ponto, SN, Wong, SL. Critical evaluation of oncology clinical practice guidelines. JCO 2013;31:2563–68. [page 50] [Qualitative Research in Medicine & Healthcare 2017; 1:6544] Article No n- co mm er cia l u se on ly