AMWA Journal / V36 N4 / 2021 / amwa.org 145 F E A T U R E ABSTRACT In 2020, the American Medical Writers Association estab- lished a working group to assess the value of the contribution of medical writers across the health sciences industry, includ- ing a subgroup tasked to gather data on the regulatory agen- cy’s perspective. We invited reviewers at regulatory agencies to participate in an anonymized survey to evaluate the effect of document quality on the regulatory review process, assess awareness among document reviewers of the contribution of medical writers to the quality of regulatory documents, and identify current strengths and opportunities to optimize docu- ment quality. This article shares the survey results and dis- cusses their implications for document quality, their impact on the regulatory review process, and the skills medical writers need to develop to bring value to this process. INTRODUCTION Medical writers bring value across the health sciences, taking the lead and driving efficient approaches for the delivery of high-quality medical communication documents targeted at diverse audiences including regulators, payors, physicians, and patients.1,2 However, the value of medical writing is not con- sistently recognized, and medical writers often still need to justify why they should have a seat at the table and be part of the team earlier in the process. Medical writing departments can also be faced with insufficient budget and resource to do their best work due to a lack of understanding of the role’s value. Given the many settings in which medical writers work and the variety of documents produced, it can be challenging to identify specific indicators of value. To address this issue, the American Medical Writers Association (AMWA) Executives Advisory Council established a taskforce to define and quantify the value of medical writing. The taskforce has 3 main areas of focus: (1) perceptions of medical writer value among medical writers and their employers, (2) key topics related to medical writer value, and (3) how the regulatory agencies view docu- ment quality and the value of medical writing. This article presents the work of the regulatory agency sub- group to evaluate the effect of document quality on the regu- latory review process and assess awareness among regulatory agency reviewers of the contribution of medical writers to the quality of regulatory documents. By understanding the regula- tor’s perspective, we hoped to demonstrate how medical writ- ers bring value to documents submitted to regulatory agencies, to identify and refine the training needs of medical writers, and to identify areas for action for the medical writing profession and for colleagues in the biopharmaceutical industry. SURVEY DESIGN AND OBJECTIVES We employed an online survey format (SurveyMonkey), targeted at participants who were actively responsible for document review at a regulatory agency, were managers of regulatory agency reviewers, or who had worked in a regu- latory agency review role in the past 6 months. Participants were eligible regardless of the specific types of documents they reviewed. We identified potential participants via con- tacts in our own networks, via our colleagues (eg, com- pany regulatory department), and via contacts of the AMWA Executives Advisory Council. Participants were also encour- aged to forward the survey to other eligible individuals within their organization. We reached out to the United States Food and Drug Administration, Health Canada, the European Medicines Agency, the Medicines and Healthcare products Regulatory Agency, the Bundesinstitut für Arzneimittel und Medizinprodukte, the Pharmaceuticals and Medical Devices Julia Cooper,1 Lisa Chamberlain James,2 Joan Affleck,3 Brian Bass,4 Julia Forjanic Klapproth,5 Dylan Harris6, on behalf of the AMWA Value of Medical Writing Working Group / 1Parexel Interna- tional, Dublin, Ireland; 2Trilogy Writing and Consulting, Cambridge, UK; 3Merck & Co, Inc, Rahway, NJ, USA; 4Bass Global Inc, Fort Myers, FL, USA; 5Trilogy Writing and Consulting, Frankfurt, Germany; 6Takeda Pharmaceutical Company Limited, Lexington, MA, USA Value of Medical Writing: The Regulator’s Perspective 146 AMWA Journal / V36 N4 / 2021 / amwa.org Agency, the National Medical Products Administration, and the Australian Therapeutic Goods Administration, although the agencies of those who actually participated are not identi- fied, as the survey was anonymous. AMWA provided an official invitation letter and cover email to explain that the survey was being conducted on behalf of AMWA, its objective, and how the results will be used and to provide confirmation that the responses remain anonymous. Being cognizant of limitations on the regulators’ availabil- ity for such a survey, we made significant effort to develop a set of 25 survey questions that we believed would capture key points from the regulators’ experience with document qual- ity and medical writing. Most of the questions were multiple choice. The survey also included a checkpoint question to eliminate participants not involved in document review, and participants were invited to take part in a follow-up inter- view. For the follow-up interviews, we prepared 7 questions to elaborate on the survey results. For example, some questions included “none of the above” as a response option. If many participants selected this option, we requested additional information during the follow-up interviews. After beta testing, the survey opened in April 2021 and was open through early August 2021. Interim views of the data were done in May/June to confirm adequate participation. Follow-up interviews were conducted during August 2021. PARTICIPANT PROFILE We received 32 responses to the survey. Although this was considerably higher than the anticipated response rate, the response rate was not uniform across all questions, and it was agreed that the sample size was appropriate for descriptive analysis only. In the following sections, we have highlighted where we believe the data should be interpreted with caution due to a lower response rate. The data on agency tenure and time spent reviewing docu- ments indicated that the survey was completed by participants meeting the target profile. Most had been employed at their current agency for over 5 years (Figure 1) and spent at least 10% of their time reviewing documents (Figure 2). Participants were also asked to indicate their department or division (omit- ting information that could identify them or their employer). Based on these responses, we were reasonably confident that we had engaged with the right people at the regulatory agen- cies for the purpose of this survey. IMPACT OF QUALITY ON REGULATOR ASSESSMENTS Medical writers will be familiar with how the work of internal and client teams is hindered when the documents they are given are poorly constructed. The survey results confirmed that the work of the regulatory reviewer is similarly impacted if documents submitted to the agency are not well written, and the responses provide important messages about the value of the medical writer. The following section also includes impor- tant information for colleagues in Regulatory Affairs or other functions involved in management of regulatory applications, as well as for corporate management. The majority (87%) of the participants confirmed that poor document quality impedes regulatory assessment (Figure 3). Of note, none of the participants disagreed that poor qual- ity impedes document review, and the remaining 13% had no opinion. When asked whether they encounter issues related to document quality during the review process, the same per- centage—87%—reported such issues either sometimes or often (Figure 4). These results show that regulatory assessors receive poor quality documents for their review relatively frequently, and regulatory assessment of the document is thereby impeded. 60% 50% 40% 30% 20% 10% 0% <1 1-5 6-10 11-20 >20 Years Responses: 32 Figure 1. How long have you been employed at your current agency? Figure 2. In your current position/role, what percentage of your time do you spend reviewing documents? 35% 30% 25% 20% 10% 5% 0% 0% 1%-10% 10%-25% 76%-100%26%-50% 51%-75% Responses: 32 AMWA Journal / V36 N4 / 2021 / amwa.org 147 To gauge whether there has been any directional change in quality of documents, the regulators were asked how docu- ment quality has changed in the past 5 years. Improvement in document quality was selected by 43% of participants. This indicates that the quality of submissions is moving in the right direction. However, there is still work to be done, because almost half (48%) responded that there has been no change in quality or they were neutral/had no opinion, and 9% believed that the quality of documents submitted to their agency has declined over the past 5 years. Note that at this point in the survey the participants had not yet been provided with exam- ples of quality issues, and so these responses likely reflect the regulators’ own concept of document quality. If documents within an application are of poor qual- ity, the regulatory reviewer may need to send the application back with questions for clarification. Over half the partici- pants (53%) said that they send over 10% of applications back or reject the application, with questions arising from poor document quality (Figure 5). Although 47% of participants send back or reject less than 10% of the applications, this still means that a sizeable number of applications are delayed. For applications that are ultimately approved (Figure 6), 77% of the regulatory reviewers agreed or strongly agreed that poor document quality will delay the approval process. These are clear messages on how poor document quality, which is an avoidable issue if proper processes are established and led by trained professionals, impacts the applicant’s goals and, per- haps of more serious consequence, leads to patients waiting longer than necessary for new medicines. To understand whether poor quality might impact other documents in the regulatory assessment process, we asked whether a poorly written document negatively influences the review of other documents from the same applicant. Almost a third (27%) of participants agreed that poor document quality Figure 3. Does poor document quality impede your ability to provide regulatory assessment? Figure 4. How often do you encounter issues related to document quality during the review process? 100% 80% 60% 40% 20% 0% Yes No Neutral/no opinion Responses: 23 Figure 5. What percentage of applications do you reject/send back to the applicant with questions due to poor document quality? 50% 40% 30% 20% 10% 0% Responses: 19 <10% 10%-25% 26%-50% >50% Figure 6. For applications that are ultimately approved, a poorly written document delays the approval process. 60% 50% 40% 30% 20% 10% 0% Strongly agree Agree Neither agree nor disagree Disagree Responses: 22 Strongly disagree 60% 50% 40% 30% 20% 10% 0% Never Rarely Sometimes Often Responses: 23 Always 148 AMWA Journal / V36 N4 / 2021 / amwa.org could negatively influence their review of the applicant’s other documents. It should be noted that we did not define what this means in practice, eg, whether the reviewer would be likely to review the applicant’s other documents in more detail or whether this approach would carry over to documents in later submissions. The same percentage (27%) disagreed with the question, and 45% neither agreed nor disagreed. This indicates that, in some cases, poor document quality can even influence the assessor’s review of the applicant’s other documents. The survey included questions around whether the regu- latory agencies collect data themselves on document quality. Three participants (13%) confirmed that their agency collects such data, 35% responded that these data are not collected, and 53% did not know. When asked what the agency does with the data, one participant stated the data are reviewed, but the majority skipped the question. Most participants (90%) responded that their agency does not keep a record of appli- cants that regularly submit poorly written documents. QUALITY ISSUES OBSERVED BY THE REGULATORS Having established that document quality has a significant effect on the regulatory assessment process, it was important to understand which kinds of document quality issues are observed by the regulators. For the questions designed to identify these quality issues, participants were provided with the following response options (Figure 7). When asked to identify all quality issues encountered (Figure 8), those most frequently reported by the regulatory reviewers were excessive length/repetition/verbosity, closely followed by lack of clarity. This will not surprise most medical writers, who expend great effort working with teams to produce documents that are clear and concise with well- organized messages. However, these results do demonstrate that the effort invested in these aspects is warranted and necessary to meet the needs of the regulatory assessors. Of note, issues such as data errors, incomplete content, broken links, and poor tables/graphs were ranked relatively low in this question, which suggests many applicants have implemented processes to catch these avoidable issues prior to document submission. In addition to the range of quality issues typically observed, we asked the regulatory reviewers to identify the one docu- ment quality issue they encountered most frequently (Figure 9). Excessive length/repetition/verbosity was ranked top here, too, closely followed by poor explanation of rationale. Once again, avoidable issues (data errors, incomplete content, poor tables/ graphs, poor language) were ranked low or not at all. Poor organization Poor language usage Lack of clarity Poorly designed/presented tables and graphs Data errors (eg, inconsistencies, transcription errors) Incomplete content Poor explanation of rationale Excessive length, unnecessary repetition, verbose Incorrect format/nonadherence to guidance Broken/incorrect or insufficient crosslinks Other None Figure 7. Examples of quality issues used in survey questions. Excessive length, repetition, verbose Lack of clarity Poor explanation of rationale Nonadherence to guidance Poor organization Poor language usage Broken/incorrect crosslinks Incomplete content Data errors Poorly designed tables /graphs None of the above Other 20%0% 40% 60% 80% Responses: 22 Figure 8. Which of the following issues related to document quality do you typically encounter? Check all that apply. Figure 9. Which one of these issues related to document quality do you encounter most frequently? Excessive length, repetition, verbose Lack of clarity Poor explanation of rationale Nonadherence to guidance Poor organization Poor language usage Broken/incorrect crosslinks Incomplete content Data errors Poorly designed tables /graphs None of the above Other 10%0% 20% 30% 40% Responses: 22 AMWA Journal / V36 N4 / 2021 / amwa.org 149 Understanding the range and frequency of quality issues will help the medical writing profession and the industry to improve processes that support document quality and to target training and skills development for authoring teams. It is also important to understand whether specific quality issues have a greater effect on the assessor’s review and appli- cation approval, regardless of how frequently they occur. Poor explanation of rationale caused the greatest negative effect on review or caused the most irritation to the regulatory reviewer, with excessive length ranked second (Figure 10). When asked to identify the one issue that has the greatest negative effect on application approval, the regulatory reviewers also ranked poor explanation of rationale at the top (Figure 11), followed by incomplete content. Poor explanation of rationale, therefore, is not only one of the most frequently observed quality issues, but also caused the most irritation to reviewers or negatively affected their review and has the greatest negative effect on approval. Clear strategic presentation of rationale supported by data should be a top area of focus for the teams responsible for documents submitted to regulatory agencies. It is also interesting that, although incomplete content is not among the most frequent quality issues, the responses sug- gest it has a large negative effect on application approval when it does occur. It is therefore important for applicants to have rigorous processes to validate documents for completeness before submission. In converse, excessive length was ranked as the most frequent and was among the top document quality issues that cause irritation or have a negative effect on regula- tory review, yet it is not among the top issues that negatively affect application approval. REGULATORS’ PERCEPTION OF MEDICAL WRITING Beyond their view of the documents themselves, we wanted to understand what the regulatory reviewers thought of medical writers, their role, and their effect on the documents sent to the regulators for review. Of those who responded, 67% were familiar with the con- tribution of medical writers to the documents they review. Importantly, 70% either agreed or strongly agreed that medi- cal writers improve the quality of these documents, and a clear majority (87%) agreed or strongly agreed that sponsor com- panies with established medical writing functions and rigor- ous document development processes and standards produce higher quality submissions. Although this last question was asked before we had given examples of quality (and so the reg- ulatory reviewers have used their own idea of a high-quality document), the responses strongly indicate that medical writ- ers improve quality and established medical writing functions and processes produce higher quality documents. We asked the regulators to indicate any areas where they believed that medical writers add value to regulatory docu- ments. Over 78% identified “adherence to standards,” and 71% identified “accuracy.” This was closely followed by 64% for each of the following: • Clarity • Completeness • Explanation of rationale • Formatting It is particularly reassuring that the regulatory reviewers believe that medical writers add value in the areas of accu- racy, adherence to standards, and also explanation of rationale, which the previous questions had clearly identified as a key area of concern for them. However, it should be noted that this Figure 10. Which one of these issues related to document quality most negatively affects your review/causes the most irritation? Excessive length, repetition, verbose Lack of clarity Poor explanation of rationale Nonadherence to guidance Poor organization Poor language usage Broken/incorrect crosslinks Incomplete content Data errors Poorly designed tables /graphs None of the above Other 10%0% 20% 30% Responses: 22 Figure 11. Which one of these issues related to document quality has the greatest negative effect on application approval? Excessive length, repetition, verbose Lack of clarity Poor explanation of rationale Nonadherence to guidance Poor organization Poor language usage Broken/incorrect crosslinks Incomplete content Data errors Poorly designed tables /graphs None of the above Other 10%0% 20% 30% Responses: 22 150 AMWA Journal / V36 N4 / 2021 / amwa.org question was only answered by 14 respondents, and so the results should be interpreted with caution. FOLLOW-UP INTERVIEWS Some of the participants indicated that they would be happy to give more detail about their survey answers. We arranged individual interviews to gather this information, which was anonymized and amalgamated and is presented below. Quality Issues and Document Type Because the survey had identified quality issues in some of the documents that the regulatory reviewers receive, it was important to understand if these were most prevalent in one document type (suggesting an issue with the template or understanding of the requirements) or were seen in all of the document types received. The regulatory reviewers confirmed that quality issues were seen generally across all document types. They explained that templates or guidance cannot address all the nuances of writing these documents and so experienced writers are needed. “Explanation of Rationale” as the Key Quality Issue Explanation of rationale was identified as a key area of impor- tance for the regulatory reviewers, and they explained that this was because it can take them a lot of time to interpret what the author intended to communicate. The review- ers often go back to the sponsor for clarification, but this depends on several factors: • The type of document being reviewed (eg, lack of clarity or other issues affecting safety are usually much more concerning than issues of lesser consequence) • Timeline (eg, whether the reviewer has the time to work through the misunderstanding/quality issue them- selves) • Complexity (eg, whether the reviewer is able to work through the quality issue in the document compared with sending it back to the sponsor) • Resources (eg, whether a specialist is available on the regulatory agency side to review the document to help with the quality issue) The impact of a document with a poorly written rationale can be significant. Some regulatory agencies could inter- pret a poorly written rationale as lack of transparency, which could then call the entire application into question (a “domino effect”), and documents with poor rationales would likely be flagged at each review step for extra investigation, which would affect the whole application. It was widely accepted that a poorly written rationale makes the entire review process more difficult and would have a negative effect on approval. Other Document Quality Issues Although we asked about the most common issues negatively affecting document quality, we wanted to know if the regulatory reviewers encountered other issues that we had not specified. Lack of transparency was identified as a key issue, par- ticularly if the regulatory agency had experienced challenges with the sponsor or their applications previously. A lack of transparency and lack of clarity around the sponsor’s objec- tives can raise regulatory reviewers’ suspicions and give the impression that the sponsor is trying to overwhelm the reviewer with a mountain of data. Transparency in terms of minutes from meetings with other regulatory agencies was also required, and a reluctance to provide these documents delays approval because it takes extra time to request them. The reviewers explained that it is important for them to see the concerns and requirements in other regions. Medical Writers’ Influence on Document Quality and Their Role We asked what influence the regulatory reviewers felt that medical writers had on document quality and the medical writer’s role. The responses were extremely heartening and reflected the aims of the medical writing profession. The regulatory reviewers felt that medical writers have a “great and positive influence on document quality; they help keep documents clear, as brief as they can be, and con- sistent.” They felt that there is “definitely a difference when medical writers have been involved” in document produc- tion and that they can tell if inexperienced writers have been used, as they see a lack of attention to detail and adherence to standards. The regulatory reviewers felt that “a professional medical writer is always welcome and is always needed” and believe that the importance and value of medical writers “continues to grow,” to the extent that some regulatory agencies have established their own medical writing teams. The regulatory reviewers felt that medical writers have a “great and positive influence on document quality; they help keep documents clear, as brief as they can be, and consistent.” AMWA Journal / V36 N4 / 2021 / amwa.org 151 One of the reviewers summed up the situation beautifully: “I know that it is a very specific profession needing training. [Sometimes] we cannot tell who has written what in the appli- cations or how much medical writers have been involved—it is invisible from the regulatory agency point of view. We don’t need to know, we just want something of good quality!” Anything Else? Finally, we asked a very open question—were there any other comments that the regulatory reviewers would like to make concerning document quality or the role of professional medi- cal writers? They explained that, beyond scientific expertise, medical writers should be involved in document production to make the information understandable and usable for the reviewer. They emphasized that they cannot “transform a bad docu- ment”—if the information they are given is not understand- able, they cannot reply to it, which they found very frustrating because their role is to encourage and facilitate drug develop- ment. Often, regulatory reviewers can see that there is excel- lent science and work behind the document, but because it has been written badly, they are forced to guess what the messages are. They believed that although the role and work of medi- cal writers may not be immediately visible to them, it was a “major” contribution. Their final comment was that there was “no negative in having medical writers involved in document development— their influence and contributions are always positive.” LOOKING FORWARD The objectives of the survey were to gain an understanding of how regulatory agencies perceive the value of medical writing and to learn where to focus the training and development of medical writers to maximize the value in, and skill set for, the preparation of regulatory documents. The survey responses showed that many regulatory review- ers understand the role of medical writers, believe that they increase the quality of the documents sent to the agencies for review, and make the job of the regulatory reviewer easier. It is unsurprising that document quality is extremely important for regulatory reviewers. Participants reiterated that poor docu- ment quality can not only hamper the ability of the reviewer to provide an assessment (delaying the drug approval process), but also has the potential to bias reviewers against subse- quent submission documents from the same sponsor. There is a clear opportunity for medical writers to improve document quality, and the survey responses can also be used to inform how medical writers present themselves within their organiza- tions—quality is clearly top of the regulatory reviewers’ list of priorities and has been recognized by them as an area where medical writers add value. Most satisfyingly, regulatory reviewers appreciated and recognized the work and importance of trained medical writ- ers; thus, addressing regulatory reviewers’ needs should con- tinue to be a priority for the profession. Training must equip medical writers to lead teams that create documents that are concise and clearly present the message supported by the data. Perhaps even more focus should be given to team manage- ment and soft skills to allow medical writers to lead and guide these teams so that the documents supporting submissions are as concise and strategic as possible to streamline and increase efficiency of the whole clinical development process. The fact that the regulator reviewers, who are often time- poor, chose to take the time to help us to understand the role and value of medical writers is a testament to the importance of our profession and the expertise that trained medical writers bring to the development of regulatory documents and their associated teams. Acknowledgment Thanks to Susan Krug, AMWA Executive Director, who pro- vided significant support setting up the survey and with com- munication to survey participants. Author declaration and disclosures: The authors note no commercial associations that may pose a conflict of interest in relation to this article. The opinions expressed in this article are the authors’ own and are not necessarily shared by their employers or AMWA. Author contact: julia.cooper@parexel.com References 1. Marchington JM, Burd GP. Author attitudes to professional medical writing support. Curr Med Res Opin. 2014;30(10):2103-2108. 2. Woolley KL, Lew RA, Stretton S, et al. Lack of involvement of medical writers and the pharmaceutical industry in publications retracted for misconduct: a systematic, controlled, retrospective study. Curr Med Res Opin. 2011;27(6):1175-1182. mailto:julia.cooper@parexel.com