V38 N3 / 2023 ©2023 American Medical Writers Association. All rights reserved. ISSN 2163-5315 AMWAJournal.org 28 Q1: What are the potential benefits and limitations of using ChatGPT in medical writing, and how can these be mitigated? Many writers in the continuing medical education (CME) field who have been exploring ChatGPT as a tool to gener- ate ideas see at least 3 possible use cases. First, if you are unfamiliar with a disease state or therapeutic area at the beginning of a project, ChatGPT can be a valuable research assistant that rapidly finds information to help organize your thinking, generate feedback on your work, and sum- marize research literature. You can ask ChatGPT to pro- vide a targeted overview and an “at-a-glance” perspective on the basics. But that’s what you’ll get. You’ll still need to do a much deeper dive into the peer-reviewed and evi- dence-based clinical practice literature. Second, some writers are using ChatGPT for efficiency gains. They create detailed prompts for ChatGPT to generate outlines for needs assessments and ideas for practice gaps. Writers I’ve spoken to who are experimenting in this way then vali- date and build out practice gaps using the literature. Third, ChatGPT is being explored to support adaptive learning by generating patient cases, providing responses to learner questions about those cases, and prescribing relevant resources for additional feedback and remediation. In all these potential use cases, the key to any degree of success is prompt engineering that shares sufficient detail and clear parameters with ChatGPT at the outset. The downside is that there is absolutely no guarantee that the information ChatGPT returns will be in any way accurate, and it certainly won’t be supported by evidence. Several journalists and scientists have now drawn attention to inac- curacies and false citations that ChatGPT generates because no one is fact-checking. If we cannot attribute claims to published sources, then we cannot use the content. So, if you are exploring ChatGPT, you’ll need to find and corrob- orate the sources and rigorously fact-check the integrity of information it returns. In my mind, the time to check facts will swallow any efficiency savings. Using ChatGPT and other artificial intelligence (AI) tools to generate ideas and content could also nudge writers into sloppy habits. It’ll be easier to lose track of your source material if that material is not anchored to specific references. Another limitation is that ChatGPT does not have the capacity to notice and process context and cannot provide nuance or perspective. These are human characteristics. As a result, medical writers will and must remain the primary drivers of content integrity in CME and continuing education for health professionals. —Alex Howson ChatGPT is a program focused on language-related tasks that “understands” natural language and can answer ques- tions and requests. GPT stands for “generative pretrained transformer.” ChatGPT was developed by OpenAI and released to the public in November 2022. The public inter- face was trained on information taken from the internet up through September 2021; thus, the information is slightly out of date, unless you input more recent information during a conversation. The public interface is free and requires no setup. Once you provide your email address and a password, you can type in a question or a request. The best answers come from questions that are as specific as possible. Benefits of ChatGPT ChatGPT has several potential benefits for medical writing. In a webinar I attended, attendees from a medical commu- nication (med comm) company noted that ChatGPT can be used as an initial research tool, for example, to ask for back- ground or an overview of a therapeutic area. Here are some examples: • “Provide peer-reviewed references for US colonoscopy guidelines within the last 10 years” • “Explain current treatments for acute myeloid leuke- mia to an oncologist in fewer than 500 words” ChatGPT can help overcome writer’s block by providing a starting point, summarizing a group of paragraphs you upload, or suggesting ideas for slide titles. Here are some examples: FREELANCE FOCUS Melissa L. Bogen Alex HowsonBrian Bass Cathryn D. Evans Ruwaida Vakil http://www.amwajournal.org https://chat.openai.com/auth/login https://networkpharma.tv/2023/05/03/ai-today-and-tomorrow-real-world-applications-for-medcomms/ AMWAJournal.org 29Freelance Focus • “Summarize the introduction and conclusions of the following publication as bullet points ” • “Rephrase the following sentence to be more concise ” ChatGPT can help repurpose material for different audi- ences. It can rewrite scientific material in lay language; adapt training modules for physicians to ultrasound techni- cians; or write a key opinion leader’s bio for a medical meet- ing from a resume that you upload. • “Write a 200-word biography of this professor of radiology to be shared with peers ” • “Rewrite this text to be language-appropriate for chil- dren ages 9-12 ” • “Turn this journal abstract into a slide presentation ” ChatGPT helps spark the creative process by providing an initial summary of information but not final draft material. Humans are needed to apply thought and judgment to the proposed answers. It’s not doing all the work for you, but it can make some tasks easier. Limitations of ChatGPT You need to check all facts and the quality and accuracy of all references. By OpenAI’s own admission, ChatGPT may produce inaccurate information about people, places, or facts. In other words, it can say things that are not true. In a recent email exchange on AMWA Engage, participants noted that ChatGPT created fake references. Because of nondisclosure agreements in place, medical writers should upload only public data to this third-party application. ChatGPT uses personal information you provide to pro- vide, administer, maintain, improve, and/or analyze its ser- vices. Although ChatGPT provides information in its privacy policy on how you can opt out of their use of your content to train the models, uploading patient data and other private information would be a mistake. —Melissa L. Bogen In my opinion, the benefits of ChatGPT are in the eyes of the beholder. I don’t personally see any benefits from the medi- cal writer’s perspective because the role of ChatGPT is to do the job of the medical writer. If we’re not doing the research, reading and synthesizing what we find, and distilling what we learn into clear, concise, and accurate language that is acces- sible to our target audiences, then what are we doing? In that vein, ChatGPT has the potential to turn medical writers into reviewers, which may not be many medical writers’ cup of tea. ChatGPT has vastly deeper, broader, and faster access to information than medical writers will ever have. But currently, ChatGPT does not have the ability to filter infor- mation from misinformation, and it cannot take responsi- bility for its content as medical writers and authors must. If a reviewer wishes to question a sentence or a paragraph written by ChatGPT, who is there to ask? ChatGPT currently doesn’t have the capacity to explain or justify itself. It simply is. It is the medical writing equivalent of “because I said so.” So, who do I think can currently benefit from ChatGPT in the medical writing space? To a large degree, content mills, because they already don’t care about accuracy or accountability. Predatory publishers are another unscru- pulous group that I think can immediately benefit from the speed and power of ChatGPT. But I do think there’s a legit- imate side to ChatGPT’s potential as a medical writing tool by limiting the information it can source and what it can do with that information. For example, limiting ChatGPT to only the information contained within a specific set of resources and giving it a prescribed template in which to apply that information is a task for which I think ChatGPT is potentially well suited. But the content ChatGPT produces must still be reviewed, scru- tinized, and when necessary, questioned. This may, in turn, create more opportunities for medical writers and medical editors who love to fact-check. —Brian Bass Benefits of ChatGPT include helping to amalgamate infor- mation related to some of various research questions. An IT friend put >10 questions to ChatGPT on my behalf (because I have not yet started using it on my own computer). The ques- tions related to medical/science issues, political topics, phil- osophical/religious topics, and simple pragmatic queries. Many factual errors as well as errors of “narrowness” resulted! As well, it was clear that the AI may provide biased answers. Serious questions, in my opinion, should not be put to ChatGPT (at least not at this time). Other alarming impedi- ments include not providing published, bona fide (respect- able) reference citations for ChatGPT’s opinions; one must specifically ask for references to be included; moreover, when we asked ChatGPT to include reference citations, some of them were incorrect ! Another danger is the risk/ temptation for a person to plagiarize ChatGPT (which has already plagiarized someone else, of course, with no attri- bution). I suspect much of this is already taking place with- out attribution. I did, however, receive helpful information in response to a certain question: I had Googled and done other searches about growing a potted rosemary plant indoors. After several unsuccessful results, we finally got a very http://www.amwajournal.org https://openai.com/policies/privacy-policy https://openai.com/policies/privacy-policy https://help.openai.com/en/articles/5722486-how-your-data-is-used-to-improve-model-performance AMWAJournal.org 30Freelance Focus helpful reply from ChatGPT, which not only amalgamated replies I had received elsewhere but elaborated more spe- cifically on my geographic location, the dry climate, and high altitude. (In the end, after killing 6 rosemary plants, I learned that, in fact, the conditions inside my home are not amenable to growing rosemary. ChatGPT was the preferred source. So, I stopped wasting money on these plants.) At this time, I do not think we should be using this AI program for serious professional medical writing. Nor do I think we can do much to mitigate the problems other than to commit ourselves to doing our homework, maintaining our integrity and ethics, and not succumbing to the temptation of laziness, ie, allowing ChatGPT to do work for us that our human critical thinking/intuitive minds should be doing. —Cathryn D. Evans AI technologies such as ChatGPT have unleashed a Pandora’s box. Recently, Congress had the opportunity to hear from Sam Altman, the founder of OpenAI, the organi- zation behind ChatGPT. Altman expressed concern about the potential for AI to spread misinformation, highlight- ing the genuine threat it poses. In his address to Congress, Altman emphasized the urgent need for regulation to miti- gate these risks. Although regulations are currently lacking, hopefully they will be implemented soon. As medical writers, we must grasp both the advantages and limitations of AI technologies. Despite the absence of regulation, AI is here to stay. We can harness the power of ChatGPT as an advanced search engine, but importantly, exercise caution and fact-check all the generated content. ChatGPT exhibits inaccuracies in developing references for its text; thereby, a critical evaluation of its generated refer- ences is necessary. Some have drawn comparisons between ChatGPT and Wikipedia, with arguments against ChatGPT reminiscent of those once made against the reliability of Wikipedia. Nonetheless, Wikipedia has persevered and flourished despite the initial skepticism. Similarly, ChatGPT will continue to propagate and evolve, irrespective of whether we choose to employ it. To illustrate the power of ChatGPT, I generated my response without the help of ChatGPT, and then I put my response in ChatGPT and asked it to expand on my answer. ChatGPT was able to summarize and expand on what I wrote while maintaining my ideas, focus, and flow. I did edit what ChatGPT generated, and what you read in the previous paragraphs is the result. Although AI may not yet replace medical writers, medi- cal writers who utilize AI effectively could outperform their peers. AI should be viewed as a tool that complements and enhances our capabilities rather than directly threaten- ing our profession. Adapting to this technology will enable medical writers to capitalize on the opportunities that AI provides while ensuring our continued relevance and suc- cess in an increasingly AI-driven world. —Ruwaida Vakil * * * Q2: What ethical considerations need to be taken into account when using ChatGPT for medical writing, partic- ularly in relation to patient privacy and data protection? Bias, copyright, and informed consent around data use are serious ethical concerns with ChatGPT. AI like ChatGPT is trained on speech, text, and images it scrapes from real- world content that is already in the public domain. Much of this content is structured by an existing inbuilt text and image bias that obscures heterogeneity in terms of gender, race/ethnicity, and other characteristics. Researchers like Timnit Gebru, founder and executive director of the Distributed Artificial Intelligence Research Institute, note that the data that large language models like ChatGPT encode are more likely to represent the perspectives of people who already occupy considerable internet real estate compared with women and people of color, who spend less time and have less access to determining online content. So, the idea starters and writing prompts that initially seem appealing as efficiency levers might already be culturally and racially biased, or at least lack cultural sensitivity and specificity. Unless medical writers are intentionally using ChatGPT and other AI tools through an equity and inequal- ity framework, we will unwittingly reproduce bias. Copyright implications for both image and text are also pressing. On the one hand, who owns AI-generated mate- rial? The US Copyright Office is actively exploring this ques- tion via a new initiative, but at the moment, if writers are using generative AI tools to create content, they do not own that content. Also, although many medical writers create content as employees or are bound by work-for-hire con- tracts, if you use AI-generated content to write blogs or books in your own name, anyone can reproduce it without your permission. On the other hand, although copyright is meant to protect material that is the product of human creativity, various ongoing lawsuits point to the ways in which copyrighted materials are already being used to train AI tools like ChatGPT without attribution, compensation, or credit to their creators. Another problem concerns the information we, as users, feed into ChatGPT. For instance, some clinicians have posted on social media how they are “testing” ChatGPT’s diagnostic capacities by feeding it anonymized history of present illness (HPI) data. Even if HPI data are anonymized, the data enter the public domain and are available to others http://www.amwajournal.org AMWAJournal.org 31Freelance Focus without patients giving consent for their own data to be shared publicly. We cannot be Luddites and push against the tide of technology. AI is going to change communication processes and practices whether we like it or not. But the direction that AI takes is not inevitable. We can and must influence how tools like ChatGPT are applied in practice. —Alex Howson I think there are several important ethical considerations to be taken into account with the use of ChatGPT for medical writing. First and foremost, ChatGPT lacks a moral com- pass. Information that’s accessible is information that’s usable, whether it is accurate or inaccurate, biased or unbi- ased, public or confidential. This is of particular concern with respect to protecting patient privacy and proprietary data in medical writing. I’m not a programmer, but it seems to me this challenge is potentially surmountable because the types of information that can reveal a patient’s identity are themselves identifiable and should therefore be able to be targeted as “off limits” to ChatGPT. It then becomes the job of reviewers to ensure that sensitive information is edited out of documents before they’re published. Another big ethical consideration with the use of ChatGPT in medical writing is accuracy. ChatGPT has every electronically accessible resource at its disposal from which to develop content, and as we all well know, not all that information is accurate. Furthermore, there’s growing evidence that ChatGPT can, and does, lie. This places an additional burden on reviewers to check and double-check ChatGPT’s work. But this brings me to my third big ethical concern regarding ChatGPT: accountability. No one—not even its inventors and keepers—knows how ChatGPT does what it does. Without the ability to question ChatGPT’s writing or hold it accountable for what it’s written, medical publishing ethicists like the Committee on Publication Ethics (COPE), organizations including the World Association of Medical Editors (WAME), and publishers such as the JAMA Network have all said that AI tools such as ChatGPT cannot be listed as the author of scholarly papers. Developers have put ChatGPT out there with all its faults and shortcomings, anticipating that real-world experience will teach and improve it. I’m sure it will. However, in a field as reliant upon sound ethical practices as medical commu- nication, I think we currently have too much to lose by put- ting too much faith in ChatGPT. —Brian Bass The 3,000 Springer-Nature journals, Elsevier’s 2,800 jour- nals, Taylor & Francis, JAMA Network, WAME (and British Medical Journal), and the International Committee of Medical Journal Editors (ICMJE) ban the listing of ChatGPT as an author. The Lancet Digital Health (owned by Elsevier) published a commentary entitled, “Generating scholarly content with ChatGPT: ethical challenges for medical pub- lishing,” complete with a response from ChatGPT in its sup- plementary material on the effect of AI on publishing ethics in medical publishing. These updated journal guidelines require that authors report the use of ChatGPT during manuscript preparation in the acknowledgement section. The burgeoning popular- ity of ChatGPT will undoubtedly lead to the development of more policies around its use and increase the importance of using human fact checkers. —Melissa L. Bogen * * * Q3: How can ChatGPT be used to assist medical writers in creating patient education materials, and what challenges need to be addressed in this context? Despite the caveats I put forth earlier in Question 1, I do feel there may be some interesting uses of ChatGPT in this con- text. (Assuming, of course, that one maintains personal and professional integrity and does one’s own work prior to que- rying and relying on ChatGPT.) First, let me point out that I include in the category of “patient education” the following types of medical writ- ing tasks with which I have personal experience—all of these require the simple clear language one should use for patients/consumers: • Pharma/biotech regulatory affairs: Informed consent forms; lay summaries (plain language summaries); standard operating procedures; and other “instruc- tional documents” for in-house or outside use • Pharma/biotech med comm, marketing communica- tion, sales, and public relations (corporate communi- cation): Patient education materials of all sorts; slide presentations or other company website information targeted to the general public (and perhaps to inves- tors as well); sales training materials; collateral advertising materials directed toward patients • Health maintenance organizations or managed care organizations: Letters to members and provider offices explaining services clearly; policies & proce- dures (P&Ps) • Hospitals, medical centers, doctors’ offices: Patient education materials; website copy intended for the layperson http://www.amwajournal.org https://publicationethics.org/cope-position-statements/ai-author https://wame.org/page3.php?id=106 https://wame.org/page3.php?id=106 https://jamanetwork.com/journals/jama/fullarticle/2801170 https://pubmed.ncbi.nlm.nih.gov/36694020/ https://www.elsevier.com/about/policies/publishing-ethics/the-use-of-ai-and-ai-assisted-writing-technologies-in-scientific-writing https://authorservices.taylorandfrancis.com/editorial-policies/defining-authorship-research-paper/ https://jamanetwork.com/journals/jama/fullarticle/2801170 https://wame.org/page3.php?id=106 https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html https://www.thelancet.com/journals/landig/article/PIIS2589-7500(23)00019-5/fulltext https://www.thelancet.com/journals/landig/article/PIIS2589-7500(23)00019-5/fulltext https://www.thelancet.com/journals/landig/article/PIIS2589-7500(23)00019-5/fulltext AMWAJournal.org 32Freelance Focus • Nonprofit health care organizations: Parts of grant applications and their interim reports; patient educa- tion fliers for their clients; P&Ps; other medical infor- mation documents they may create for their clients • Medical journalism: Articles for publication in lay magazines and trade journals reporting new informa- tion about diseases and medicines; website copy tar- geted to the layperson Certainly, there are other areas, but all the mentioned examples require a voice similar to that required for “patient education” materials. One might use ChatGPT to assist in formatting and organizing a plain language summary—first by asking for samples of various companies’ formats for such sum- maries (some of which are offered on the Food and Drug Administration website and elsewhere, but ChatGPT would offer several samples). This could help an inexperienced medical writer to understand how such summaries are best designed. One might also submit to ChatGPT the summary one has already written and request improvement. One might submit a less simple example of any of the mentioned documents one has already written and ask ChatGPT to revise it—just to see if the AI program does indeed offer improvement in structure or language (always, of course, being careful not to allow plagiarism to creep in). The main challenge, it seems to me, relates to discrim- ination in fact-checking and ensuring that plagiarism has not occurred. Likely there are other caveats, but I have not used the AI program(s) extensively enough to identify other difficulties or challenges. —Cathryn D. Evans Additional Information: A Developer Interface In addition to ChatGPT, OpenAI also has a developer inter- face available for US $20/month. In a webinar I attended, attendees from a med comm company noted this private interface could be used by pharmaceutical or medical com- munication companies on behalf of clients. Its advantages are that data could be kept confidential, larger data sets can be uploaded, and the user has more control over the responses. Here are some uses and examples. The developer interface could generate internal reports from multiple data sources and summarize advisory boards (eg, executive summaries). However, it will be a while before it can be trained to ignore the nonsensical chatter in an ad board transcript. The developer interface could be trained to act as a medical information chatbot (not for public release). The user could input prescribing information, a clinical study report, and published studies. These data could be used to draft medical letters or answers/replies. The interface could rewrite existing content from, say, a pharmacist geared toward another audience. The private interface could also provide consistency to omnichannel materials. It could ensure that the informa- tion is relevant to different audiences and could repurpose material into new formats (eg, from video to slide deck) much faster than a person could. But again, with the noted limitations, humans are needed for their thought, judgment, and discernment for accuracy and relevance. —Melissa L. Bogen General Principles of Word Usage www.amwa.org/online_learning Choose the right word for accuracy and clarity. http://www.amwajournal.org https://networkpharma.tv/2023/05/03/ai-today-and-tomorrow-real-world-applications-for-medcomms/