PMMB 2023, 6, 1; a0000337. doi: 10.36877/pmmb.0000337 http://journals.hh-publisher.com/index.php/pmmb Original Research Article Applications, Benefits, and Risks of ChatGPT in Medical and Health Sciences Research: An Experimental Study Yaser Mohammed Al-Worafi1,2*, Andi Hermansyah3*, Ching Siang Tan4*, Chee-Yan Choo5, Abdelhakim Bouyahya6, Ganesh Sritheran Paneerselvam7, Kai Bin Liew8, Khang Wen Goh9, Long Chiau Ming3,10,11 Article History 1College of Medical Sciences, Azal University for Human Development, Sana’a, Yemen 2College of Pharmacy, University of Science and Technology of Fujairah, Fujairah, United Arab Emirates 3Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, 60115, Indonesia 4School of Pharmacy, KPJ Healthcare University, 71800 Nilai, Malaysia 5Faculty of Pharmacy, Universiti Teknologi MARA, Puncak Alam, Malaysia; choo715@uitm.edu.my (C-YC) 6Laboratory of Human Pathologies Biology, Department of Biology, Faculty of Sciences, Mohammed V University in Rabat, Morocco; a.bouyahya@um5r.ac.ma (AB) 7School of Pharmacy, Taylor’s University, Subang Jaya, Selangor, Malaysia; ganeshsritheran.Paneerselvam@taylors.edu.my (GSP) 8Faculty of Pharmacy, University of Cyberjaya, Cyberjaya, Selangor, Malaysia; liewkaibin@cyberjaya.edu.my (KBL) 9Faculty of Data Science and Information Technology, INTI International University, 71800 Nilai, Malaysia; khangwen.goh@newinti.edu.my (KWG) 10PAP Rashidah Sa’adatul Bolkiah Institute of Health Sciences, Universiti Brunei Darussalam, Gadong, Brunei Darussalam 11School of Medical and Life Sciences, Sunway University, Bandar Sunway 47500, Selangor, Malaysia; longchiauming@gmail.com (LCM) *Corresponding author: Ching Siang Tan; School of Pharmacy, KPJ Healthcare University, 71800 Nilai, Malaysia; tcsiang@kpjuc.edu.my (TCS); Andi Hermansyah; 3Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, 60115, Indonesia; andi-h@ff.unair.ac.id (AH); Yaser Mohammed Al-Worafi; College of Medical Sciences, Azal University for Human Development, Sana’a, Yemen; yworafi@yahoo.com (YMA) Received: 26 February 2023; Received in Revised Form: 13 June 2023; Accepted: 18 June 2023; Available Online: 21 June 2023 PMMB 2023, 6, 1; a0000337 2 of 13 Abstract: This article explored the potential applications, benefits, and risks of using ChatGPT in medical and health sciences research. The experimental study was performed with content analysis of the potential applications, benefits and risks of using ChatGPT in medical and health sciences research. This study shows many potential applications, benefits, and risks of using ChatGPT in medical and health sciences research. The average experts’ ChatGPT appropriateness and accuracy rates in the eight research themes were between 60% and 95%. This concludes that ChatGPT could help medical and health sciences researchers, especially new researchers, with caution in many aspects of research. The ChatGPT is still in the early phase of use by researchers worldwide, and its ability to help in research will be better soon. Attending training workshops about ChatGPT and AI is very important and highly recommended. The practice of ChatGPT in medical and health sciences research is important and recommended to explore the potential uses, benefits, risks and suggest recommendations for the best practice. Keywords: artificial Intelligence; ChatGPT; medical; health sciences 1. Introduction Researching the various aspects of medical and health sciences education and practice is very important to evaluate the current practices, identify the challenges and suggest recommendations to address the identified challenges. The evolution of technology has contributed effectively to the medical and health sciences education, practice, and research during the last decades; one of the new technologies is Artificial Intelligence (AI) which has been implemented in many fields, including medical and health sciences education, practice and research [1-9]. AI can be defined as the “science and engineering of making intelligent machines”, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence. Still, AI does not have to confine itself to methods that are biologically observable” [9], or as “a field of science and engineering concerned with the computational understanding of what is commonly called intelligent behavior, and with the creation of artifacts that exhibit such behavior” [10]. One of the recent advances in AI development is the launch of a model called ChatGPT, which interacts conversationally. It is an innovative kind of AI that can produce writing that seems to have been written by a person [11]. The dialogue format allows ChatGPT to answer follow-up questions, admit mistakes, challenge incorrect premises, and reject inappropriate requests; ChatGPT is a general Large Language Model (LLM) developed recently by OpenAI. While the previous class of AI models has primarily been Deep Learning (DL) models, which are PMMB 2023, 6, 1; a0000337 3 of 13 designed to learn and recognize patterns in data, LLMs are a new type of AI algorithm trained to predict the likelihood of a given sequence of words based on the context of the words that come before it [12]. The ChatGPT can be used to assist researchers and health professionals in generating real time information dissemination [13]. For instance, in a study done on the Covid-19 pandemic among Brunei Darussalam patients [14], by using ChatGPT, the researcher can quickly gather data and help in answering queries about the latest case numbers, government regulations and guidelines, testing centers, vaccination sites and trends in vaccination. In a systematic review, the potential benefits of ChatGPT health care education, research, and practice have been highlighted in improving writing, research efficiency, streamlined workflow, and personalized learning [15]. Moreover, it can assist in drug use pattern-related studies, where researchers conducting a comparative analysis of the factors that impact the pattern of procurement and usage of antithrombotic drugs can leverage the insights gained to inform their medical and health sciences research [16]. Additionally, they can harness the capabilities of ChatGPT, an AI-powered language model, to further enhance their studies. By synergistically combining the findings from their comparative analysis with the support of ChatGPT, medical and health sciences researchers can effectively advance their understanding of the subject matter and contribute to evidence-based practices in this domain. Furthermore, ChatGPT can be considered a supportive tool to facilitate the literature review process. It can help the researcher access a wide range of literature sources and provide an overview of studies from various disciplines and publications. For instance, in gathering data related to Mangrove Antinobacterial diversity [17], use of ibrutinib in multiple myeloma [18], or in identifying the prevalence of Listeria monocytogens [19] related studies, ChatGPT can provide relevant summaries of the studies, helping to identify important findings, methodologies, and identifying the research gaps in existing research. Besides that, by leveraging ChatGPT, researchers can access up-to-date information on topics like Mycobacterium ulcerans [20], facilitating data gathering and answering queries related to pathogenesis, diagnosis, and treatment. Additionally, ChatGPT can aid in exploring breakthroughs in Actinobacteria drug discovery research [21] by assisting in data analysis, literature reviews, and generating insights. Furthermore, in studying the chemistry of the gut microbiome [22], ChatGPT can assist researchers in retrieving relevant studies, analyzing complex interactions, and generating hypotheses for further investigation. Overall, ChatGPT can be a valuable tool in medical research, supporting researchers with information retrieval, data analysis, and knowledge synthesis [23]. PMMB 2023, 6, 1; a0000337 4 of 13 As massive studies show that ChatGPT has the potential to revolutionize medical research in various ways [24-27], many medical and health sciences researchers started using ChatGPT at the end of 2022 for many purposes, including research and scientific writing. With its potential to revolutionize the clinical and translational medicine fields, ChatGPT offers benefits like access to current information, enhanced patient engagement, and reduced healthcare provider workloads [28]. This article aimed to explore the potential applications, benefits, and risks of using ChatGPT in medical and health sciences research. 2. Materials and Methods 2.1. Study Design The researchers will perform the content analysis of the potential applications, benefits, and risks of using ChatGPT in medical and health sciences. 2.2 Data collection The research was conducted between 01 January 2023 and 20 February 2023 to explore the potential applications, benefits, and risks of using ChatGPT in medical and health sciences research. Questions related to the medical and health sciences research were asked to explore the ability of ChatGPT to answer them; the questions were divided into the following themes: 2.2.1. Theme 1. Questions related to the research ideas The questions were to suggest ideas for research projects; eight questions were related to pharmacy practice, public health, and patient care. 2.2.2. Theme 2. Questions related to the research proposal The questions were to write a proposal for the research projects. Three projects were selected from what ChatGPT suggested in the previous step (Theme 1) related to pharmacy practice, public health, and patient care. 2.2.3. Theme 3. Questions related to the research background The questions were to write a background for the research projects. Three projects were selected from what ChatGPT suggested in the previous step (Theme 2) related to pharmacy practice, public health, and patient care. PMMB 2023, 6, 1; a0000337 5 of 13 2.2.4. Theme 4. Questions related to the literature review The questions were to write a literature review for the selected three projects. 2.2.5. Theme 5. Questions related to the research questions and objectives The questions were to suggest research questions and objectives for the selected three projects. 2.2.6. Theme 6. Questions related to the research hypothesis The questions were to suggest null and alternative hypotheses for the selected three projects. 2.2.7. Theme 7. Questions related to the significance of the study and expected outcomes The questions were to write the significance of the study and expected outcomes for the three selected projects. 2.2.8. Theme 8. Questions related to the methodology The questions were to suggest different study designs, data collection, populations, sample size calculation, sampling methods, and data analysis for the three selected projects. 2.3. Data Analysis Four professors with long experience in medical and health sciences research (two from pharmacy school and two from medicine school) independently evaluated the answers by ChatGPT and rated the appropriateness and accuracy of each question as a percentage out of 100; the average of four professors’ evaluations was considered in this study. Moreover, they will comment on the potential applications, benefits, and risks of using ChatGPT in medical and health sciences research. 3. Results 3.1. Theme 1 - Questions related to the research ideas Analysis of the expert’s opinion shows that ChatGPT was able to suggest more than ten research ideas for each specialty. Moreover, the ideas were good. The average of experts’ rates of appropriateness and accuracy was 95%. PMMB 2023, 6, 1; a0000337 6 of 13 3.1.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers, select good ideas. 3.1.2. Potential risks ChatGPT can generate general ideas but cannot generate specific ideas. 3.1.3. Recommendations Medical and health sciences researchers, especially new researchers, can use ChatGPT as a guide for generating research ideas. 3.2. Theme 2 - Questions related to the research proposal Analysis of the expert’s opinion shows that ChatGPT could write brief research proposals for the three selected proposals. The average experts’ rates of appropriateness and accuracy were 60%. 3.2.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers, as a guide in writing brief research proposals. 3.2.2. Potential risks There are many steps in writing good proposals, which ChatGPT will not be able to do. 3.2.3. Recommendations Medical and health sciences researchers cannot use ChatGPT to write good research proposals. 3.3. Theme 3 - Questions related to the research background Analysis of the expert’s opinion shows that ChatGPT could write research background with the rationality for the three selected projects. However, ChatGPT was not able to cite what was written appropriately. The average of experts’ rates of appropriateness and accuracy was 75%. PMMB 2023, 6, 1; a0000337 7 of 13 3.3.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers, as a guide in writing background about research projects. 3.3.2. Potential risks The background should include the appropriate references, which ChatGPT will not be able to do it. 3.3.3. Recommendations ChatGPT can help medical and health sciences researchers, especially new researchers, as a guide in writing the research background. 3.4 Theme 4 - Questions related to the literature review Analysis of the expert’s opinion shows that ChatGPT was able to write a research literature review for the three selected projects. However, ChatGPT could not cite what was written appropriately. Moreover, the literature review was not up-to-date. The average experts’ rates of appropriateness and accuracy were 70%. 3.4.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers, as a guide in writing literature reviews. This was in line with the finding by Vaishya et al. (2023), which states that the current ChatGPT version appears helpful to medical professionals [29]. 3.4.2. Potential risks Writing a literature review should include the appropriate references and update the literature, which ChatGPT will not be able to do. 3.4.3. Recommendations ChatGPT can help medical and health sciences researchers, especially new researchers, as a guide in writing the literature review. PMMB 2023, 6, 1; a0000337 8 of 13 3.5. Theme 5 - Questions related to the research questions and objectives Analysis of the expert’s opinion shows that ChatGPT was able to suggest good objectives and study questions. The average experts’ rates of appropriateness and accuracy were 90%. 3.5.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers writing the research objectives and study questions. 3.5.2. Potential risks ChatGPT cannot generate specific objectives and research questions. 3.5.3. Recommendations Medical and health sciences researchers, especially new researchers, can use ChatGPT as a guide for writing the study objectives and questions. 3.6. Theme 6 - Questions related to the research hypothesis Analysis of the expert’s opinion shows that ChatGPT was able to suggest good null and alternative hypotheses for the research projects. The average of experts’ rates of appropriateness and accuracy was 90%. 3.6.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers, write the null and alternative hypotheses for research projects. 3.6.2. Potential risks ChatGPT cannot generate specific null and alternative hypotheses for the research projects. 3.6.3. Recommendations Medical and health sciences researchers, especially new researchers, can use ChatGPT as a guide for writing the null and alternative hypotheses for research projects. PMMB 2023, 6, 1; a0000337 9 of 13 3.7. Theme 7 - Questions related to the significance of the study and expected outcomes Analysis of the expert’s opinion shows that ChatGPT was able to write the significance of the study and expected outcomes for the research projects. The average experts’ rates of appropriateness and accuracy were 80%. 3.7.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers, write the study's significance and expected outcomes for the research projects. 3.7.2. Potential risks ChatGPT cannot generate specific significance of the study and expected outcomes for the research projects. 3.7.3. Recommendations Medical and health sciences researchers, especially new researchers, can use ChatGPT as a guide for writing the significance of the study and expected outcomes for the research projects. 3.8. Theme 8. Questions related to the methodology Analysis of the expert’s opinion shows that ChatGPT could suggest different study designs, data collection, populations, sample size calculation, sampling methods, and data analysis for the selected projects. The average experts’ rating of appropriateness and accuracy was 60%. 3.8.1. Potential benefits ChatGPT can help medical and health sciences researchers, especially new researchers suggest different study designs, data collection, populations, sample size calculation, sampling methods, and data analysis for the selected projects. Moreover, ChatGPT can elaborate on each method, data collection, population, sample size calculation, sampling methods, and data analysis. PMMB 2023, 6, 1; a0000337 10 of 13 3.8.2. Potential risks ChatGPT cannot generate valid and reliable methods. There are many steps to design valid and reliable questionnaires or qualitative interviews, which ChatGPT will not be able to do. 3.8.3. Recommendations Medical and health sciences researchers, especially new researchers, in many can use ChatGPT as a guide for methodology. However, Medical and health sciences researchers cannot use ChatGPT to develop valid and reliable questionnaires and qualitative interviews. 4. Discussion This study explored the potential applications, benefits, and risks of using ChatGPT in medical and health sciences research. The findings of this study can be classified into the following themes. 4.1. Theme 1 - Research ideas This study's findings show that ChatGPT could suggest good research ideas. Research ideas are critical to start any research project; this could help new researchers in their early careers as a guide. However, researchers should apply critical thinking and search for new research ideas to fill the gap in the literature. Moreover, more information and research about this issue must be needed to compare and conclude that ChatGPT or the new AI technologies positively impact generating ideas. In conclusion, many medical and health sciences researchers, especially new researchers, can use ChatGPT as a guide for developing research ideas. For example, using a medical mobile phone application linked to ChatGPT assists in collecting a wealth of data and analyzing and interpreting this data into meaningful patterns and insights to support medical and health sciences research [30]. 4.2. Theme 2 - Research proposal This study's findings show that ChatGPT could write brief research, which is unacceptable to depend on for acceptable research proposals. However, ChatGPT is still in the early development phase, and it could be better soon and be able to generate better proposals. PMMB 2023, 6, 1; a0000337 11 of 13 4.3. Theme 3 - Research background and literature review This study's findings show that ChatGPT could write research background without the appropriate references and updated literature. This could be due to many reasons, such as the nature of ChatGPT as a new language model and the need for access to scientific databases. 4.4. Theme 4 - Research questions, objectives and hypothesis This study's findings show that ChatGPT was able to suggest good research objectives, questions, and hypotheses. Therefore, many medical and health sciences researchers, especially new researchers, can use ChatGPT as a guide for generating research objectives, questions, and hypotheses. 4.5. Theme 5 - Expected outcomes and significance of study This study's findings show that ChatGPT could write the significance of the study and expected outcomes for the research projects but cannot generate specific significance of the study and expected outcomes for the research projects. This could be due to the nature of ChatGPT as a new language model. ChatGPT is still in the early development phase and could be better soon. 4.6. Theme 6 - Research methodology This study's findings show that ChatGPT cannot generate valid and reliable methods. However, ChatGPT was able to suggest different study designs, data collection, populations, sample size calculation, sampling methods, and data analysis, which could help medical and health sciences researchers, especially new researchers. 5. Conclusions In conclusion, ChatGPT could help medical and health sciences researchers, especially new researchers, with caution in many aspects of research. ChatGPT is still in the early phase of use by researchers worldwide, and its ability to help in research will be better soon. Attending training workshops about ChatGPT and AI is very important and highly recommended. Practice ChatGPT in medical and health sciences research is very important and highly recommended to explore the potential uses, benefits, and risks and suggest recommendations for the best practice. PMMB 2023, 6, 1; a0000337 12 of 13 Author Contributions: All authors contributed equally to this study. Funding: No external funding was provided for this research. Conflicts of Interest: The authors declare no conflict of interest. References 1. Petitti DB, Crooks VC, Buckwalter JG, et al. Blood pressure levels before dementia. Arch Neurol 2005; 62(1): 112– 116. 2. Masters K. Artificial intelligence in medical education. Medical Teacher. 2019; 41(9):976-80. 3. Loeckx J. Blurring boundaries in education: Context and impact of MOOCs. International Review of Research in Open and Distributed Learning. 2016;17(3):92-121. 4. Ahuja AS. The impact of artificial intelligence in medicine on the future role of the physician. PeerJ. 2019;7:e7702. 5. Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. InArtificial Intelligence in healthcare. Academic Press. 2020; 1:25-60. 6. Secinaro S, Calandra D, Secinaro A, et al. The role of artificial intelligence in healthcare: a structured literature review. BMC medical informatics and decision making. 2021;21:1-23. 7. Davenport TH, Ronanki R. Artificial intelligence for the real world. Harvard business review. 2018 1;96(1):108-16. 8. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJ. Artificial intelligence in radiology. Nature Reviews Cancer. 2018;18(8):500-10. 9. Roll I, Wylie R. Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education. 2016;26:582-99. 10. McCarthy J. What is artificial intelligence. Available from : https://cse.unl.edu/~choueiry/S09-476- 876/Documents/whatisai.pdf . 2007 11. Javaid M, Haleem A, Singh RP. ChatGPT for healthcare services: An emerging stage for an innovative perspective. BenchCouncil Transactions on Benchmarks, Standards and Evaluations 2023; 3(1):100105. 12. Shapiro SC. Encyclopedia of artificial intelligence second edition. New Jersey: A Wiley Interscience Publication. 1992. 13. King MR. The future of AI in medicine: a perspective from a Chatbot. Ann Biomed Eng 2023; 51(2):291-295. 14. WS Ang, JW-Fei, V Letchumanan, YS Ong, Y Kumari, LC Ming, LT Tan. COVID-19 Pandemic in Brunei Darussalam. 2023. Prog Microbes Mol Biol. 6, 1; a0000326. 15. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare 2023; 11(6): 887. 16. MM Manan, AY Baharudin, QY Lean, JS Dhaliwal, SK Sodhi, MJ Farrukh, CS Tan, LC Ming. Comparison of the factors that influence the pattern of procurement and usage of antithrombotic drugs. Prog Microbes Mol Biol. 2022, 5, 1; a0000283. 17. Letchumanan V, Wong PC, Goh BH, et al. A review on the characteristics, taxanomy and prevalence of Listeria monocytogenes. Prog Microbes Mol Biol. 2018; 1(1): a0000007. 18. Law JWF, Pusparajah P, Ab Mutalib NS, et al. A Review on Mangrove Actinobacterial Diversity: The Roles of Streptomyces and Novel Species Discovery. Prog Microbes Mol Biol 2019; 2(1): a0000024. 19. Bong ST, Law LNS, Law JWF. A Review on the Potential of Bruton’s Tyrosine Kinase (Btk) Inhibitor - Ibrutinib for Treatment of Multiple Myeloma (MM). Prog Microbes Mol Biol 2019; 2(1): a0000028. 20. Tan LTH, Raghunath P, Ming LC, et al. Mycobacterium ulcerans and Mycobacterium marinum: pathogenesis, diagnosis and treatment. Prog Microbes Mol Biol 2020; 3(1). 21. Thye AY, Letchumanan V, Tan LTH, et al. Malaysia’s Breakthrough in Modern Actinobacteria (MOD-ACTINO) Drug Discovery Research. Prog Microbes Mol Biol 2022; 5(1). PMMB 2023, 6, 1; a0000337 13 of 13 22. Lau AWY, Tan LTH, Ab Mutalib NS, et al. The chemistry of gut microbiome in health and diseases. Prog Microbes Mol Biol 2021; 4(1). 23. Van Dis EA, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. 2023; 614(7947):224-226. 24. Ruksakulpiwat S, Kumar A, Ajibade A. Using ChatGPT in Medical Research: Current Status and Future Directions. J Multidiscip Healthc 2023; 1513-1520. 25. Cascella M, Montomoli J, Bellini V, et al. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst 2023; 47(1):33. 26. Khan RA, Jawaid M, Khan AR, et al. ChatGPT-Reshaping medical education and clinical management. Pak J Med Sci 2023; 39(2):605. 27. Wang DQ, Feng LY, Ye JG, et al. Accelerating the integration of ChatGPT and other large‐scale AI models into biomedical research and healthcare. MedComm Futur Med 2023; 2(2):e43. 28. Xue VW, Lei P, Cho WC. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med 2023;13(3). 29. Vaishya R, Misra A, Vaish A. ChatGPT: Is this version good for healthcare and research? Diabetes Metab Syndr 2023; 17(4):102744. 30. Loy MJ, Goh KW, Osili N, et al. Features and Functionalities of Medical Mobile Applications for the Endemic Phase of COVID‐19: Review and Content Analysis. Prog Microbes Mol Biol 2022;5(1). Author(s) shall retain the copyright of their work and grant the Journal/Publisher right for the first publication with the work simultaneously licensed under: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). This license allows for the copying, distribution and transmission of the work, provided the correct attribution of the original creator is stated. Adaptation and remixing are also permitted.