Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Ronak Hamzehei1, Masoumeh Ansari2, Shahabedin Rahmatizadeh3, Saeideh Valizadeh-Haghi4* 1. Clinical Research Development Unit of Shahid Beheshti hospital, Hamadan University of Medical Sciences, Hamadan, Iran 2. Medical Educational and Therapeutic center of Kowsar, Kurdistan University of Medical Sciences, Sanandaj, Iran 3. Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran 4. Department of Medical Library and Information Sciences, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran Abstract Objectives: Health service providers use internet as a tool for the spreading of health information and people often go on the web to acquire information about a disease. A wide range of information with varying qualities and by authors with varying degrees of credibility has thus become accessible by the public. Most people believe that the health information available on the internet is reliable. This issue reveals the need for having a critical view of the health information available online that is directly related to people's life. The Ebola epidemic is an emergency situation in the international public health domain and the internet is regarded as an important source for obtaining information on this disease. Given the absence of studies on the trustworthiness of health websites on Ebola, the present study was conducted to assess the trustworthiness of websites which are focused on this disease. Methods: The term "Ebola" was searched in Google, Yahoo and Bing search engines. Google Chrome browser was used for this purpose with the settings fixed on yielding 10 results per page. The first 30 English language websites in each of the three search engines were evaluated manually by using the HONcode of conduct tool. Moreover, the official HONcode toolbar was used to identify websites that had been officially certified by HON foundation. Results: Almost the half of the retrieved websites were commercial (49%). Complementarity was the least-observed criterion (37%) in all the websites retrieved from all three-search engines. Justifiability, Transparency and Financial Disclosure had been completely observed (100%). Discussion: The present study showed that only three criteria (Justifiability, Transparency and Financial Disclosure) out of the eight HON criteria were observed in the examined websites. Like other health websites, the websites concerned with Ebola are not reliable and should be used with caution. Conclusion: Considering the lack of a specific policy about the publication of health information on the web, it is necessary for healthcare providers to advise their patients to use only credible websites. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI Introduction Nowadays the internet has become one of the dominant ways of obtaining health information and ranks first among the sources of health information. The internet is regarded as a reliable and accessible source of information for patients and other individuals [1], and one out of every three people uses the internet to obtain their health information [2]. The internet is used as a tool for the spreading of health information by health service providers and as a source for obtaining health information by patients. People often go on the web as the first step in acquiring some rudimentary data about a disease [1]. The health information available on the web allows for an interactive communication between the producers and consumers of the information. The positive features of the internet as the leading source of health information does not mean that we can overlook its negative attributes, because not all internet users know the proper method of searching for information, and the information stored on the web also lacks a proper classification, and these issues can make the search for health information difficult [3]. Moreover, the internet is uncontrollable, and there is no authority for controlling the credibility and accuracy of the information available through it. Furthermore, putting information on the web is easy, inexpensive or free, and anyone with any level of expertise can easily post information on this medium. A wide range of information with varying qualities and by authors with varying degrees of credibility has thus become accessible by the general public [4]. In addition, more than 80% of people believe that the health information available on the internet is reliable [5], and many of these consumers of health information do not consult with health specialists about the health information retrieved on the web[6]. This issue reveals the need for having a critical assessment of the health information available online that is directly related to people's life and health as well as for the evaluation of health websites by organizations and individuals. The health information published on the internet affects people's perceived health and the patients’ decisions about treatment choices [7]. The Ebola epidemic is an emergency situation in the international public health domain [8]. The first outbreak of this disease started in 1976 in Democratic Republic of Congo, and the other in South Sudan in West Africa. The 2014–2016 outbreak in West Africa was the largest and most complex Ebola outbreak since the virus was first discovered in 1976[9]. Ebola virus disease is a seriously fatal. There is currently no standard treatment for this disease [10], and no vaccines have yet been developed to prevent it. Since prevention is always better than cure, it is highly important Furthermore, teaching them the criteria for assessing the trustworthiness of health websites would be helpful. Keywords: Patient portals, internet, online health information, Ebola, self-care, patient education *Correspondence: Saeideh Valizadeh-Haghi, email: saeideh.valizadeh@gmail.com DOI: 10.5210/ojphi.v10i3.9544 Copyright ©2018 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. mailto:saeideh.valizadeh@gmail.com Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI for people to know about this disease, its development as well as prevention methods. In this regard, the internet is considered as an important source for obtaining this information. Nonetheless, not all health websites are trustable, and some of them contain incorrect and unreliable information [3]. Given the absence of studies on the trustworthiness of health websites in relation to Ebola, the present study was conducted to assess the credibility of health websites that are focused on this disease. Methods Study samples and setting Search engines are the first and main tools used to search for information on the web [11] and have a major role in obtaining medical and health information by non-specialists and specialists in medicine [12]. Google, Yahoo and Bing are the three most popular search engines used by people around the globe [13,14]. For the present study, the term "Ebola" was searched in these three search engines. Google Chrome browser was used for this purpose with the settings fixed on yielding 10 results per page. Considering that most of search engine users only view the results appearing on the first three pages of their search [15,16], the first 30 websites in each of the three search engines were reviewed, making for a total of 90 results. The non-English websites, repetitive websites, articles in medical journals, non-relevant websites and inaccessible links were excluded from the assessment, and 43 out of the 90 retrieved websites were thus assessed (Figure 1). Data were collected through direct observation on Dec. 5, 2017. Figure 1: Internet search flow diagram Data collection tools There are various instructions and guidelines available for assessing the trustworthiness of health websites, and the Health on the Net Foundation Code of Conduct (HONcode) was selected for this research. This code is used in 102 countries for more than 7300 websites and 10 million pages as a reference for publishing health information [17]. The research tool consisted of a checklist Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI developed according to the HONcode (Figure 2). HONcode has eight criteria: Authoritative, Complementarity, Privacy, Attribution, Justifiability, Transparency, Financial Disclosure and Advertising Policy[18]. This tool has been used in many studies for assessing the credibility of health websites[19-22]. The websites to be evaluated were divided into four categories: University, Governmental, Commercial and Organizational. They were then manually assessed by MA and RH, and the validity of the resulting data was reassessed by SV and SR. The official HONcode toolbar was used to identify websites that had been officially assessed. The data obtained were analyzed in SPSS-17. Figure 2: HONcode principles. The figure information is adopted from the HON website[18] Results: Of the 90 retrieved websites, 43 were unique and were assessed in this study. Table 1 shows the frequency of the range of websites retrieved from Google, Bing and Yahoo search engines compared to each other. With 20 results, Google had the most unique pages retrieved. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI Table 1: The frequency of retrieved websites range in three search engines of Bing, Yahoo and Google Search Engine Number of Retrieved Sites Duplicate Websites No. of Unique Websites HON verified Bing 30 3 8 2 Yahoo 30 6 15 4 Google 30 3 20 2 Total 90 12 43 8 The reviewed websites were divided into four categories by domain: University, Commercial, Organizational and Governmental (Figure 3). Many of the retrieved websites were commercial (49%). 49% 28% 16% 7% Commercial Organizational Governmental Educ ational Figure 3: Distribution of websites by Domain Address Table 2 presents the consistency of the websites assessed with the HON criteria. Complementarity was the least-observed criterion in all the websites retrieved from all three search engines. Justifiability, Transparency and Financial Disclosure had been completely observed (100%). In addition, of the 43 websites assessed, only eight had been officially approved by the HONcode, and none of the other websites had fully observed the eight criteria of the HON. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI Table 2: Evaluation results based on the components of HON criteria categorized by search engines Search Engine Quality criterion Google (n = 20) Yahoo (n = 15) Bing (n = 8) No. of Websites (N = 43) Authority 11 (55%) 8 (53%) 5 (62%) 24 (56%) Complementarity 6 (30%) 7 (47%) 3 (37%) 16 (37%) Privacy 18 (90%) 15 (100%) 8 (100%) 41 (95%) Attribution 18 (90%) 15 (100%) 7 (87%) 40 (93%) Justifiability 20 (100%) 15 (100%) 8 (100%) 43 (100%) Transparency 20 (100%) 15 (100%) 8 (100%) 43 (100%) Financial disclosure 20 (100%) 15 (100%) 8 (100%) 43 (100%) Advertising policy 11 (55%) 12 (80%) 4 (50%) 27 (63%) Discussion The present findings showed that the websites providing information about Ebola have a poor degree of credibility, which agrees with the results of other studies conducted on health websites focusing on different issues [21,23-26]. Governmental and university websites focused on health generally seek to provide educational information [27], but only 16% of the websites retrieved in the present study belonged to governmental organizations and 7% to universities. As in line with the results of previous studies [28,29], the majority of websites retrieved at the present study (43%) were commercial. Compared to other websites, commercial websites have poor quality and credibility [30,31]. Therefore, while searching information on Ebola, people come across websites that are less valid than other websites and that may obtain incorrect information that could put their health at risk. It should be noted that merely being a university website does not ensure the higher quality of the information contained [32], and the accuracy of the information available on these websites should also be assessed. In the present study, of the 43 websites assessed, only eight had been officially assessed by the HON foundation, which agrees with the results of other studies conducted on similar subjects [33,34]. These websites are examples of websites that users will come across when searching information on Ebola. To help empower patients for facing various diseases, including Ebola, access to valid websites that contain high-quality information will be beneficial. Non-compliance with the HON criteria in the examined websites shows that users will come across less credible websites that may contain poor-quality information, which affects their proper decision-making about the prevention and treatment of Ebola. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI Health information written by specialists are more reliable [35]. Nevertheless, in the present study, just 56% of the websites had specified the name and specialization of the author(s). Also in a similar research that investigated testicular websites, only 32% of the examined websites had specified the author's name [33] while observing this criterion is indicative of the validity and trustworthiness of the information source [35]. The medical information provided on health websites should not replace the direct doctor-patient relationship. In fact, the information provided on websites is for the purpose of support and education and cannot replace consultation with a doctor who is directly in contact with the patient. This point should be clearly stated on health websites. In the present study, however, only 37% of the examined websites had declared this point. Given that only a small percentage of people consult with their doctor regarding the health information retrieved online [36], it is imperative for health websites to pay greater attention to this criterion. So that people can be well informed and refrain from replacing their doctor with the medical information obtained from health websites and will use online information after further consultation with the doctors. The present study showed that only three criteria (Justifiability, Transparency and Financial Disclosure) out of the eight HON criteria were observed in the examined websites (Table 2). Same as other health websites [24,25,37,38], the websites concerned with Ebola are not reliable and should be used with caution. Nevertheless, it should be noted that the HON criteria do not necessarily show the quality of the information published on a website and merely indicate the credibility of the website itself. Patients and other users of online health information should carefully assess the quality of the information retrieved through websites, even if this information has been obtained from credible websites. Given the importance of the internet in spreading health information and its extensive use for obtaining health information and given that only a small percentage of people consult with their doctor about the medical information obtained on the web, thus, clinicians have a key role in guiding patients to using trustable websites, so that they can make informed decisions about diseases and their health. Conclusions Considering the lack of a specific law or policy about the publication of health information on the web, it is necessary for healthcare providers to advise their patients to use only credible websites that contain quality information. Furthermore, it is necessary to teach them the criteria for assessing the trustworthiness of health websites. People's knowledge of health website evaluation tools such as the HONcode for identifying and using websites with a higher credibility will help them use better and higher-quality information. People will thus be able to have a better understanding of their health and can make more informed decisions about their health and illness. Also, since people use the internet to obtain health information and in the absence of a unique tool used globally for the assessment of health websites, it is essential for doctors to know about their patients’ use of online information, so that they can guide them to trustable and high-quality websites. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI Financial Disclosure This research was funded by the School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences (grant number 10571). References 1. Zickuhr K, Smith A. Digital differences. Pew Internet and American Life Project. Available at: http://pewinternet. org/Reports/2012/Digital-differences. aspx (accessed February 2014); 2012. 2. Van de Belt TH, Engelen LJ, Berben SAA, Teerenstra S, Samsom M, et al. 2013. Internet and social media for health-related information and communication in health care: preferences of the Dutch general population. J Med Internet Res. 15(10), e220. PubMed https://doi.org/10.2196/jmir.2607 3. Cline RJW, Haynes KM. 2001. Consumer health information seeking on the Internet: the state of the art. Health Educ Res. 16(6), 671-92. PubMed https://doi.org/10.1093/her/16.6.671 4. Kaicker J, Debono VB, Dang W, Buckley N, Thabane L. 2010. Assessment of the quality and variability of health information on chronic pain websites using the DISCERN instrument. BMC Med. 8(1), 59. PubMed https://doi.org/10.1186/1741-7015-8-59 5. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet: The case of carpal tunnel syndrome. J Bone Jt Surg - Ser A. 2000;82(11):1540–3. 6. Fox S. Online health search 2006. Pew Internet & American Life Project; 2006. 7. Nan X, Madden K. 2012. HPV Vaccine Information in the Blogosphere: How Positive and Negative Blogs Influence Vaccine-Related Risk Perceptions, Attitudes, and Behavioral Intentions. Health Commun. 27(8), 829-36. PubMed https://doi.org/10.1080/10410236.2012.661348 8. Love CB, Arnesen SJ, Phillips SJ. 2015. Ebola outbreak response: the role of information resources and the National Library of Medicine. Disaster Med Public Health Prep. 9(1), 82- 85. PubMed https://doi.org/10.1017/dmp.2014.108 9. Ebola virus disease [Internet]. WHO. 2018 [cited 2018 Nov 26]. Available from: http://www.who.int/news-room/fact-sheets/detail/ebola-virus-disease 10. Tseng C-P, Chan Y-J. 2015. Overview of Ebola virus disease in 2014. J Chin Med Assoc. 78(1), 51-55. PubMed https://doi.org/10.1016/j.jcma.2014.11.007 11. Bilal D. 2012. Ranking, relevance judgment, and precision of information retrieval on children’s queries: Evaluation of Google, Yahoo! Bing, Yahoo! Kids, and ask Kids [Internet]. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI J Am Soc Inf Sci Technol. 63(9), 1879-96. http://doi.wiley.com/10.1002/asi.22675. doi:10.1002/asi.22675. 12. Wang L, Wang J, Wang M, Li Y, Liang Y, et al. 2012. Using Internet search engines to obtain medical information: a comparative study. J Med Internet Res. 14(3), e74. PubMed https://doi.org/10.2196/jmir.1943 13. Alexa. The top 500 sites on the web [Internet]. [cited 2010 Aug 20]. Available from: https://www.alexa.com/topsites%0Ahttp://www.alexa.com/topsites 14. Gkouskou K, Markaki A, Vasilaki M, Roidis A, Vlastos I. 2011. Quality of nutritional information on the Internet in health and disease [PubMed]. Hippokratia. 15(4), 304. 15. Promislow S, Walker JR, Taheri M, Bernstein CN. 2010. How well does the Internet answer patients’ questions about inflammatory bowel disease? [PubMed]. Can J Gastroenterol. 24(11), 671-77. 16. iProspect Blended Search Results Study 2008 [Internet]. [cited 2017 Sep 10]. Available from: http://www.iprospect.com 17. The commitment to reliable health and medical information on the internet [Internet]. Health on the Net Foundation. 2018. Available from: http://healthonnet.org/HONcode/Pro/Visitor/visitor.html 18. The HON Code of Conduct for medical and health Web sites (HONcode) [Internet]. [cited 2017 Nov 10]. Available from: http://www.hon.ch/HONcode/ 19. Saraswat I, Abouassaly R, Dwyer P, Bolton DM, Lawrentschuk N. 2016. Female urinary incontinence health information quality on the Internet: a multilingual evaluation. Int Urogynecol J. 27(1), 69-76. PubMed https://doi.org/10.1007/s00192-015-2742-5 20. Grohol JM, Slimowicz J, Granda R. 2014. The quality of mental health information commonly searched for on the Internet. Cyberpsychol Behav Soc Netw. 17(4), 216-21. PubMed https://doi.org/10.1089/cyber.2013.0258 21. Hanna K, Brennan D, Sambrook P, Armfield J. 2015. Third molars on the Internet: a guide for assessing information quality and readability. Interact J Med Res. 4(4), e19. PubMed https://doi.org/10.2196/ijmr.4712 22. Lawrentschuk N, Sasges D, Tasevski R, Abouassaly R, Scott AM, et al. 2012. Oncology health information quality on the Internet: a multilingual evaluation. Ann Surg Oncol. 19(3), 706-13. PubMed https://doi.org/10.1245/s10434-011-2137-x 23. Ellsworth B, Patel H, Kamath AF. 2016. Assessment of quality and content of online information about hip arthroscopy. Arthroscopy. 32(10), 2082. PubMed https://doi.org/10.1016/j.arthro.2016.03.019 Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI 24. Valizadeh-Haghi S, Rahmatizadeh S. 2018. Evaluation of the Quality and Accessibility of Available Websites on Kidney Transplantation [PubMed]. Urol J. 15(5), 261-26. 25. Rahmatizadeh S, Valizadeh-Haghi S. 2018 Evaluating the trustworthiness of consumer- oriented health websites on diabetes. Libr Philos Pract. 4 26. ShahrabiFarahani N. Shekofteh M. Kazerani M, Emami Z. An Evaluation of Persian Diabetes Websites based on WebMedQual (2016). Iran J Endocrinol Metab [Internet]. 2018 Oct 15 [cited 2018 Nov 1];20(3):142–50. Available from: http://ijem.sbmu.ac.ir/article-1-2352- en.html 27. Cancer Information on the Internet [Internet]. Vol. 2017. American Cancer Society; 2016 [cited 2018 Feb 20]. Available from: http://www.cancer.org/cancer/cancer-basics/cancer- information-on-the-internet.html 28. López-Jornet P, Camacho-Alonso F. The quality of internet information relating to oral leukoplakia. Med Oral Patol Oral Cir Bucal [Internet]. 2010 [cited 2018 Mar 19];15(5):727– 31. Available from: http://www.medicinaoral.com/medoralfree01/v15i5/medoralv15i5p727.pdf 29. Bruce-Brand RA, Baker JF, Byrne DP, Hogan NA, McCarthy T. 2013. Assessment of the Quality and Content of Information on Anterior Cruciate Ligament Reconstruction on the Internet. Arthroscopy. 29(6), 1095-100. PubMed https://doi.org/10.1016/j.arthro.2013.02.007 30. Khazaal Y, Chatton A, Cochand S, Zullino D. 2008. Quality of Web-based information on cocaine addiction [Internet]. Patient Educ Couns. 72(2), 336-41. http://www.ncbi.nlm.nih.gov/pubmed/18423952. PubMed https://doi.org/10.1016/j.pec.2008.03.002 31. Ostry A, Young ML, Hughes M. 2007. The quality of nutritional information available on popular websites: a content analysis. Health Educ Res. 23(4), 648-55. PubMed https://doi.org/10.1093/her/cym050 32. Lee S, Shin JJ, Haro MS, Song SH, Nho SJ. 2014. Evaluating the Quality of Internet Information for Femoroacetabular Impingement. Arthroscopy. 30(10), 1372-79. PubMed https://doi.org/10.1016/j.arthro.2014.04.102 33. Prasanth AS, Jayarajah U, Mohanappirian R, Seneviratne SA. 2018. Assessment of the quality of patient-oriented information over internet on testicular cancer. BMC Cancer. 18(1), 491. PubMed https://doi.org/10.1186/s12885-018-4436-0 34. Perzel S, Huebner H, Rascher W, Menendez-Castro C, Hartner A, et al. Searching the web: A survey on the quality of advice on postnatal sequelae of intrauterine growth restriction and the implication of developmental origins of health and disease. In: Journal of Developmental Origins of Health and Disease. 2017. p. 604–12. Websites as a tool for public health education: determining the trustworthiness of health websites on Ebola disease Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(3):e221, 2018 OJPHI 35. Evaluating Health Information [Internet]. University of California. 2016 [cited 2018 Feb 20]. Available from: https://www.ucsfhealth.org/education/evaluating_health_information/ 36. Fast AM, Deibert CM, Hruby GW, Glassberg KI. 2013. Evaluating the quality of Internet health resources in pediatric urology [Internet]. J Pediatr Urol. 9(2), 151-56. http://www.ncbi.nlm.nih.gov/pubmed/22281281. PubMed https://doi.org/10.1016/j.jpurol.2012.01.004 37. Elliott ADAD, Bartel AFPAF, Simonson D, Roukis TSTS. Is the Internet a Reliable Source of Information for Patients Seeking Total Ankle Replacement? J Foot Ankle Surg. 2015/03/10. 2015;54(3):378–81. 38. Sullivan TB, Anderson JT, Ahn UM, Ahn NU. 2014. Can Internet Information on Vertebroplasty be a Reliable Means of Patient Self-education? Clin Orthop Relat Res. 472(5), 1597-604. PubMed https://doi.org/10.1007/s11999-013-3425-5