Vol 15 No 05 September-October 2018 261 KIDNEY TRANSPLANTATION Evaluation of the Quality and Accessibility of Available Websites on Kidney Transplantation Saeideh Valizadeh-Haghi1, Shahabedin Rahmatizadeh2* Purpose: (i) to assess the quality of health websites on kidney transplant and (ii) to evaluate the accessibility of these websites and their concordance with the existing guidelines. Materials and Methods: The terms “kidney transplantation” and “renal transplantation” were searched in the three most popular search engines Google, Yahoo, and Bing. 58 unique websites were eligible for the analysis . The Websites accessibility was evaluated using the AChecker tool. Kruskal–Wallis test was performed to examine any significant difference between accessibility issues across different domains. The eligible websites were screened for quality based on the HONcode of conducts. Moreover, the daily traffic data of each website was determined by Alexa. The correlation of known accessibility problems with website popularity was examined too. Result: The main reported known problems belonged to “scripts must have functional text,” “text equivalents,” “accessible forms,” and “text links for server-side image map”. Although the mean accessibility errors in gov- ernmental (10.25 ± 7.274) and organizational (12.31 ± 9.469) websites were less than those in the other domains, the differences were not significant (P = 0.60). Findings showed no significant correlation (P > 0.05) between the extent of known problems (16.50 ± 12.18) and Alexa ranking (253675.07 ± 534690.947). Furthermore, most websites on kidney transplant were not certified by the HONcode. Conclusion: The health websites designers should be aware of accessibility problems, because there is a growing population of potential users with disabilities. This study indicated the need to ensure the compliance of kidney transplant websites with accessibility guidelines such as Section 508. Furthermore, most surveyed websites were of poor quality and unreliable. Therefore, physicians should warn their patients about unqualified online health information and guide them to websites which are more reliable. Keywords: kidney transplantation; consumer health information; web accessibility; HONcode; health education; health informatics; website popularity; Alexa ranking INTRODUCTION End-stage renal disease (ESRD), which causes an irreversible impairment in renal function, may be fatal if transplantation or dialysis is discarded(1). Despite major advances in diagnostic and surgical methods for kidney transplant, the associated complications remain a major clinical problem, which can improve the risk of hospitalization and morbidity and increase medical costs (3). Therefore, the patients who need a kidney trans- plant should decide informatively. To do so, in addition to medical consultations, they need extra information about the conditions, side effects and postsurgical care. Health information can be obtained from various sourc- es and a person’s choice of the source of information would affect their future health-related decisions(4). Fol- lowing the increasing demand for health awareness, the Internet has become a vital source of information. This highlights the need for reliable health websites that help users understand their health status and make appropri- ate decisions (5). Despite its advantages, e.g., availability, online health information may not always be reliable(6). Previous 1Assistant professor, Department of Medical Library and Information Sciences, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. 2Assistant professor, Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. *Correspondence: 4th floor, No 21, Darband st., Tajrish sq., Tehran. Iran; Email: sh.rahmatizadeh@sbmu.ac.ir. Received November 2017 & Accepted March 2018 studies have reported the poor quality of many websites due to the dissemination of misleading, inaccurate, in- complete, and inappropriate information. Grewal and Alagaratnam assessed the quality of colorectal disease websites for colorectal cancer. Their study showed that colorectal cancer websites were potentially unrelia- ble(7). In another study, Haymes assessed the quality of health information regarding rhinoplasty on the inter- net. The findings showed that, the quality of informa- tion available on the Internet with regard to rhinoplasty was generally of low quality and unreliable(8). In an- other study, the information on total ankle replacement (TAR) available to the general public through the Inter- net were evaluated. The study has demonstrated a low quality of TAR information available across all website types(9). Considering the significant effects of the use of online health information on an individual’s overall health, the provision of unreliable or inappropriate information would increase the risk of negative consequences, such as ineffective treatments or delays in seeking medical care(10). Therefore, evaluation and identification of the quality of health websites are necessary. However, considering the growing number of poten- tial users with disabilities(11), the accessibility of many health websites to physically challenged users is very limited(12). Through web accessibility, people can comprehend, navigate, and interact with the Internet, regardless of their limitations(13). Therefore, web de- signers should take accessibility into account to satisfy needs of such users. To understand the accessibility bar- riers of health websites, web accessibility evaluation is needed, which refers to the evaluation of Internet use by physically challenged individuals. Currently, there are no studies assessing the quality and accessibility of health websites on kidney transplant. In this study, by considering the Internet as an information source for patients, we (i) assessed the quality of health websites on kidney transplant and (ii) evaluated the ac- cessibility of these websites and their concordance with the existing guidelines. MATERIALS AND METHODS The terms “kidney transplantation” and “renal trans- plantation ” were searched in the three most popular search engines, i.e., Google, Yahoo, and Bing. the first three pages of search results provided by each of the above-mentioned search engines (180 URLs) were evaluated in this study. All URLs were analyzed, and redundant websites (containing links to portable document files, repeated unreachable addresses, non-English websites, and ad- vertising websites) were excluded. After exclusion, 58 unique websites were eligible for the analysis . Each re- trieved website was classified as governmental (.gov), educational (.edu), commercial (.com), and organiza- tional (.org). In this study, website accessibility was evaluated using the AChecker automatic tool(16), because it has been ac- credited by the World Wide Web Consortium and has been introduced in the consortium portal (“Web Ac- cessibility Evaluation Tools List,” n.d.). Furthermore, AChecker is a reliable, cost-effective tool and has been used in several studies to examine website accessibil- ity(17,18). Achecker defines three levels of problems, including “known,” “likely,” and “potential”. Known problems are identified as certain accessibility barriers and should be resolved by website owners. Likely prob- lems are identified as probable barriers, which should be identified by an individual. Finally, potential prob- lems cannot be identified by AChecker and require a human decision(19). In this study, known problems were reported as per the Section 508 guidelines. Non-para- metric Kruskal–Wallis test was performed to examine any significant difference between accessibility issues across different domains. The eligible websites were sequentially screened for quality based on the HONcode of conducts, which has set regulations to make website developers adhere to ethical standards in presenting information and to assist readers in identifying the purpose and source of data. For this purpose, HON principles were applied using the HONcode toolbar (http://www.hon.ch)(20). This toolbar function, which has been used and examined in different studies, is considered valid(21). Kruskal–Wallis test was performed to examine possi- ble differences in the mean ranking of known problems between HONcode-verified and unverified websites. Moreover, the daily traffic data of each website, deter- mined by Alexa as an index of popularity of websites, was used by the researchers. Alexa’s Traffic Ranks are based on the traffic data provided by users in Al- exa’s global data panel over a 3 month rolling period (22). To examine the correlation of known accessibili- ty problems with website popularity, non-parametric Spearman’s test was conducted. For statistical analyses, SPSS version 24 was used. The level of statistical sig- nificance was set at P < 0.05 . RESULTS All URLs were analyzed, and redundant websites were excluded. After exclusion, 58 unique websites were el- igible for the analysis . Four out of 58 websites were not responsive to online evaluation. The most frequent accessibility error types reported by AChecker, along with the percentage of websites with these errors, are presented in Table 1. The main reported known problems belonged to: “scripts must have functional text,” “text equivalents,” “accessible forms,” and “text links for server-side im- age map” (Table 2). To investigate the relationship between the extent of ac- Table 1. Most frequent accessibility error types Check ID Description Error category Webpages (%) 90 Script must have a nonscript section Script must have functional text *91.4 1 Img element missing alt attribute Text equivalents 56.9 57 Input element, type of "text", missing an associated label Accessible forms 46.6 7 Image used as anchor is missing valid Alt text Text equivalents 36.2 91 Select element missing an associated label Accessible forms 8.6 58 Image used for input element is missing Alt text Accessible forms 6.9 121 Input element, type of "radio", missing an associated label Accessible forms 3.4 91 Select element missing an associated label Accessible forms 3.4 118 Input element, type of "password", missing an associated label Accessible forms 1.7 13 Client-side image map missing duplicate text links Text links for server-side image map 1.7 119 Input element, type of "checkbox", missing an associated label Accessible forms 1.7 *Most of the websites showed error in check90. Error category Websites n (%) Script must have functional text 53 (91.4%) Text equivalents 41 (70.7%) Accessible forms 30 (51.7%) Text links for server-side image map 1 (1.7%) Table 2. Accessibility error rate by category Evaluation of Kidney Transplant Websites- Valizadeh Haghi et al. Kidney Transplantation 262 Vol 15 No 05 September-October 2018 263 cessibility errors and domain of studied websites, first, quantitative normalization of the extent of errors was investigated by Smirnov–Kolmogorov test. Consid- ering the absence of normal distribution, comparisons were made by Kruskal–Wallis test in different domains. According to Table 3, although the mean accessibility errors in governmental (10.25 ± 7.274) and organiza- tional (12.31 ± 9.469) websites were less than those in the other domains, the differences were not significant (P = 0.60). Spearman’s correlation coefficient test showed no sig- nificant correlation (P > 0.05) between the extent of known problems and Alexa ranking. Most websites on kidney transplant (70.7%) were not certified by the HONcode toolbar. The mean of known problems was lower in the HON-verified websites (11.41 ± 7.78) than in the verified websites (18.61 ± 13.10). Differences were statistically significant (X2 = 4.428; P = 0.035 on Kruskal–Wallis test; Table 4). Table 4. Kruskal–Wallis test results comparing verified and unverified websites DISCUSSION To the best of our knowledge, this is the first study to investigate the status of accessibility as well as quality of websites containing information regarding kidney transplant. Generally, websites with complex interac- tion modalities and user interfaces expose physically challenged people to new opportunities as well as chal- lenges(23). Currently, attention to the issue of website accessibility is very important. Adherence to the guide- lines of web accessibility is essential to reduce the gap between digitally underserved (e.g., physically chal- lenged people) and information-affluent people(24). Unfortunately, several websites are inaccessible to most people. Overall, web accessibility status is largely un- known, particularly in health information websites, be- cause limited studies have assessed the level of compli- ance of health websites. Accordingly, the present study aimed to examine the current status of accessibility to health websites on kidney transplant for people with disabilities. The accessibility status of health websites is varia- ble, depending on different health topics. The results of accessibility evaluation in prostate cancer websites showed that the majority of websites (92%) were acces- sible(25). The results of accessibility evaluation of aortic aneurysm treatment websites also showed moderate quality in terms of accessibility(26). Some other studies on health website accessibility in different subjects have shown that most health websites are not accessible enough to people with disabilities(27). Similarly, the present research showed that the majority of kidney transplant websites (96.6%) had accessibility barriers to physically challenged people. Findings are discouraging because most kidney transplant websites do not fulfill the criteria. The majority of accessibility problems, particularly those related to scripts, are addressed by web develop- ers(28). Similarly, the present study revealed that most of health websites on kidney transplant (91.4%) have problems in “script must have functional text” cate- gory. Therefore, health website designers should use scripting languages (for displaying content or creating interface elements), which can be read using assistive technologies(29). A developer can increase website accessibility by in- cluding alternate texts for video files, images, and audio files; the idea is to present a textual description(30). The present study revealed that 70.7% of surveyed websites had accessibility issues in the “text equivalent” cate- gory, mainly “img element missing alt attribute” error type (Table 2). Therefore, in health websites on kidney transplant, textual equivalents should be presented for all non-text elements that convey information to make the websites more accessible to physically challenged people. Moreover, providing alternate texts for image maps is necessary(30). Client-side image maps, instead of server-side maps, should be used to improve accessibility ad in the present study, most websites had taken this point into account. In fact, without a text alternative for each section, serv- er-side image maps are not accessible(29). Special considerations should be taken regarding the design of health website accessible forms since much of the information retrieved from the Internet is gath- ered in online forms(29). However, our study showed that 51.7% of surveyed websites had problems in “acces- sible forms” category (Table 2). Therefore, physically challenged users may encounter problems while using these websites. Since government-sponsored websites and educational institutions present reliable health information and are trustable(31), it is expected that people, including phys- ically challenged patients, use these types of websites more than the commercial and private ones. Therefore, it is expected that designers of these health websites consider equal access to the information for all users. Zeng and Bambang (2003) revealed that governmental and educational health websites exhibit better perfor- mance on web accessibility than other domains(32). In the present study, governmental and educational web- sites were speculated to show better performance re- Table 3. Mean and standard deviation of known problems of web- site accessibility across domains Domain Number of Known Problems Websites (n) Mean Standard Deviation Commercial (com) 19 22.16 12.807 Educational (edu) 9 19.44 15.018 Organizational (org) 26 12.31 9.469 Governmental (gov) 4 10.25 7.274 Total 58 16.50 12.180 HON not verified HON verified P-value Known Problem Mean (SD) Median Mean (SD) Median 0.035 18.61 (13.10) 19.00 11.41 (7.78) 8.00 Table 4. Kruskal–Wallis test results comparing verified and unverified websites Evaluation of Kidney Transplant Websites- Valizadeh Haghi et al. garding accessibility. Nevertheless, the results showed no significant differences among various domains with regard to accessibility barriers (Table 3). In this study, accessibility barriers exist in all catego- ries of websites (e.g., educational and governmental), especially commercial websites. Because physically challenged people prefer to use websites with less ac- cessibility barriers, they may visit accessible websites, which contain unreliable health information and can negatively affect their health. In general, accessibility may have positive effects on a website’s popularity(24). Nevertheless, in the present study, the results revealed that the correlation between accessibility barriers and website popularity is not sig- nificant (r = 0.172, P = 0.205). This implies that people with disabilities may encounter accessibility barriers, even if they visit popular websites. Because physically challenged people are among the Internet users, popular websites should pay special attention to the accessibility guidelines to make their websites more accessible; this can in turn increase the website visits and popularity. Based on the results, various factors may be respon- sible for the limited compliance of websites with ac- cessibility guidelines. In some studies, one of the main problems was that many developers did not prioritize accessibility(33). Another reason is that websites may not be evaluated or modified after the design, based on ac- cessibility guidelines. While web design strategies should be in line with ac- cessibility needs assessment of users with different dis- abilities (e.g., cognitive, visual, auditory, and motor disabilities)(34). In addition to accessibility, quality of health websites is important, because it may affect the patients’ decision-making. Therefore, we assessed the quality of health websites on kidney transplant, as well. Our study showed that kidney transplant websites are of poor quality, as only 17 out of 58 (29.3%) websites were HON-verified, which is in line with the studies in various health topics(8,9); therefore, patients should use these websites with more caution. Moreover, accessibility errors in HON-verified websites were fewer than the unverified ones, and differences were statistically significant (P = 0.035). Authoritative websites on kidney transplant had made more efforts to make their websites more accessible. Therefore, physi- cally challenged people who use authoritative websites to fulfill their information needs on kidney transplant, are able to use assistive technologies more effectively and encounter fewer barriers while obtaining health in- formation. CONCLUSIONS The health websites designers, as well as owners, should be aware of accessibility problems, because there is a growing population of potential users with disabilities. This study indicated the need to ensure the compliance of kidney transplant websites with accessibility guide- lines such as Section 508. Furthermore, because most surveyed websites were of poor quality and unreliable, there is a need to pay special attention to this problem. Physicians should warn their patients about unqualified online health information and guide them to websites which are more reliable. Because limited studies have examined the accessibil- ity of health websites, besides the present study, it is recommended to pay more attention to the assessment of website accessibility on different topics. This effort is expected to increase awareness on web accessibility issues in health information websites. ACKNOWLEDGEMENT The authors would like to thank Dr. Mir Davood Omrani, and appreciate his kindly supports for this study. This article is extracted from the research project (code 13886) which is funded by the School of Allied Medical Sciences, Shahid Beheshti University of Med- ical Sciences. CONFLICT OF INTEREST The authors report no conflict of interest. REFERENCES 1. Abbasi MA, Chertow GM, Hall YN. End- stage renal disease. BMJ Clin Evid. 2010. 2. Garcia GG, Harden P, Chapman J, World Kidney Day Steering Committee 2012 for the WKDSC. The global role of kidney transplantation. Kidney Blood Press Res. 2012;35:299–304. 3. Humar A, Matas AJ. Surgical Complications After Kidney Transplantation. Semin Dial. 2005;18:505–10. 4. Hornik R, Parvanta S, Mello S, Freres D, Kelly B, Schwartz JS. Effects of Scanning— Routine Health Information Exposure—on Cancer Screening and Prevention Behaviors in the General Population. J Health Commun. 2013;18:1422. 5. Coulter A. Engaging Patients in Healthcare. McGraw-Hill Education (UK); 2011. 6. Mun YY, Yoon JJ, Davis JM, Lee T. Untangling the antecedents of initial trust in Web-based health information: The roles of argument quality, source expertise, and user perceptions of information quality and risk. Decis Support Syst. 2013;55:284–95. 7. Grewal P, Alagaratnam S. The quality and readability of colorectal cancer information on the internet. Int J Surg. 2013/03/26. 2013;11:410–3. 8. Haymes AT. The Quality of Rhinoplasty Health Information on the Internet. Ann Plast Surg. 2016/01/14. 2016;76:143–9. 9. Elliott AD, Bartel AF, Simonson D, Roukis TS. Is the internet a reliable source of information for patients seeking total ankle replacement? J Foot Ankle Surg. 2015/03/10. 2015;54:378–81. 10. Hu X, Bell RA, Kravitz RL, Orrange S. The prepared patient: information seeking of online support group members before their medical appointments. J Health Commun. 2012;17:960–78. 11. Harrison S, Barlow J, Williams G. The content and interactivity of health support group websites. Health Educ J. 2007;66:371–81. 12. Chiang MF, Starren J. Evaluation of consumer Evaluation of Kidney Transplant Websites- Valizadeh Haghi et al. Kidney Transplantation 264 Vol 15 No 05 September-October 2018 265 health website accessibility by users with sensory and physical disabilities. Stud Health Technol Inform. 2004;107:1128–32. 13. de Santana VF, de Oliveira R, Almeida LDA, Baranauskas MCC. Web accessibility and people with dyslexia: a survey on techniques and guidelines. In: Proceedings of the International Cross-Disciplinary Conference on Web Accessibility - W4A ’12. New York, New York, USA: ACM Press; 2012. p. 1. 14. Cebi S. Determining importance degrees of website design parameters based on interactions and types of websites. Decis Support Syst. 2013;54:1030–43. 15. PURCELL K, BRENNER J;, RAINIE LEE. Search Engine Use 2012. Washington, D.C.: Pew Research Center’s Internet & American Life Project; 2012. 16. IDI Web Accessibility Checker : Web Accessibility Checker. 17. AkgÜL Y, Vatansever K. Web Accessibility Evaluation of Government Websites for People with Disabilities in Turkey. J Adv Manag Sci. 2016;201–10. 18. Alahmadi T, Drew S. An evaluation of the accessibility of top-ranking university websites: Accessibility rates from 2005 to 2015. In: There and back: Charting flexible pathways in open, mobile and istance education. Hamilton; 2016; 224–33. 19. Gay G, Li CQ. AChecker: open, interactive, customizable, web accessibility checking. In: Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) - W4A ’10. New York, New York, USA: ACM Press; 2010; 1. 20. The HON Code of Conduct for medical and health Web sites (HONcode) [Internet]. [cited 2017 Nov 10]. Available from: http://www. hon.ch/HONcode/ 21. Memon M, Ginsberg L, Simunovic N, Ristevski B, Bhandari M, Kleinlugtenbelt YV. Quality of Web-based Information for the 10 Most Common Fractures. Interact J Med Res. 2016;5:e19. 22. Website Traffic, Statistics and Analytics - Alexa. 23. Harper S, Yesilada Y. Web accessibility: a foundation for research. Harper S, Yesilada Y, editors. Springer Science & Business Media; 2008; 364 p. 24. Zeng X, Parmanto B. Web Content Accessibility of Consumer Health Information Web Sites for People with Disabilities: A Cross Sectional Evaluation. J Med Internet Res. 2004;6:e19. 25. Borgmann H, Wölm J-H, Vallo S, Mager R, Huber J, Breyer J, et al. Prostate Cancer on the Web—Expedient Tool for Patients’ Decision- Making? J Cancer Educ. 2017;32:135–40. 26. San Norberto EM, Taylor J, Salvador R, Revilla Á, Merino B, Vaquero C. The Quality of Information Available on the Internet About Aortic Aneurysm and Its Endovascular Treatment. Rev Española Cardiol (English Ed. 2011;64:869–75. 27. Niu L, Luo D, Liu Y, Xiao S. The Accessibility, Usability, and Reliability of Chinese Web-Based Information on HIV/ AIDS. Int J Environ Res Public Health. 2016 Aug;13(8):834. 28. Harper S, Yesilada Y, editors. Web accessibility: a foundation for research. Springer Science & Business Media; 2008;364. 29. Shneiderman B. Research-Based Web Design & Usability Guidelines. Igarss. 2014:1-5. 30. Sierkowski B, Brian. Achieving web accessibility. In: Proceedings of the 30th annual ACM SIGUCCS conference on User services - SIGUCCS ’02. New York, New York, USA: ACM Press; 2002; 288–91. 31. Finding and Evaluating Online Resources. National Center for Complimentary and Integrative Health. 2014. 32. Zeng X, Parmanto B. Evaluation of web accessibility of consumer health information websites. AMIA . Annu Symp proceedings AMIA Symp. 2003;743–7. 33. Lazar J, Dudley-Sponaugle A, Greenidge K-D. Improving web accessibility: a study of webmaster perceptions. Comput Human Behav. 2004;20:269–88. 34. Barricelli BR, Sciarelli P, Valtolina S, Rizzi A. Web accessibility legislation in Italy: a survey 10 years after the Stanca Act. Univers Access Inf Soc. 2017;1–12. Evaluation of Kidney Transplant Websites- Valizadeh Haghi et al.