Stesura Seveso Archivio Italiano di Urologia e Andrologia 2024; 96(1):12179 1 ORIGINAL PAPER information on YouTube about some urological pathologies. In particular, the quality of information about neoplastic bladder disease has been evaluated (2, 3). Bladder cancer is the 10th most frequent cancer in the world (4). A high number of patients with this disease use social media for information and support (5). The quality of information available on YouTube about this disease appears mostly moderate or poor (2) and this leads to a high risk of disinformation for users. Other social media, such as Twitter, also collect information about this disease (6). The therapeutic opportunities and the possible surgical indication determine a strong emotional impact for these patients, for instance, the possible urinary diversion after radical cystectomy. The possibility of obtaining reliable information about these aspects, using social tools such as YouTube, therefore, appears an important factor in the path of understanding and acceptance of the pathology by these patients. YouTube platform is very dynamic with thousands of hours of contents up-loaded per second. Deep learning has recently had a huge impact on the YouTube video recommendations system, therefore, YouTube’s algorithm selects output based on different fac- tors like user’s history and context (7, 8). Nowadays, the absence of information about the charac- teristics of YouTube users has not allowed authors who investigated this field to evaluate the impact of these videos on patient’s decision making (2), furthermore we have no data about the quality of information related to YouTube user’s profile. The primary end point of this paper is to evaluate quality of informations available on the YouTube platform in 2023, with focus on informations about the bladder can- cer treatment, evaluating the relation between numbers of views and other point of interest, such as age of the videos, level of misinformation, DISCERN score and PEMAT-AV score. Authors also wanted to explore a secondary endpoint in order to evaluate the differences in term of quality of information and level of disinformation of contents Background: Social media are widely used information tools, including the medical/health field. Unfortunately, the levels of misinformation on these platforms seem to be high, with a medium-low quality of the proposed content, as evidenced by previous studies. You Tube is one of the most important platforms for audio/video con- tent. It shows content to users through a recommendation algo- rithm system. Materials and methods: We have classified in two cohorts the first results obtained by researching "bladder tumor treatment" on You Tube through two different user profiles: "Cohort A" with a not logged-in session in incognito mode (46 videos enrolled) and "Cohort B" with a logged-in session with a physi- cian profile (50 videos enrolled). The videos were evaluated using validated instruments such as DISCERN and PEMAT-AV Furthermore, we used a Likert’s scale for the evaluation of levels of misinformation. Results: Overall quality of information was moderate to poor (DISCERN 3) in 54% of cohort A and 24% of cohort B. Moreover, a high degree of misinformation (Likert score 3) was found in 52% of cohort A cases and 32% of cohort B. Conclusions: Levels of misinformation in both cohorts are positively correlated to the number of views per month. Globally, the levels of information quality, understandability and actionability are lower for the results obtained from searches performed with anonymous user profile (Cohort A). KEY WORDS: Bladder cancer; YouTube; Misinformation; Social media; Bladder. Submitted 11 December 2023; Accepted 24 December 2023 INTRODUCTION Since YouTube platform was established, on February 14th 2005, the world of information and communication has been revolutionized. This social media is a widely used tool, with more than 1.5 billion users (1), allowing to find information on many areas of interest. The quality of information available on YouTube is evoking considerable interest in scientific literature in recent years. Several authors have already assessed the reliability of the Quality of bladder cancer treatment information on YouTube: May the user’s profile affect the quality of results? Pier Paolo Prontera 1, Francesca Romana Prusciano 2, Marco Lattarulo 1, Emanuele Utano 1, Francesco Schiralli 1, Carmine Sciorio 3, Lorenzo Romano 4, Francesco Saverio Grossi 1 1 Department of Urology, “S.S. Annunziata” Hospital, Taranto, Italy; 2 Department of Emergency and Organ Transplantation-Urology, Andrology and Kidney Transplatation Unit, University of Bari, Bari, Italy; 3 Department of Urology, “Alessandro Manzoni” Hospital of Lecco, Lecco, Italy; 4 Department of Neurosciences, Reproductive Sciences and Odontostomatology, University of Naples “Federico II”, Naples, Italy. DOI: 10.4081/aiua.2024.12179 Summary Archivio Italiano di Urologia e Andrologia 2024; 96(1):12179 P. Paolo Prontera, F. Romana Prusciano, M. Lattarulo, et al. 2 emerged from researches made by different user’s profile (in particular comparing the results obtained using a logged-in session with a physician profile (Urologist) with those obtained using a not logged-in session (in incogni- to mode). MATERIALS AND METHODS YouTube algorithm was queried independently of the two different authors, on November 19th, 2023, at 10 o'clock. The keywords used were: "bladder tumor treatment". Authors used two different user setting: a logged-in ses- sion with a physician profile (Urologist) and a not logged- in session in incognito mode. We decided to exclude from the analysis advertising video and ten "very short videos", which by definition can be a maximum of 60 seconds long and should be a minimum of 15 seconds long, without any information about num- ber of views and comments. We received first 60 videos for both cohorts so we enrolled 50 videos in the first cohort (logged-in session with physician profile) and 46 videos in the second cohort (not logged-in session in incognito mode). Were excluded ten and fourteen videos respectively in the two groups, given the exclusion criteria. All contents enrolled were analysed using validate instru- ments: the Patient Education Materials Assessment Tool for Audio-Video (PEMAT-AV) and DISCERN quality criteria for consumer health information (9, 10). The Patient Education Materials Assessment Tool (PEMAT) is a system- atic method to evaluate and compare the understandabil- ity and actionability of patient education materials. It is designed as a guide to help determine whether patients will be able to understand and act on information. Separate tools are available for use with print and audio- visual materials (11). DISCERN is a brief questionnaire which provides users with a valid and reliable way of assessing the quality of written information on treatment choices for a health problem. DISCERN can also be used by authors and pub- lishers of information on treatment choices as a guide to the standard which users are entitled to expect (12). Moreover, we rated the level of misinformation using a Likert scale (13). We also evaluated other parameters according to other authors (2): Length of video (less than 4 minutes, 4 to 20 minutes, or more than 20 minutes), presence of advertising during viewing, age of video, num- ber of views, type of publisher (academic journal/compa- ny, commercial/industrial, consumer/patient, physician, foundation, health/wellness channel, hospital/clinic, med- ical education, news, source/media outlet, professional society, university, other), number of thumbs-up and thumbs-down, audience (anyone/public, medical educa- tion), audience (any/general public, patients, healthcare providers, caregivers), characters in the video (anima- tion/drawing, celebrity/public figure, doctor/healthcare professional, patient, other). We also assessed the relation between numbers of views (globally and per months) and other variables using Pearson correlation coefficients. We also evaluated the total number of comments per video. Analysis for both cohorts lasted twelve days. RESULTS From now on the group of videos searched using the user profile not logged-in and with session in incognito mode will be called "Cohort A", and the group of videos obtained using logged-in physician user profile will called "Cohort B" (Table 1). The total length of videos was 271.31 minutes (median length of 5.51 minutes) for the cohort A and 268.45 min- utes (with a median length of 5.35 minutes) for cohort B. The majority of the videos was less than 4 minutes in both cohorts (43.5% and 56% respectively). The median numbers of views for both groups were respectively 19.6k and 57.5k for Cohort A and Cohort B, with a median age of each videos calculated of 5.21 and 3.66 years respectively. A median of 111.65k thumbs up were found in the Cohort A and 545.24k in Cohort B. While no thumbs down were funded in both Cohorts. Most of the contents in Cohort A was aimed at anyone (83%) and to Healthcare providers (15%). Only 1% of contents of the Cohort A was specifically for patients. Table 1. Property of the analysed YouTube® videos. Value Cohort A Cohort B Lenght of video Total lenght 271.31 268.45 Median lenght (h:min:sec) (range) 5.51 (0.49-51.04) 5.36 (48.3-0.52) Less than 4 min. (n°) 20 (43.5%) 28 (56%) 4-20 min. (n°) 19 18 More than 20 min., n° (%) 6 4 Views, median (range) 19.6k (26-247k) 57.5K (0-362k) Thumbs up, median (range) (tot) 111.65 (0-2.39k) (4801k) 545,24 (0-3.58k) (27.26k) Thumbs down, median (range) 0 (0) 0 (0) Comment, median (range) (tot) 35.3 (0-196) (1658) 43.4 (0-242) (2040) Intended audience, n° (%) Anyone 38 (83%) 37 (74%) Specifically for patients 1 (2%) 12 (24%) Healtcare providers 7 (15%) 1 (2%) Caregivers 0 0 (0) Others 0 0 (0) Publisher type, n° (%) Academic journal/company 6 (13%) 0 (0) Commercial/industry 1 (2%) 4 (8%) Consumer/patient 0 (0) 0 (0) Doctor 11 (24%) 4 (8%) Foundation 2 (4%) 9 (18%) Healt/wellness channel 15 (33%) 11 (22%) Hospital/clinic 5 (11%) 6 (12%) Medical/education 0 (0) 8 (16%) News source/media 1 (2%) 2 (4%) Professional society 4 (9%) 3 (6%) University 0 (0) 2 (4%) Others 1 (2%) 1 (2%) Who is in?, n° (%) Doctor/healthcare professional 40 (87%) 34 (68%) Others 6 (13%) 16 (32%) Presence of advertising, n° (%) 18 (39%) 13 (26%) Presence of “very short videos” 4 (9%) 3 (6%) Median age of the videos (years) 5.21 3.66 Archivio Italiano di Urologia e Andrologia 2024; 96(1):12179 3 Bladder cancer treatment information on YouTube Similar results emerged from the analysis of Cohort B with the 74% of contents directed to anyone, but with an inversion of proportion regarding Healthcare providers and patients (2% and 24% respectively). Doctors or health care professional were the most fre- quent characters in the contents in both Cohorts (87% for Cohort A and 68% for Cohort B). The presence of advertisements that interrupted the view- ing of the video was found in 39% of cases in cohort A and in 26% of cases in Cohort B. According to the DISCERN score, the quality of informa- tion was moderate to poor for the Cohort A (median range 42.4) and moderate to high for the cohort B (medi- an range 59.56), and the score 1 appears in 13% of the cohort A while it was never present in the Cohort B. Moreover, in both groups the DISCERN most representa- tive score was 4 with a prevalence of 64% for the cohort B instead of the 29% for the Cohort A. In the cohort A the understandability score and the actionability score, calculated using PEMAT-AV tool, were median to poor (respectively 49% and 44%), whereas the same parameters were evaluated moderate to high for the cohort B (respectively 76% and 77.7%). The application of a Likert scale revealed a low level of misinformation in cohort B if compared with cohort A. In particular, we found 28% vs 6% of score 5 and 0% vs 27% of score 1, in cohort B and A respectively (Table 2). Application of Pearson correlation coefficients highlight- ed a moderate positive correlation between overall views and the level of misinformation, considering cohort B, and a weak negative correlation considering cohort A for the same factors. The relation between overall views and PEMAT-AV score (understandability and actionability both) was weakly positive for both groups, while we discovered a weak neg- ative correlation between overall views and DISCERN score for the cohort B versus a weak positive correlation found for the cohort A (Table 3). Investigating relationships of DISCERN score, PEMAT-AV score and misinformation levels to the views per months, we found a strong negative correlation between DISCERN score and views per months in Cohort B and a weak pos- itive correlation in cohort A (Table 4). These findings are in line with the results of other authors (2, 3). DISCUSSION Social media today represent a mass information tool also regarding the demand for information in the healthcare sector. The same health professionals (healthcare profes- sionals) use social media for scientific dissemination. Unfortunately, the massive presence of disinformation on the Web is a serious social problem (2, 14), which, in our opinion, can have an impact both on the psychological sphere of the patient undergoing medical treatment, espe- cially if it is invasive, and on the ability of the profession- al to convey the appropriate information in the most cor- rect way. In recent years several authors have investigated the quality and levels of misinformation present in con- tent published on various social media, through the use of validated tools (9, 10, 13], showing a high percentage of erroneous content in the publications (2, 3, 14). YouTube is one of the leading platforms for visual audio content and uses a search algorithm that, thanks to deep learning, offers results selected according to the charac- teristics of the user and his activity on the site (8). Therefore, we decided to evaluate the results obtained from the research of "bladder tumor treatment" performed through two different user accounts: Cohort A and Cohort B. Overall, the quality of the contents obtained in cohort B was higher (96% of medium-high quality compared to 69% obtained in cohort A). On average the contents of cohort B have higher levels of understandability and Table 2. Results of analysis with validated tools (DISCERN, PEMAT-AV). Value Cohort A Cohort B DISCERN overall rating, n° (%) 1 6 (13%) 0 (0) 2 8 (17%) 2 (4%) 3 11 (24%) 10 (20%) 4 13 (29%) 32 (64%) 5 8 (17%) 6 (12%) DISCERN sum (median range) 42.4 (19-78) 59.56 (22-76) PEMAT-AV, median (range) Understandability 49% (0-100) 76% (0-100%) Actionability 44% (0-100) 77.7% (0-100%) MISINFORMATION score (Likert Scale) no. (%) 1 3 (6%) 14 (28%) 2 15 (32%) 20 (40%) 3 10 (22%) 10 (20%) 4 6 (13%) 6 (12%) 5 12 (27%) 0 (0%) Table 3. Correlation between the number of total views and the variables examined. Tot view Variables Pearson Index (r) (n°) Cohort A Cohort B * Age of the video 0.144 (weak) 0.25 (weak) * Misinformation (Likert scale) -0.028 (weak) 0.400 (moderate) * DISCERN score 0.166 (weak) -0.252 (weak) * PEMAT-AV understandability 0.009 (weak) 0.150 (weak) * PEMAT-AV actionability 0.031 (weak) 0.170 (weak) * Rounded to the three decimal places. Table 4. Correlation between the number of views/month and the variables examined. View per Variables Pearson Index (r) month (n°) Cohort A Cohort B * Age of the video -0.26 (weak) 0.25 (weak) * Misinformation (Likert scale) 0.012 (weak) 0.007 (weak) * DISCERN score 0.153 (weak) -0.75 (strong) * PEMAT-AV understandability -0.004 (very weak) -0.550 (moderate) * PEMAT-AV actionability 0.073 (weak) 0.097 (weak) * Rounded to the three decimal places. Archivio Italiano di Urologia e Andrologia 2024; 96(1):12179 P. Paolo Prontera, F. Romana Prusciano, M. Lattarulo, et al. 4 actionability (76% and 77.7% respectively compared to 49% and 44% observed in cohort A) (Table 5). This data seems to confirm that the same search per- formed by a user profile closely related to the field of interest of the search, returns results of higher quality and with higher level of understandability and actionability. However, this could be a contradiction because if on the one hand it is logical for the algorithm to propose higher quality videos based on the level of affinity of the user profile to the search, on the other hand it seems contro- versial to show videos with a lower level of understand- ability and actionability to an average user not associated with a health profile and therefore potentially with a lower ability to understand the content. As for the level of misinformation, obtained through the use of a Likert scale, our data are almost in line with pre- vious published studies. In fact, the calculation of the linear correlation coefficient shows a positive correlation between the number of views of the videos and their degree of misinformation; this is particularly evident and relevant if we look at the results obtained by relating the number of total views and the degree of misinformation in cohort B, which is associated with the average health profile. This correlation becomes weakly negative if we consider the results obtained with the same variables in cohort A. This might seem like a paradox since from this data we could deduce that the videos with more views proposed by the YouTube algo- rithm to a health profile are at the same time those with a higher degree of misinformation. If instead we consider as variable the visualizations for month this correlation becomes weakly positive in both cohorts. It is also inter- esting to note that there is a strong negative correlation between the number of views per month and the DIS- CERN score in cohort B, unlike a weak positive correla- tion present in cohort A. From this, we could then deduce that the videos proposed to the user with health profile with a higher number of views per month are actually those with lower quality. On average, enlisted videos have a medium-low quality level of 30% and 4% in cohorts A and B respectively. This figure is apparently not in line with the findings of other authors (2, 3) who report a low average quality level of about 67% (2) and 66,7% (3). This discrepancy, howev- er, could be linked to different research criteria: in the study of García Cano Fernández et al. videos were enlisted exclusively in Spanish and with a duration of less than four minutes, which instead represent 43,5% and 56% of cohorts A and B respectively in our paper. In the paper by Loeb at al. videos with a range of duration from 21 sec- onds to 76 minutes were enlisted. However, in both papers, the type of user profile from which the search was made has not been specified and this, in our opinion, rep- resents an important limitation. The limitations of this study are linked to the exclusive use of YouTube, and not of other social media, although this platform represents the mainly used. Another limit, as probably for all the studies presented on this issue to date, is intrinsically related to the huge number of vari- ables that affect the results offered by YouTube’s algorithm. The evaluation of the results obtained by two profiles with deeply different characteristics (medical profile and anonymous profile) is a first step in the understanding and interpretation of the health information offered to users on social media, and the consequent impact on the clinical practice of these instruments. CONCLUSIONS According to our study, in accordance with the literature data available today, the level of misinformation of the YouTube’s contents about bladder cancer treatment is pos- itively related to the number of views per month. The same search performed by a user profile related to the field of interest of the search itself offers higher quality results. The quality of information provided by YouTube regarding bladder cancer therapy, as well as levels of understand- ability and actionability, is, in general, medium-low if the research is performed with anonymous profile (Cohort A), while such results are reversed if the research is per- formed by a profile with high level of affinity with the field of interest of the research. This could be an important step in better understanding the correlation between the results suggested by YouTube's video recommendation algorithm and the characteristics of the user profile from which the search is performed. We also hope that future studies will investigate the pos- itive correlation that appears to exist between higher lev- els of misinformation and more views. REFERENCES 1. Borgmann H, Cooperberg M, Murphy D, et al. Online profession- alism— 2018 update of european association of urology (@uroweb) recommendations on the appropriate use of social media. Eur Urol 2018; 74: 644-50. 2. Loeb S, Reines K, Abu-Salha Y, et al. Quality of Bladder Cancer Information on YouTube. Eur Urol. 2021; 79:56-59. 3. García-Cano-Fernández AM, Szczesniewski-Dudzik JJ, García- Tello A, et al. Quality of bladder cancer information on YouTube. Cent European J Urol. 2022; 75:248-251. 4. International Agency for Research on Cancer. Bladder cancer GLOBOCAN. In: https://gco.iarc.fr/today/data/factsheets/can- cers/30-Bladder-fact-sheet.pdf2018. 5. Kamat AM, Agarwal P, Bivalacqua T, et al. Collaborating to move research for-ward: proceedings of the 10th annual bladder cancer think tank. Bladder Cancer 2016; 2:203-13. 6. Tariq A, Khan SR, Vela I, Williams ED. Assessment of the use of the internet and social media among people with bladder cancer and their carers, and the quality of available patient-centric online. BJU Int. 2019; 123(Suppl 5):10-18. Table 5. Spread of contents about treatment of bladder tumor on YouTube®. Cohort A Cohort B Moderate to poor quality (DISCERN ≤ 3) 54% 24% High Misinformation (Likert score ≥ 3) 52% 32% High Understandability (> 50%) 49% 76% High Actionability (> 50%) 44% 77.7% Archivio Italiano di Urologia e Andrologia 2024; 96(1):12179 5 Bladder cancer treatment information on YouTube 7. Covington C, Adams J, Sargin E. Deep neural networks for youtube recommen-dations. RecSys '16: Proceedings of the 10th ACM Conference on Recom-mender Systems 2016; pp. 191-198. 8. Davidson J, Liebald B, Liu J, et al. The youtube video recommen- dation system. In Proceedings of the Fourth ACM Conference on Recommender Sys-tems, RecSys’10, pages 293-296, New York, NY, USA, 2010. ACM. 9. AHRQ. The patient education materials assessment tool (pemat) and user’s guide. https://www.ahrq.gov/professionals/prevention- chronic-care/improve/self-mgmt/pemat/pemat-av.html. 10. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health infor- mation on treatment choices. J Epidemiol Commun Health 1999; 53:105-11. 11. Available from: http://www.ahrg.gov/healt-literacy/patient-edu- cation/pemat.html 12. Available from: http://www.discern.org.uk 13. Herbert AS, Nemirovsky A, Hess DS, et al. Pelvic organ prolapse on YouTube: evaluation of consumer information. BJU Int. 2020; 125:759-760. 14. Fode M, Jensen CFS, Østergren PB. How Should the Medical Community Respond to the Low Quality of Medical Information on Social Media? Eur Urol. 2021; 79:60-61. Correspondence Pier Paolo Prontera, MD (Corresponding Author) pierpaolo.prontera@asl.taranto.it Marco Lattarulo, MD marco.lattarulo@asl.taranto.it Emanuele Utano, MD emanuele.utano@asl.taranto.it Francesco Schiralli, MD Francesco Saverio Grossi, MD, PhD grossifs@libero.it Department of Urology, “S.S. Annunziata” Hospital, Taranto (Italy) Via Bruno Francesco 1 - 74010 Taranto (Italy) Francesca Romana Prusciano, MD Department of Emergency and Organ Transplantation, Urology, Andrology and Kidney Transplantation Unit, University of Bari, 70124 Bari (Italy) Carmine Sciorio, MD carminesciorio@gmail.com Department of Urology, “Alessandro Manzoni” Hospital of Lecco, Lecco (Italy) Lorenzo Romano, MD loryromano@hotmail.it Department of Neurosciences, Reproductive Sciences and Odontostomatology, University of Naples “Federico II”, 80131 - Naples (Italy) Conflict of interest: The authors declare no potential conflict of interest.