Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 1 OJPHI Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking Eden Shaveet1, Marrissa Gallegos1, Jonathan Castle1, Lisa Gualtieri1 1. Tufts University School of Medicine, Department of Public Health and Community Medicine Abstract The pervasiveness of online mis/disinformation escalated during the COVID-19 pandemic. To address the proliferation of online mis/disinformation, it is critical to build reliability into the tools older adults use to seek health information. On average, older adult populations demonstrate disproportionate susceptibility to false messages spread under the guise of accuracy and were the most engaged with false information about COVID-19 across online platforms when compared to other age-groups. In a design-thinking challenge posed by AARP to graduate students in a Digital Health course at Tufts University School of Medicine, students leveraged existing solutions to design a web browser extension that is responsive to both passive and active health information-seeking methods utilized by older adults in the United States. This paper details the design-thinking process employed, insights gained from primary research, an overview of the prototyped solution, and insights relating to the design of effective health information-seeking platforms for older adults. Keywords: Internet, Older Adults, Inventions, Curriculum Abbreviations: AHT2: AARP Health Tools 2.0, TUSM: Tufts University School of Medicine DOI: 10.5210/ojphi.v14i1.12593 Copyright ©2022 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. Background Adults 50 years and older are the largest consumer population of healthcare in the United States [1]. Consequently, the methods older adults use to seek health information that may impact their healthcare decisions is noteworthy for healthcare providers, older adult communities, and public health stakeholders. The number of older adults seeking health information online has increased in recent years, which is consistent with this population’s growing adoption of smartphone, tablet, and social media use [2], [3]. However, increased reliance on online health information seeking has heightened concern around rapid dispersion of health misinformation and disinformation [4]. Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 2 OJPHI Online Health Information Seeking During COVID-19 Disruption of routine healthcare service delivery in the U.S. during the COVID-19 pandemic prompted increased patient reliance on internet-enabled technologies for virtual visits with providers, consumer-grade digital health application use, and vaccination appointment scheduling [5], [6]. This abrupt dependence on web-based resources for healthcare purposes coincided with increased adoption of online health information-seeking behaviors [7], [8]. In 2020, over 80% of U.S. adults aged 50 years or older claimed that the internet had been an essential resource to them throughout the COVID-19 pandemic; however, older adults were also found to be the most engaged with false information about COVID-19 across online platforms when compared to other age-groups [9], [10]. Misinformation and Disinformation Misinformation and disinformation refer to false messaging spread under a guise of accuracy [11]. While misinformation is defined as false information spread regardless of intent, disinformation is the distinct sect of misinformation that is deliberately propagated [11]. In recognition of their mutual relevance and harm, we use the terms collectively throughout this paper. The proliferation of online mis/disinformation in older adult communities has garnered considerable public attention in recent years, as older adult populations who experience digital exclusion are, on average, disproportionately susceptible to mis/disinformation when encountered [4]. While its presence has drastically increased since the early 2010’s, pervasiveness of online health mis/disinformation escalated during the COVID-19 pandemic [12]. In response, efforts have been dedicated to reducing online health mis/disinformation on the part of information-sharing platforms like Google and Facebook, asserting that optimal solutions to mitigating proliferation include both reducing the amount of false content in online circulation and promoting better health and digital literacy skills [13], [14]. These are vital endeavors that require considerable time and resources. Proposed in this paper is a concurrent strategy conceptualized using design thinking with few barriers to implementation. Design Thinking Design thinking is an approach to innovation and validation used to develop effective solutions to complex problems [15]. Distinct from other development methods, design thinking relies on human-centered design principles to observe how people interact with their environments and iteratively design solutions to a population’s expressed needs. Interdisciplinary student teams at Tufts University School of Medicine (TUSM) engaged in a design-thinking process adapted to a 14-week Digital Health course in collaboration with AARP (a United States member-based interest and advocacy group focusing on issues affecting adults over the age of 50) to address a set of obstacles facing older adults in the U.S. Teams engaged in a design-thinking process encompassing review of the literature and existing solutions, primary research, problem explication, ideation, prototyping, and solution validation. Additional details on the use of design thinking as a teaching medium at TUSM are reported in ref [16]. Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 3 OJPHI Academic-Industry Partnership with AARP An academic-industry partnership between TUSM and AARP to teach graduate students about design thinking for healthcare innovation has spanned over eight semesters. While student teams are not required to collaborate closely with AARP to design their solutions, our team determined that building upon AARP’s existing resources would be advantageous and lead to a more feasible and trusted solution prototype. Trust is foundational to solution adoption and behavior change in older adult communities [17]. AARP’s tenure in the older adult interest and advocacy space has earned the organization recognition as one of the most notable and respected membership-based organizations in the U.S. At a membership count of over 38 million people, AARP is a household name frequently referenced in pop-culture, academic, and industry spaces [18]. In their efforts to mitigate online mis/disinformation about COVID-19, Google’s early decision to promote COVID-19 information sourced from AARP in 2020 amplified the organization’s credibility as a trusted source for health information [19]. For this reason, along with high utilization of their website by older adults, AARP was an ideal entity with which to prototype a high-impact design-thinking solution to the problem of unreliable health information retrieval among older adults online. Intended User Population Older adults who seek health information online are often prompted by receipt of new diagnoses, progression of symptoms, or prescription of new medications [20], [21]. Recognizing that 85% of adults 50 years and older live with at least one chronic condition, are likely to experience changes in their conditions, or be prescribed new medications to manage these conditions [22], [23], we identified this population as the intended user base for a formulated design-thinking solution. Additional considerations were given to the 35-48% of older adults aged 55 years and older who experience digital exclusion as digital mistrust and limited digital literacy skills may result in misuse of or aversion to technology [17], [24]. In recognition of family members, caregivers, and healthcare professionals who support older adults with health information seeking, we identified members of these populations as secondary users. Under the precept of universal design, when digital solutions are designed to be accessible to members of specific communities, like older adults, they are also likely to be accessible to the population at large [25]. Existing Solutions The internet is a primary source of health information gaining traction among older adults [1]; however, health and digital literacy skills are often poor among older adults who seek health information online [2]. Trusted searches that yield valid health information may be achieved by employing methods that in some way limit or verify the trustworthiness of search results [26]. Past work in this area includes the development of specialized search engines and content verification indicators. Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 4 OJPHI Specialized Search Engines Search engines are software programs that carry out web search queries [27]. Specialized search engines are search engines that specialize in retrieval of web-based information relating to a particular topic or category [28]. While several specialized health search engines have emerged in the last two decades, these search engines have been unsuccessful in reaching wide-audiences when compared to Google Search or in mitigating the proliferation of health-related mis/disinformation online [29], [30]. Additionally, advertisement-based monetization models or use of relevance algorithms which leverage data scraping mechanisms and access user cache and cookies jeopardize user privacy and may disincentivize use among older adults who express digital mistrust and privacy concerns [31]. Content Certification Programs Online health content certification programs, like HONcode, are programs which use visual indicators to inform users of when they are accessing websites which house valid health information [32]. While widely recognized in academic circles whose interests are in health information reliability [33], it's unknown whether certification indicators like HONcode are recognized across the general population. Content certification indicators which rely on widespread recognition are effective only if enough online health information seekers recognize and find value in them. AARP Health Tools In 2021, AARP hosted a suite of 12 web-based health tools on their website including a pill identifier, symptom checker, and health encyclopedia populated by content licensed from Healthline’s Health Reference Library [34]. Some of these tools have since been removed or modified; however, we will refer to them throughout this paper as they existed between May - August 2021. AARP Perks™ Browser Extension The AARP Perks™ browser extension notifies members of AARP benefits while browsing online [35]. Though not designed for the purpose of health information-seeking, the extension’s development was informed by the online information-seeking habits of older adults, including passive information acquisition while browsing for other information. Seeking User Perspectives Design thinking emphasizes understanding the needs and constraints of an intended user population. In pursuit of designing a solution that fit the needs of our intended user population, we conducted informal key informant interviews. We convened a non-probability sample of key informants (n=4) consisting of three intended user population members and one secondary population member (Table 1). Informants were asked about their health information seeking habits and experience with AARP. Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 5 OJPHI Informants largely preferred seeking health information offline for queries about their own health or the health of others (50-75%). They also expressed low trust in commercial search engines and that privacy while browsing for health information online is of high importance to them (75%). However, despite mistrust in commercial search engines, most informants expressed interest in using online health-information solutions, such as health search engines (75%). Half of the informants were AARP members, but none had ever visited the AARP website and had thus never used AARP Health Tools. Additionally, no respondents were familiar with the Perks Browser Extension. These insights suggested a need for health information solutions that preserve privacy and are accessible in both online and offline formats. Table 1: Key Informant Interview Insights Health Information Seeking Behavior (N=4) Primary health information source (own health) % Primary care provider 75 Commercial search engine 25 Primary health information source (others’ health) Primary care provider 25 Library/books 25 Commercial search engine 25 Health information websites 25 Prompts to search for health information New medication 50 Change in symptoms 50 New diagnosis 50 Last searched for health information < 2 weeks 25 < 6 months 50 >12 months 25 Search Engines and Privacy Trust in commercial search engines Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 6 OJPHI Low trust 75 Medium trust 25 Importance of privacy online for general queries Highly important 75 Fairly important 25 Importance of privacy online for health queries Highly important 75 Fairly important 25 Health Search Engines Familiar with health search engines Yes 25 No 75 Interest in using health search engines Yes 75 Unsure 25 AARP Experience AARP member Yes 50 No 50 Visited AARP website No 100 Familiar with AARP Perks Browser Extension No 100 Table 1: proportion of key informants who indicated agreement with each corresponding item Platform Research The design thinking process emphasizes “learning by doing” [36]. We embodied this principle by pursuing platform research from a user’s perspective. Our interest in the Perks browser extension Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 7 OJPHI and AARP’s existing partnerships for hosting valid health information online informed the focus of our platform research. Following review of the literature and assessment of present solutions, we evaluated these AARP platform features to observe their underlying mechanisms, identify barriers and facilitators of health information access, and identify opportunities to leverage these features in a feasible design thinking solution. We tested how AARP’s main search bar recalled content in response to health queries and how the AARP Perks browser extension behaved when accessing credible and non-credible online health information resources. Additionally, we assessed the readability of AARP’s existing health content, as well as the click path and scroll depth necessary to access them. Search Bar Evaluation We evaluated AARP’s main website search bar to observe content recall and page rank. Over the course of seven days, we utilized three cookie and cache cleared web browsers to submit 126 queries of six distinct health-related terms in the following categories: medical conditions, medications, and health products (Table 2). Forty-two queries on AARP.org were conducted on a location-tracking disabled web browser (DuckDuckGo) and 84 queries were conducted on two location-tracking enabled web browsers (Microsoft Edge, Google Chrome). Different browsers with different tracking permissions were utilized to observe whether health content recall and page rank would be affected by the presence or lack of permissions. Other differences in browser operation and performance were not accounted for. Page rank remained identical for all queries across testers and browsers. Notably, information sourced from AARP Health Tools (Health Encyclopedia, Symptom Checker, etc.) was not included in this content recall. These insights led us to suspect use of a fixed index that excludes validated health content from AARP Health Tools, but no use of location-based relevance algorithms to modify page rank. We did, however, suspect use of a data-scraping mechanism to utilize user cookies and cache for location-specific advertising and product placement which may deter use in older adults who experience digital mistrust. Table 2: Search Bar Evaluation Search Terms Location-Tracking Enabled Web Browser Location-Tracking Disabled Web Browser Tester Locations: MA, WA (USA) Tester Location: OR (USA) Medical Conditions Medications Health Products Medical Conditions Medications Health Products Diab etes CO PD Gerit ol® Eliqu is® Pulse oxim eter MetLi fe® Diab etes CO PD Gerit ol® Eliqu is® Pulse oxim eter MetLi fe® Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 8 OJPHI # Que ries 14 14 14 14 14 14 7 7 7 7 7 7 Table 2: number of queries submitted per search term by tester location and browser location permissions Browser Extension Evaluation We analyzed the behavior of the AARP Perks™ web browser extension to infer the mechanics of the extension’s search function and search-activated notification tab. Though the notification tab was intended to notify AARP members of member benefits while browsing online, we were interested in whether health content was ever supplied through the extension and, if so, how to leverage this in a solution focused on health information access. We tested responsiveness to health and non-health related search terms and quantified instances of tab presence across search contexts to surmise an algorithm flow that could be used in a design thinking solution. Six health and six non-health related terms were submitted to the Perks web browser extension search bar. Health-related search terms returned zero results, while non-health-related terms returned 15-30 results. Multi-word non-health related search terms returned 0-2 results. These observations led us to suspect that the parsing mechanism favored single-word search terms and used a fixed index that excluded validated health content from AARP Health Tools. Table 3: Browser Evaluation Search Terms Health Search Terms Non-Health Search Terms (Single word) Non-Health Search Terms (Multi-word) “Diabetes” “COPD” “Geritol” ® “Eliquis” ® “Pulse oximeter” “Metlife” ® “Vacation” “Money” “Shopping” “Going shopping” “Saving money” “Going on vacation” Table 3: search terms submitted by category to the AARP Perks™ web browser extension When assessing valid sources for health information about chronic obstructive pulmonary disease (COPD), the Perks browser extension notification tab appeared upon access to some reliable online sources for health information, including the Mayo Clinic and U.S. Centers for Disease Control and Prevention websites (Figure 1). It also appeared upon access to WebMD, a popular online health content publisher who employs and collaborates with health care providers to validate their Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 9 OJPHI content [37]. It did not appear when accessing social media platforms or community-maintained general information sharing platforms (ex. Reddit). The notification tab did not appear upon access to American Lung Association or the federal MedlinePlus websites. The extension notification tab also did not appear upon access to or submission of search terms to web browsers and search engines. The information displayed in the notification tab while accessing health-related websites was not inclusive of validated health information sourced from AARP’s health encyclopedia, however, a link to access the “symptom checker” health tool was sometimes returned, demonstrating a health information-seeking use case for the extension. Behavior of the notification tab led us to suspect use of a whitelisting algorithm to prompt appearance of the tab upon accessing whitelisted universal resource locators (URLs). We were unsure whether the whitelist was manually populated and maintained or if the algorithm relied on other website performance metrics for whitelist classification. Figure 1: AARP Perks Browser Extension Presence Across Accessed COPD Health Websites, August 19, 2021 Figure 1: COPD webpage screenshots and presence/absence of AARP Perks™ web browser extension on August 19, 2021 Click path and scroll depth analysis Click path and scroll depth are metrics which describe a user’s experience on a website or platform [38]. Click path characterizes the number of clicks needed to access information of interest from Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 10 OJPHI a defined start point (often the website homepage), while scroll depth characterizes the amount of scrolling necessary to access content between clicks [39]. To minimize transaction costs which dissuade user retention and limit accessibility, short click path and low scroll depth are desirable [40]. Our user experience analyses assessed the shortest possible click path and scroll depth needed to access clinically validated health information hosted on the AARP website. The shortest path to validated health information on the AARP website through Health Tools contained a total of nine steps: seven clicks and two scroll events comprising 75% total scroll depth (Figure 2). Notably, this path required that users know and submit exact search terms. These observations led us to suspect that website users who do not use exact search terms are unsuccessful in locating validated health content passively or actively. This conclusion was consistent with the lack of key informants who had ever accessed Health Tools content. Figure 2: “High Blood Pressure” Content Click Path, August 2021 Figure 2: click path diagram of the steps needed to access clinically validated health encyclopedia content about high blood pressure on the AARP website in August, 2021 Content readability comparison test Access to valid health information is mediated by its readability [41]. Since we aimed to design a solution that leveraged health information already available through AARP channels, we decided to examine AARP health content readability. To do this, we used open-access readability scoring software, TextCompare [42], to assess online content hosted by AARP pertaining to the COVID- 19 vaccination [43] compared to online anti-vaccination content [44] via six readability indices: Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, Gunning-Fog Score, SMOG Index, Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 11 OJPHI Coleman Liau Index, and Automated Readability Index. Information about each index and its formula appears below. Flesch Kincaid Reading Ease Point Scale: 100-90 89-80 79-70 69-60 59-50 49-30 29-10 10-0 Easiest to read Most difficult to read Formula: 206.835 − 1.015 ( 𝑤𝑜𝑟𝑑𝑠 𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠 ) − 84.6 ( 𝑠𝑦𝑙𝑙𝑎𝑏𝑙𝑒𝑠 𝑤𝑜𝑟𝑑𝑠 ) Flesch Kincaid Grade Level Point Scale: < 5.0-5.9 6.0-6.9 7.0-7.9 8.0-9.9 10.0-12.9 13.0-15.9 16.0- 17.9 > 18.0 Easiest to read Most difficult to read Formula: 0.39 ( 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 𝑡𝑜𝑡𝑎𝑙 𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠 ) + 11.8 ( 𝑡𝑜𝑡𝑎𝑙 𝑠𝑦𝑙𝑙𝑎𝑏𝑙𝑒𝑠 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 ) − 15.59 Gunning Fog Score Point Scale: 0-5 6 7 8 9-12 13-16 17 18-20 Easiest to read Most difficult to read Formula: 0.4 [( 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 𝑡𝑜𝑡𝑎𝑙 𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠 ) + 100 ( 𝑐𝑜𝑚𝑝𝑙𝑒𝑥 𝑤𝑜𝑟𝑑𝑠 ∗ 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 )] Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 12 OJPHI *contains ≥ 3 syllables SMOG Index Point Scale: 0-6 7-20 21-42 43-90 91-132 133-182 183-210 ≥ 211 Easiest to read Most difficult to read Formula: 3 + √𝑝𝑜𝑙𝑦𝑠𝑦𝑙𝑙𝑎𝑏𝑖𝑐 𝑐𝑜𝑢𝑛𝑡 Coleman Liau Index* Point Scale: ≤ 4 5 6 7 7-10 11-12 13-16 ≥ 17 Easiest to read Most difficult to read Formula: 0.0558(𝐿) − 0.296(𝑆) − 15.8 *L=avg. number of letters per 100 words, S=avg. number of sentences per 100 words Automated Readability Index Point Scale: ≤ 1 2 3 4 5 6 7 8 9 10 11 12 13 ≥ 14 Easiest to read Most difficult to read Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 13 OJPHI Formula: 4.71 ( 𝑐ℎ𝑎𝑟𝑎𝑐𝑡𝑒𝑟𝑠 𝑤𝑜𝑟𝑑𝑠 ) + 0.5 ( 𝑤𝑜𝑟𝑑𝑠 𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠 ) − 21.43 When compared to an online anti-vaccination article, an AARP article about COVID-19 vaccination options ranked less readable by all six readability indices: Flesch Kincaid Reading Ease (41.74, 65.11), Flesch Kincaid Grade Level (12.82, 7.45), Gunning Fog Score (15.75, 10.26) SMOG Index (14.33, 10.45), Coleman Liau Index (13.17, 9.06), and Automated Readability Index (13.65, 6.87) (Table 4). Table 4: Readability Index Scores Readability Index Article Source AARP Article Anti-Vaccination Blog Flesch Kincaid Reading Ease 41.74 65.11 Flesch Kincaid Grade Level 12.82 7.45 Gunning Fog Score 15.75 10.26 SMOG Index 14.33 10.45 Coleman Liau Index 13.17 9.06 Automated Readability Index 13.65 6.87 Table 4: Readability scores between AARP COVID-19 vaccination article and anti-vaccination article Solution Design Insights gained from intended user perspectives and platform research informed our iterative ideation process. Knowing that the Perks browser extension was designed around the browsing habits of older adults, we recognized that a feasible solution could leverage this existing AARP platform architecture to 1.) directly provide AARP’s existing validated health content and 2.) validate the reliability of other health content platforms while browsing online. AARP Health Tools 2.0 AARP Health Tools 2.0 (AHT2) is a web browser extension prototype for health information validation and delivery designed to be responsive to both passive and active health information- seeking methods employed by older adults. Leveraging the existing Perks web browser extension Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 14 OJPHI architecture, AHT2 would employ a similar whitelisting algorithm to notify users of trusted health information sources while browsing online. Proposed modifications to the existing architecture include 1.) redirection of the content repository path to a new fixed index that houses AARP’s health encyclopedia entries and 2.) a rewrite of whitelisting conditions to prompt notification tab appearance on non-whitelisted URLs rather than whitelisted URLs. By rewriting the whitelisting conditions, the extension’s function changes from member benefits notifications to content validation and supply. Populating a fixed content index with content from the health encyclopedia benefits the existing parsing mechanism which favors single-word search terms as health condition entries in the encyclopedia were stored with keywords. Prototype Overview Once installed, the AHT2 browser extension is enabled by a user within the partner browser’s extension manager. AHT2 relies on a whitelisting algorithm to validate URLs as whitelisted or not whitelisted. If whitelisted, the URL is deemed a trusted source for health information. If not whitelisted, the URL is deemed an unverified source for health information. The AHT2 notification tab, whose index is populated by health encyclopedia content, is prompted upon access to non-whitelisted URLs. To aid in intuitive use, along with prompting the notification tab on access of non-whitelisted URLs, AHT2 leverages prior work on content certification indicators [32] to communicate the perceived “trustworthiness” of a website to a user. To accomplish this, AHT2 returns a “checkbox” indicator upon accessing whitelisted URLs and returns a “null” indicator upon accessing non- whitelisted URLs coupled with the notification tab. In this design iteration, the whitelist would be manually populated and maintained rather than relying on other website performance metrics to inform the whitelist classifier. Figure 3: AARP Health Tools 2.0 (AHT2) Extension Whitelisting Algorithm Flow Diagram Figure 3: Flow diagram of proposed AHT2 extension whitelisting algorithm Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 15 OJPHI Addressing User Needs To address the need for online and offline health information access, AHT2 may be supplemented with health information-seeking tips available in print media through AARP’s popular print publications. Quick response (QR) codes with step-by-step instructions may also provide an opportunity to support older adults in building digital literacy skills and encouraging reader access to reliable health information online through the AHT2 extension. Additionally, in the interest of privacy-preservation, AHT2 departs from Perks by not requiring member sign-in prior to installation or use and inhibits health information queries to be tied back to an AARP member profile. Finally, to maximize the impact of AHT2, focus on content accessibility is vital. It’s suggested that online health information be written at the sixth-grade reading level to accommodate accessibility needs [41]. Findings of the readability comparison tests revealed that a published COVID-19 vaccination article on the AARP website ranked less readable in six out of six readability indices when compared to online anti-vaccination content. We propose that AHT2 adoption be contingent on review and revision of existing health encyclopedia content to the sixth- grade reading level to broaden access. Limitations This work has several limitations. First, the AHT2 protype described in this paper has not yet been developed, only designed. Therefore, we are unable to evaluate hypotheses related to the use of the extension as an intervention. Second, our key informants comprised a small convenience sample. Thus, the insights gleaned from these interviews are likely not representative of all members of our intended user population. Third, while we describe that the extension relies on manual intervention for whitelist population and maintenance, as well as content revision to an appropriate reading level, we did not consider the maintenance workflow required by AARP personnel to undertake this work, nor associated operational costs. Finally, we did not consider any manual or automated feedback mechanisms to inform extension performance improvement, which is vital for the continual refinement of any design thinking intervention. Conclusion Web-based health information seeking is on the rise across older adult populations in the US and beyond [1]. In the absence of health information seeking solutions that accommodate the health information-seeking habits of older adults, the potential for exposure to health mis/disinformation escalates [45], [46]. The COVID-19 pandemic provides a case study of an infodemic in which older adults, a population disproportionately susceptible to COVID-19, were also the most engaged with mis/disinformation that encouraged actions which enabled its spread [10], [47]. In commercial settings, refinement of existing solutions to meet evolving consumer needs has led to feasible product and service implementation [48]. Our work acknowledges this idea by presenting a case in which existing solutions may be used for alternate purposes when modified through a lens of design thinking. Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 16 OJPHI To promote adoption in older adult communities, it is beneficial for solutions to be operated or sponsored by entities that older adults already trust [17]. AARP is a trusted entity offering valid health information solutions and products which appeal to the online browsing habits of older adults; however, these products are disparate and may not be broadly accessible or delivered at the appropriate reading level. By consolidating these tools and making slight modifications to their function, a new solution emerges with a proposed ability to mitigate the spread of mis/disinformation online. Acknowledgements The authors express gratitude to Ted Moffatt, AARP Digital Analytics Architect, and Raymond Deschenes, AARP Senior Advisor of Enterprise Search, for their guidance and consultation following our research and preliminary ideation phase. The authors also thank Alison Bryant, former Senior Vice President of Research & Enterprise Lead for Technology & Digital Equity at AARP for her guidance and feedback during the project upon which this paper was based. Financial Disclosure No funding was received for this work. The authors report no competing financial interests. Competing Interests The authors declare no conflicts of interest. References 1. Turner AM, Osterhage KP, Taylor JO, Hartzler AL, Demiris G. 2018. A Closer Look at Health Information Seeking by Older Adults and Involved Family and Friends: Design Considerations for Health Information Technologies. AMIA Annu Symp Proc. 2018, 1036-45. PubMed 2. Berkowsky RW, Czaja SJ. “Challenges associated with online health information seeking among older adults,” in Aging, Technology and Health, Elsevier, 2018, pp. 31–48. doi: https://doi.org/10.1016/B978-0-12-811272-4.00002-6. 3. Vogels E. “Millennials stand out for their technology use, but older generations also embrace digital life,” Pew Research Center, Washington, D.C., Sep. 2019. [Online]. Available: https://www.pewresearch.org/fact-tank/2019/09/09/us-generations-technology-use/ 4. Swire-Thompson B, Lazer D. 2020. Public Health and Online Misinformation: Challenges and Recommendations. Annu Rev Public Health. 41(1), 433-51. doi:https://doi.org/10.1146/annurev- publhealth-040119-094127. PubMed https://pubmed.ncbi.nlm.nih.gov/30815147 https://pubmed.ncbi.nlm.nih.gov/30815147 https://doi.org/10.1016/B978-0-12-811272-4.00002-6 https://doi.org/10.1146/annurev-publhealth-040119-094127 https://doi.org/10.1146/annurev-publhealth-040119-094127 https://pubmed.ncbi.nlm.nih.gov/31874069 Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 17 OJPHI 5. De’ R, Pandey N, Pal A. 2020. Impact of digital surge during Covid-19 pandemic: A viewpoint on research and practice. Int J Inf Manage. 55, 102171. doi:https://doi.org/10.1016/j.ijinfomgt.2020.102171. PubMed 6. Miyawaki A, Tabuchi T, Ong MK, Tsugawa Y. 2021. Age and Social Disparities in the Use of Telemedicine During the COVID-19 Pandemic in Japan: Cross-sectional Study. J Med Internet Res. 23(7), e27982. doi:https://doi.org/10.2196/27982. PubMed 7. B. H. R. 2022. C. S. J, R. Pradeep, and D. M. Philip, “Online Health Information Seeking Behaviour Due to COVID-19 Pandemic-Induced Health Related Anxiety among the General Population in India. J Assoc Physicians India. 70(1), 11-12. 8. Neely S, Eldredge C, Sanders R. 2021. Health Information Seeking Behaviors on Social Media During the COVID-19 Pandemic Among American Social Networking Site Users: Survey Study. J Med Internet Res. 23(6), e29802. doi:https://doi.org/10.2196/29802. PubMed 9. Vogels E, Perrin A, Rainie L, Anderson M. “53% of Americans Say the Internet Has Been Essential During the COVID-19 Outbreak,” Pew Research Center, Apr. 2020. [Online]. Available: https://www.pewresearch.org/internet/2020/04/30/53-of-americans-say-the-internet-has-been-essential-during- the-covid-19-outbreak/ 10. Ontario Medical Association. “Report on COVID Research in Ontario for the OMA,” Ontario Medical Association, Toronto, Canada, Mar. 2021. [Online]. Available: https://www.oma.org/uploadedfiles/oma/media/public/covid-myths-asi-research-summary-for-oma.pdf 11. “Misinformation, Disinformation, and Online Propaganda,” in Social media and democracy: the state of the field and prospects for reform, Cambridge New York Port Melbourne New Delhi Singapore: Cambridge University Press, 2020, pp. 10–27. 12. Huijstee D, Vermeulen I, Kerkhof P, Droog E. 2022. Continued influence of misinformation in times of COVID-19. Int J Psychol. 57(1), 136-45. doi:https://doi.org/10.1002/ijop.12805. PubMed 13. “Addressing Health Misinformation with Health Literacy Strategies.” National Academies of Sciences, Engineering, and Medicine, Dec. 17, 2020. [Online]. Available: https://www.ncbi.nlm.nih.gov/books/NBK565935/ 14. Purtill J. “Facebook, Google, Twitter release industry code to fight spread of disinformation,” ABC Science News, Australia, Feb. 22, 2021. [Online]. Available: https://www.abc.net.au/news/science/2021-02-22/facebook-google-release-voluntary-industry-code- disinformation/13178488 15. Razzouk R, Shute V. 2012. What Is Design Thinking and Why Is It Important? Rev Educ Res. 82(3), 330-48. doi:https://doi.org/10.3102/0034654312457429. 16. Gualtieri L, Shaveet E, Kim D. 2022. Digital Health Innovation for Older Adults through Design Thinking (in press). J Commun Healthc. •••. doi:https://doi.org/10.1080/17538068.2022.2060665. https://doi.org/10.1016/j.ijinfomgt.2020.102171 https://pubmed.ncbi.nlm.nih.gov/32836633 https://doi.org/10.2196/27982 https://pubmed.ncbi.nlm.nih.gov/34259641 https://doi.org/10.2196/29802 https://pubmed.ncbi.nlm.nih.gov/34043526 https://doi.org/10.1002/ijop.12805 https://pubmed.ncbi.nlm.nih.gov/34448200 https://doi.org/10.3102/0034654312457429 https://doi.org/10.1080/17538068.2022.2060665 Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 18 OJPHI 17. McMurray J, et al. 2017. The Importance of Trust in the Adoption and Use of Intelligent Assistive Technology by Older Adults to Support Aging in Place: Scoping Review Protocol. JMIR Res Protoc. 6(11), e218. doi:https://doi.org/10.2196/resprot.8772. PubMed 18. Laura M, Rainville C, Skufca L. 2017. 2017 AARP Cognitive Activity and Brain Health Survey. AARP Research. (Jul). doi:https://doi.org/10.26419/res.00044.001. 19. E. Moffatt and R. Deschenes, “Expert consult interview,” Jun. 28, 2021. 20. Ghazanfari A, Chin J. “What Triggers Health Information Search: An Explorative Analysis on the Information Needs, Situations, Technologies, and Habits in Health Information Search Across the Lifespan,” Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, vol. 8, no. 1, pp. 125–125, Sep. 2019, doi: https://doi.org/10.1177/2327857919081028. 21. Cutilli C. 2020. Patient Education Corner: Seeking Health Information: What Sources Do Your Patients Use? Orthop Nurs. 29(3), 214-19. PubMed https://doi.org/10.1097/NOR.0b013e3181db5471 22. Rahkovsky I, Anekwe T, Gregory C. 2018. Chronic Disease, Prescription Medications, and Food Purchases. Am J Health Promot. 32(4), 916-24. doi:https://doi.org/10.1177/0890117117740935. PubMed 23. NIH. “Supporting Older Patients with Chronic Conditions,” National Institute on Aging, May 17, 2017. https://www.nia.nih.gov/health/supporting-older-patients-chronic-conditions 24. Martínez-Alcalá CI, et al. 2018. Digital Inclusion in Older Adults: A Comparison Between Face- to-Face and Blended Digital Literacy Workshops. Front ICT. 5, 21. doi:https://doi.org/10.3389/fict.2018.00021. 25. Carr K, Weir PL, Azar D, Azar NR. 2013. Universal Design: A Step toward Successful Aging. J Aging Res. 2013, 1-8. doi:https://doi.org/10.1155/2013/324624. PubMed 26. Luo G, Tang C, Yang H, Wei X. “MedSearch: a specialized search engine for medical information retrieval,” in Proceeding of the 17th ACM conference on Information and knowledge mining - CIKM ’08, Napa Valley, California, USA, 2008, p. 143. doi: https://doi.org/10.1145/1458082.1458104. 27. Seymour T, Frantsvog D, Kumar S. 2011. History Of Search Engines. IJMIS. 15(4), 47. doi:https://doi.org/10.19030/ijmis.v15i4.5799. 28. Steele R. “Techniques for Specialized Search Engines.” University of Technology, Sydney, Australia, 2007. [Online]. Available: https://opus.lib.uts.edu.au/bitstream/10453/6844/1/2005001150.pdf 29. Lyengar R. “Why it’s so hard to dethrone Google,” CNN, Oct. 22, 2020. [Online]. Available: https://www.cnn.com/2020/10/21/tech/google-antitrust-lawsuit-better-search/index.html https://doi.org/10.2196/resprot.8772 https://pubmed.ncbi.nlm.nih.gov/29097354 https://doi.org/10.26419/res.00044.001 https://doi.org/10.1177/2327857919081028 https://pubmed.ncbi.nlm.nih.gov/20505493 https://doi.org/10.1097/NOR.0b013e3181db5471 https://doi.org/10.1177/0890117117740935 https://pubmed.ncbi.nlm.nih.gov/29284282 https://pubmed.ncbi.nlm.nih.gov/29284282 https://doi.org/10.3389/fict.2018.00021 https://doi.org/10.1155/2013/324624 https://pubmed.ncbi.nlm.nih.gov/23431446 https://doi.org/10.1145/1458082.1458104 https://doi.org/10.19030/ijmis.v15i4.5799 Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 19 OJPHI 30. Ilic D. 2003. Specialized medical search-engines are no better than general search-engines in sourcing consumer information about androgen deficiency. Hum Reprod. 18(3), 557-61. doi:https://doi.org/10.1093/humrep/deg154. PubMed 31. Fox G, Connolly R. 2018. Mobile health technology adoption across generations: Narrowing the digital divide. Inf Syst J. 28(6), 995-1019. doi:https://doi.org/10.1111/isj.12179. 32. “HONcode Certification,” Health on the Net. https://www.hon.ch/en/certification.html#certification 33. Cuan-Baltazar JY, Muñoz-Perez MJ, Robledo-Vega C, Pérez-Zepeda MF, Soto-Vega E. 2020. Misinformation of COVID-19 on the Internet: Infodemiology Study. JMIR Public Health Surveill. 6(2), e18444. doi:https://doi.org/10.2196/18444. PubMed 34. AARP. “Health Tools,” AARP Health Tools. https://www.aarp.org/health/health_tools/ 35. “AARP PerksTM Browser Extension,” AARP, 2021. https://www.aarp.org/benefits- discounts/perks/install/ 36. Rehm, John, Joffres, Kal, and Fajardo, Glenn, “Turning Design Thinking to Design Doing,” 2012, doi: https://doi.org/10.48558/WBTR-DX32. 37. “About WebMD,” WebMD. https://www.webmd.com/about-webmd-policies/default.htm 38. Sinha T, Jermann P, Li N, Dillenbourg P. “Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions,” arXiv:1407.7131 [cs], Sep. 2014, Accessed: Sep. 17, 2021. [Online]. Available: http://arxiv.org/abs/1407.7131 https://doi.org/10.3115/v1/W14-4102 39. Flaherty R. “Scroll Depth,” Parsnip, 2018. https://scrolldepth.parsnip.io/ 40. U.S. General Services Administration. “First Click Testing,” usability.giv, 2013. https://www.usability.gov/how-to-and-tools/methods/first-click-testing.html 41. Daraz L, et al. 2018. Readability of Online Health Information: A Meta-Narrative Systematic Review. Am J Med Qual. 33(5), 487-92. doi:https://doi.org/10.1177/1062860617751639. PubMed 42. TextCompare. TextCompare, 2022. [Online]. Available: https://www.textcompare.org/about/ 43. Nania R, Crouch M. “10 Myths About Coronavirus Vaccines,” AARP Health, Jan. 10, 2022. https://www.aarp.org/health/drugs-supplements/info-2020/covid-vaccine-myths.html 44. Noel T. “Why COVID Vaccines Are a Really Bad Idea,” The Richardson Post, Sep. 21, 2021. https://richardsonpost.com/ted-noel-md/23566/why-covid-vaccines-are-a-really-bad-idea/ 45. Lee K, Hoti K, Hughes JD, Emmerton LM. 2015. Consumer Use of ‘Dr Google’: A Survey on Health Information-Seeking Behaviors and Navigational Needs. J Med Internet Res. 17(12), e288. doi:https://doi.org/10.2196/jmir.4345. PubMed https://doi.org/10.1093/humrep/deg154 https://pubmed.ncbi.nlm.nih.gov/12615824 https://doi.org/10.1111/isj.12179 https://doi.org/10.2196/18444 https://pubmed.ncbi.nlm.nih.gov/32250960 https://doi.org/10.48558/WBTR-DX32 https://doi.org/10.3115/v1/W14-4102 https://doi.org/10.1177/1062860617751639 https://pubmed.ncbi.nlm.nih.gov/29345143 https://doi.org/10.2196/jmir.4345 https://pubmed.ncbi.nlm.nih.gov/26715363 Designing a Browser Extension for Reliable Online Health Information Retrieval Among Older Adults Using Design Thinking 20 OJPHI 46. J. Greene, J. Hibbard, and M. Tusler, “How Much Do Health Literacy and Patient Activation Contribute to Older Adults’ Ability to Manage Their Health?,” p. 30. 47. Mueller AL, McNamara MS, Sinclair DA. 2020. Why does COVID-19 disproportionately affect older people? Aging (Albany NY). 12(10), 9959-81. doi:https://doi.org/10.18632/aging.103344. PubMed 48. Wang G. 2022. Digital reframing: The design thinking of redesigning traditional products into innovative digital products. J Prod Innov Manage. 39(1), 95-118. doi:https://doi.org/10.1111/jpim.12605. https://doi.org/10.18632/aging.103344 https://pubmed.ncbi.nlm.nih.gov/32470948 https://pubmed.ncbi.nlm.nih.gov/32470948 https://doi.org/10.1111/jpim.12605