390 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Electronic health literacy among adults with chronic pain: A descriptive, cross-sectional survey Original Research Hyejin Park1, Geraldine Martorella1, Glenna Schluck,1 1College of Nursing, Florida State University, Tallahassee, FL, USA Corresponding author: H. Park (hpark5@fsu.edu) ABSTRACT Background: Approximately 100 million American adults are living with chronic pain, which costs the healthcare system an average of $560–635 billion each year. Levels of health literacy and ehealth literacy are important factors in determining a patient’s capacity to manage pain and the multidimensional impact of pain. To our knowledge, few studies have specifically examined the level of ehealth literacy and its association with health literacy among chronic pain patients. The purpose of this study was to 1) assess the levels of health literacy and ehealth literacy in adults with chronic pain, and 2) examine the relationship between health literacy and ehealth literacy skills among adults diagnosed and living with chronic pain. Methods: A non-experimental, descriptive cross-sectional survey was distributed to adults with chronic pain. A total of 196 participants were asked to complete questionnaires related to demographic characteristics, ehealth literacy (eHEALS), and health literacy (HLQ). Descriptive statistics were calculated to summarize data from all the scales used in the study. Results: The average level of ehealth literacy was 32.6 (SD 4.4) out of 40. The level of health literacy was measured by four subscales: having sufficient information to manage my health (mean=2.8; SD=0.55), appraisal of health information (mean=3.27; SD=0.41), ability to find good health information (mean=3.68; SD=0.45) and understanding health information well enough to know what to do (mean=3.66; SD=0.48). Two subscales (i.e., appraisal of health information, ability to find good health information) were significant in predicting ehealth literacy total score. Discussion and Conclusions: Examining ehealth literacy and health literacy can assist in the dissemination of accessible and understandable chronic-pain-related health information for individuals of all health literacy levels. In addition, this will allow the development of interventions for enhancing ehealth literacy skills and/or usability of web-based information for adults with chronic pain. KEYWORDS Chronic Pain, Ehealth Literacy, Health Literacy, Online Health Information INTRODUCTION Chronic pain is one of the most common chronic conditions. According to data from the 2019 National Health Interview Survey (NHIS), the prevalence of chronic pain was 20.4%, (Zelaya et al., 2020), with an estimated national economic cost of $560–635 billion annually (Gaskin & Richard, 2012). People with chronic pain are required to manage their condition daily and are often on a waiting list for referral to a pain specialist or consultation at a pain clinic. Although caseloads and wait times are difficult to estimate, it is generally recognized that treatment availability for chronic pain is particularly scarce across the United States (Fashler et al., 2016; Schatman, 2012). As a result, individuals often get discouraged and search for temporary solutions online to assist in the management of their pain. As a result, health literacy, which is “the degree to which individuals have the ability to find, understand, and use information and services to inform health-related decisions and actions for themselves and others” (Santana et al., 2021, s259) has become an important factor in determining their capacity to manage their Several studies have examined the impact of health literacy levels on health outcomes (Berkman et al., 2011; Sheridan et ai., 2011; Jacobs et al., 2016; Kim, & Xie, 2017), with results suggesting low health literacy may have an impact on the management of chronic pain. Specifically, Devraj, Herndon and Griffin (2013) reported individuals who had lower levels of health literacy levels were not able to 391 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 effectively engage in self-care, as they did not fully understand when and how to take their pain medication, the various types of non- pharmacological strategies to use to manage pain, and how to access care from health professionals during periods of pain flare-ups (Devraj, Herndon, & Griffin, 2013). Additionally, approximately 90 % of Americans use the Internet on a regular basis (Pew Research Center, 2017) with on average, one in three adults searching for health information online (Fox & Duggan, 2013). The notion of accessing health information online is known as electronic health (ehealth) information. As the rate of ehealth information continues to increase, it has become more important than ever to determine ehealth literacy of individuals who not only access this information but use it to influence their overall health status. Electronic health literacy is the ability to seek, find, understand, appraise, and implement online health information to address or solve a health problem (Norman & Skinner, 2006a; Norman & Skinner, 2006b). This type of health literacy has become an important aspect of self-management intervention design (Kim & Xie, 2017; Watkins, & Xie, 2014; Chesser et al., 2016). Table 1 compares the definitions of health literacy and ehealth literacy. Currently, it is unclear as to whether low ehealth literacy is related to low health literacy, low digital literacy or a combination of these factors (Del Giudice et al., 2018). Although, it is generally admitted that both variables are related based on Norman and Skinner’s theoretical model, studies have reported contradictory results depending on the population sample. For example, among students, these variables did not demonstrate a statistically significant association (Monkman et al., 2017). However, among individuals living with chronic conditions, these variables were found to predict different clinical outcomes (Neter & Brainin, 2019; Stellefson et al., 2019). Few studies exist that have either demonstrated an association between health literacy and ehealth literacy or considered the impact of health literacy on ehealth literacy. One study suggested individuals with low health literacy skills experienced increased difficulties evaluating online health information (Diviani et al., 2015). Additionally qualitative reports indicate individuals with high health literacy were more likely to engage in online searches for health information (Ellis et al., 2012). Given the limited research in this area, there appears to be a need to examine the link more closely between health and ehealth literacy. By examining this connection, health departments and organizations will be better informed about the best method for disseminating chronic-pain-related health information that can be accessed and understood by individuals of all health literacy levels. In addition, this will allow the development of interventions for enhancing ehealth literacy skills and/or improving the usability of Web-based information for adults with chronic pain. PURPOSE To our knowledge, no study has looked at the relationship between health literacy and ehealth literacy levels in adults with chronic pain. The purpose of this study was to 1) assess the levels of health literacy and ehealth literacy in adults with chronic pain, and 2) examine the relationship between health literacy and ehealth literacy skills among adults diagnosed and living with chronic pain. METHODS Design A non-experimental, descriptive cross-sectional survey was distributed to adults diagnosed and living with chronic pain. Sample Size The sample size was determined using Green’s rule of thumb (Green, 1991) which states the minimum number of subjects is 50+8m where m is the number of predictors. With 9 predictors, the minimum required sample size was calculated to be 122. Procedure A convenience sample of 196 participants was recruited using Amazon’s Mechanical Turk (MTurk). Amazon’s MTurk is a popular Internet crowdsourcing tool that makes it possible to recruit large and diverse samples of research participants from across the United States and around the world (Paolacci, Chandler, & Ipeirotis, 2010) quickly and at relatively low cost (Horton, & Chilton, 2010). MTurk is also effective in reaching populations that are typically underrepresented through traditional recruitment techniques (Chandler, & Shapiro, 2016). Participant responses through MTurk are anonymous to requesters, which protects participant 392 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 anonymity and increases response rates (O’Neil, & Penrod, 2001). Responses have also been shown to be as reliable and valid as those of participants recruited through traditional sampling methods within a variety of research domains (Goodman, Cryder, & Cheema, 2013). MTurk workers browse human intelligence tasks (HITs) by title, keyword, reward, availability, and so on, and complete HITs of interest, for which they are paid upon completion (Chandler, & Shapiro, 2016). The format is well suited to the collection of survey data, and since requesters can discretionally reject work, worker reputation has a direct impact on future HITS that workers can complete, leading to strong norms of honesty and accuracy (Rand, 2012). Sampling Criteria Participants in this study were restricted to U.S. residents with a history of at least a 90% task approval rate for their previous HITS. Participants who were willing to be involved in this study were asked to complete questionnaires related to 1) demographic characteristics (i.e., age, gender, ethnicity, marital and work status, education level, duration, and location of chronic pain), 2) Internet use and ehealth literacy, and 3) health literacy. Instruments To assess participants’ ehealth literacy level, an ehealth literacy scale titled: eHEALS was used (Norman, & Skinner, 2006b). The eHEALS is an 8- item self-report questionnaire that focuses on knowledge and understanding of what health information is available on the Internet, where one can find helpful health resources, how to access this information, how to use the Internet, how to evaluate online health information, and how to discern the difference between high- and low-quality health resources on the Internet. Each item is rated on a 5- point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Total scores of the eHEALS are summed, ranging from 8 to 40, with higher scores representing higher self-perceived ehealth literacy. Internal consistency reliability ranges from .89 to .97 and has good test-retest reliability (Norman, & Skinner, 2006b). Two supplemental items recommended by the authors of eHEALS were included to assess the perceived usefulness of the Internet for making health decisions (a 5-point Likert scale ranging from 1=not useful at all to 5=very useful) and perceived importance of being able to access health resources on the Internet (also a 5-point Likert scale ranging from 1=not important at all, to 5=very important). To assess health literacy, the Health Literacy Questionnaire (HLQ) was used after obtaining a licence agreement from the authors (Osborne et al., 2013). This instrument identifies health literacy strengths and weaknesses. The HLQ includes 9 subscales, for a total of 44 items. Each subscale provides a score. Four out of nine subscales that are specifically related to the ability to search, appraise, understand, and use health information were included. The HLQ has been shown to have strong psychometric properties (Osborne et al., 2013). The four subscales include having sufficient information to manage health (subscale 2), appraisal of health information (subscale 5), ability to find good health information (subscale 8) and understanding health information well enough to know what to do (subscale 9). Ethics After IRB approval was obtained from Florida State University, the survey offer was listed on MTurk. After they accessed the offer, respondents were redirected to a link to the informed consent form. By starting the survey, the participants were agreeing to participate. Respondents received $2.00 in compensation after completing the survey. Analysis Descriptive statistics were calculated to summarize data from all scales used in the study. Normality was assessed through examination of histograms and by conducting Kolmogorov-Smirnoff tests. The relationship between ehealth literacy and health literacy was assessed by examining scatterplots, calculating Spearman’s rank-order correlation between eHEALS and each HLQ subscale, and conducting linear regression predicting eHEALS scores from the HLQ subscales. Residual plots were examined to determine if there were any violations to the assumptions of linear regression. Missing Data The two subscales for the health literacy questionnaire (subscales 8 and 9: finding and understanding health information) contained missing data. The missing values were imputed using an expectation maximization (EM) algorithm according to scoring instructions and SPSS code supplied by the authors of the HLQ tool (Osborne, Batterham, Elsworth, Hawkins, & Buchbinder, 2013). RESULTS 393 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Demographics and Clinical Characteristics The sample included 196 participants with an average age of 40 years (SD 12.1 years). More than half (n=110, 56.1%; Table 2) were female, white (n=162, 82.7%), achieved a 2-year degree or higher (n=124, 63.3%), or were employed full time (n=136, 69.4%). Approximately half were married (n=97, 49.5%). Most participants reported suffering from chronic pain for more than 2 years with 20.9% of them indicating that they have been suffering from chronic pain for more than 10 years. Approximately half (n=94, 48%) suffered from chronic back pain. Levels of Health Literacy and ehealth Literacy The level of health literacy was measured by four subscales of the HLQ (having sufficient information to manage my health [Mean= 2.8 ; SD=0.55], appraisal of health information [Mean= 3.27 ; SD=0.41], ability to find good health information [Mean= 3.68 ; SD=0.45], understanding health information well enough to know what to do [Mean= 3.66 ; SD=0.48] (Table 4). Table 5 describes the level of ehealth literacy in a sample of individuals suffering from chronic pain. When asked “How useful do you feel the internet is in helping you make decisions about your health?”, 82.1% indicated that the internet is useful (n=137, 69.9%) or very useful (n=24, 12.2%). When asked “How important is it for you to be able to access health resources on the internet?”, 88.7% indicated that it is important (n=91, 46.4%) or very important (n=83, 42.3%). The average ehealth literacy (eHEALS) was 32.6 (SD 4.4). out of 40. Slightly fewer than half (n=93, 47.4%) of all participants had high ehealth literacy (defined as an eHEALS score above the mean). Among the eight items in eHEALS, “I can tell high quality health resources from low quality health resources on the Internet” rated the highest and “I feel confident in using information from the Internet to make health decisions” rated lowest (Table 5). Cronbach’s alpha was calculated for each health literacy subscale and for the ehealth literacy scale to assess internal consistency of the measures. These values are displayed in Table 3. All scales reported adequate internal consistency. The Relationship between ehealth Literacy and Health Literacy To examine the relationship between ehealth literacy and health literacy in adults with chronic pain, scatterplots between the eHEALS total score and each of the four HLQ subscales were fit. Scatterplots indicated weak to moderate positive bivariate relationships. The correlations between the eHEALS total score and each of the four HLQ subscales were also weak to moderate (Having sufficient information to manage my health: 0.36, Appraisal of health information: 0.47, Ability to find good health information: 0.27, Reading and understanding health information: 0.18). Preliminary examination of the bivariate scatterplots between the eHEALS total score and each of the four HLQ subscales indicates that a quadratic relationship may be present. Therefore, both the linear terms and the quadratic terms were included for each of the four subscales. Backwards elimination was used to eliminate 3 of the four squared terms from the model. After this first step, the four linear terms and the squared term for HLQ subscale 8 (i.e., finding health information) remained in the model. Since the primary interest of this research question is to determine the relationship between the subscales themselves and the eHEALS total score, the linear terms were included regardless of significance level or standardized coefficient size. Upon inspection of residual diagnostics, collinearity issues were present for HLQ8 and its quadratic term. Standardizing this subscale and recalculating the quadratic term solved the collinearity issue. Additionally, two cases had large, standardized residuals (outside +/- 3 standard deviations). The analysis was repeated without these two cases to determine their impact on the analysis. While the p- values and estimates were impacted for HLQ9 (understanding health information), the conclusions about the relative importance for the variables does not change. To remain conservative, the results are presented for the full data set. While the HLQ subscales are non-Normal, examination of the histogram of residuals and the Normal probability plot indicated no violations to the normality of residuals assumption. No additional violations to regression assumptions were noted. The linear regression ANOVA results indicate that this model is useful for prediction (F[131, 5]=13.234, p<.0005) with the model explaining 31% of the variance in the eHEALS total score. In the final model, two subscales significantly predict eHEALS total score (HLQ5: appraisal of health information, and HLQ8: ability to find good health information). Additional terms in the model are HLQ2 (Having sufficient information), HLQ9 (Understanding health 394 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 information), and the quadratic HLQ8 (ability to find good health information) term. Coefficients, p-values, and confidence intervals are presented in Table 6. DISCUSSION Health consumers’ health literacy plays an important role in seeking online health information because it includes the ability to evaluate health information online (Schulz & Nakamoto, 2013). This study is the first to explore the level of self-reported health literacy and ehealth literacy in adults with chronic pain, and the association between health literacy and ehealth literacy among these patients. Based on these study results, most participants indicated that they found the Internet useful and very important for searching for health information. The average ehealth literacy was high as illustrated by a mean score of 32.6 out of 40. Of the 8 items in eHEALS, participants scored highest on the ability to tell high from low quality health resources on the Internet, and lowest on feeling confident using online information to make health decisions. One study (Stellefson et al., 2018) found that overall, participants with chronic obstructive pulmonary disease (COPD) have moderate ehealth literacy, with more than 70% feeling confident in their ability to find online health information, but those participants felt much less confident in their ability to tell high from low quality online health information. These results differed slightly from our study findings. Another study revealed that the average ehealth literacy was 26.3/40 among patients with rheumatic diseases, and that higher ehealth literacy was related to younger age, experience with app use, belief in using mobile apps, and current Internet use for seeking online health information (Knitza et al., 2020). Chronic pain is one of the most common chronic conditions, and adults with chronic pain are a growing population of health care consumers. Health literacy in patients with chronic pain will play a critical role in their development of better self- management of pain and finding adequate coping strategies for chronic pain. Patients with higher skills in the areas of searching, finding, understanding, and critiquing information have previously shown better performance with their chronic pain management and lower pain intensity (Köppen et al., 2018). A cross- sectional study (n=131), however, found that individuals with chronic pain have inadequate health literacy (Mackey et al., 2019). In this study, we also found that the mean score for some areas of health literacy were inadequate or low. The lowest overall scores occurred for the scales “having sufficient health information to manage my health” and “appraisal of health information.” The highest score was seen for “understanding health information.” The results are similar to those of another study (Beauchamp et al., 2015). As in our study, that study found that the lowest score was for “appraisal of health information.” Our findings suggest that health literacy efforts that focus on “appraisal of health information” and “having sufficient health information to manage my health” for patients with chronic pain are an important strategy for utilizing quality of online health information to make health related decision. As technology-driven health consumers search for health information online, digital (ehealth) literacy has become an emerging concept. In terms of the relationship between levels of ehealth and health literacy, a weak to moderate correlation was identified. Two categories of health literacy such as “appraisal of health information” and “ability to find good health information” significantly predicted ehealth literacy. This means that participants with lower scores on these two subscales had lower levels of perceived skills in finding or appraising digital (online) health information. These findings are similar to those of other studies. One study identified levels of health literacy among students, and less health literate students had significantly lower eHEALS scores than those with adequate health literacy (Ghaddar et al., 2012). Another study also found that participants with lower health literacy scored poorly on finding high quality online information and high on finding low quality online information (Benotsch et al., 2004). While health literacy is one factor that influences ehealth literacy, other potential factors can also be associated with ehealth literacy. For example, education levels influence ehealth literacy positively (Neter & Brainin, 2012; van Deursen & van Dijk, 2011). This association has been previously demonstrated in chronic pain patients living in a low-income country (Shiferaw et al., 2020) and it was also the case in our study, where the vast majority of participants had completed at least some college and reported a fairly high level of ehealth literacy. Another study showed that reading ability is a factor that influences ehealth literacy (Benotsch et al., 2004). LIMITATIONS Although these study findings provide some suggestions, they also are subject to two limitations. Firstly, data were collected via a self-reported Web- based survey (MTurk). This may affect the study 395 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 findings due to the nature of self-selection and self- reporting. Another limitation is the small sample size with a racially homogeneous sample. Therefore, representativeness and generalization are potential limitations of this study. CONCLUSION Despite its limitations, this study has provided important insights about health literacy and ehealth literacy for patients with chronic pain. Our findings showed that health literacy plays an important role in ehealth literacy, and this topic is worth scholarly attention. Future studies in this area should focus on high quality interventions that improve the ability of individuals with low health literacy and low ehealth literacy to better self-manage their pain. In addition, health departments and organizations need better information about methods they can use to disseminate chronic-pain-related health information that people of all health literacy levels can access and understand. REFERENCES Beauchamp, A., Buchbinder, R., Dodson, S., Batterham, R. W., Elsworth, G. R., McPhee, C., ... & Osborne, R. H. (2015). Distribution of health literacy strengths and weaknesses across socio-demographic groups: a cross- sectional survey using the Health Literacy Questionnaire (HLQ). BMC Public Health, 15(1), 1-13. Benotsch, E. G., Kalichman, S., & Weinhardt, L. S. (2004). HIV-AIDS patients' evaluation of health information on the internet: the digital divide and vulnerability to fraudulent claims. Journal of Consulting and Clinical Psychology, 72(6), 1004. Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., & Crotty, K. (2011). Low health literacy and health outcomes: an updated systematic review. Annals of Internal Medicine, 155(2), 97-107. Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., Viera, A., Crotty, K., ... & Viswanathan, M. (2011). Health literacy interventions and outcomes: an updated systematic review. Evidence Report/Technology Assessment, (199), 1-941. Bailey, S. J., LaChapelle, D. L., LeFort, S. M., Gordon, A., & Hadjistavropoulos, T. (2013). Evaluation of chronic pain-related information available to consumers on the internet. Pain Medicine, 14(6), 855-864. Chesser, A., Burke, A., Reyes, J., & Rohrberg, T. (2016). Navigating the digital divide: a systematic review of ehealth literacy in underserved populations in the United States. Informatics for Health and Social Care, 41(1), 1-19. Chandler, J., & Shapiro, D. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12, 53-81. Corcoran, T. B., Haigh, F., Seabrook, A., & Schug, S. A. (2009). The quality of internet-sourced information for patients with chronic pain is poor. The Clinical journal of pain, 25(7), 617- 623.Devraj, R., Herndon, C. M., & Griffin, J. (2013). Pain awareness and medication knowledge: a health literacy evaluation. Journal of Pain & Palliative Care Pharmacotherapy, 27(1), 19-27. Diviani, N., van den Putte, B., Giani, S., & van Weert, J. C. (2015). Low health literacy and evaluation of online health information: a systematic review of the literature. Journal of Medical Internet Research, 17(5), e112. Del Giudice, P., Bravo, G., Poletto, M., De Odorico, A., Conte, A., Brunelli, L., ... & Brusaferro, S. (2018). Correlation between ehealth literacy and health literacy using the ehealth literacy scale and real-life experiences in the health sector as a proxy measure of functional health literacy: cross-sectional web-based survey. Journal of Medical Internet Research, 20(10), e281. Ellis, J., Mullan, J., Worsley, A., & Pai, N. (2012). The role of health literacy and social networks in arthritis patients' health information-seeking behavior: a qualitative study. International Journal of Family Medicine, 2012. Fox S, Duggan, M (2013) Health Online 2013: Internet, Science and Tech Report Retrieved from https://www.pewresearch.org/internet/201 3/01/15/health-online-2013/ Fashler, S. R., Cooper, L. K., Oosenbrug, E. D., Burns, L. C., Razavi, S., Goldberg, L., & Katz, J. (2016). Systematic review of multidisciplinary chronic pain treatment facilities. Pain Research and Management, 2016. Gaskin, D. J., & Richard, P. (2012). The economic costs of pain in the United States. The Journal of Pain, 13(8), 715-724. Ghaddar, S. F., Valerio, M. A., Garcia, C. M., & Hansen, L. (2012). Adolescent health literacy: the importance of credible sources for online 396 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 health information. Journal of School Health, 82(1), 28-36. Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26(3), 213-224. Green, S. B. (1991). How many subjects does it take to do a regression analysis? Multivariate Behavioral Research, 26(3), 499-510. Honey, M. L., Bycroft, J., Tracey, J., Boyd, M. A., & McLachlan, A. (2010). Quality processes that maximise the health navigator web portal as an enabler for consumers and health professionals. Health Care and Informatics Review Online, 14(1), 12-18. Horton, J. J., & Chilton, L. B. (2010, June). The labor economics of paid crowdsourcing. In Proceedings of the 11th ACM conference on electronic commerce (pp. 209-218). Jacobs, R. J., Lou, J. Q., Ownby, R. L., & Caballero, J. (2016). A systematic review of ehealth interventions to improve health literacy. Health Informatics Journal, 22(2), 81- 98. Kaicker, J., Debono, V. B., Dang, W., Buckley, N., & Thabane, L. (2010). Assessment of the quality and variability of health information on chronic pain websites using the DISCERN instrument. BMC Medicine, 8(1), 1-8. Knitza, J., Simon, D., Lambrecht, A., Raab, C., Tascilar, K., Hagen, M., ... & Hueber, A. J. (2020). Mobile health usage, preferences, barriers, and ehealth literacy in rheumatology: patient survey study. JMIR mHealth and uHealth, 8(8), e19661. Kim, H., & Xie, B. (2017). Health literacy in the ehealth era: A systematic review of the literature. Patient Education and Counseling, 100(6), 1073-1082. Köppen, P. J., Dorner, T. E., Stein, K. V., Simon, J., & Crevenna, R. (2018). Health literacy, pain intensity and pain perception in patients with chronic pain. Wiener klinische Wochenschrift, 130(1), 23-30. Mackey, L. M., Blake, C., Casey, M. B., Power, C. K., Victory, R., Hearty, C., & Fullen, B. M. (2019). The impact of health literacy on health outcomes in individuals with chronic pain: a cross-sectional study. Physiotherapy, 105(3), 346-353. Monkman, H., Kushniruk, A. W., Barnett, J., Borycki, E. M., Greiner, L. E., & Sheets, D. (2017). Are health literacy and ehealth literacy the same or different? Studies in Health Technology and Informatics, 245:178- 182. PMID: 29295077. Neter, E., & Brainin, E. (2012). ehealth literacy: extending the digital divide to the realm of health information. Journal of Medical Internet Research, 14(1), e19. Neter, E., & Brainin, E. (2019). Association between health literacy, ehealth literacy, and health outcomes among patients with long-term conditions. European Psychologist. Norman, C. D., & Skinner, H. A. (2006a). ehealth literacy: essential skills for consumer health in a networked world. Journal of medical Internet Research, 8(2), e506. Norman, C. D., & Skinner, H. A. (2006b). eHEALS: the ehealth literacy scale. Journal of Medical Internet Research, 8(4), e27. O’Neil, K. M., & Penrod, S. D. (2001). Methodological variables in Web-based research that may affect results: Sample type, monetary incentives, and personal information. Behavior Research Methods, Instruments, & Computers, 33(2), 226-233. Osborne, R. H., Batterham, R. W., Elsworth, G. R., Hawkins, M., & Buchbinder, R. (2013). The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health, 13(1), 1-17. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on amazon mechanical turk. Judgment and Decision Making, 5(5), 411- 419. Pew Research Center (2017). Record shares of Americans now own smartphones, have home broadband. Retrieved from https://www.pewresearch.org/fact- tank/2017/01/12/evolution- oftechnology/?utm_content=bufferdaf6c&utm_m edium=social&utm_source=twitter.com&utm_ca mpaign=buffer Rand, D. G. (2012). The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments. Journal of Theoretical Biology, 299, 172-179. Santana, S., Brach, C., Harris, L., Ochiai, E., Blakey, C., Bevington, F., ... & Pronk, N. (2021). Updating Health Literacy for Healthy People 2030: Defining Its Importance for a New Decade in Public Health. Journal of Public Health Management and Practice. Schulz, P. J., & Nakamoto, K. (2013). Health literacy and patient empowerment in health communication: the importance of separating conjoined twins. Patient Education and Counseling, 90(1), 4-11. 397 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Shiferaw, K.B., Tilahun, B.C., Endehabtu, B.F. et al. (2020). E-health literacy and associated factors among chronic patients in a low-income country: a cross-sectional survey. BMC Medical Information Decision Making, 20, 181. https://doi.org/10.1186/s12911-020-01202-1 Stellefson, M., Hanik, B., Chaney, B., Chaney, D., Tennant, B., & Chavarria, E. A. (2011). ehealth literacy among college students: a systematic review with implications for ehealth education. Journal of Medical Internet Research, 13(4), e102. Stellefson, M. L., Shuster, J. J., Chaney, B. H., Paige, S. R., Alber, J. M., Chaney, J. D., & Sriram, P. S. (2018). Web-based health information seeking and ehealth literacy among patients living with chronic obstructive pulmonary disease (COPD). Health Communication, 33(12), 1410-1424. Stellefson, M., Paige, S. R., Alber, J. M., Chaney, B. H., Chaney, D., Apperson, A., & Mohan, A. (2019). Association between health literacy, electronic health literacy, disease-specific knowledge, and health-related quality of life among adults with chronic obstructive pulmonary disease: cross-sectional study. Journal of Medical Internet Research, 21(6), e12165. Sheridan, S. L., Halpern, D. J., Viera, A. J., Berkman, N. D., Donahue, K. E., & Crotty, K. (2011). Interventions for individuals with low health literacy: a systematic review. Journal of Health Communication, 16(sup3), 30-54. Schatman, M. E. (2012). Interdisciplinary chronic pain management: international perspectives. Pain: Clinical Updates, 20(7), 1-5. van Deursen, A. J., & van Dijk, J. A. (2011). Internet skills performance tests: are people ready for ehealth?. Journal of Medical Internet Research, 13(2), e1581. Watkins, I., & Xie, B. (2014). ehealth literacy interventions for older adults: a systematic review of the literature. Journal of Medical Internet Research, 16(11), e225. Washington, T. A., Fanciullo, G. J., Sorensen, J. A., & Baird, J. C. (2008). Quality of chronic pain websites. Pain Medicine, 9(8), 994-1000. Zelaya, C. E., Dahlhamer, J. M., Lucas, J. W., & Connor, E. M. (2020). Chronic pain and high- impact chronic pain among US adults, 2019. 398 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Table 1. Definitions Health Literacy ehealth Literacy The degree to which individuals can find, understand, and use information and services to inform health-related decisions and actions for themselves and others” (Santana et al., 2021, s259) The ability to seek, find, understand, and appraise online health information and apply such knowledge to address or solve a health problem (Norman, & Skinner, 2006a; Norman, & Skinner, 2006b) 399 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Table 2. Participants’ Demographics Variable Level n % Gender Female 110 56.1 Male 86 43.9 Ethnicity White 162 82.7 Not White 34 17.3 Marital Status Married 97 49.5 Not Married 99 50.5 Work Status Full-Time 136 69.4 Part-Time 34 17.3 Not Working 26 13.3 Education Level HS Graduate 23 11.7 Some College 49 25.0 2 Year Degree 32 16.3 4 Year Degree 66 33.7 Graduate Degree 26 13.3 Duration of Chronic Pain Less Than 2 Years 51 26.1 2 – 5 Years 61 31.1 5 – 10 Years 43 21.9 More Than 10 Years 41 20.9 Location of Chronic Pain Back 94 48.0 Neck 17 8.7 Head 15 7.7 Knees 14 7.1 Feet/Ankles 13 6.6 Shoulders 10 5.1 Other 8 4.1 General 6 3.1 Legs 6 3.1 Hands 5 2.6 Arms 4 2 Hips 4 2 Mean S.D. Age 40.1 12.1 400 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Table 3. Internal consistency of the health literacy measures Measure/Scale Cronbach’s Alpha HLQ – Having Sufficient Information to manage my health (HSI) .85 HLQ – Appraisal of Health Information (AHI) .70 HLQ – Ability to Find Good Health Information (AFGHI) .82 HLQ – Understanding Health Information well enough to know what to do (UHI) .80 eHEALS – ehealth Literacy (8 items) .86 401 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Table 4. Health literacy summary statistics Measure n Mean Median Std. Dev. Min. Max. *HLQ – HSI (Having Sufficient Information to manage my health) 196 2.89 3 0.55 1.5 4 *HLQ – AHI (Appraisal of health information) 196 3.27 3.2 0.41 2.4 4 **HLQ – AFGHI (Ability to Find Good Health Information) 168 3.68 3.8 0.45 1 4 **HLQ – UHI (Understanding Health Information well enough to know what to do) 142 3.66 3.8 0.48 1 4 * Rating from 1( strongly disagree) to 4 (strongly agree); **Rating from 1( always difficult) to 5 (always easy) 402 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Table 5. eHEALS item frequencies N (%) Strongly Disagree Disagree Undecided Agree Strongly Agree I know what health resources are available on the Internet. 0 (0%) 6 (3.1%) 18 (9.2%) 124 (63.3%) 48 (24.5%) I know where to find helpful health resources on the Internet. 1 (0.5%) 7 (3.6%) 17 (8.7%) 119 (60.7%) 52 (26.5%) I know how to find helpful health resources on the Internet. 1 (0.5%) 5 (2.6%) 13 (6.6%) 126 (64.3%) 51 (26%) I know how to use the Internet to answer my questions about health. 0 (0%) 3 (1.5%) 10 (5.1%) 126 (64.3%) 57 (29.1%) I know how to use the health information I find on the Internet to help me. 1 (0.5%) 5 (2.6%) 21 (10.7%) 121 (61.7%) 48 (24.5%) I have the skills I need to evaluate the health information I find on the Internet. 1 (0.5%) 7 (3.6%) 21 (10.7%) 108 (55.1%) 59 (30.1%) I can tell high quality health resources from low quality health resources on the Internet. 1 (0.5%) 8 (4.1%) 27 (13.8%) 93 (47.4%) 67 (34.2%) I feel confident in using information from the Internet to make health decisions. 2 (1%) 20 (10.2%) 30 (15.3%) 105 (53.6%) 39 (19.9%) 403 IHTP, 1(3), 390-403, 2021 CC BY-NC-ND 4.0 ISSN 2563-9269 Table 6. Regression coefficients for predicting ehealth literacy (i.e., eHEALS total score) Unstandardized Coefficients Standardized Coefficient 95% Confidence Interval for B B Std. Error Beta t p-value Lower Bound Upper Bound Intercept 14.667 3.916 --- 3.746 <.0005 6.921 22.413 HLQ2 1.214 0.678 0.144 1.79 0.076 -0.127 2.556 HLQ5 4.117 0.852 0.373 4.833 <.0005 2.432 5.801 HLQ8 (Std) 1.649 0.495 0.413 3.329 0.001 0.669 2.629 HLQ9 0.026 0.777 0.003 0.034 0.973 -1.511 1.564 HLQ8 (Std)2 0.263 0.135 0.229 1.952 0.053 -0.004 0.53 Monkman, H., Kushniruk, A. W., Barnett, J., Borycki, E. M., Greiner, L. E., & Sheets, D. (2017). Are health literacy and ehealth literacy the same or different? Studies in Health Technology and Informatics, 245:178-182. PMID: 29295077.