key: cord-317849-i80zf67c
authors: Isautier, J. M.; Copp, T.; Ayre, J.; Cvejic, E.; Meyerowitz-Katz, G.; Batcup, C.; Bonner, C.; Dodd, R.; Nickel, B.; Pickles, K.; Cornell, S.; Dakin, T.; McCaffery, K. J.
title: Lessons from the COVID-19 pandemic: People's experiences and satisfaction with telehealth during the COVID-19 pandemic in Australia
date: 2020-09-11
journal: nan
DOI: 10.1101/2020.09.10.20192336
sha: 
doc_id: 317849
cord_uid: i80zf67c

Objectives: To determine how participants perceived telehealth consults in comparison to traditional in-person visits, and to investigate whether people believe that telehealth services would be useful beyond the pandemic. Design: A national cross-sectional community survey. Participants: Australian adults aged 18 years and over (n=1369). Main outcome measures: Telehealth experiences. Results: Of the 596 telehealth users, the majority of respondents (62%) rated their telehealth experience as "just as good" or "better" than a traditional in-person medical appointment. On average, respondents perceived that telehealth would be moderately to very useful for medical appointments after the COVID-19 pandemic is over (M=3.67 out of 5, SD=1.1). Being male (p=0.007), having a history of both depression and anxiety (p=0.037), or lower patient activation (individuals' willingness to take on the role of managing their health/healthcare) (p=0.037) were associated with a poorer telehealth experience. Six overarching themes were identified from free-text responses of why telehealth experience was poorer than a traditional in-person medical appointment: communication is not as effective; limitations with technology; issues with obtaining prescriptions and pathology; reduced confidence in doctor; additional burden for complex care; and inability to be physically examined. Conclusions: Telehealth appointments were reported to be comparable to traditional in-person medical appointments by most of our sample. Telehealth should continue to be offered as a mode of healthcare delivery while the pandemic continues and may be worthwhile beyond the pandemic.

• The COVID-19 pandemic has resulted in an increase in telehealth services.

• The majority of telehealth users (62%) perceived their experience to be just as good or better than traditional in-person medical care; and that telehealth would be at least somewhat useful beyond the pandemic.

• Having a history of both depression and anxiety was associated with a poorer telehealth experience and in-person visits were frequently preferred over telehealth visits for mental health appointments.

• Telehealth should continue to be offered while the COVID-19 pandemic continues and may be worthwhile beyond the pandemic, however, telehealth may be less effective for mental health services.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The coronavirus (COVID-19) outbreak was officially declared a pandemic by the World Health Organization on March 11 2020. To help minimize the spread of the virus, healthcare systems have rapidly adopted alternative models for healthcare delivery, including telehealth services (1) . This type of healthcare delivery minimizes the spread of the virus by providing health care services without the need for face-to-face contact, reducing the risk of exposure to COVID-19 for both patients and clinicians.

In response to the COVID-19 pandemic, the Australian Government introduced a temporary telehealth scheme on March 30 2020 to enable subsidised access to healthcare services via telephone or videoconferencing (2) . Prior to the pandemic, telehealth consultations were restricted to rural and remote communities. This new scheme allowed all medical appointments with a variety of health professionals to be conducted via telehealth, regardless of rurality. As a result of this scheme, telehealth consults accounted for 36% of all services provided in April 2020 compared to 1.3% before the pandemic (3) . At the end of April 2020, a nationally representative survey by the Australian Bureau of Statistics (ABS) (N=1,022) reported that 1 in 6 people used a telehealth service (17%); women were almost twice as likely as men to use telehealth (22% vs 12%); and persons with a chronic/mental health condition were twice as likely to have used a Telehealth service as those without (25% vs 13%).

However, 1 in 10 people (11%) reported to have a GP or health professional appointment cancelled or postponed (4) . Cancelling or postponing appointments is concerning because reduced healthcare utilization during pandemics has been associated with poorer health outcomes, for example during the Ebola virus outbreak and SARS epidemic (5, 6) . The increased uptake of telehealth services and increased cancelling or postponing of medical appointments warrants further investigation to better understand people's experiences and satisfaction with accessing telehealth services during the pandemic. This is particularly necessary given the long-term outlook of COVID-19 -while some health services have returned to normal, continuing outbreaks may deter patients from accessing in-person care for some time (7) .

Despite the increase in telehealth, little is known about people's experience of telehealth services compared to traditional-in person visits during the COVID-19 pandemic in Australia.

We investigated the experiences related to telehealth in a sample of Australians during the COVID-19 pandemic. Our aims were to determine how participants perceived telehealth consults in comparison to traditional in-person visits, and to investigate whether people believe that telehealth services would be useful after the pandemic. Furthermore, we investigated the sociodemographic and health-related factors associated with negative telehealth experiences.

The data used in this study are from a prospective longitudinal national survey launched in April 2020 exploring variation in understanding, attitudes, and uptake of COVID-19 health advice during the 2020 pandemic (8) . Here, we report on data from the survey wave conducted over a 1-week period in Australia from June 5 to June 12 2020, using the online platform Qualtrics. Participants were aged 18 years and over, could read and understand English and currently residing in Australia. Recruitment was via paid targeted advertisements on social media (Facebook and Instagram). More details on recruitment are given elsewhere (8) . Participants were given the opportunity to enter into a prize draw for the chance to win one of ten $20 gift cards upon completion of the survey. This study was approved by The University of Sydney Human Research Ethics Committee (2020/212).

Sociodemographic variables including age, gender and educational status were collected, as well as self-reported chronic diseases and overall health. We assessed health literacy using the Newest Vital Sign (9) and digital health literacy using the eHealth Literacy Scale (eHeals) (10) . The Consumer Health Activation Index (CHAI) (11) was used to determine patient activation (individuals' willingness to take on the role of managing their health and healthcare). Remoteness and socioeconomic status of place of residence were derived from participant postcode (12) . Participants were asked to indicate whether they had used telehealth services, and if so, how telehealth services compared to traditional in-person . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. . visits, if they experienced any barriers to using telehealth, and whether they have cancelled or postponed an appointment with a health professional (Box 1).

Quantitative data were analysed using Stata/IC v16. 1Descriptive statistics were calculated for sample characteristics and used to summarise the telehealth experiences of the sample since COVID-19 restrictions commenced. A generalised linear model utilising a modified Poisson approach (log link function with robust standard errors) was used to estimate adjusted relative risks (with 95% confidence intervals) of having a poorer telehealth experience compared to traditional in-person medical visits across various sociodemographic and health-related factors. An independent samples t-test was used to compare perceived usefulness of telehealth medical appointments once the COVID-19 emergency ends between those who rated their telehealth experience as worse, and those who reported the experience to be the same or better, than an in-person medical visit.

Statistical significance for these exploratory analyses was set at p<0.05.

Qualitative data was analysed using content analysis (13) which combines both qualitative and quantitative methods, allowing both the frequency of categories to be reported and the content. JI and TC familiarised themselves with the content and generated a list of recurring themes; these were discussed and then checked by an additional researcher (JA). JI and TC then applied the final coding framework to all the data. The level of agreement was tested using Cohen's kappa and indicated substantial agreement (κ=0.76) (14) . Discrepancies were discussed until consensus was obtained. Descriptive statistics are provided to summarise the frequency of each code.

Of the 1,369 respondents who completed the June survey, 596 (45%) reported using telehealth services since the start of the pandemic. Those who used telehealth were slightly older, more likely to be female, had higher levels of education, greater prevalence of chronic health conditions (including mental health history), and poorer self-reported general health.

Sample characteristics are summarised in Table 1. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. .

There were 272 respondents (19.9%) who cancelled or postponed an in-person appointment with a health professional. The reasons for cancelling appointments were due to: concerns about catching COVID-19 at a clinic or hospital (n=85, 31.3%), isolating due to COVID-19 symptoms or risk (n=31, 11.4%), concerns about travelling on public transport (n=21, 7.7%), feeling too busy (n=20, 7.4%), cost (n=9, 3.3%), and other reasons (n=106, 39%). Less frequent reasons for cancelling or postponing an in-person appointment included: border closures, elective surgery postponed, and the appointment seemed non-essential. A total of 219 respondents (16%) felt they needed to see a health professional in-person in the last 4 weeks but chose not to go, due to: concerns about catching COVID-19 at a clinic or hospital (n=72, 32.9%), feeling too busy (n=37, 16.9%), isolating due to COVID-19 symptoms or risk (n=18, 8.2%), concerns about travelling on public transport (n=13, 5.9%), other reasons (n=67, 30.6%). Less common reasons listed for choosing not to see a health professional included: only telehealth available, limited in-person appointment availability, and feeling too complicated.

The characteristics of the telehealth users experience are shown in Table 2 . Of those who used telehealth, over half reported having more than one telehealth appointment (54.7%) most of which were delivered by telephone (72%). The majority of respondents (62%) rated their telehealth experience as "just as good" or "better" than a traditional in-person medical visit. On average, respondents perceived telehealth as moderately to very useful for medical appointments after the COVID-19 pandemic is over (M=3.67 out of 5, SD=1.1). Individuals who responded that their telehealth experience was worse than traditional in-person medical visits (34.4%) also rated the usefulness of telehealth after the COVID-19 emergency is over as significantly lower (2.86 vs 4.17, Mean difference=1.31, 95%CI: 1.14 to 1.47, t(572)=15.62, p<0.001) than those whose experience was just as good or better than an inperson visit.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. .

Multivariable analysis exploring factors associated with poorer telehealth experience compared to in-person appointments are displayed in Table 3 . Male gender (p=0.007), having a history of both depression and anxiety (p=0.037), or a lower patient activation score (p=0.037) were associated with a poorer telehealth experience after controlling for all other variables in the model.

Six overarching themes (outlined in Table 4 ) emerged from the free-text data; 1)

Communication is not as effective, 2) Limitations with technology, 3) Issues with obtaining prescriptions and pathology, 4) Reduced confidence in doctor, 5) Additional burden for complex care, and 6) Inability to be physically examined. Overall, the most recurrent theme was that communication was not as effective as traditional in-person visits, due to lack of visual cues, eye contact and body language.

Less than 2% of respondents reported not being able to access a telehealth service (n=19, 1.4%). The reported barriers were: telehealth was not available from their GP or health professional (n=4), they did not have internet (n=2), the appointment was not available when required (n=8) and that using telehealth felt too complicated (n=5).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. .

Our findings showed that more than half of the respondents rated their telehealth experience as just as good or better than traditional in-person medical care. This is encouraging considering that community transmission of COVID-19 across Australia may continue to persist for some time. Of those who used telehealth, most anticipated that telehealth would be at least somewhat useful after the COVID-19 pandemic is over, suggesting telehealth may be a viable long-term option for healthcare delivery. Our findings are consistent with a survey reporting that 85% of older Australians found their telehealth experience to be similar or better than face-to-face consults (15) . We found that telehealth modality (telephone vs video) was not associated with a poorer telehealth experience compared to in-person appointments, consistent with other studies showing telephone and videoconferencing to be comparable in terms of patient satisfaction (16) . Perhaps unsurprisingly, individuals who rated their telehealth experiences as worse were less likely to perceive that telehealth would be useful beyond the COVID-19 pandemic. Those who rated their telehealth experience as worse than traditional-in person visits were more likely to be male, have lower patient activation (individuals' willingness to take on the role of managing their health and healthcare), or have a history of both depression and anxiety. This last observation is also supported by the content analysis, which highlighted that traditional in-person visits were frequently preferred over telehealth visits for mental health appointments.

Our findings suggest that telehealth is less effective for mental health services. This is of concern as mental health problems such as depression and anxiety were at least twice as prevalent in the first month of the COVID-19 pandemic compared to before the pandemic in Australia (17), a problem that is only expected to grow (18). Our findings are similar to previous research on telehealth and mental health (19), which is concerning as negative experiences of telehealth may result in no mental health care for patients at all if face-to-face services are unavailable.

The most common theme for why respondents perceived telehealth to be worse than inperson medical care was less effective communication. This issue could be addressed by encouraging the use of established strategies to improve communication between health professionals and patients. For example, the teach-back method, also known as "show me" . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. . or "closing the loop", has shown to increase people's understanding of health information by asking the patient to repeat back health information in their own words (20). In addition, providing a written lay summary of the visit via a patient letter or patient portal may improve the telehealth experience for patients. Patients have reported improved patient-provider communication as a result of using a patient portal (21). Other approaches to address this issue may include online education or mobile applications, both of which have been used to enhance patient understanding of content and improve healthcare services (22, 23). It should be noted though that the Australian government is currently fast-tracking electronic prescribing which may improve communication between GP, patients and the pharmacist (24). This process allows easy electronic sharing of prescriptions, eliminating issues related to the challenge of obtaining prescriptions from telehealth appointments.

Additional issues identified in our study were that physical examination was not possible;

people were less confident in their doctor/healthcare professional during telehealth; and additional burden was experienced for complex health conditions. These issues could be addressed by setting clear expectations for telehealth when scheduling appointments and expectations about which types of appointments are suitable for telehealth. For example, when appointments are scheduled, patients should be notified that depending on the complexity of their medical appointment, additional in-person consults may be required, or that telehealth may not be appropriate for mental health issues. In addition, video conferencing could be offered as it may help with more reliable visual assessment and greater diagnostic accuracy (16) . Overall, in order to improve patients' experiences with telehealth, strategies should be implemented to ensure that patients are aware of what to expect from telehealth appointments. Importantly, our study identified that ~20% of respondents had cancelled or postponed an inperson health appointment, with the main reason for cancelling due to concerns about catching COVID-19 at health or hospital clinics. Our study identified a higher proportion of people who cancelled or postponed a health appointment than the survey conducted by the ABS (11%) (4) . Similarly, a study in Israel of 151 women with breast cancer found that 31% of people cancelled a health appointment, with the most common reason being due to fear of contacting COVID-19 (25). This is worrying as continuing outbreaks may deter patients from . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted September 11, 2020. . accessing essential in-person medical care for some time. Therefore, our results suggest that telehealth services should continue to be offered while community transmission of COVID-19 persists. Future studies should investigate whether patients who are cancelling or postponing health appointments are seeking telehealth services and monitor the long-term impact of health service utilization on health outcomes Whilst the study sample was large and diverse, it is not statistically representative of the Australian population, in particular including a higher proportion of females, a higher level of education, and potentially higher levels of digital literacy. In addition, our survey did not collect any information on the type of telehealth services that participants attended (e.g. allied health, specialist). Future surveys should investigate how people's experience with telehealth compares to traditional in-person visits depending on the health service type (e.g. GP, Specialist, Allied Health) and determine the impact of different health service modalities and utilization on health outcomes.

Overall, we found that telehealth was reported to be comparable to traditional in-person visits by most of our participants. We identified the most common reasons for a poor experience of telehealth and provided strategies which could help to improve the experience of telehealth users. Universal telehealth should continue to be offered as a mode of healthcare delivery while the pandemic continues and may be worthwhile beyond the pandemic. Shinan-Altman S, Levkovich I, Tavori G. Healthcare utilization among breast cancer patients during the COVID-19 outbreak. Palliat Support Care. 2020:1-7.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted September 11, 2020. .

Response Scale Since the COVID-19 restrictions started, have you had a telemedicine/telehealth appointment (appointment with your health provider by video or phone instead of an in-person appointment)? How many telehealth appointments have you had? Was/were your telemedicine/telehealth visit(s) done by:

Numerical (free-text) Telephone / Videoconference / Both

How did your telemedicine/telehealth visit compare to a traditional in-person medical visit?

If, telemedicine/telehealth was worse, please tell us why:

Better than traditional visit / Just as good as a traditional visit / Worse than a traditional visit / Not sure Free text

How useful do you think it will be to have medical appointments with telemedicine/telehealth after the COVID-19 emergency is over?

Have you cancelled or postponed an appointment with a health professional in the last 4 weeks because of COVID-19? Why?

Concerned about the cost / I am isolating due to COVID-19 symptoms or risk / I was worried about travelling on public transport because of the COVID-19 risk / I did not want to go to a health or hospital clinic because of concerns about catching COVID-19 there / Too busy / Other (please tell . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Concerned about the cost / I am isolating due to COVID-19 symptoms or risk / I was worried about travelling on public transport because of the COVID-19 risk / I did not want to go to a health or hospital clinic because of concerns about catching COVID-19 there / Too busy / Other (please tell us)

Have you needed to access a telehealth service in the last 4 weeks but could not? What was the main reason that you could not access a telehealth service in the last 4 weeks?

Telehealth not available from GP or other health professional / Do not have internet / I am not able to use the internet / dislike or fear of the service / appointment not available when required / Other (please detail) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. . Health Activation Index (CHAI); 0-79 low activation, 80-94 moderate activation, 95-100 high activation. eHealth literacy was measured on a 5-point Likert scale, a higher score reflects a higher level of eHealth literacy. Based on Index of Relative Socio-economic Advantage and Disadvantage (IRSAD) quintile (1 is most disadvantaged and 5 most advantaged).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. . . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. . Table 3 . Multivariable^ analysis of factors associated with poorer % telehealth experience compared to in-person appointment (n=574). Values displayed are relative risks (with 95% confidence intervals). Adjusted relative risks less than 1 indicate a reduced risk of reporting a poorer telehealth experience (relative to the reference group).

Adjusted relative risk (95% CI) .51) ^ analysis also controls for number of telehealth visits since lockdown. % outcome is defined as rating telehealth experience as worse (compared to just as good or better) than traditional in-person medical visit. Individuals who responded "unsure" (n=22) were excluded from the analysis.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. . . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020 *Full text could have more than one theme applied . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 11, 2020. .

The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence

Prime Minister of Australia. $1.1 billion to support more mental health, medicare and domestic violence services 2020

Centre for Online Health, The University of Queensland

Household Impacts of COVID-19 Survey

Utilization of non-Ebola health care services during Ebola outbreaks: a systematic review and meta-analysis

Health care utilization and health outcomes: a population study of Taiwan

Warnings GP surgeries could close as pandemic hits doctor income

Disparities in COVID-19 related knowledge, attitudes, beliefs and behaviours by health literacy

Quick assessment of literacy in primary care: the newest vital sign

eHEALS: The eHealth Literacy Scale

Development and Validation of the Consumer Health Activation Index

California: Sage: Thousand Oaks

The measurement of observer agreement for categorical data

Global Centre for Modern Ageing

Videoconference compared to telephone in healthcare delivery: A systematic review

We would like to acknowledge the members of the Australian public who participated in this survey.No funder had any role in the study.There are no conflicts of interest.All authors had full access to all of the data including statistical reports and tables.Words: 2479