Farmeco 2014;15(4)147-153.html
Farmeconomia. Health economics and therapeutic pathways 2014; 15(4): 147-153
http://dx.doi.org/10.7175/fe.v15i4.979
Original research
How do European patients feel about the listening skills of primary care physicians? Results from the EUprimecare Project
Carlos Alberto Sánchez Piedra 1, Sonia García-Pérez 1,2, Francisco Javier Prado-Galbarro 1, Antonio Sarría-Santamera 1,2
1 Agency for Health Technology Assessment, Instituto de Salud Carlos III, Madrid, Spain
2 Red de Investigación en Servicios, Red de Servicios de Salud Orientados a Enfermedades Crónicas (REDISECC), Madrid, Spain
Abstract
OBJECTIVE: EUprimecare is a study funded by the 7th Framework Programme of the European Union aimed at analyzing the quality of the different models of primary care (PC) in Europe. The objective of this study was to describe and analyze the determinants associated with patient satisfaction with the listening skills of their PC physicians.
METHODS: Telephone population survey in each EUprimecare consortium countries (Germany, Spain, Estonia, Finland, Hungary, Italy and Lithuania) among PC users. The questionnaire included sociodemographic variables, health status, use and satisfaction with PC services. The survey was conducted in 3020 patients. It was developed descriptive analysis, bivariate correlations and ordinal regression model to study the direct relation between levels of satisfaction and the explanatory variables on demographics, state and health services for patients. We show the regression coefficients (β) with 95% confidence interval and statistical significance associated.
RESULTS: We found significant relation between the level of satisfaction and age (β = 0.016), visits to specialist (β = -0.040), having a general practitioner (GP) (β = 0.619), having a chronic disease (β = 0.255), measuring weight, cholesterol and blood pressure (β = 0.650), countries (β1 Estonia= 0.938; β2 Germany = 0.469; β3 Lithuania = 0.483; β5 Italy = 0.544 and β6 Hungary = 1.010) and a better perception of health status (β = 0.388). Specialist visits have a negative influence with the higher degree of satisfaction.
CONCLUSIONS: Overall, the results indicate some areas that may be related to greater doctor-patient satisfaction. Different factors are converging to explain satisfaction with listening skills.
Keywords
Primary Health Care; Patient satisfaction; Health Care Surveys
Corresponding author
Carlos Alberto Sánchez Piedra
carlos.sanchez@isciii.es
Disclosure
The authors have no conflict of interest to report
Introduction
Primary Care (PC) is considered the basic structure of health care systems. PC has to provide whole-person care, which is longitudinal, comprehensive and coordinated care. Patient-centred care, characterized by continuous healing relationships, shared understanding, emotional support, trust, patient enablement and activation, and informed choices, has been identified as one of the main elements of high-quality care. The touchstone for developing successful patient-centred care is that patients and professionals develop an effective relationship. Patient-centred care seeks to increase health care providers’ understanding of patients’ individual needs, perspectives, and values; to give patients the information they need to participate in their care; and to build trust and understanding between physicians and patients. Listening is a central clinical function in building this relationship, and, therefore, in the delivery of high-quality health care [1,2].
Patient define medical skills as a mixture of technical knowledge and communication skills [3]. The quality of communication during consultation affects emotional health, resolution of symptoms, and some physiologic measures [4]. Empathy and use of open-ended questions are some examples of skilful communication, which increase patient involvement and adherence to recommended therapy; influence patient satisfaction, adherence, and health care utilization; and improve quality of care and health outcomes [5,6]. Difficult clinical encounters are associated with poor patient-doctor communication. These encounters often leave both physician and patient frustrated with its potential leading consequences in terms of clinical outcomes [7].
European countries show large variation in the specific structures of their PC systems. European PC services differ from each other as the result of different social, historical, economic and cultural factors. Therefore, it might be convenient to identify indicators which allow to compare systems and to explore the effect that those difference could have to improve quality and outcomes of the services provided, including in listening skills of PC professionals [8-12]. A better understanding of the potential modifiable determinants contributing to a higher satisfaction with listening skills could lead to more effective consultations, increase in adherence, better health outcomes, and avoiding difficult encounters.
The aim of this article is to study patient satisfaction with the listening skills of PC doctors in 7 European countries, and to explore the factors which could be associated with the level of satisfaction with listening skills in those countries.
Methods
Population survey and sample
EUprimecare is a study funded by the 7th Framework Programme of the European Union aimed at analyzing the quality of the different models of PC in Europe [13]. A population survey among users of PC services was conducted in seven countries in Europe. The final questionnaire included the variable satisfaction to assess the perception of patients with the quality of services in PC.
The data for this study comes from the population survey conducted in EUprimecare project in order to assess quality in PC in Europe. Citizens from Estonia, Lithuania, Hungary, Finland, Spain, Italy, and Germany were selected according to a stratified sampling plan developed for each country considering gender, age, household net monthly income, education level of householder, occupation of householder, and regions. Inclusion criteria were: having had at least one visit to a PC doctor in the previous year and being 18 years or older. Individuals were interviewed during April 2012. The final sample used for analysis in the study consisted of 3020 (1502 men and 1518 women) PC users selected in a random sample of seven countries. We conducted a multistage sampling design, with the first step based on regional division; a second stage based on randomly selected municipalities, and the last step selecting people between 25 and 75 years that in the last twelve months had visited a general practitioner or primary care physician for consulting about a problem, pain or illness. During sampling we took into account the age and gender ratios considering the distribution of these variables in the population of each country.
A questionnaire with clinical and no clinical indicators was developed by consensus of researchers from the EUprimecare partners. The dimensions of quality were developed in a process which started from focus groups both with patients and with professionals. The questionnaire was designed in English. Afterwards, translated into the language of the participating countries using a dual focus method [14] that aims at conceptual equivalence, besides dealing with grammar and wording aspects. To ensure internal validity and comprehensibility of wording, the instrument was piloted on 25 individuals in each country and some variations were done to refine the final instrument. The final version of the questionnaire included 24 items. The survey questionnaire included variables related to socio-demographic, health status and health services utilization.
Data were collected through Computer-Assisted Telephone Interview (CATI). Random digit dialing in strata ensured a randomized representative sample of the seven countries populations. With this procedure, sampling continues until a previously defined number of complete interviews are achieved. Therefore a country-dependent number of calls were made until having approximately 431 complete interviews from all countries. In total, 3020 questionnaires were completed, corresponding to an average response rate of 15.98%.
Users of PC services were investigated regarding the use of health care services (GP, specialists, others), their health status, and access to specific services: counselling/health education (tobacco, diet, exercise), control of chronic diseases and their perception of Quality of PC through patient satisfaction.
The survey questionnaire included variables related to socio-demographic, health status and health services utilization.
Statistical analysis
We developed descriptive analysis, using bivariate correlations and ordinal regression analysis to model the association between satisfaction and the possible explanatory variables. In order to evaluate listening skills, EUprimecare researchers agreed to use the proxy variable “Listening skills of your PC doctor” which could be answered in a scale from 1 through 5, meaning 1 Very Dissatisfied and 5 Totally Dissatisfied. For this work, the values of this variable were recorded into 3 categories, low (1 and 2), mild (3), and high (4 and 5).
The independent variables were also items from the questionnaire. The following quantitative characteristics were studied: number of visits to PC, number of visits to the specialist and age. The qualitative variables were sex, self-perceived health, rural residence, and activities conducted in PC during the last year (measuring blood pressure, weight and height) during the last year, chronic disease diagnosis and countries.
First, univariate ordinal logistic regression models were conducted to identify the effect of each independent variable on satisfaction. Second, multivariate analysis using ordinal regression was used to establish the independent effect of patient characteristics associated with patient satisfaction with services provides in PC. The model includes the regression coefficients (B) by the 95% confidence interval and statistical significance associated. In the ordinal regression analysis Spain was entered as reference category as the value of crude satisfaction with lower satisfaction.
Results
A total of 3,020 patients using PC services responded to the telephone survey. Table I summarized baseline characteristics including demographic, socio-economic and health characteristics.
Variable
Listening skill satisfaction
p
Satisfied (n. = 2,255)
Mild (n. = 560)
Disatisfied (n. = 167)
Age
<0.001
Mean
51.45
47.52
48.52
SE
14.15
13.75
13.39
Visits to the GP
0.370
Mean
4.34
3.95
4.52
SE
5.47
4.08
5.64
Visits to the specialist doctor
0.145
Mean
1.83
1.89
2.45
SE
3.32
3.24
3.52
Country (%)
<0.001
Spain
74.36
21.12
4.43
Estonia
88.71
8.94
2.35
Germany
84.42
11.16
4.42
Lithuania
82.94
13.08
3.97
Italy
86.01
10.26
3.73
Hungary
88.34
9.56
2.10
Finland
76.40
16.82
6.78
Gender (% man)
49.82
48.97
51.26
0.902
Rural residence (%)
35.80
38.20
36.13
0.655
Do you consider you have a GP? (% yes)
93.45
88.46
79.83
<0.001
Your health is good or bad (% fair, good or very good)
92.34
92.29
88.98
0.412
Do you have a chronic disease? (% yes)
51.51
47.69
52.10
0.364
Measuring blood pressure, weight, height during the last year (%)
78.46
63.33
59.66
<0.001
Table I. Baseline characteristics
The average age was 51 years (SD: 14.1). More than three quarters of participants were satisfied with listening skills of their GPs (83.02%). Table II shows baseline characteristics depending on country. The lowest level of satisfaction with listening skills of GP was found in Spain (74.36%), whereas the opposite was found in Estonia were 88.71% of patients claim to be satisfied.
Bivariate analysis of the selected variables showed their relationship with the listening skills satisfaction (Table III).
Variable
Germany
Lithuania
Finland
Spain
Estonia
Hungary
Italy
Total
P
Age
<0.001
Mean
50.35
48.13
50.70
48.48
53.84
48.37
56.04
50.85
SE
14.28
13.85
13.79
14.86
13.96
14.33
11.80
14.14
Visits to the GP
<0.001
Mean
3.88
4.67
2.73
3.96
3.50
5.73
5.53
4.29
SE
5.29
6.12
3.16
4.26
3.67
7.13
5.72
5.31
Visits to the specialist doctor
<0.001
Mean
1.64
2.49
1.12
1.27
2.13
2.84
1.54
1.86
SE
2.63
3.71
2.02
2.58
3.80
4.56
2.83
3.31
Listening skills satisfaction (%)
<0.001
Dissatisfied
4.42
3.97
6.78
4.43
2.35
2.10
3.73
3.94
Mild
11.16
13.08
16.82
21.12
8.94
9.56
10.26
12.91
Satisfied
84.42
82.94
76.40
74.36
88.71
88.34
86.01
83.02
Gender (% man)
49.88
49.88
49.77
48.72
50.00
49.88
50.00
49.74
0.212
Rural residence (%)
28.07
29.93
34.72
57.08
36.11
43.16
24.77
36.26
<0.001
Do you consider you have a GP? (% yes)
100
98.84
48.61
100
99.54
98.38
100
92.20
<0.001
Your health is good or bad (% fair, good or very good)
93.50
88.84
96.52
95.58
85.71
89.74
94.68
92.10
<0.001
Do you have a chronic disease (% yes)
44.32
63.81
53.24
36.89
57.18
59.86
41.90
51.00
<0.001
Measuring blood pressure, weight, height during the last year (%)
80.51
87.01
63.89
65.66
81.94
77.03
73.61
75.70
<0.001
Table II. Characteristics depending on country
Variables
OR
CI 95%
Wald
OR
p
Age
0.018
0.011 – 0.025
27.758
1.018
<0.001
Visits to GP
0.009
-0.010 – 0.029
0.862
1.009
0.012
Visits to specialist
-0.017
-0.044 – -0.009
1.655
0.983
0.013
Sex: Men
0.009
-0.181 – 0.199
0.009
1.009
0.389
City/town (Ref: city)
-0.079
-0.275 – 0.118
0.620
0.924
0.013
Do you consider you have a GP?: Yes
-2.434
0.550 – 1.138
232.287
1.540
0.004
Health status: Fair, good, very good
0.127
-0.216 – 0.470
0.529
1.135
0.490
Measuring weight, cholesterol and blood pressure during the last year
0.783
0.581 – 0.985
57.680
2.188
<0.001
Chronic disease: no
-0.017
-0.294 – 0.086
1.155
0.901
0.387
Table III. Univariate analysis
CI 95% = confidence interval 95%; OR = odds ratio
Satisfaction showed statistically significant and positive associations with age; visits to the GP; visits to the specialist; residence (city/town); if patient consider he/she had a GP; and the measurement of different clinical parameters in PC during the previous year. Sex, health self-perception and having being diagnosed of a chronic disease had no statistically significant effect on satisfaction.
We studied the effect of independent variables selected on satisfaction in a multivariate model. The reference category for the dependent variable of “Listening skills of your PC doctor” was higher satisfaction, all Odds Ratio were expressed in relation to this category. Table IV presents the ordinal logistic regression results.
Variables
β
Wald
p
OR
CI 95%
Threshold (Ref. Satisfied)
Dissatisfied
-0.513
4.608
0.025
0.599
0.271 – 0.756
Mild
1.162
8.463
0.004
3.196
0.884 – 3.713
Location
Age
0.016
15.910
<0.001
1.016
1.008 – 1.024
Visits to GP
0.001
0.016
0.900
1.001
0.981 – 1.022
Visits to specialist
-0.040
7.640
0.006
0.961
0.934 – 0.988
Sex: Men
0.053
0.272
0.602
1.054
0.865 – 1.284
City/town (Ref: city)
-0.027
0.063
0.801
0.973
0.788 – 1.202
Do you consider you have a GP?: Yes (Ref: No)
0.619
7.652
0.006
1.857
1.197 – 2.878
Health status: Fair, good, very good
0.388
4.101
0.043
1.474
1.012 – 2.147
Measuring weight, cholesterol and blood pressure during the last year (Ref: No)
0.650
32.232
<0.001
1.916
1.531 – 2.396
Chronic disease: yes (Ref: No)
0.255
4.991
0.025
1.290
1.031 – 1.613
Country (Ref: Spain)
Estonia
0.938
20.820
<0.001
2.555
1.707 – 3.823
Germany
0.469
6.869
0.009
1.598
1.125 – 2.270
Finland
0.384
3.183
0.074
1.468
0.963 – 2.237
Lithuania
0.483
7.102
0.008
1.621
1.136 – 2.237
Italy
0.544
8.446
0.004
1.723
1.193- 2.484
Hungary
1.010
26.640
<0.001
2.746
1.870 – 4.027
Table IV. Ordinal regression results (R2 Nalgerkerke= 0.07; Chi2 Model: 131.155; gl: 15; p: <0.001; Parallel lines test: p=0.511; Goodness-of-fit: 0.730)
There was a positive and statistically significant effect of respondent’s age (β = 0.016), visits to specialists (β = -0.040), having a doctor that considered as his GP (β = 0.619), control of weight, cholesterol and blood pressure (β = 0.650), having a chronic disease (β = 0.255), countries (β1 Estonia = 0.938; β2 Germany = 0.469; β3 Lithuania= 0.483; β5 Italy = 0.544 and β6 Hungary= 1.010). Estonian and Hungarian patients are more likely to be classified as more satisfied with listening skills of GP. Visits to specialists had a negative influence on patient satisfaction. The model fits the parallel lines test.
Discussion
Results from this study indicate that, overall, levels of satisfaction with the listening skills of PC physicians were high (83.02%). These data also show also the existence of differences in the quality of perceived listening skills across the different countries included in the analysis, as well as the association between patient satisfaction with GP listening skills and individual factors such as older age or better health status perception, or variables related to organizational characteristics or standard practice in PC.
Variables related to doctor-patient relationship had a strong impact on the satisfaction results. Having a reference GP doctor is one of the most relevant variables that seem to be influencing satisfaction with listening skills of PC physicians. Long-term relationships facilitate to consolidate effective communication with a PC doctor as well as a better knowledge of patient’s medical history can. Patients who have a consistent long-term relationship with a single physician [15,16] were more satisfied than those who are seen by different doctors [17], reflecting the importance of having a identified PC doctor to integrate their overall care valuing to initiate care for new medical problems with these physicians.
The experience of being referred to a specialist is a significant issue for patients. A specialist referral requires coordination and continuity of care, some of the dimensions of patient-centered care. A referral to a specialist could be seen by patients as a failure of PC to resolve complex problems [18], because of a lack of clarity about reasons for referrals or unclear follow-up plans. Referrals have been identified as strong predictors of patients giving their PC physicians low trust, confidence, and satisfaction ratings [19]. On the other hand, patients with a higher intensity of tests and procedures received in PC, have a better perception of listening skills of their PC physicians.
Our data shows that patients with chronic diseases indicate that they have a higher value higher findings are similar to other studies that show the importance of good communication to treat chronic diseases [3,20,21]. The closer relationship that patient with chronic diseases tend to establish with their GP could explain these results, something essential regarding shared decisions in PC.
The results of the study suggest a potential implication of organizational factors of each country to explain part of the variance in listening skill satisfaction. A wide range of factors are involved in the possible equation to decode the keys to understand the relation between listening skills and satisfaction with PC services. Assessing listening skills depends on individual factors, but there are also variables that are not usually measured such as the capabilities of every physician, their training or even the expectations of each individual patient that could be influencing the results.
Despite all the evidence available, the impact of improving listening skills is still controversial. Our results differ from Jabaaij L et al., who did not find any specific influence related to GP patient familiarity [22].
After all, we have to consider if this information could be useful in order to introduce changes and improvements. A recent systematic review concluded that there is insufficient evidence to determine the effectiveness of interventions aim to improve communication skills and trust in doctors [23]. On the other hand, another systematic review establish the basis for a better understanding about doctor-patient communication, considering proximal, intermediate, and long-term consultation outcomes [24]. The interpretation and management of communication skills is still under consideration.
The strength of this study lies in the use of a stratified sample of users of PC services in seven countries with data collected through a questionnaire developed from focus groups with patients and professionals. To our knowledge there are no studies based on PC patients in Europe that evaluate determinants associated to patient-provider communication from the perspective of “Listening skills”.
Results are based on data from seven countries and the study is novel both in its aim and dimension. However, it does not cover all European countries. Due to language and logistics aspects the survey was conducted by an outside company, which made it difficult controlling for potential selection bias. Limitations of this study include the response rate of the questionnaire. The sampling process was designed considering a final sample representative of the total population. However, the sample size and the low response rate could limit the representativeness of these data. Moreover, “listening skills” could be representing only a minor component of the broad concept of patient-provider communication. Conclusions and comparisons with other studies could be affected by this consideration. Finally, we have no information to study cultural expectations that could explain satisfaction in terms of listening skills [25-27].
These results suggest that patient satisfaction with GP listening skills seem to be a combination of different factors, and, consequently, could provide information to address patient-physician relationship from new perspectives. Organizational characteristics of each PC system may be a key variable to understand the relationship between patient and GP. However, other modifiable variables like having a reference PC doctor or even the measurement different clinical parameters regularly (weight, cholesterol and blood pressure) have a relevant contribution to explain patient satisfaction with listening skills.
References
1. Epstein RM. Whole mind and shared mind in clinical decision-making. Patient Educ Couns 2013; 90: 200-6; http://dx.doi.org/10.1016/j.pec.2012.06.035
2. Epstein RM, Street RL, Jr. Shared mind: communication, decision making, and autonomy in serious illness. Ann Fam Med 2011; 9: 454-61; http://dx.doi.org/10.1370/afm.1301
3. Prieto Rodriguez MA, Danet Danet A, Escudero Carretero MJ, et al. Definition of medical competence. The point of view of chronically-ill patients in the Andalusian public healthcare system (Spain). Gac Sanit 2012; 26: 450-6; http://dx.doi.org/10.1016/j.gaceta.2012.02.007
4. Stewart MA. Effective physician-patient communication and health outcomes: a review. CMAJ 1995; 152: 1423-33
5. Zolnierek KB, Dimatteo MR. Physician communication and patient adherence to treatment: a meta-analysis. Med Care 2009; 47: 826-34; http://dx.doi.org/10.1097/MLR.0b013e31819a5acc
6. Levinson W, Lesser CS, Epstein RM. Developing physician communication skills for patient-centered care. Health Aff (Millwood) 2010; 29: 1310-8; http://dx.doi.org/10.1377/hlthaff.2009.0450
7. Cannarella Lorenzetti R, Jacques CH, Donovan C, et al. Managing difficult encounters: understanding physician, patient, and situational factors. Am Fam Physician 2013; 87: 419-25
8. Schafer WL, Boerma WG, Kringos DS, et al. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Fam Pract 2011; 12: 115; http://dx.doi.org/10.1186/1471-2296-12-115
9. Schafer WL, Boerma WG, Kringos DS, et al. Measures of quality, costs and equity in primary health care instruments developed to analyse and compare primary care in 35 countries. Qual Prim Car 2013; 21: 67-79
10. Sanchez-Piedra CA, Prado-Galbarro FJ, Garcia-Perez S, et al. Factors associated with patient satisfaction with primary care in Europe: results from the EUprimecare project. Qual Prim Care 2014; 22: 147-55
11. Garcia-Perez S, Sanchez-Piedra CA, Sarria-Santamera A. Patient-centred care in Spain: there is still work to be done. Aten Primaria 2013; 45: 333-4; http://dx.doi.org/10.1016/j.aprim.2012.12.012
12. Mira JJ, Aranaz J. Patient satisfaction as an outcome measure in health care. Med Clin (Barc) 2000; 114 Suppl 3: 26-33
13. EUprimecare: Quality and Cost of Primary Care in Europe Available at http://www.euprimecare.com
14. Erkut S, Alarcon O, Garcia Coll C, et al. The Dual-Focus Approach to Creating Bilingual Measures. J Cross Cult Psychol 1999; 30: 206-18; http://dx.doi.org/10.1177/0022022199030002004
15. Wun YT, Lam TP, Goldberg D, et al. Reasons for preferring a primary care physician for care if depressed. Fam Med 2011; 43: 344-50
16. Rincon-Gomez M, Bernabeu-Wittel M, Bohorquez-Colombo P, et al. Perceived quality of healthcare in a multicenter, community-based population of polypathological patients. Arch Gerontol Geriatr 2011; 52: 142-6; http://dx.doi.org/10.1016/j.archger.2010.03.003
17. Ha JF, Longnecker N. Doctor-patient communication: a review. Ochsner J 2010; 10: 38-43
18. Maeng DD, Martsolf GR, Scanlon DP. Care coordination for the chronically ill: understanding the patient’s perspective. Health Serv Res 2012; 47: 1960-79; http://dx.doi.org/10.1111/j.1475-6773.2012.01405.x
19. Grumbach K, Selby JV, Damberg C, et al. Resolving the gatekeeper conundrum: what patients value in primary care and referrals to specialists. JAMA 1999; 282: 261-6; http://dx.doi.org/10.1001/jama.282.3.261
20. Infante FA, Proudfoot JG, Powell Davies G, et al. How people with chronic illnesses view their care in general practice: a qualitative study. Med J Aust 2004; 181: 70-3
21. Hirsh AT, Atchison JW, Berger JJ, et al. Patient satisfaction with treatment for chronic pain: predictors and relationship to compliance. Clin J Pain 2005; 21: 302-10; http://dx.doi.org/10.1097/01.ajp.0000113057.92184.90
22. Jabaaij L, Fassaert T, van Dulmen S, et al. Familiarity between patient and general practitioner does not influence the content of the consultation. BMC Fam Pract 2008; 9: 51; http://dx.doi.org/10.1186/1471-2296-9-51
23. Rolfe A, Cash-Gibson L, Car J, et al. Interventions for improving patients’ trust in doctors and groups of doctors. Cochrane Database Syst Rev 2014; 3: CD004134; http://dx.doi.org/10.1002/14651858.CD004134.pub3
24. Labrie N, Schulz PJ. Does argumentation matter? A systematic literature review on the role of argumentation in doctor-patient communication. Health Commun 2014; 29: 996-1008; http://dx.doi.org/10.1080/10410236.2013.829018
25. DeVoe JE, Wallace LS, Fryer GE, Jr. Measuring patients’ perceptions of communication with healthcare providers: do differences in demographic and socioeconomic characteristics matter? Health Expect 2009; 12: 70-80; http://dx.doi.org/10.1111/j.1369-7625.2008.00516.x
26. Cooper-Patrick L, Gallo JJ, Gonzales JJ, et al. Race, gender, and partnership in the patient-physician relationship. JAMA 1999; 282: 583-9; http://dx.doi.org/10.1001/jama.282.6.583
27. Blair IV, Steiner JF, Fairclough DL, Hanratty R, Price DW, Hirsh HK, et al. Clinicians’ implicit ethnic/racial bias and perceptions of care among Black and Latino patients. Ann Fam Med 2013; 11: 43-52; http://dx.doi.org/10.1370/afm.1442