16 F is io te ra p ie , F e b ru a rie 1 9 8 6 , d e e l 4 2 n o 1 Sampling Bias in Physiotherapy Research J. A. HEN D RY SUMMARY In this article the difference between random j error (due to sampling variability) and syste­ matic error (or bias) is briefly illustrated. Major j types of systematic error which o ccur due to j errors in sampling are discussed briefly with reference to the following study designs: the randomized clinical trial, cohort analytical-, cross-sectional, case control and before-after studies. B o th q u a lifie d an d s tu d e n t p h y sio th e ra p ists c o n d u c tin g o r p la n n in g research p ro jects sh o u ld be aw are o f the m a n y fo rm s o f sy stem atic e rr o r o r bias w hich m ay o c c u r a t a n y stag e o f th e research process, th e re b y th re a te n in g th e v a lid ity o f th e clinical o b se rv a tio n s and fin d in g s. A n aw areness o f th e p o te n tia l biases th a t can occu r d u rin g p a tie n t tre a tm e n t, o r d u rin g th e e v a lu a tio n of th e efficacy and effectiveness o f p h y sio th e ra p y m o d a ­ lities an d tr e a tm e n t regim ens is also o f value to th e clinically o rie n te d p h y sio th e ra p ist, as th is m ay fo ster a m ore discerning an d critical a ttitu d e to w a rd s th e existing lite ra tu re an d c u rre n t clinical practice. R e se a rc h d a ta is u su ally o b ta in e d o n a sam ple o f p a tie n ts w ith th e disease, d is a b ility o r c h a ra c te ristic o f in te re st. S eld o m is th e en tire p o p u la tio n o f in terest stu d ied . H ow ever, d u e to b iologic v a ria tio n am o n g in d iv id u als, a n d ch an ce fa c to rs in th e sam p lin g process, o b se rv a tio n s on a sam p le never c o rre sp o n d ex actly to th e tr u e p o p u la tio n v alu e. T his ra n d o m e rr o r d u e to sa m p lin g v a ria b ility can never be to ta lly elim in ated , th o u g h it m ay be e stim a te d by sta tistic a l p ro ced u res an d m inim ised th r o u g h p ro p e r research d e s ig n .1 A m o re im p o rta n t an d in sid io u s fo rm o f e rr o r th a t m a y o c c u r is sy ste m a tic e r r o r o r b ias. Bias gives rise to c o n siste n t discrepancies betw een th e tru e p o p u la tio n v a lu e an d t h a t a c tu a lly o b ta in e d a n d is d u e to all J. A. Hendry, B.A., N at. Dipl, in Physiotherapy, T T H D . Lecturer, D ep artm en t of P hysiotherapy, U niversity of Stellen­ bosch OPSOMMING Hierdie artikel illustreer kortliks die verskil tussen die begrippe ‘ewekansige of toevalsfout' (as gevolg van variabiliteit in steekproefneming) en ‘sistematiese fout’ of sydigheid. Voorbeelde van sydigheid wat ontstaan as gevolg van fou- tiewe steekproefneming word bespreek met verwysing na die volgende studie strukture: die ewekansige kliniese proef, kohort anali- tiese-, dwarsnit-, geval kontrole- en voor-na studies. causes o th e r th a n sa m p lin g v a ria b ility .2 A sy stem atic e rr o r o f sufficient m a g n itu d e m ay d is to rt a s tu d y ’s co n clu sio n s in a clinically im p o rta n t w ay an d severely d a m a g e th e re s e a rc h e r’s credibility. T h ese tw o m a jo r so u rces o f e rr o r are n o t m u tu ally exclusive an d in m o st in stan ces occu r sim u ltan eo u sly to a greater or lesser degree. P h y sio th erap y research centres larg ely on th e ra p e u tic tria ls w hich a tte m p t to ev alu a te p h y s i o th e r a p y m o d a litie s a n d t r e a t m e n t re g im e n s. Because o f the wide v a ria tio n s in th e in d iv id u al p a tie n t’s responses to tre a tm e n t, large sam ples are o ften required. R a n d o m e rr o r (sam p lin g v aria b ility ) is th e re b y m in i­ m ised. H o w ev er, if serio u s fo rm s o f sy ste m a tic e rro r are p re s e n t in th e re s e a rc h d esig n an d e x e c u tio n , thiS| bias o n ly increases in m a g n itu d e w hen th e sam p le size is increased! Bias can o c c u r a t an y stage o f th e research process: d u rin g th e lite ra tu re review , in selecting a n d specifying th e s a m p le , d u r i n g th e e x e c u t i o n o f th e c lin ic a l m an o e u v re , d u rin g m ea su re m e n t o f th e o u tc o m e , in d a ta analysis an d in te rp re ta tio n an d finally, in the p u b lic a tio n o f th e re s u lts .3 T his article d eals o n ly w ith th e m a jo r fo rm s o f bias th a t c a n o c c u r d u e to e rro rs in selecting a n d specifying th e stu d y sam ple. R a n d o m iz e d c lin ic a l tr ia ls a n d c o h o r t a n a ly tic a l stu d ie s ra n k highest in th e h ie ra rc h y o f re se a rc h stu d y d e sig n s. N e v e rth e le s s , if o b s e r v a tio n s are m a d e on g ro u p s o f p a tie n ts th a t a re to ta lly in c o m p a ra b le o r th a t have been selected in c o rre c tly , serious sa m p lin g bias m ay o c c u r.4,5 V ario u s fo rm s o f bias th a t m ay o ccu r are: R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 01 3. ) p h y s io th e ra p y , F e b ru a ry 1 9 8 6 , vo l 4 2 n o 1 17 1. V olunteer bias3'6 M a n y p h y sio th e ra p y studies use v o lu n teers as sam ple su b jects, larg ely fo r e th ic a l re a s o n s. H o w ev er, m a n y stu d ies have sh o w n v o lu n te e rs to d iffer sy stem atically from n o n -re sp o n d e n ts ,6 in th a t th e y ten d to be h e a lth ie r an d m o re c o m p lia n t. T h e in ferences w hich can be m ade fro m th e resu lts a re lim ited in a p p lic a tio n referrin g o n ly to th e efficacy o f th e tr e a tm e n t a n d n o t its effec­ tiveness in th e tru e clinical s itu a tio n , w here n o t all p a tie n ts will be eq u ally m o tiv a te d o r co m p lian t. 2. Procedure selection bias3,6 T his m ay o c c u r in th e a llo c a tio n o f p a tie n ts to ce rta in clinical p ro c e d u re s o r tr e a tm e n ts . A certain tre a tm e n t m ay be p referen tially offered to th o se p a tie n ts w ho are co n sid ered h ig h r i s k /p o o r p ro g n o sis o r a lte rn a ­ tively low r is k /g o o d p ro g n o sis. T h e re s u lta n t a p p a re n t efficacy o f o n e tr e a tm e n t over a n o th e r m ay be d u e to sy stem atic d ifferen ces in th e d eg ree o f h e a lth betw een th e tw o c o h o rts. E x a m p le s are th e a llo c a tio n o f ce rta in p a tie n ts to m ed ical vs. su rg ical th e r a p y o r th e selection o f p a tie n ts fo r p h y s io th e ra p y exercise regim ens (p o s t m y o c a r d ia l i n f a r c t i o n / c h r o n i c o b s t r u c ti v e a irw a y s disease). 3. D iagn ostic vogue bias3 S am p le subjects sh o u ld be selected acco rd in g to p re ­ selected c rite ria . D ia g n o stic labels sh o u ld be clearly specified an d d iag n o ses sh o u ld b e co n firm e d by several sources to av o id m isclassificatio n o f sa m p le subjects. T h e sam e illness m ay receive d iffe re n t d ia g n o s tic labels at d iffe re n t stages o r in d iffe re n t g e o g ra p h ic reg io n s. A c o m m o n ex a m p le is th e B ritish “ b ro n c h itis ” vs. th e N o rth A m erican “e m p h y se m a ”.3,6 4. M em bership bias3 M e m b e rsh ip o f a g ro u p (e.g. jo g g e rs, th e e m p lo y ed ) m ay im ply a degree o f health w hich differs system atically fro m th a t o f th e g en eral p o p u la tio n . T h e re s e a rc h e r m ay choose to select a h o m o g en o u s sam ple in preference to a h e te ro g e n o u s sam p le, b u t s h o u ld th e n realise th a t he is m o re lim ited in generalising a b o u t th e fin d in g s.2 5. M igration bias4/lo s s o f sam ple subjects bias3,5 In n e a rly all stu d ies so m e m em b ers o f th e o rig in al c o h o rt v o lu n ta rily d ro p o u t o f th e stu d y , are w ith d raw n (fo r a v a rie ty o f reaso n s) o r are lo st to fo llo w -u p . T o ta l o u tco m e re p o rtin g on all su b jects, at th e end o f th e study, is essential. Loss o f subjects w ith e q u a l freq u en cy in b o th c o h o rts in tro d u c e s n o bias. H o w ev er, th is seldom h a p p e n s. U n d e rly in g re a s o n s fo r p a tie n t w ith ­ d ra w a l/lo s s o f subjects are o ften related to th e o u tco m e of prognosis. It is th erefo re necessary to o b ta in w hatever in fo rm a tio n is possible on th e ch a ra c te ristic s o f these p atien ts. It is also a d v isab le to select a ra n d o m sam ple o f w ith d ra w a ls fo r in ten siv e fo llo w -u p to a sc e rta in w hether system atic differences ex ist betw een th e p atien ts re m a in in g in th e stu d y a n d th o s e w ho are w ith d ra w n / lost to th e in itial follow -up. T h e in v e s tig a to r sh o u ld be co n serv ativ e an d assum e th e w o rst possible o u tco m e f o r these p a tie n ts w hen a n a ly sin g th e re s u lts .4,6 A lte r­ n ativ ely a b ro a d e s tim a te o f th e effects o f these g ro u p s o f p a tie n ts on the o v erall fin d in g s m ay be c a lc u la te d by d e te rm in in g th e tw o e x trem es o f a ran g e — one based on th e w o rst p o ssib le o u tc o m e an d o n e based o n the b est p o ssib le o u tc o m e .7 L oss o f sam ple o b jects w hich affects o ne c o h o rt m o re th a n th e o th e r m ay in tro d u c e a serio u s fo rm o f bias, as c o h o rts w hich w ere c o m p a ra b le at the o u tset o f the stu d y b eco m e less so as tim e passes. 6. N on-respondent b ias3,5,7 T h is is th e a n tith e s is o f v o lu n te e r bias. A m in im u m o f 80% in th e re sp o n se ra te is re q u ire d fo r resu lts to be re g a rd e d as valid. (C o c h ra n ’s ru le ).7 A s in th e prev io u s e x a m p le , sy ste m a tic d ifferen ces betw een n o n -re s p o n ­ d en ts and resp o n d en ts should be ascertained by selection o f a ra n d o m sam p le fo r intensive fo llo w -u p .4 7. L ead-tim e b ias/startin g tim e bias2,4,6 U n d e rly in g g r o u p d iffe re n c e s s h o u ld alw ay s be se a rc h e d fo r w hen n o n -r a n d o m c o n tr o l and e x p e ri­ m e n ta l g ro u p s are u sed. S y stem atic d ifferen ces betw een c o h o rts could occu r d u e to • d ifferen ces in th e e x te n t/s e v e rity o f th e disease (e.g. gr. I vs. gr. IV d y sp n o ea) • th e presence o f o th e r diseases (co n fo u n d in g variables) • differen ces in tim e in th e co u rse o f th e disease (o r tre a tm e n t o f th e disease). F a ilu re to id en tify a c o m m o n s ta rtin g tim e fo r the illness u n d e r in v e stig a tio n o r th e tre a tm e n t being e v a lu ­ ated m ay lead to e rro n e o u s co n c lu sio n s reg a rd in g the b enefit o f th e ra p y , e.g. p a r t o f the a p p a r e n t im p ro v e ­ m en t in in -h o sp ita l m o rta lity rates fro m m y o card ial in fa rc tio n e x p erien ced by p a tie n ts in c o ro n a ry care u n its m ay be re la te d to the fa c t th a t m a n y h e a rt-a tta c k victim s die sh o rtly afte r o n se t o f the a tta c k , while p a tie n ts in c o ro n a r y c are u n its have a lre a d y survived th e s h o r t d elay b etw een a d m issio n to th e h o sp ita l an d a d m issio n to th e u n it .1 8. Other S ev eral m a jo r m e th o d o lo g ic a l p ro b le m s m ay arise in h o sp ital- (o r p riv ate p ractice) b ased stu d ies, p a rtic u la rly w ith regard to p a tie n t selection: T h e ad m issio n of p a tie n ts to c e rta in in s titu tio n s m ay be influenced by the in te re st stirred up by th e p resen tin g c o n d itio n (p o p u ­ la rity bias). D ia g n o stic o r th e ra p e u tic access bias m ay o ccur, as individuals differ in th e ir geo g rap h ic, te m p o ra l and econom ic access to various diagnostic or th erap eu tic procedures. Sim ilarly, the re p u ta tio n o f certain clinicians o r p h y sio th e ra p ists m ay cause in d iv id u als w ith specific d iso rd e rs, to g ra v ita te to w a rd s th em (cen trip etal bias). T h e re a d e r is re ferred to the stu d y by O re n s te in ,8 as it p ro v id es several excellent ex am p les o f sy stem atic e rro rs in p a tie n t sam p lin g . C a s e -c o n tro l stu d ie s an d cro ss-se c tio n a l a n aly tic s u r­ veys are stu d y designs w hich are b eco m in g in creasin g ly p o p u la r as tim e, co st a n d eth ical p ro b lem s are m inim al. M a tc h in g cases an d c o n tro ls fo r facto rs such as ag e/ R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 01 3. ) 18 F is io te ra p ie , F e b ru a rie 1986, d e e l 42 n o 1 s e x / r a c e is c o m m o n p r a c t ic e , a s t h es e a r e o f t e n s t r o n g l y r e l a t e d t o d i s e a s e p r o g n o s i s . H o w e v e r , t h e m a t c h i n g p r o c e s s o n l y c o n t r o l s f o r b ia s f o r t h e s e f a c t o r s t a k e n i n t o a c c o u n t a n d o f w h i c h th e r e s e a r c h e r is a w a r e . 1,2A5'6 T h e d a n g e r o f o v e r - m a t c h i n g m a y a l s o o c c u r 1 r e s u l t i n g in m a s k i n g o f i m p o r t a n t d i f f e r e n c e s b e t w e e n th e t w o g r o u p s w i t h r e g a r d t o t h e c h a r a c t e r i s t i c o f in te r e s t . F i n d i n g c o n t r o l p a t i e n t s w h o m e e t all m a t c h i n g c r i t e r i a m a y a l s o p r e s e n t m a j o r p r a c t i c a l d i f f ic ul tie s . It is t h e r e ­ f o r e f r e q u e n t l y h e lp f u l t o h a v e a d i a g n o s t i c a l l y h e t e r o ­ g e n o u s c o n t r o l g r o u p a n d w h e r e p o s s ib l e m o r e t h a n o n e c o n t r o l g r o u p i.e. o n e d r a w n f r o m th e s a m e m e d i c a l f a ci lit y a n d o n e d r a w n f r o m o u t s i d e th e fa cili ty ( n e i g h ­ b o u r s , f e l l o w - e m p l o y e e s , f a m i l y o r f r i e n d s ) . 1 T h e m a j o r f o r m o f b i a s e n c o u n t e r e d in c a s e - c o n t r o l a n d c r o s s - s e c t i o n a l s t u d i e s , is 9. P rev alen ce — in cid en ce (N ey m an ) b ias3 S a c k e t t d e f i n e d t h is a s “ a l a t e l o o k a t t h o s e e x p o s e d ( o r a f fe c t e d ) e a r l y will m is s f a ta l a n d o t h e r s h o r t e p i ­ s o d e s p l u s m i l d o r s i l e n t c a s e s . . ( t h e r e a d e r is a g a i n r e f e r r e d t o t h e e x a m p l e u n d e r 7) e.g. a r e t r o s p e c t i v e i n v e s t i g a t i o n i n t o t h e f r e q u e n c y o f s o f t - t i s s u e in jur ie s a m o n g a t h l e t e s w o u l d re s u l t in a b i a s e d a c c o u n t o f t h e p r e v a l e n c e o f t h e s e i n j u r i e s as a l a r g e p r o p o r t i o n o f m i n o r , m i l d , a c u t e i n j u r i e s o f s h o r t d u r a t i o n w o u l d be m iss ed . R e t r o s p e c t i v e s t u d i e s ( s u c h as c a s e - c o n t r o l ) a lso h a v e o t h e r i m p o r t a n t , p o t e n t i a l s o u r c e s o f b ia s s u c h as recall b i a s 1,3,4 a n d m i s s i n g cli n ic a l d a t a bias. M i s s i n g d a t a m a y s e r i o u s l y b i a s r e s u l t s as it is u n k n o w n w h e t h e r th e d a t a is n o r m a l , n e g a t i v e , n e v e r m e a s u r e d , o r m e a s u r e d b u t n e v e r r e c o r d e d . 3 T h e s t u d y d e s i g n w h i c h r a n k s as o n e o f t h e l o w e s t in th e h i e r a r c h y o f s t u d y d e s i g n s is t h e “ b e f o r e - a f t e r ” s t u d y ( f r e q u e n t l y u se d by p h y s i o t h e r a p i s t s ! ) . H a v i n g a g r o u p as its o w n c o n t r o l s e e m s e sp e c i a l l y a t t r a c t i v e , 2 s in c e this a p p e a r s t o e l i m i n a t e v i r t u a l l y all g r o u p d i f f e r ­ e nce s a n d a v o i d m a n y o f the p o t e n t i a l f o r m s o f s a m p l i n g b ia s o f t e n e n c o u n t e r e d in o t h e r s t u d y de si g n s. H o w e v e r , t h e c o n t r o l a n d e x p e r i m e n t a l o b s e r v a t i o n s a r e m a d e d u r i n g d i f f e r e n t t i m e p e r i o d s a n d t h e r e is t h e r e a l d a n g e r t h a t w i t h t h e p a s s a g e o f t im e e x t r a n e o u s f a c t o r s o u t s i d e t h e c o n t r o l o f t h e i n v e s t i g a t o r h a v e in f lu e n c e d t h e s t u d y g r o u p , l e a d i n g t o t h e a p p e a r a n c e o f b e n e f i t w h e n n o n e e x is ts , o r c o n v e r s e l y , m a s k i n g t r u e benef its . P h y s i o t h e r a p i s t s e n g a g e d in r e s e a r c h , o r s t u d e n t p h y ­ s i o t h e r a p i s t s p l a n n i n g r e s e a r c h p r o j e c t s , s h o u l d n o t be d i s c o u r a g e d by th e e x a m p l e s o f s a m p l i n g b i a s t h a t h a v e b e e n d i s c u s s e d , n o r by t h e fa ct t h a t th is is n o t a c o m p r e h e n s i v e list! ( S a c k e t t h a s listed 22 e x a m p l e s ! ) 3 S a m p l i n g b i a s e s c a n , a n d s h o u l d be, a n t i c i p a t e d d u r i n g r e s e a r c h p l a n n i n g a n d c a n be g r e a t l y m in i m is e d t h r o u g h t h e use o f r a n d o m i z a t i o n a n d s t r a t i f i c a t i o n o f s a m p l e s u b j e c t s , s t a n d a r d i s a t i o n a n d m u l t i v a r i a t e a d j u s t m e n t in d a t a a n a l y s i s a n d , m o s t i m p o r t a n t , the c o r r e c t c h o i c e o f r e s e a r c h s t u d y d e s i g n a n d r i g o r o u s e x e c u t i o n . R e f e r e n c e s 1. F r i e d m a n G D . P rim er o f e p id e m io lo g y 2 n d e d. N e w Y o r k : M c G r a w H i l l , 1980. 2. C u r r i e r D P . E le m e n ts o f re sea rch in p h y s ic a l th e r a p y 2 n d e d . B a l t i m o r e : W i l l i a m s & W i l k i n s , 1984. 3. S a c k e t t D . B i a s in a n a l y t i c r e s e a r c h . J C h ro n D is 1979; 32- 5 1 -6 3. 4. F l e t c h e r R H el at. C linical e p id e m io lo g y - th e essentials. B a l t i m o r e : W i l l i a m s a n d W i l k i n s , 1982. 5. L i l i e n f e l d A M . F o u n d a tio n o f e p id e m io lo g y . N e w Y o r k : O x f o r d U n i v e r s i t y P r e s s , 1976. 6. M a u s n e r J S a n d B a h n A K . E p id e m io lo g y - an in tro d u c to ry te x t. P h i l a d e l p h i a : W B S a u n d e r s , 1974. 7. A l d e r s o n M . A n in tr o d u c tio n to e p id e m io lo g y . L o n d o n : M a c m i l l a n P r e s s , 1976. 8. O r e n s t e i n D M et al. E x e r c i s e c o n d i t i o n i n g a n d c a r d i o p u l ­ m o n a r y f i t n e s s in c y s t i c f i b r o s i s . C hest O c t 1981; 80(4)- 3 9 2 - 3 9 8 . Rand M edical Supplies R a ndburg Cen tre 449, Jan Sm uts Ave., B la irg o w rie , R andburg 2194 Box 7166, Jo h a n n e s b u rg . Tel. 7 8 9 -2 2 0 3 /7 8 9 -2 2 8 6 Try New Flotation Pads to keep you free from bedsores Now ‘ NEW B O N M A T ” and “ N E W P A R A M A T ” co m e up w ith b e tte r q u a lity yet low er p ric e s than co n v e n tio n a l flotatio n R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 01 3. )