Microsoft Word - Vol10_MAliceShilling#2AF69B.doc 29 Graduate Student Journal of Psychology Copyright 2008 by the Department of Counseling & Clinical Psychology 2008, Vol. 10 Teachers College, Columbia University ISSN 1088-4661 An Exploratory Study of the Influences on Response Stability of the Interactive Dominic/Terry Questionnaire M. Alice Shillingsburg Marcus Institute Emory School of Medicine Kevin Ragsdale Lee County Youth Development Center Steven K. Shapiro Auburn University Jason Sikorski Central Connecticut State University The Dominic Interactive is a pictorial structured child interview screener with features to address devel- opmental issues that are often problematic for young informants. It includes items depicting non- symptomatic activities as well as activities demonstrating DSM-IV (American Psychiatric Association, 1994) symptoms of several childhood disorders. We examined the reliability of the Interactive Dominic/Terry, as well as potential factors influencing response stability, with a sample of 59 inpatient adolescents. Response stability refers to the tendency for the same response to be given over time. Analyses yielded good to excellent reliability for the majority of the scales. Response stability was similar regardless of age, gender, or race. Results also suggested an inverse relationship between Verbal IQ and rate of response change. Overall, these preliminary results suggest that this instrument is suitable for children and adolescents of different racial backgrounds, although the impact of intelligence requires further investigation. Children and adolescents referred to inpatient psychiat- ric facilities are frequently the focus of extensive medical and psychological assessment. The primary diagnostic tool often is the clinical interview even though interviewer- and/or diagnostic criteria variance are common (Silverman & Ollendick, 2005). To overcome this limitation a number of diagnostic interviews, either highly structured or semi- structured, have been developed (McClelland, 2004). In the inpatient setting a “respondent-based” interview (An- gold & Fisher, 1999) yielding information regarding the thoughts and feelings of the youth is desirable to comple- ment observational material from staff reports. However, difficulties with traditional interview meth- ods such as the use of abstract language, cumbersome Author note: We thank Todd Smitherman, Natalie Arnette, and Kanika Bell for their assistance with data collection and J. P. Valla for his helpful suggestions on previous drafts of this article. Appreciation is extended to the participants and staff of Lee County Youth Development Center. This paper is based on a Master's thesis conducted by the first author while at Auburn University. Correspondence concerning this article should be directed to Steve Shapiro, Department of Psychology, 226 Thach, Auburn University, Alabama 36849-5214; shapisk@auburn.edu length, and references to time have been noted (Edelbrock & Bohnert, 2000; Fallon & Schwab-Stone, 1994). Studies have shown that up to half of adolescents are not interested in traditional interviews and one-third has difficulty paying attention (Shaffer et al., 1993). Adolescents have also re- ported difficulty understanding interview questions (Edel- brock & Bohnert, 2000). These factors may impede the utility of the information gathered. In order to increase in- terest and attention, a diagnostic interview may necessitate a concrete format to which young informants can relate. Instruments showing pictures representing psychopa- thology, thereby combining auditory and visual modalities, seem appropriate. As demonstrated by Furman and Bierman (1983), pre- senting a combination of both auditory and visual informa- tion when posing questions to children as young as four years old results in increased comprehension of the ques- tion (see also Bierman, 1984). This dual-modality approach is consistent with Paivio’s dual-coding theory, which posits two distinct subsystems, an auditory and a visual system, that process cognitive information (Paivio, 1986). Such a combined approach has been shown to aid comprehension and learning, enhance children’s memory (Pressley & Miller, 1987; Schneider & Pressley, 1997), improve atten- tion, and stimulate interest (Calvert, 1999; Peeck, 1987; Pezdek & Stevens, 1984). In addition, according to Bender and Levin (cited in Pressley & Miller, 1987) the dual- modality approach significantly improved learning in a SHILLINGSBURG, SHAPIRO, RAGSDALE, & SIKORSI 30 group of educable mentally retarded children. Given previ- ous research indicating that adolescents, similar to younger children, have difficulty paying attention during structured interviews, the combination of visual and auditory stimuli may also be beneficial when interviewing adolescents, par- ticularly those with lower intelligence. Among pictorial instruments, the Dominic is one of the oldest and most widely tested (Bidaut-Russell, Valla, Tho- mas, Bergeron, & Lawson 1998; Ederer, 2004; Valla, Bergeron, Berube, Gaudet, & St-Georges, 1994). The Dominic demonstrates robust psychometric properties in the general population and outpatient clinic children (Cot- tler, Reich, Rourke, Cunningham-Williams, & Compton, 2000; Loney & Frick, 2003; Murphy, Cantwell, Jordan, Lee, Cooley-Quille, & Lahey, 2000). It has, however, not been validated with inpatient youths. The Dominic Interactive (Valla, 2000) is part of a relatively new set of structured child interview screeners (Dominic-R, Interactive Dominic/Terry) developed in an attempt to resolve many of the problems with traditional face-to-face interviews (Valla et al., 1994; Valla, Bergeron, Bidaut-Russell, St-Georges, & Gaudet, 1997; Valla, Bergeron, & Smolla, 2000). This screening tool is a picto- rial structured interview that, in addition to blending visual and verbal information, has a short administration time (10- 20 minutes), presents the child with simple response op- tions, and does not include questions regarding time and frequency. As the interviewee views the pictures he or she simultaneously hears a voice describing the pictured activ- ity. This interview portrays, in still-frame, a boy or girl named Dominic (Caucasian), or Terry (African-American) engaging in various activities and situations. Some of the activities and situations demonstrate non-symptomatic ac- tivities and are included on a Strengths and Competencies Scale, which has received little empirical investigation. Other activities and situations presented demonstrate DSM- IV (American Psychiatric Association, 1994) symptoms of Attention-Deficit/Hyperactivity Disorder (ADHD), Oppo- sitional Defiant Disorder (ODD), Conduct Disorder (CD), Major Depressive Disorder (MDD), Separation Anxiety Disorder (SAD), Generalized Anxiety Disorder (GAD), and Specific Phobias (SPh). Upon presentation of the items, the child either answers “yes” or “no” as to whether s/he acts similarly to Dominic/Terry (Valla et al., 2000). Currently, an incomplete set of versions is available across age, language, and race. Thus, this instrument has consid- erable potential as a tool for cross-cultural applications. Valla and colleagues (1994, 1997, & 2000) have re- viewed the construction and psychometric properties of the paper-based version of the Dominic and Dominic-R. Of 42 symptom scales (7 diagnostic-approximates, or “tenden- cies,” for six age groups) included on the Dominic-R, 33 obtained an Intraclass Correlation Coefficient (ICC) above .70. Seven were between .60 and .69 and 2 were below .60. The 2 symptom scales below .60, Specific Phobia and Conduct Disorder, were obtained from the group of 6-year- olds. Internal consistency for both internalizing and exter- nalizing Tendency scales were .89. Alphas for the individ- ual symptom scales ranged from .64 to .83. Bidaut-Russell and colleagues (1998) introduced Dominic-R versions depicting African Americans (Terry). Using 36 referred and non-referred children, these re- searchers obtained test-retest kappas for Terry tendencies from .70 to .76 and ICCs between .77 and .88. Cronbach’s alphas ranged from .78 to .90. These findings are commen- surate with those obtained using the Dominic-R and indi- cate that the measure can be used reliably with African American children. Still, as noted by Valla et al. (2000), it is necessary to further document the psychometric charac- teristics of the Terry. This is consistent with the need to develop more ecologically sensitive instruments to facili- tate identification with the material and enhance self- disclosure (Merrell, 2003). The computer-based Interactive Dominic was first in- troduced by Valla, Bergeron, and Smolla (1997). Unpub- lished research conducted by Bergeron and Smolla (2002), using 609 non-clinical and clinical French and English- speaking children, report psychometric properties that were similar to or better than the paper version. Valla et al. (2002), demonstrated the measure’s transcultural appropri- ateness and ability to discriminate between clinically re- ferred and non-referred children, as well as referred chil- dren with and without a clinical judgment-based diagnosis. The psychometric characteristics of the Interactive Terry have not been reported. Previous findings suggest that the Dominic-R elicits responses that demonstrate better test-retest reliability than those yielded from the K-SADS, the DISC, and the DISC-2 (Valla et al. 1994). However, initial studies with the Dominic-R using both outpatient clinic and community samples resulted in limitations in the calculation of some kappa values due to the low base rate of some symptoms. Because the Dominic already demonstrated acceptable psychometric qualities in other settings, and instruments tend to be more reliable in clinical settings (Boyle et al., 1993; Schwab-Stone, Fisher, Piacentini, Shaffer, Davies, & Briggs, 1993; Welner, Reich, Herjanic, Jung, & Amado, 1987), the current study was not designed as a comprehen- sive validation study. Instead, we first aimed to extend the reliability evidence of the Interactive Dominic to an inpa- tient setting and its parallel version (Terry) with a sample of African Americans. Second, we were interested in ex- amining whether measured intelligence affected response stability. A plethora of literature has documented adaptive behavior deficits (e.g., difficulties with hygiene care, daily living skills) and the full range of psychopathology in chil- dren presenting with intellectual deficits (see Dykens, 2000 for a review). It follows, therefore, that children with intel- lectual delays would be well represented in inpatient psy- chiatric settings. Comorbid cognitive delays and behav- ioral/ emotional problems pose particular assessment chal- lenges (Dykens, 2000; Hodapp & Dykens, 1996; Singh, Oswald, & Ellis, 1998). Indeed, previous reliability studies on traditional structured diagnostic interviews frequently INFLUENCES ON RESPONSE STABILITY 31 have excluded children due to low intelligence (Edelbrock, Costello, Dulcan, Conover, & Kalas, 1986; Herjanic & Reich, 1982; Moretti, Fine, Haley & Marriage, 1985; Wel- ner et al., 1987). Third, the influence of gender, age, race, test-retest interval, and attenuation was explored as possi- ble factors affecting response stability. Method Participants Participants were 59 inpatient adolescents consecu- tively referred between July 2002 and April 2003 to a resi- dential treatment and diagnostic center, which provides short-term emergency shelter in East Alabama. Males (52.5%) and females (47.5%), African Americans (46%) and Caucasians (54%) were equally represented in the sample, and ranged in age from 12 to 17 (M = 14.3, SD = 1.63). Inclusion criteria were: age between 12 and 17, in- formed consent/assent, and presence during the period of data collection. There were no exclusion criteria. The range of presenting problems was wide and these teens partici- pated in a variety of treatment programs, in addition to an evaluation. Measures Interactive Dominic/Terry Interactive (Valla, 2000). We used the 6-11 version of the instrument with our inpa- tient sample since the African-American version (Terry) is not yet available for adolescents. Downward extensions of adult questionnaires have been widely criticized because children’s language comprehension may differ markedly from that of adults due to factors previously outlined. In the present study, however, the measure has been developed with young children, and the more recent adolescent ver- sion (Smolla, Valla, Bergeron, Berthiaume, & St-Georges (2004), with few exceptions, utilizes the same formulations as the 6-11 version (J. P. Valla, personal communication, July 2006). The child version of the Dominic/Terry yields diagnostic-approximate information (tendencies) based on a normative sample aged 6-11. Cut-off points obtained with that sample were irrelevant for our adolescent participants and no attempt was made to study diagnostic prevalence estimates. Wechsler Intelligence Scales. The Wechsler Intelli- gence Scale for Children, Third Edition (WISC-III; Wechsler, 1991) and Wechsler Adult Intelligence Scale, Third Edition (WAIS-III; Wechsler, 1997) represent the most widely used measures of intellectual abilities (Kauf- man & Lichtenberger, 2000), and yield a Full Scale IQ, Verbal IQ, Performance IQ, and four factor scores. Reli- ability and validity of these instruments are well docu- mented (Sattler, 2001). Procedures Upon arrival at the facility each resident received a standard diagnostic battery, which among other measures, included a semi-structured clinical interview, the WISC-III or WAIS-III, and the Interactive Dominic/Terry. Nothing was mentioned regarding Dominic's/Terry's youthful ap- pearance. Interviewers presented the task using the stan- dard prompt provided in the test. The test-retest interval for the Dominic/Terry ranged from 3 to 17 days (Mdn = 7; M = 8.33; SD = 3.3). Informed consent and assent were secured from the parents/guardians and adolescents, respectively, prior to the data being obtained in an anonymous fashion from the participants’ file. Analyses The overall sample of 59 participants was used to de- termine the internal consistency (Cronbach’s alpha) of the Interactive Dominic/Terry tendency scales. Of these 59 youths, 10 left the facility before completing retest. No differences on any study variable existed between youths with retests and those without. Therefore, test-retest analy- ses were based on a sample of 49 adolescents (24 females [49%], 23 African Americans [46.9%]). Kappa coefficients were calculated to assess the test-retest reliability of symptoms. Intra-class Correlations (ICCs) were computed to examine the test-retest reliability of the seven tendency scales, the Strengths and Competencies scale, and the total instrument. Of these 49 subjects, 47 (23 females, 21 African- Americans) were administered the WISC-III or WAIS-III, depending on age (12-17; M = 14.4; SD = 1.62). We com- puted Pearson product-moment correlations in order to examine the relationship between the amount of response change on the Interactive Dominic/Terry and intelligence. Response change at retest was calculated for every ten- dency scale and every participant. Response change refers to the change from one response to another from the first to second administration. Less response stability results in greater response change. The amount of response change was computed by summing the number of responses that were changed from test to retest and dividing by the num- ber of possible responses per tendency scale. In order to further examine differences in terms of amount of response change and IQ, participants were grouped according to IQ. Those with IQs of 79 and below were classified as “Moderately or Significantly Below Av- erage,” those with IQs between 80 and 89 were classified as “Below Average,” and those with IQs of 90 and above were classified as “Average or Above.” We conducted an analysis of variance (ANOVA) for each diagnostic scale with post-hoc comparisons (Tukey HSD). These IQ groups were also used to examine ICC patterns among the seven tendency scales, the Strengths and Competencies scale, and the total instrument. Univariate analyses explored other possible factors affecting response change, and examined SHILLINGSBURG, SHAPIRO, RAGSDALE, & SIKORSI 32 attenuation differences according to gender and two age groups (12-14 and 15-17). Results Acceptability of the Pictorial Approach Participants were observed to be quite interested in the computer-based interview relative to other components of the assessment. There was no reaction to the age of Dominic with the exception of two participants who com- mented on how young Dominic/Terry appeared. Internal Consistency Cronbach’s alphas ranged from .66 (Strengths) to .91 (ADHD). Alphas for the internalizing and externalizing tendency scales were both very high (.94). Test-retest Reliability ICCs were calculated for the overall sample as well as for groups based on gender, race, and age. For the overall sample, ICCs of the eight tendency scales ranged from .81 to .96. The ICC for the total instrument was .97. Except for the Strengths and Competencies scale and the Specific Phobia tendency scale, all ICCs were above .90. ICCs ranged from .79 to .97 for the younger group (12 to 14) and .84 to .96 for the older group (15-17). ICCs for the two groups for each scale and the total test were similar; how- ever, the younger group obtained an ICC of .83 on the Spe- cific Phobia tendency scale while the older group obtained an ICC of .94. Because the Interactive Dominic/Terry provides slightly different characters based on gender and race, test- retest reliability was examined depending on the version given (see Table 1). ICCs for both genders were similar except for the Specific Phobia tendency scale: males ob- tained a lower ICC (.69) than females (.91) on that scale. ICCs for the African American and Caucasian groups were similar with the exception of the Strengths and Competen- cies scale where the African American group obtained a lower ICC (.71) than the Caucasian group (.91). Of the 91 symptoms, kappas were .70 or above for 42 symptoms, between .60 and .69 for 17 symptoms, between .50 and .59 for 14 symptoms, and below .50 for 18 symp- toms. Using the classification approach offered by Landis and Koch (1977), the Interactive Dominic/Terry yielded 27 symptoms with excellent reliability (above .75), 34 with good reliability (.59 to .74), 19 with fair reliability (.40 to .58), and 5 with poor reliability. Six items had low base rates at both test and retest. Intelligence and Response Stability Full Scale IQ (FSIQ) scores ranged from 54 to 116 (M = 84.13, SD = 12.76), Verbal IQ (VIQ) scores ranged from 52 to 126 (M = 84.02, SD = 14.23), Performance IQ (PIQ) scores ranged from 62 to 112 (M = 87.11, SD = 11.09), and Verbal Comprehension Index scores ranged from 54 to 102 (M = 84.16, SD = 11.85). FSIQ and PIQ were negatively correlated with response change for four of the eight ten- dency scales (SPh: r = -.35 and -.34, p < .05; SAD: r = -.40 and -.40, p < .01; CD: r = -.31 and -.29, p < .05; Strengths: r = -.44, p < .01 and r = -.30, p <.05) and the overall in- strument (r = -.50 and -.30, p < .01). Significant negative associations were also obtained between VIQ and response change for all but one (ODD) tendency scale (p < .05: SPh r = -.31, SAD r = -.35, GAD r = -.29, MDD r = -.29, CD r = -.29, ADHD r = -.29; p < .01: Strengths r = -.48) as well as the overall instrument (r = -.53, p <.01). The Verbal Comprehension Index was negatively associated with five tendency scales (SAD r = -.42, p < .05; MDD r = -.38, p < .05; CD r = -.35, p < .05; ADHD r = -.46, p < .01; Strengths r = -.59, p < .01) and the overall instrument (r = - .56, p < .01). Table 1 Intraclass Correlation test-retest reliability of scales of the Interactive Dominic/Terry based on gender and race Males N=25 Females N=24 African American N=23 Caucasian N=26 Tendencies ICC a CI b ICC CI ICC CI ICC CI SPh .69 .29, .86 .91 .80, .96 .86 .68, .94 .89 .76, .95 SAD .95 .88, .98 .94 .86, .97 .94 .86, .97 .95 .89, .98 GAD .96 .91, .98 .91 .80, .96 .95 .89, .98 .93 .85, .97 MDD .95 .88, .98 .96 .91, .98 .97 .93, .99 .95 .89, .98 ADHD .95 .88, .98 .97 .93, .99 .96 .91, .98 .96 .91, .98 ODD .88 .72, .95 .94 .87, .98 .95 .87, .98 .89 .75, .95 CD .96 .90, .98 .88 .71, .95 .94 .86, .97 .94 .87, .97 Overall Total .96 .91, .98 .97 .94, .99 .98 .94, .99 .96 .92, .98 Strengths .83 .62, .93 .80 .53, .91 .71 .31, .88 .91 .80, .96 a Intraclass Correlation Coefficient b 95% confidence interval Based on the ANOVA between VIQ groups (“Moder- ately or Significantly Below Average,” “Below Average,” INFLUENCES ON RESPONSE STABILITY 33 and “Average or Above”), the lowest VIQ group tended to show higher response changes, but only significantly so on the CD and SAD tendency scales as well as on the Strengths and Competencies scale (see Table 2). Similarly, all scale ICCs remained above .90 regardless of VIQ group, except for the lowest VIQ group’s SPh scale (.79) and the lower two VIQ groups’ Strengths and Competen- cies scale (.35 and .84, respectively). Table 2 VIQ group differences in amount of response change Mean Amount of Response Change Tendencies MSBA VIQ a N=13 BA VIQ b N=19 AA VIQ c N=15 F p Tukey HSD SPh .128 .076 .052 1.72 .191 NS SAD .163 .145 .050 4.43 .018 MSBA, BA>AA GAD .226 .140 .142 2.33 .109 NS MDD .181 .158 .093 2.64 .083 NS ODD .188 .164 .178 .097 .908 NS CD .165 .071 .109 3.52 .038 MSBA>BA ADHD .170 .130 .081 2.65 .080 NS Overall Total .177 .127 .089 7.39 .002 MSBA > AA Strengths .231 .168 .053 8.40 .001 MSBA, BA>AA a Moderately or Significantly Below Average VIQ = 79 and below b Below Average VIQ = 80-89 c Average or Above VIQ = 90 and above Note. SPh=Specific Phobia, SAD=Separation Anxiety Dis- order, GAD=Generalized Anxiety Disorder, MDD=Major Depressive Disorder, ADHD=Attention- Deficit/Hyperactivity Disorder, ODD=Oppositional Defiant Disorder, CD=Conduct Disorder, Strengths=Strengths and Competencies Other Factors Affecting Response Stability Age, gender, race, and test-retest interval were ex- plored as possible factors affecting amount of response change. Univariate analyses resulted in no significant dif- ferences for gender, age, or test-retest interval indicating that these factors did not affect response change. Only one scale (Strengths) resulted in a significant difference when comparing African Americans (M = .19, SD = .14) and Caucasians (M = .11, SD = .11), t (47) = 2.18, p < .05. Attenuation is the tendency to change positive re- sponses at the first administration to negative responses at the second administration. Thus, attenuation is an important phenomenon to consider when examining response change. Attenuation was seen for all scales. On the SPh scale, mean rate of attenuation for males (M = 1.5, SD = 0.97) was lower than females (M = 2.71 SD = 1.23), t (47) = 2.33, p < .05. All other comparisons showed no differences. No cor- relation was found between age and attenuation. Discussion The goals of the current study were to: 1) test the reli- ability evidence of the Interactive Dominic in an inpatient setting and its parallel version with African American youth; and 2) explore the relationship between response stability, intelligence, and demographic variables. Reliability Internal consistency statistics derived in the present study are generally commensurate with those obtained on the paper-based version (Bidaut-Russell et al, 1998; Valla, Bergeron, Bidaut-Russell et al., 1997), though notably higher values were found on the GAD/OAD scale (.86 ver- sus .66) and the CD scale (.85 versus .64). Furthermore, the test-retest reliability results of the present study are higher than those obtained previously by the authors of the Inter- active Dominic/Terry both for symptoms and symptom scales. ICCs obtained in the present study also compare favorably with those obtained by Valla and colleagues. Such improved reliability was expected in an inpatient sample of older youth, since younger children tend to be less consistent in their reports than older ones (Edelbrock et al, 1985). In addition, community samples are unlikely to endorse a sufficient number of the more severe symptoms (Shaffer, Fisher, & Lucas, 1999), which influences statisti- cal power. Thus, the higher rates of symptom endorsement in our inpatient sample could have accounted for the higher reliabilities. Consistent with this, test-retest reliabilities obtained in the present study with inpatient adolescents were considerably higher than those reported by Fisher et al. (1997; see also Shaffer et al., 2000), using the DISC-IV on a sample of psychiatric outpatient children and adoles- cents (9-17 years old). In an early study of the Child and Adolescent Psychiat- ric Assessment self-report (CAPA-C) reliability, Angold and Costello (1995) also found typically higher ICCs for inpatients relative to outpatients. Our results compare fa- vorably with these authors’ ICCs for symptoms of Major Depression; in contrast, our ICCs were considerably higher for symptoms of Separation Anxiety Disorder, Opposi- tional Defiant Disorder, and Conduct Disorder. Few other studies of inpatients are available, making further compari- sons difficult. Regardless, no other diagnostic screening instrument has combined auditory and visual stimuli. Audio computer-assisted self-interviewing has been shown SHILLINGSBURG, SHAPIRO, RAGSDALE, & SIKORSI 34 to foster adolescent disclosure of socially undesirable symptoms (Turner et al., 1998). In addition, the Voice DISC-IV has shown utility in juvenile justice settings, where the use of evidence-based instruments is highly vari- able (Wasserman, McReynolds, Lucas, Fisher, & Santos, 2002). Attention should, therefore, be directed towards establishing the incremental utility of the Dominic Interac- tive as part of an evidence-based diagnostic evaluation process. The tendency scales on the Interactive Dominic/Terry demonstrated acceptable reliability for both genders and for African-Americans and Caucasians, indicating that the in- strument may be quite useful for children and adolescents of different racial backgrounds. Similar to previous studies of the Dominic (Bidaut-Russell et al, 1998; Valla, Bergeron, Bidaut-Russell et al., 1997), attenuation was seen for all tendency scales. This phenomenon has been docu- mented by other interview developers as common (e.g., Angold & Costello, 1995; Piacentini et al, 1999). The Specific Phobia (SPh) and Strengths and Compe- tencies scales demonstrated lower ICCs and internal con- sistency than other scales, and of the items with poor reli- ability, four were on the Strengths scale and three were on the SPh scale. Various types of specific phobias (e.g., dogs, bugs, heights) are represented on the SPh scale, as opposed to the severity of any one specific experience required in the DSM-IV. Specific phobias tend to diminish as children age (Barrios & O’Dell, 1998); thus, for our adolescent sample, endorsement base rate was very low. Valla, Bergeron, Bidaut-Russell et al. (1997), reported adequate internal consistency and good test-retest reliability for the SPh scale in their sample aged 6-11 years. Items on the Strengths scale refer to having fun with friends, feeling good about school, and liking the place where the youth lives. Affirmative responses may be ex- pected from well adjusted children living in the commu- nity. However, due to the upheaval associated with a resi- dential placement, this scale may not translate into inpa- tients’ current situation. As noted, African-Americans ob- tained a lower ICC on this scale compared to Caucasians. This finding may indicate less cultural applicability of this scale to African-American youths’ perception of their strengths and competencies. Previous studies did not report the psychometric properties of this scale. Therefore, clarifi- cation of the utility and integrity of this scale is needed. Response Stability and IQ Our adolescent participants did not have a parental re- port available and were the sole providers of information during the diagnostic evaluation, in addition to historical records. This limited access to information is also a con- cern in juvenile justice settings (see Wasserman et al, 2002). For adolescents of lower cognitive functioning who do not have parental report available to supplement their self-report, the method of eliciting information is crucial. The Interactive Dominic/Terry was developed with the receptive and expressive limitations of children in mind. It relies less on the respondents’ verbal ability than sentence- based interviews that ignore limitations, and as a result, is likely less influenced by measured “verbal intelligence.” Although age of our participants would suggest that a sen- tence-based interview is acceptable, the cognitive abilities of many participants suggest otherwise. Verbal IQ is still an associated factor in respondents’ ability to provide sta- ble responses over time on the Interactive Dominic/Terry. Although there is an impact of IQ on the Dominic’s reli- ability, this impact seems minimal and should not preclude the use of this instrument with intellectually delayed indi- viduals. Systematic research on the impact of intelligence on reliability of traditional questionnaires and interviews is surprisingly sparse. The fact that many studies investigat- ing the properties of sentence-based interviews exclude children based on cognitive ability suggest that those inter- views (and other self report measures) may not have an adequate base of evidence on which to claim reliability. The format of the Interactive Dominic/Terry may allow for useful diagnostic screening that might otherwise not have been possible. Limitations and Future Directions The main limitation of our preliminary study is the small sample size. A larger sample size allows for better examination of symptoms rarely endorsed. Further, larger samples of each race and gender would have been desir- able. Another clear limitation of the present study is the use of the 6-11 year old version of the instrument with an ado- lescent population. At the time of the current investigation, the adolescent version of the measure was in the process of being created and had not yet been made available. How- ever, the authors of both versions of the measure indicated that with few exceptions the adolescent version used the same formulations as the 6-11 version (J. P. Valla, personal communication, July 2006). One primary difference is the older appearance of the child in the adolescent version. Given that only two of the participants made statements about the age of the child depicted, it is likely that this is an insignificant factor. In addition, given that diagnostic in- formation was not relevant to the current study, cut-off scores based on a younger normative sample were not used. Lastly, although the age of our participants would suggest that a sentence-based interview would be sufficient, the cognitive abilities of many participants and lack of an ado- lescent version suggests that use of the child version of the instrument was appropriate. This study did not examine the Interactive Dominic/Terry and a DSM-IV-, sentence-based interview simultaneously, and thus the extent of the improvement presumably brought about by the combination of verbal and visual information with adolescents of lower cognitive functioning could not be determined. It would be important INFLUENCES ON RESPONSE STABILITY 35 to examine the relationship between Verbal IQ and both types of interview in order to determine whether the dual modality presentation improves response stability of low functioning adolescents. As noted by Klein, Dougherty, and Olino (2005), most studies have not directly compared different diagnostic interviews in the same sample, thus making it difficult to identify advantages with particular instruments (see also Angold & Fisher, 1999). It might also be worthwhile to investigate whether order effects are pre- sent with the Interactive Dominic, consistent with findings that youths (and parents) report more symptoms for diag- noses assessed early in the more traditional structured in- terviews (Jensen, Watanabe, & Richters, 1999). Although an inverse relationship between IQ and response change was seen with some scales for lower functioning individu- als relative to higher functioning individuals, the impact seems minimal and may be less of a factor when using a pictoral interview compared to more traditional structured interviews. The current investigation did not directly com- pare the effects of IQ on response stability using the two interview formats. Future studies should directly compare these two modalities to determine the impact of IQ when using a dual-modality approach compared to traditional interviews. Lastly, although the length of the retest interval was examined as a potential factor affecting response sta- bility, we did not specifically examine the effect on at- tenuation. Future investigations should include this analysis to fully understand the effects of the length of the retest interval. Clinical Implications As implied above, this study provides preliminary evi- dence on the reliability of the Interactive Dominic/Terry with both African-American and Caucasian adolescent in- patient populations, yielding useful information in an effi- cient manner. This novel assessment approach based on both visual and auditory stimuli may be indicated with low functioning adolescents for whom traditional sentence- based interviews and inventories pose particular challenges. Attention might also be directed towards establishing the incremental utility of the Dominic Interactive as part of an evidence-based diagnostic evaluation process. Investiga- tions of convergent and divergent validity and factors that influence the validity of obtained Dominic/Terry responses are essential. This information would provide support for the usefulness of this type of interview with an adolescent population. In addition, similar investigations of test-retest reliability with the adolescent version of the instrument are warranted. References American Psychiatric Association (1994). Diagnostic and Statistical Manual of Mental Disorders (4 th ed.). Wash- ington, DC: Author. Angold, A. & Costello, E. J. (1995). A test-retest reliability study of child-reported psychiatric symptoms and diag- noses using the child and adolescent psychiatric assess- ment (CAPA- C). Psychological Medicine, 25, 755-762. Angold, A., & Costello, E. J. (1999). Comorbidity. Journal of Child Psychology and Psychiatry & Allied Disciplines, 40, 57-87. Angold, A., & Fisher, P. W. (1999). Interviewer-based in- terviews. In D. Shaffer, C. P. Lucas & J. E. Richters (Eds.), Diagnostic assessment in child and adolescent psychopathology, (pp.34-64). New York: The Guilford Press. Barrios, B.A., & O’Dell, S. L. (1998). Fear and anxieties. In E. J. Mash & R.A. Barkley (Eds). Treatment of child- hood disorders. (2 nd ed., 249-337). New York: The Guil- ford Press. Bergeron, L., & Smolla. N. (2002). Etude de la fidelite et de al validite de critere du Dominique Interactif chez les enfants de 6 a 11 ans. Rapport de recherché remis au Fonds de la recherché en sante du Quebec (FRSQ), juillet. Bidaut-Russell, M., Valla, J.P, Thomas, J.M., Bergeron, L., & Lawson, E. (1998). Reliability of the Terry: A mental- health cartoon-like screener for African-American chil- dren. Child Psychiatry and Human Development, 28, 249-263. Bierman, K. L. (1984). Cognitive development and clinical interviews with children. In B. Lahey & A. Kazdin (Eds.), Advances in clinical child psychology (Vol. 6, pp. 217-250). New York: Plenum Press. Boyle, M.H., Offord, D. R., Racine, Y., Sanford, M., Szat- mari, P., Fleming, J.E., & Price-Munn, N. (1993). Evaluation of the diagnostic interview for children and adolescents for use in the general population samples. Journal of Abnormal Child Psychology, 21, 663-681. Calvert, S. L. (1999). The form of thought. In I. E. Sigel (Ed.), Development of mental representation: Theo- ries and applications (pp. 453-470). Mahwah, NJ: Law- rence Erlbaum Associates. Cottler, L. B., Reich, W., Rourke, K., Cunningham- Williams, R. M., Compton, W. M. (2000). Assessing psychopathology among children aged four to eight. In G. Andrews & S. Henderson (Eds). Unmet need in psy- chiatry: Problems, resources, responses. (pp. 345-355). New York, NY: Cambridge University Press. Dykens, E.M. (2000). Annotation: Psychopathology in children with intellectual disability. Journal of Child Psychology and Psychiatry, 41, 407-417. Edelbrock, C. & Bohnert, A. (2000). Structured interviews for children and adolescents. In G. Goldstein & M. Hersen (Eds.), Handbook of psychological assessment. (3 rd ed., pp 369-386). Oxford, England: Elsevier Science. Edelbrock, C., Costello, A. J., Dulcan, M. K., Conover, N. C. & Kalas, R. (1986). Parent-child agreement on child psychiatric symptoms assessed via structured interview. Journal of Child Psychology and Psychiatry, 27, 181- 190. SHILLINGSBURG, SHAPIRO, RAGSDALE, & SIKORSI 36 Edelbrock, C., Costello, A. J., Dulcan, M. K., Kalas, R. & Conover, N. C. (1985). Age differences in the reliability of the psychiatric interview of the child. Child Develop- ment, 56, 265-275. Ederer, E.M. (2004). Mental health problems in young children: Self-report and significant others as informants. Psychology Science, 46, 123-140. Fallon, T. Jr. & Schwab-Stone, M. (1994). Determinants of reliability in psychiatric surveys of children aged 6-12. Journal of Child Psychology and Psychiatry, 35, 1391- 1408. Fisher, P. W., Lucas, C., Shaffer, D. et al (1997). Diagnos- tic Interview Schedule for ChildrenVersion IV (DISC- IV): Test-retest reliability in a clinical sample. Presented at the 44 th Annual Meeting of the American Academy of Child and Adolescent Psychiatry, Toronto. Furman, W. & Bierman, K. L. (1983). Developmental changes in young children’s conceptions of friendship. Child Development, 54, 549-556. Herjanic, B. & Reich, W. (1982). Development of a struc- tured psychiatric interview for children: Agreement be- tween child and parent on individual symptoms. Journal of Abnormal Child Psychology, 10, 307-324. Hodapp, R. M. & Dykens, E. M. (1996). Mental retarda- tion. In E.J. Mash & R.A. Barkley (Eds). Child psycho- pathology. (362-389). New York, NY: Guilford. Jensen, P. S., Watanabe, H. K., & Richters, J. E. (1999). Who’s up first? Testing for order effects in structured interviews using a counterbalanced experimental design. Journal of Abnormal Child Psychology, 27, 439-445. Kaufman, A. S. & Lichtenberger, E. O. (2000). Essentials of WISC-III and WPPSI-R assessment. New York: John Wiley & Sons, Inc.. Klein, D. N., Dougherty, L. R., & Olino, T.M. (2005). To- ward guidelines for evidence-based assessment of de- pression in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 412-432. Landis, J. & Koch, G. G. (1977). The measurement of ob- server agreement for categorical data. Biometrics, 33, 159-174. Loney, B.R., & Frick, P.J. (2003). Structured diagnostic interviewing. In C.R. Reynolds & R.W. Kamphaus (Eds). Handbook of psychological and educational as- sessment of children: Personality, behavior, and context. (2 nd ed; pp. 235-247). New York, NY: Guilford Press. McClelland, J. (2004). Diagnostic interviews. In J.M. Wie- ner & M.K. Dulcan (Eds). American Psychiatric Pub- lishing Textbook of child and adolescent psychiatry (3 rd ed, pp. 137-148). Washington, DC: American Psychiatric Association. Merrell, K.W. (2003). Behavioral, social, and emotional assessment of children and adolescents. (2 nd ed.). Mah- wah, New Jersey, Lawrence Erlbaum Associates. Moretti, M. M., Fine, S., Haley, G. & Marriage, K. (1985). Childhood and adolescent depression: Child-report ver- sus parent-report information. Journal of the American Academy of Child Psychiatry, 24 (3), 298-302. Murphy, D. A., Cantwell, C., Jordan, D. D., Lee, M. B., Cooley-Quille, M. R., & Lahey, B. B. (2000). Test-retest reliability of Dominic anxiety and depression items among young children. Journal of Psychopathology and Behavioral Assessment, 22, 257-270. Paivio, A. (1986). Mental representations: A dual coding approach. Oxford, England: Oxford University Press. Peeck, J. (1987). The role of illustrations in processing and remembering illustrated text. In D. M. Willows & H. A. Haughton (Eds.), The psychology of illustration: Vol. 1, Basic research (pp. 115-151). New York: Springer- Verlag. Pezdek, K. & Stevens, E. (1984). Children’s memory for auditory and visual information on television. Develop- mental Psychology, 20, 212-218. Piacentini, J., Roper, M., Jensen, P., Lucas, C., Fisher, P., Bird, H., Bourdon, K., Schwab-Stone, M., Rubio-Stipec, M., Davies, M., & Dulcan, M. (1999). Informant-based determinants of symptom attenuation in structured child psychiatric interviews. Journal of Abnormal Child Psy- chology, 27, 417-428. Pressley, M. & Miller, G. E. (1987). Effects of illustrations on children’s listening comprehension and oral prose memory. In D. M. Willows & H. A. Haughton (Eds.), The psychology of illustration: Vol. 1, Basic research (pp. 87-114). New York: Springer-Verlag. Sattler, J.M. (2001). Assessment of Children: Cognitive Applications (4 th ed). San Diego, CA: Jerome M. Sattler, Publishers, Inc. Schneider, W. & Pressley, M. (1997). Memory development between two and twenty. (2 nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Schwab-Stone, M., Fisher, P., Piacentini, J., Shaffer, D., Davies, M. & Briggs, M. (1993). Test-retest reliability. Journal of the American Academy of Child and Adoles- cent Psychiatry, 32, 651-657. Shaffer, D., Fisher, P. W. & Lucas, C. P. (1999). Respon- dent-based interviews. In D. Shaffer, C. P. Lucas & J. E. Richters (Eds.), Diagnostic assessment in child and adolescent psychopathology (pp.3-33). New York: The Guilford Press. Shaffer, D., Fisher, P.W., Lucas, C. P., Dulcan, M.K., & Schwab-Stone, M. E. (2000). NIMH Diagnostic Inter- view Schedule for Children Version IV (NIMH DISC- IV): Description, Differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psy- chiatry, 39, 28-38. Shaffer, D., Schwab-Stone, M., Fisher, P., Cohen, P. Piacentini, J., Davies, M., Conners, C. K. & Regier, D. (1993). Preparation, field testing, interrater reliability, and acceptability. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 643-650. Silverman, W. K., & Ollendick, T. H. (2005). Evidence- based assessment of anxiety and its disorders in children and adolescents. Journal of Clinical Child and Adoles- cent Psychology, 34, 380-411. INFLUENCES ON RESPONSE STABILITY 37 Singh, N. N., Oswald, D. P. & Ellis, C. R. (1998). Mental retardation. In T.H. Ollendick & M. Hersen. (Eds). Handbook of child psychopathology. (pp. 91-116). New York, NY: Plenum. Smolla, N., Valla, J.P., Bergeron, L., Berthiaume, C., & St- Georges, M. (2004). Development and reliability of a pictorial mental disorders screen for young adoles- cents. Canadian Journal of Psychiatry, 49 (12), 828-837. Turner, C. F., Ku, L., Rogers, S.M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F.L. (1998). Adolescent sexual be- havior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867-873. Valla, J. P. (2000). Instruction Manual for the Dominic Interactive. D.I.M.A.T. Inc. Valla, J. P., Bergeron, L., Berube, H., Gaudet, N. & St- Georges, M. (1994). A structured pictorial questionnaire to assess DSM-III-R-Based diagnoses in children (6-11 years): Development, validity, and reliability. Journal of Abnormal Child Psychology, 22, 403-423. Valla, J. P., Bergeron, L., Bidaut-Russell, M., St-Georges, M. & Gaudet, N. (1997). Reliability of the Dominic-R: A young child mental health questionnaire combining vis- ual and auditory stimuli. Journal of Child Psychiatry, 38, 717-724. Valla, J. P., Bergeron, L., & Smolla, N. (1997). The Inter- active Dominic. Paper presented at the Annual Meeting of the American Academy of Child and Adolescent Psy- chiatry, Toronto. Valla, J. P., Bergeron, L. & Smolla, N. (2000). The Dominic-R: A pictorial interview for 6- to 11- year-old children. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 85-93. Valla, J. P., Kovess, V., Chee, C. C., Berthiaume, C., Van- talon, V., Piquet, C., Gras-Vincedon, A., Martin, C., & Alles-Jardel, M. (2002). A French Study of the Dominic Interactive. Social Psychiatry and Psychiatric Epidemi- ology, 37, 441-448. Wasserman, G.A., McReynolds, L.S., Lucas, C. P., Fisher, P., & Santos, L. (2002). The Voice DISC-IV with incar- cerated male youths: Prevalence of disorder. Journal of the American Academy of Child and AdolescentPsychia- try, 41, 314-321. Wechsler, D. (1991). Wechsler intelligence scale for chil- dren – third edition (WISC-III) manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1997). Wechsler adult intelligence scale – third edition (WAIS-III) technical manual. San Antonio, TX: Hartcourt Brace and Co. Welner, Z., Reich, W., Herjanic, B., Jung, K. G. & Amado, H. (1987). Reliability, validity, and parent-child agree- ment studies of the diagnostic interview for children and adolescents (DICA). Journal of the American Acad- emy of Child and Adolescent Psychiatry, 26, 649-653. .