SATEB NR Vol 1 (1998) Nr 1 85 The Portability of American Job Involvement and Job Satisfaction Scales to Non-English Speaking South Africans L Kamfer and D Venter University of Pon Elizabeth A B BoshotT University of Pretoria ABSTRACT The scales discussed in Boshoff and Hoole (above) were applied to a sample of non-English mother tongue speakers in South Africa to test their "portability" between America and South Africa. Where more than one possible structure was obtained, they were compared by means of confirmatory factor analysis. To reduce error variance and improve goodness of fit indices, items were aggregated by taking the mean of random item clusters, and the confirmatory factor analyses repeated. The best fit solution for each of the scales was identified and discussed. Indications are that both the Minnesota Satisfaction Questionnaire and the Kanungo Job Involvement Scale can be used with confidence in South Africa, even on respondents who are not home language English speakers. INTRODUCTION It is not uncommon for a researcher to use a scale developed in one country to gather data in another, on the assumption that the scale would be measuring the same construct in its new area of application. Edwards & Leger (1995) list various studies in which this has been done by researchers in Africa. The term "portability" has been applied to this aspect of construct validity, and was introduced into the South African literature by Boshoff. Julyan and Botes R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . 86 SAJEMS NS Vol I (1998) No I (1996) in their study of the Kanungo Job Involvement Questionnaire and the Minnesota Satisfaction Questionnaire (MSQ) from this perspective. However, concerns have been expressed as to whether instruments developed in one cultural sening would be appropriate when used in another (Anastasi, 1982; Berry, Pooninga, Segall & Dasen, 1992; Bhagat & McQuaid, 1982; Bhagat, Kedia, Crawford & Kaplan, 1990; Retief, 1988; Taylor, 1987; Triandis, 1972). Retief (1988) suggests a three-part framework (comparability and equivalence, translation of meaning, and bias) to describe the pitfalls that exist when applying tests across cultures. In an attempt to address these concerns, the International Test Commission (1994) has developed guidelines to be followed when using psychological instruments in cross-cuItural studies. At least two studies performed in this country, albeit involving different scales, have shown that different psychometric patterns sometimes occur when scales developed for one language group are applied to another (Edwards & Riordan, 1994; Edwards & Leger, 1995). A major problem in the cross-cultural use of psychometric instruments appears to lie in the systems used to attribute meaning to events in different cultures (Poortinga, 1983; Relief, 1992). Berry & Triandis (1980) and Berry & Lonner (1986) state that for scale scores obtained from different cultures to be comparable, the instrument used must show three types of equivalence: junctional, ie the behaviour referred to should have a similar function in each culture; conceptual, an item should have the same meaning to members of both cultures, and metric, the scale used should have the same psychometric propenies in both the cultural groups concerned. This paper investigates the metric equivalence of the two scales being studied. In a study which also focuses on the metric equivalence of these same two scales, Boshoff et a!. (1996) used exploratory and confirmatory factor analyses to examine the structures produced for the scales on a South African sample which consisted of white collar professionals. They found the nine items which they retained from the Kanungo scale to measure one confirmed factor which accounted for 44.1 % of the variance. The scale had an Alpha of .83. They also found the MSQ to be essentially unidimensional. One factor explained 36.3% of the variance and the scale alpha was .90. They concluded that both scales were robust as far as ponability between the United States and South Africa is concerned. R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . SATEB NR Vol I (1998) Nr I 87 In this study, the portability of these two instruments was again examined, on a South African sample that differed in an important way from that used in the study of Boshoff and his colleagues: scales were applied to respondents who did not have English as their mother tongue METHOD Instruments The instruments used in this study were again the two scales used in the previous study, ie the MSQ and Kanungo's Job Involvement scale. MSQ Median reliabilities of .90 have been reported for the short, 20 item form of the Minnesota Satisfaction Questionnaire (Weiss, Dawis, England and Lofquist, 1967). The MSQ measures job satisfaction in two dimensions, extrinsic and intrinsic, with 10 items for each dimension. A total satisfaction score is also provided by the responses to all 20 items. All items are positively worded. Kanungo's Job Involvement Scale The Kanungo (1982) job involvement scale contains 10 items and has developed as an improvement on the previously widely used measure of Lodahl and Kejner (1965), measuring job involvement as a unidimensional construct (Boshoff, Bennett & Kellerman, 1994). Kanungo reported an Alpha coefficient of .83 for the scale. Two items are negatively phrased and have to be reverse scored. Sample The sample for this study was obtained from a South African government organization, the Department of Correctional Services. The sample frame was 540 employees to whom English was a second language. The two scales used were combined into one questionnaire which was applied by the human resources staff of the client organization. Completed questionnaires were received from 237 respondents (43.9%). Non-response was mainly the result of unavailability of respondents on the day that the data gathering had been scheduled. R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . 88 SAJEMS NS Vol 1 (1998) No 1 Four of the respondents were Indian, 97 coloured, and 136 black. All were matriculated (had twelve years of schooling). Males numbered 213 (90%) while 24 (10%) were female. Mean length of service for the sample was 10.01 years, SO = 7.29. Mean age for the Indian respondents was 32.5 years, SO = 10.5; for the coloured respondents, 37.0, SO = 10.3 and for the black respondents 3L5 years, SO = 10.3. The men had an average age of 30.5, SO = ILl and the women 33.5, SO 0= 12.3. Average age for the sample frame was 33 years, SO = 13.7, and age range was 20 to 59 years. Analyses performed Program 4M of the BMOP (1990) statistical package was used to factor analyze the data. The extraction method was principal component factor analysis with direct quartamin oblique factor rotation to allow for intercorrelation between factors. Factor loadings greater or equal to .30 were regarded as significant. Both Kaiser's rule, ie factors with eigenvalues greater than one, and scree tests were used to aid in decisions on the optimal number of factors to retain. Items with maximum factor loadings less man .30 were omined as well as items that loaded significantly on more than one factor. Items were inverted where necessary to ensure that all were scored in the same direction. Cronbach's alpha was used to determine internal consistency. Confirmatory factor analysis, using the EQS computer package (Bentler, 1996) was performed for one-, two- and three-factor solutions for both scales. first for models consisting of the original items. and thereafter for models with aggregated scores. The following fit indices were inspected to determine which model provided the best fit: R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . SATED NR Vol I (1998) Nr 1 89 (a) Independence Model Chi-square; (b) Model Chi-square; (c) Akaike's Information Criterion (AIC); (d) Bozdogan's Consistent Version of the AIC (CAlC); (e) Bentler-Bonnet Normed Fit Index (BNFI); (f) Bentler-Bonnet Nonnormed Fit Index (BNNFI); (g) Comparative Fit Index (CFI); (h) Residual Normed Index (RNI); (0 Robust Comparative Fit Index (RCFI); (j) Bollen Fit Index (lFI); (k) McDonald Fit Index (MFI); (I) Lisrel Fit Index (GFI); (m) Lisrel Fit Index (AGFI); (n) Root Mean Square Residual (RMR): (0) Root Mean Square Error of Approximation (RMSEA); According to Bentler (1996), Chi-square values are inversely related to model fit. The RMR and RMSEA are also inversely related to model fit. The model that produces the minimum AIC or CAlC may be considered, in the absence of other substantive criteria, to be a potentially useful model. Regarding the fit indices. values greater than .90 are desirable. RESULTS MSQ The indices obtained from the various confirmatory factor analyses conducted on the outcomes of the exploratory factor analyses for the MSQ are shown in Table 1. R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . T: -_._.- - btained fl _ .. _- n f: Indepen- Akaike's Model denee Information Chi- Model Criterion Square Chi-Square (AIC) ~c Three- 203.58 850.97 75.58 factor d/f = 78 solution Two- 410.90 1574.11 72.90 factor d/f=169, d/f = 190 solution 2<·00 One- 503.73 1574.11 163.73 factor d/f=170. d/f = 190 solution ~<.OO MSQ - Bozdogan's Bentler- Bentler- ConsistentVe Bonnett Bonnett rsion of TIle Normed Nonnormed AIC Fit Index Fit Index (CAlC) (BNFI) (BNNFI) -209.01 .76 .78 -678.60 .74 .80 -592.22 .68 .73 Compa- ratlve Fit Index (CFI) .82 .83 .76 Residual Normed Index (RNI) .82 .82 .76 • ~ II) ~ m ~ ~ ~ ~ ~ R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . Table I (continued) Robust Bollen McDonald Lisrel (RCFI) (IFI) (MFI) (GFI) Three- .83 .82 .74 .89 factor solution Two- .85 .83 .59 .85 factor solution One- .78 .76 .49 .81 factor solution Root mean Usrel ( square residual AGFI) (RMR) .85 .21 .81 .09 .76 .10 Root mean square error of approxi- ation (RMSEA) .10 .08 .09 --- til ~ ~ g: ~ j ~ 10 R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . 92 SAJEMS NS Vol 1 (1998) No 1 MSQ From Table 1 it can be seen that the three-factor solution is best on four of the indices reponed. BNFI, MFI. GFI and AGFI. The two-factor solution has the best score on eight indices, AIC. CAlC, CFt BNNFI, RCFI, IFI, RMR and RMSEA. The one-factor has no bests. The two-factor solution is shown in Table 2. In Table 2 (and later in Table 6) the factor loading matrix has been rearranged so that the columns appear in decreasing order of variance explained by factors. The rows have been rearranged so that for each successive factor. loadings greater than .30 appear first. For ease of reading, loadings less than .30 have been omitted. The two-factor solution is preferred because it performs best on more of the goodness of fit indices produced by the confinnatory factor analysis than either of the other two solutions, although the three-factor solution also does well. Also in favour of the two-factor solution is the fact that it retains all of the original 20 items (compared with the II retained in the three-factor solution). It also has higher Cronbach's Alphas, .87 and .75 compared to the .79, .66 and .80 of the three-factor solution. Its configuration also closely resembles the Extrinsic- Intrinsic aspect of the MSQ in its original form. The two factors explain 40.62 % of the total variance. Factor 1 contains 11 items, being all eight of the original "extrinsic" items, plus three "intrinsic" items. This factor explains 31.51 % of total and 77.57% of common variance. Factor 2 contains 9 items, all "intrinsic". It explains 9.11 % of total and 22.43% of common variance. The two factors correlate .41 with each other. In their analysis of the MSQ, Boshoff et aI. (1996) also retained all 20 items. They preferred a one-factor solution which had an alpha of .90, the one factor explaining 36.3% of total variance. Their confinnatory factor analysis fit indices were acceptable and better than the ones portrayed in Table I, which are not very good. In an attempt to improve the fit indices by reducing the error variance (Bagozzi & Heatherton, 1994), the items in the MSQ were aggregated into subscales. This was done by taking the mean of random item clusters. The three-factor structure was represented by the following MSQ items (factors in square brackets and clusters in parantheses) [(15, 19, 18), (16, 20)]; [(9, 7, 3), (4, 2, 8),]; [6, 5]; two-factor structure was [(19, 6, 5, 16), (14, 13, 12, 17), (15, 18,20)]; [(10, 1,9), (7, 3, 2), (8, lI, 4)]. The one-factor structure contained all 20 items clustered as follows: [(13,3,7, 10), (8, 15,4,5), (2, l, 14, 19), (12.6, 17,9), (16, 11,20, 18)]. R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . SATED NR Vol 1(1998) Nr I Tablel Soned rotated factor loadings for best fit. exploratory factor analysis on MSQ MSQ Item ~ wo,kio, oood;,"', 19 The praise) get for doing a good job 13 18 12 6 16 IS 5 14 20 9 4 8 3 \0 11 2 7 I My pay and the amount of work 1 do The way my co-workers get along with each other The way company policies are put into practice The competence of my supervisor in making decisions The chance to try my own methods of doing the job The freedom to use my own judgement The way my boss handles his workers The chances for advancement on this job The feeling of accomplishment) get from the job The chance to do things for other people The chance to be "somebody" in the community The way my job provides for steady employment The chance to do different things from time to time The chance to tell people what to do The chance to do something that makes use of my abilities The chance to work alone on the job Being able to do things that don't go against my conscience Being able to keep busy all the time % Common variance % Total variance % Total variance explained E .. Item measures extnnslC Job satisfactIOn Item measures intrinsic job satisfaction 93 two- factor solution after Fac- 9 tor 1 (E) .77 (E) .72 I (E) .71 (E) .67 (E) .63 (E) .63 (I) .61 (I) .59 (E) .58 (E) .50 (I) .42 (I) .78 (I) .61 (I) .60 (I) .58 (I) .57 (l) .54 (I) .42 (I) .41 (I) .37 77.57 22.43 31.51 9.11 40.62 R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . 94 SAJEMS NS Vall (1998) No I Confirmatory factor analyses were then performed to compare the degrees of fit produced for these aggregated structures. Results appear in Table 3. Table 3 shows that the three-factor solution can be discarded as it performs significantly worse than the one- and two-factor solutions. Its model chi-square is significant. All its fit indices are unacceptably low, well below those of the other solutions. The two-factor solution appears to be better than the one-factor solution: it does better on the model chi-square, AIC and CAlC whilst all the other fit indices are identical to those for the one-factor solution. For both two- and one-factor solutions model chi-square probabilities are not significant, which indicates a non-significant deviation from the postulated factor model. In the two-factor solution, the two factors explain 71.58% of total variance. Factor 1 explaining 55.72% and Factor 2, 15.86%. Factor I explains 77.84% of common variance, and Factor 222.16%. The factors correlate .55. Cronbach's Alpha for Factor 1 is .86, and for Factor 2 •. 73. To summarize: the two-factor solution for the MSQ is preferred in both the non-aggregated and the aggregated analyses. In the non-aggregated analysis, the two factors explain 40.62% of total variance. Factor 1 accounts for 31.51 % of total and 77.57% of common variance. Factor 2 explains 9.11 % of total and 22.43% of common variance. The Cronbach's Alphas are .87 for Factor 1 and .75 for Factor 2. The correlation between the factors is.41. In the aggregated analysis, the two factors explain 71.58% of total variance, Factor 1 accounting for 55.72% and Factor 2 for 15.86%. Factor 1 explains 77.84% and Factor 2, 22.16% of common variance. The factors correlate .55. Cronbach's Alphas are .86 for Factor I and .73 for Factor 2. R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . ._- - -------- -- ,---_._.- fi n . f: Indepen- Akaike's Model dence InfonnatiOn Chi- Model Criterion Square Chi- Square (AIC) Three-factor 84.40 525.55 68.40 solution d/f=8 d/f 15 1'< .00 Two-factor 5.96 559.95 -10.04 solution d/f=8 d/f 15 1'=.65 One-factor 4.36 624.15 -5.64 solution d/f = 5 d/f = 10 p =.50 .-.... _., .. _ .. _- - fMSQ - Bozdogan's Bentler- Bentler- ConsistentVe Bonnett Bonnett rsion of The Nonncd Nonnonned AIC Fit Index Fit Index (CAlC) (BNFI) (BNNFI) 32.82 .84 .72 -45.62 .99 1.00 -27.87 .99 1.00 Compa- rath'e Fit Index (CFI) .85 1.00 1.00 Residual Nonned Index (RNI) .85 1.00 1.00 ~ Z ~ ~ '"' j ~ ::5: R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . Table 3 (continued) Robust Bollen Mc Donald Lisrel (ReFI) (lFI) (MFI) (GFI) Three-factor .87 .85 .85 .91 solution Two-factor 1.00 1.00 1.00 .99 solution One-factor 1.00 1.00 1.00 .99 solution Root mean Lisrel ( square residual AGFI) (RMR) .75 .25 .98 .01 .98 .01 Root mean square error of approxi- atlon (RMSEA) .20 .00 .00 I I I ~ en c: tTl ~ en Z en 2: ~ Z o R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . SATES NR Vol 1 (1998) Nr 1 97 Kanungo's Job Involvement Scale Exploratory factor analyses were performed on Kanungo's Job Involvement scale producing three-, two- and one-factor solutions. Confirmatory factor analysis was conducted to determine the goodness of fit of the three solutions. Results appear in Table 4. R ep ro du ce d by S ab in et G at ew ay u nd er li ce nc e gr an te d by th e P ub lis he r (d at ed 2 00 9) . Table 4 Indices obtained from confirmatory factor analyses on Kanungo Bozdogan's Bentler- Bentler- Akalke's Consistent V Bonnett Bonnett Indepen- ersion or Nonned Nonnonne dence Inronnatio The AIC Fit Index d Fit Model nCriterion Chl- Model Index Square Chi- (CAlC) (BNFI) Square (AIC) (BNNFI) Three- 35.76 316.83 19.76 -15.81 .89 .83 factor d/f=8 solution p