iassist quarterly - 3 Techniques for secondary analysis: Unfolding analysis of "PICK K/N" and "PICK ANY/N" data by Wijbrandt H. van Schuur' Faculty of Social Sciences University of Groningen Oude Boteringestraat 23 9712 GC Groningen The Netherlands Introduction In survey research we regularly encounter the following type of question: "which of these stimuli do you prefer most? which of the remaining ones do you now prefer most?" etcetera. Sometimes a full rank order of preferences is asked in this way, more often only a partial rank order is obtained. 'Paper prepared for presentation at the IFDO/IASSIST Conference, Workshop on Techniques for Secondary Analysis, Amsterdam, May 20 - 23, 1985. Comments are welcomed by the author. Sometimes the question asked is only: "which k of these n stimuli do you prefer most?" or, even more generally, "which of these n stimuli do you prefer?" Such questions can be referred to as 'ramk n/n', 'rank k/n', 'pick k/n' and 'pick any/n' data, respectively. Stimuli may be political parties, candidates, career possibilities, or brand names of some consumer good. Rather than asking about 'preference', the questions may also be phrased in terms of other evaluative concepts, such as 'sympathy' or 'importance'. In this paper I will be concerned with analyzing data of the form 'pick k/n' or 'pick any/n'. Generally these data types are difficult to analyze. Often responses to such data are only reported in the form of frequency distributions of the number of times a stimulus is mentioned as most preferred, second most preferred, etcetera. Trying to find structure in these responses with the help of standard techniques, such as factor analysis or cumulative scaling, is not possible either because no full set of responses to all stimuli is available, or because the responses given are not independent It is then difficult to determine whether or not all responses given were based on the same underlying criterion. In this paper an analysis technique is presented that allows one to look for structure in the responses to 'pick k/n' or 'pick any/n' questions. Since complete or partial rank orders can always be recoded to the 'pick k/n' form, and since survey questions with independent responses, such as five-point Likert items, can be recoded to the 'pick any/n' form, the type of data analysis presented here can have very general applicatioa The data analysis technique presented here is a dichotomous version of the unfolding model, proposed by Coombs (1950, 1964), as 'parallelogram analysis'. It differs from Coombs' original proposal in the following ways: the technique proposed here allows for some error (i.e., it conforms to a stochastic model), and it is an exploratory technique to search for Summer 1986 4 - iassist quarterly maximal subsets of stimuli that can be represented in a unidimensional unfolding scale. In both these aspects the 'parallelogram analysis' model proposed here resembles the stochastic unidimensional aunulative scaling technique developed by Mc^en (1971). The reader should be warned that the technique presented here is not an all-purpose technique for analyzing 'pick k/n' or 'pick any/n' data, but only for those types of data which can be expected to conform to the unfolding model! The perfect unidimensional unfolding model for complete rank orders of preference In this section I wiU first simimarize some basic ideas behind unfolding analysis, by using an example from Meerling (1981). In an investigation by Ritzema and Van de Kloot, preference rank orders were collected for the following statements: : People can be changed in any conceivable direction, provided that the environment is manipulated in the proper way (O = omgeving, envirormient); 1 : The major condition for people to change is for them to have a clear understanding of their situation (I = inzicht, understanding); E : Behaviour is determined much more strongly by emotions than by rational considerations (E = emoties, emotions); A : Inborn characteristics determine to a large extent what kind of person someone becomes (A = aangeboren, inborn). These four statements were shown to psychologist colleagues, and the following six types of preference rank orders were foimd: OIEA. lOEA. lEOA, HOA. EAIO, and AHO. In applying the unfolding model we assume that there is a latent dimension on which each of these statements can be represented. Meerling suggests for these statements and these preference rank orders that a 'nurture-nature' dimension may be appropriate, in which the statements are arranged in the order OIEA. When the location of each of the statements on this dimension is established, the dimension can be divided into two areas for each pair of statements (I,J): the first area, in which the first statement is preferred over the second, and the second area, in which the second statement is preferred over the first The boundary between these two areas lies in the middle between these two stimuli, and is called the 'midpoint of the pair of stimuli', m(IJ). This midpoint allows us to locate individuals who give their preference rank order along this dimension. An individual, PI, who prefers statement O to statement I will be located to the left of midpoint m(OI), whereas another individual, P2, who prefers statement I to statement O, will be located to the right of that midpoint (see Figure 1) Figure 1 Midpoint m(OI) divides the dimension areas into two PI ; P2 ! 1 m(OI) I The four statements, together, have six midpoints. These divide the dimension into seven areas, the areas that are separated by the midpoints. Each of these areas is characterized by a special preference rank order, and is called an 'isotonic region', (see figure 2) Summer 1966 iassist quarterly - 5 Figure 2 4 stimuli, 6 midpoints, and 7 isotonic areas or subject types A subject is usually represented on the scale by a single point, called his 'ideal point'. The preference order of the subject is called his 'Individual scale', or 'I-scale' for shorL The representation of all subjects and all stimuli jointly on the same dimension is called the 'Joint-scale', or 'J-scale' for short The I-scale gives the order of the stimuh in terms of their distance from the ideal point of the individual. In other words: the I-scale has to be 'unfolded' at the ideal points to produce the J-scale. Unfolding analysis is designed to find a joint representation of stimuli and subjects in one dimension, that is, to find a unidimensional J-scale on the basis of the preference rank orders of the individual 1-scales. Finding a J-scale brings us two things. The first is an unfoldable order of the stimuli which can generally be used to infer the criterion used by the subjects in determining their preference order (e.g., the nurture-nature criterion). Secondly, having a J-scale allows us to combine a subject's answers to the n survey questions in a single rank order which can be used to measure the preference of the subject in terms of his ideal point on the criterion dimensioiL By measuring a subject's preference in this way we can create a new variable which can be related to other characteristics of the subject The purpose of creating such a new variable is to try to explain why people differ in their preferences, or to explain other attitudes or behaviours on the basis of scale values on the J-scale. If we have perfect data, such as we usually find in textbooks on scaling (and by perfect data I mean I-scales that can be perfectly represented in a unidimensional unfolding scale) it is no problem to find the J-scale that represents the I-scales. Problems only arise when the data are not perfect, which is in most cases. The major reason why the unfolding technique has so far been relatively unpopular and why it has as yet not been incorporated into most standard statistical packages, is that up to now we have not been able to imfold imperfea data in a satisfactory way. If we could find a usable unfolding technique, interest in it should be great, since the model is plausible, and there is a great deal of interest in measuring the preferences of subjects. Discussion of some alternative proposals for unfolding models Before introducing my own model, I will first consider five strategies that have been developed in the literature and which attempt to find useful and interpretable unfolding results. These strategies are all derived from a description of the ideal type of unfolding analysis, namely the perfect representation of a complete rank order of preferences in a unidimensional space, in which all stimuli and all individuals can be represented. These five strategies are: 1. Analyze the I-scales after they have been dichotomized into the k most preferred and n-k least preferred stimuli; 2. Relax the criterion of perfect representation to allow stochastic representation; 3. Find a representation in more than one dimension; Summer 1986 6 - iassist quarterly 4. Find a representation for a maximal subset of the stimuli; 5. Find a representation for a maximal subset of the subjects. The first strategy is to dichotomize full or partial rank orders of stimuli into the k most prefened and the n-k least preferred stimuli. The unfolding analysis of such data, parallelogram analysis, can be defended with the argument that the stimuli a subject prefers most will be the most salient ones for him, and a subject will therefore be able to single them out more rehably than the remaining ones. Moreover, although the imfolding model assumes that successively chosen stimuli are in a sense substitutes for the subjects' most preferred stiumlus to a deaeasing degree, graudally, in the course of giving a full rank order of preference, a subject may begin to use other criteria. Coombs (1964) talked about the 'portfoUo model* in this respect, and Tversky (1972, 1979) suggested an 'Elimination by Aspects' model, in which different criteria for preference are hierarchically ordered. If we are interested in finding the dominant criterion that is used first by all respondents, then we should restrict ourselves to analyzing only the first few most preferred stimuli, lest we run the risk of introducing idiosynaatic noise. Two more practical advantages of this strategy can be mentioned. First, if applying an unfolding model in which the distinction between the k most preferred and n-k least prefered stimuli does not lead to a good-fitting representation, it is no use trying more sophisticated models that require the full rank order, or that may even require metric preference information. Second, the unfolding of dichotomous data implies that essentially all types of data can be used in a preference analysis, as long as the most preferred responses can be distinguished from the others. The second strategy is to relax the criterion of perfect representation to allow stochastic representatioa I regard it as obvious that preference judgments reflect so many idiosyncratic influences, that we should be happy to fmd that a rather heterogeneous group of subjects agrees on at least a dominant criterion. Stochastic models have been proposed before (SixU, 1973; Zinnes and Griggs, 1974; Bechtel, 1976; Jansen, 1981). I regard tiiese proposals as inferior to the model 1 propose for at least two reasons. Firstiy, many of the probabilistic unfolding models assume that the order of stimuh along the J-scale is already known, and only parameter estimation of subjects and stimiili on the basis of the known order is needed. In many cases such an approach is begging the question, as often the order of the stimuli is not known in advance. Secondly, other stochastic unfolding models require that for each subject, we need the probability of his preferring one stimulus to another. In many practical applications this information is impossible to obtain: it is expensive and time consuming enough to ask respondents one single time to compare all pairs of stimuli with respect to preference. A third strategy to analyze data that are not unfoldable in one dimension is to try to represent them in more than one dimension. It is possible that subjects did not use a single criterion in making their preference judgments, they may instead have used two or three criteria simultaneously. Multidimensional models have been proposed by Bennett and Hays (1960), Roskam (1968), SchOnemann (1970), Carroll (1972), Young (1972). Gold (1973), Kruskal et al (1973), and Reiser (1981), among others. They are appealing, because the use of more than one dimension implies the possibility of using a number of additional models that differ in the way in which the various dimensions are combined: the vector model, the weighted distance model, or the compensatory distance model, to mention only a few. Summer 1986 iassist quarterly - 7 There are at least four possible problems with the multidimensional unfolding model. First, in applying a nonmetric multidimensional unfolding model, we may fmd an almost degenerate solution, in which most subjects are close together in the centroid of the space, and most stimuli lie in a circle around iL Secondly, also with respect to nonmetric multidimensional unfolding, there is a fundamental difference between the nonmetric analysis of similarities data and the nonmetric analysis of preference data, even though both models are based on the same principle. In the multidimensional analysis of siniilaiities, the isotonic region in which a stimulus falls becomes so small that for a sufficient number of stimuh each stimulus can only be represented by a point in the space, rather than by a region. But in multidimensional unfolding, the representation of some respondents in the form of such isotonic regions is different; some isotonic regions do not shrink to points, but remain open. Such respondents cannot be uniquely represented by one point in the space. Thirdly, multidimensional unfolding assumes that all dimensions are used simultaneously, rather than in a hierarchical order. This is an empirical question, rather than an untestable assumptioa Fourth, the assumption that all dimensions are appropriate for all stimuli is equally an empirical question, rather than an untestable assimiption. We are told that reality is not unidiraensional. Indeed, a chair has a colour, a weight, and a nimiber of sizes. A person has an age, a sex, and a preference for certain drinks. And a pohtical party may be large, religious and right wing. Still, we never analyze reaht>'. We analyse aspects of reality! We do not compare chairs, subjects or political parties, but sizes of chairs, ages of subjects and ideological positions of political parties. That objects or subjects have more aspects than the ones in which we are interested, does not at all imply that our analyses need to be multidimensional. They may be, but that is an empirical question, and not an untestable assimiption from the outset I do not fundamentally object to a multidimensional representation of the preferences of a group of subjects. There may be instances in which this is indeed the best model. But the utility of different models will have to be shown in their practical applicability. With respect to the last two strategies for salvaging the imfolding model, selecting a maximal subset of stimuh and selecting a maximal subset of subjects, it is estabhshed practice in multidimensional unfolding analysis to assign stress values to subjects. This implies that any difficulties in fmding a representation can be explained by pointing at suljjects who used different criteria, or who perhaps even behaved completely at random. A possible procedure, given this assumption, is to delete respondents whose stress values are too high. However, it may be the case that large stress values occur because one or more stimuli caimot be represented since they do not belong in the same imiverse of content as the other stimuli. Subjects are allowed to differ in their evaluation of the stimuh, but for unfolding to be applicable, they must agree on the cognitive aspects of the stimuh; whether gentlemen prefer blondes or brunettes is a different matter from estabUshing whether Marilyn is blonde or brunette. If there is no agreement among the subjects on the characteristics of a stimulus, differences in preference will be difficult to represent Often, subjects are selected as representatives of a larger population. Deleting subjects lowers the possibility of generalizing from a sample to a population. Stimuli, on the other hand, are often not so much a random sample of a population of stimuli, but are more often intended to serve as the best and most prototypical indicators of a latent trait; we are often not so much interested in the actual stimuli, but rather in their implications for measuring subjects along this latent trait This Summer 1986 iasast quarterly means that we generally can delete stimuli with less harm that when we delete subjects. The discussion of these strategies is intended to justify the strategy adopted in the technique to be described below, of finding a stochastic representation of a maximal subset of stimuli and all subjects in one dimension, using the first few preferences of each subject Unfolding dichotomous data: the concept of 'error' We generally do not know in advance which stimuli can be represented in an imfolding scale, nor in which order they can be represented. The approach used here is a form of hierarchical cluster analysis, in which first the best, smallest unfolding scale is found, and then is extended by more stimuli, as long as they continue to satisfy the criteria of an unfolding scale. The smallest unfolding scale consists of three stimuli, since it takes at least three stimuli to falsify the unfolding model. If stimuli A, B, and C form a perfect unfolding scale in this order, then subjects who prefer A and C but not B, do not exist For the unfolding scale ABC the response patterns in which A and C are prefened but B is not, is defined as the 'error pattern' of that triple of stimuli. But since we do not know in advance in what order the stimuli form an unfolding scale, we must take into account the three permutations in which each of the three stimuli is the middle one: BAG, ABC, and ACB. If a subject prefers A and B, but not C, for example, he makes an enor according to the unfolding scale ACB. For each triple of stimuli, given a dichotomous response to each stimulus, eight response patterns are possible: 111, 110, 101, Oil, 100, 010, 001, and 000. If these stimuli form pan of an unfolding scale, then one of these eight patterns cannot occur: the pattern '101' (see Table l)^ This pattern is called the 'error response'. For each triple of stimuli, in each of its three possible permutations, the frequency of occurrence of the error pattern can be counted. Counting frequencies of enor response patterns can be extended to larger response patterns, in which each subject evaluates more stimuli. Table 2 gives five response patterns in which two or three stimuli are preferred from a set of four. In the first two response patterns only one triple is in error. In the last three response patterns two triples are in error. The amount of error in a response pattern is defined as the number of triples in that response pattern that are in error. The last three response patterns therefore contain twice as much error as the first two. In the second example, four subjects prefer six out of seven stimuli. It makes an enormous difference to the amount of error in their response patterns whether the stimulus not preferred is D, C, B, or A. In the case of D, the amount of error is maximal, whereas in the case of A there are no errors at all. Stochastic unfolding The stochastic aspect of the unfolding strategy proposed here lies in comparing the amount of error observed with the amount of error expected under statistical independence. In the deterministic unfolding model, the k stimuli that are preferred by a subject are found within the symmetric closed interval around the subject's ideal point The probability of preferring a set of stimuli (e.g., two, three, or more) will be '1' if all stimuli fall within the subject's preference ' Editor's note: Tables are gathered together at end of article Summer 1986 iassist quarterly - 9 interval, and '0' if at least one stimulus falls outside this interval. The null model differs from the deterministic model in two ways. First, local independence is assumed among preference responses for different stimuli. This means that for each subject the probability of a preference response pattern to a set of stimuli is the product of the positive (preferential) response to each of the stimuli. Second, the null model assumes that there are no individual differences in the probabilities of giving a positive preference response to the stimuh. The expected frequency with which a set of stimuli is preferred will therefore be the product of the relative frequencies with which each stimulus is prefened times the number of cases, if subjects are free to selea as many 'most prefened' stimuli as they wish: Exp.Freq(ijk,101) = p(i).(l - pO)).p(k).N where p{i) is the relative frequency with which stimulus i is preferred and N is the number of cases. The expected number of errors under the null model for 'pick k/n' data is first explained for 'pick 3/n' data. It consists of two steps: 1. determine the expected frequency of the '111' response pattern by applying the n-way simple quasi-independence model (e.g., Bishop et al, 1975); 2. from the '111' responses to each triple, other response patterns - like 110, 101, or Oil - cjm be deduced. In a data matrix, in which each of the N subjects picks exactly 3 of n stimuli as most prefened, the relative frequency p(i) with which each stimulus is picked can be found. In the null model, these p(i)'s are derived from the addition of the expected frequency of triples (ijJc) for all combinations of j and k with a fixed i. This expected frequency of triples ijjc, a(ijk), is the product of the item parameters f(i), f(j), f(k) times a general scaling factor f. without interaction effects: a(ijk) = f fi;i).flj).f(k). The values of f, and each f(i) are found iteratively. (see Table 3) The details of this procedure are given in Van Schuur (1984). Once the expected frequency of the '111' pattern of all triples is known, the expeaed frequency of the other response patterns can be foimd, given that each subject picked exactly 3 stimuli as most prefened. For example: Consider the situation in which there are five stimuli. A, B, C, D. and E, and each subject chooses three stimuh as most preferred. For the unfolding scale ABC the error response pattern is the pattern 101, in which stimuh A and C are picked, but stimulus B is not If B was not one of the subject's choices, then D or E must have been. We can therefore calculate the expected frequency across all respondents of the response pattern 101 for the triple ABC by summing the expected 'HI' responses of the triples ACD and ACE In general: Exp.Freq.(ijk,101) = ff[i).n:k).I fi[s) This procedure can easily be generalized to the 'pick k/n' case, where k = 2, or where k > 3. First, the expeaed frequency of each k-tuple, ranging between 1 and (") is found. Second, the expected frequency oT the enor response pattern of an unfolding scale of three stimuli is foimd by calculating: Exp.Freq.(ijk,101) = ff^i).fIk).Q where Q is the sum over all (?_-) k-2 tuples of the product of their f(s)'s, where s is not equal to i, j, or k. Once we know the frequency of the error response observed, Obs.Freq.(ijk,101), as well as the frequency expected imder the null model. Summer 1986 10 - iassist quarterly Exp.Freq.(ijk,101), for each triple of stimuli in each of its three essentially different permutations, we can compare the two using a scalability coefTicient analogous to Loevinger's H (Loevinger. 1948; Mokken. 1971): H... = 1 - Obs.Freq.{iJk.l01) Exp.Freq.(ijk,101) For each triple of stimuli (ij, and k), three coefficients of scalability can be found: H(ijk), H(ikj). and H(jik). Perfect scalabihty is defined as H = 1. This means that no error is observed. When H = the amount of error observed is equal to the amount of error expeaed under statistical independence. The scalability of an unfolding scale of more than three stimuli can also be evaluated. In this case we can simply calculate the sum of the error responses to all relevant triples of the scale, for both the observed and expected enor frequency, and then compare them, using the coefficient of scalability H: 3 J3 Obs. Freq. (i jk , 101 ) H = 1 _„iii>l=li 3 j; E;;p. Freq. (i jk , 101 ) < V jk= 1 > MUDFOLD: Multiple Unidimensional unFOLDing, the search procedure After having obtained all relevant information about each triple of stimuli in each of its three different permutations (e.g., Obs.Freq.(ijk,101), Exp.Freq.(ijk,101), and H(ijk), we can begin to construct an unfolding scale. This is a two-step procedure. First, the best elementary scale is found, and second, new stimuli are added, one by one. to the existing scale. The best triple of stimuU that conforms to the following criteria is the best elementary scale: 1. its scalability value should be positive in only one of its three permutations, and negative in the other two. This guarantees that the best triple has a imique order of representation; 2. its scalability value must be higher than some user specified lower boundary. This guarantees that if the scalability value is positive, it can be given a substantively relevant interpretation. 3. the absolute frequencies of the perfect patterns with at least two of the three stimuli (i.e.. Ill, 110, and Oil) is highest among all triples fulfilling the first two criteria. This guarantees the representativeness of the largest group of respondents. The scalability of single stimuli in the scale can equally be evaluated, by adding up the frequencies of the enor patterns observed and expected, respectively, in only those triples that contain the stimulus under consideration, and then comparing these frequencies using the scalability coefficient for each stimulus separately. Once the best elementary scale is found, each of the remaining n-3 stimuli is investigated to determine whether or not it might make the best fourth stimulus. The fourth stimulus (e.g., D) may be added to the three stimuli of the best triple (e.g., ABC) in any one of four places: DABC, ADBC. ABDC, or ABCD, denoted as place 1 through place 4, respectively. The best fourth - or, more generally. Summer 1986 iassist quarterly - 11 the criteria: p+l-st _ stimulus must fulfill the follOMdng All new (P) triples, including the p+l-st stimulus and two stimuli from the existing p-stimulus scale, must have a positive H(ijk)-value. This guarantees that all stimuli are homogeneous with respect to the latent dimensioa The p+l-st stimulus should be uniquely representable, in only one of the p possible places in the p-stimulus scale. This guarantees the later usefulness and interpretability of the order of the stimuli in the scale. The H(i)-value of the new stimulus, as well as the H-value of the scale as a whole, must be higher than some user-specified lower boimdary (see second criterion for the best elementary scale). If more than one stimulus conforms to the criteria mentioned above, that stimulus will be selected which leads to the highest overall scalability value for the scale as a whole. The dominance and adjacency matrices: visual inspection of model conformity Once a maximal subset of unfoldable stimuli is found, a final visual check of model conformity can be performed by inspecting the dominance and adjacency matrices. The dominance matrix is a square, asymmetric matrix which contains in its cells (ij) the proportion of respondents who preferred stimulus i but not stimiJus j. If the stimuh are in their order along the J-scale, then for each stimulus i the proportions p(ij) should decrease from the first column toward the diagonal and increase from the diagonal to the last column. The adjacency matrix is a lower triangle that contains in its ceUs (iJ) the proportion of respondents who preferred both i and j. If the stimuli are in their order along the J-scale, then for each stimulus i the proportions p(ij) should inaease from the first column to the diagonal and deaease from the diagonal to the last row. This pattern is called a 'simplex pattern*. Stimuli that disturb these expected characteristic monotonidty patterns should be considered for deletion from the scale.(see Table 4). This procedure, of extending a scale with additional stimuli, can continue as long as the criteria mentioned above are met If, however, no stimulus conforms to these criteria, the p-stimulus scale is a maximal subset of unfoldable stimuli. A new procedure then starts which begins by selecting the best triple among the remaining n-p stimuli. This procedure, in which, for a given pool of stimuli, more than one maxima] subset of unidimensionally unfoldable stimuli can be found, is called 'multiple scaling'. Scale values Once an unfolding scale of a maximal subset of stimuli has been found, scale values for stimuli and subjects must be foimd. The scale value of a stimulus is defined as its rank number in the unfolding scale. The scale value of a subject is defined as the mean of the scale values of the stimuli that the subject chose as most preferred. Subjects who did not pick any stimulus from the scale cannot be given a scale value, and must be treated as missing data. An example of the assignment of scale values is shown in Table 5. Summer 1986 12 - iassist quarterly Respondents may have different response patterns, but be assigned the same scale value. This can be seen by comparing subjects 1, 2. and 3. Subjects 4 and 5 show that a scale value for a subject does not need to be an integer value. Respondent 6 shows that a scale value is assigned to a subject regardless of the amount of error in his response pattern, which in his case is maximal. Subject 7 does not pick any of the 7 stimuli and therefore cannot be represented on this scale. An example: Pick the 2 most sympathetic of 6 European party groups As part of the Middle Level Elite Project (e.g.. Van Schuur, 1984), sympathy scores for six European party groups in the European Parliament of 1979 were elicited from party activists from 50 political parties in the European Community. The responses of 1786 subjects about their two most sympathetic party groups were analyzed. The six party groups are, with the letter by which they will be denoted, and with the frequency with which they were mentioned as sympathetic in brackets: A: Communists (359); B: Social Demoaats (747); C: European Democrats for Progress (366); D: European Liberals and Demoaats (662); E: Christian Democrats (792); and F: Conservatives (646). The frequency with which each pair of parties was mentioned as most sympathetic is: AB(341) AC(9) AD(3) AE(2) AF(4) BC(106) BD(202) BE(86) BF(12) CD(124) CE(50) CF(77) DE(217) DF(116) EF(437). On the basis of this information, a labeled matrix can be constructed that contains, for each triple of stimuli in each of its three essentially different permutations, the values Obs.Freq.(ijk,101), or E(o), Exp.Freq.(ijk,101), or E(e), and H(ijk). This information is given in Table 6. Table 6 provides all the necessary information for constructing an unfolding scale. First, the best elementary imfolding scale is found among those triples that have a positive scalabihty value in only one of its three permutations. This leaves the ordered triples ABC, ABD, BCF, BDE, CDE, DCF, CFE, and DEF. Triple ABD is the best triple, since the sums of the pairs (A3) and (Bj)) is highest. The H-value of triple ABD is 0.96, which is well above the recommended default user specification of 0.30. On the basis of scale ABD, stimulus C cannot be represented in this scale in any position, since the triple B,CJ) has negative H-values in all three permutations. Stimulus E is uniquely representable in place 4, forming scale ABDE, whereas stimulus F is representable in either place 1 (scale FABD) or place 4 (scale ABDF). Stimulus E is selected because it is the only one uniquely representable. The four-stimulus scale is ABDE, its H-value is 1 - 93/485 = 0.81. which is acceptably high. For the best fifth stimulus, we need only consider stimulus F. This is now only representable in place 5, which gives the final scale ABDEF. Its H-value is 1 - 245/1185 = 0.79. In the process of scale construction, the H-values of individual stimuli are also calculated. For the triple ABD these values are the same: H(A) = H(B) = H(D) = H(ABD) = 0.96. For the four- and five-stimulus scales these values must be computed separately. The resulting H-values for the final scale are shown in Table 7, along with the dominance matrix and the adjacency matrix for the stimuli in the order of the final scale. Neither matrix shows any violation of the expeaed characteristic monotonicity pattern. Five of the six European party groups can be included in an unfolding scale based on party Summer 1986 iassist quarterly - 13 activists' sympathy scores for these party groups. The scale can be interpreted as a left-right dimension, with the Communists represented in the left-most place and the Conservatives in the right-most place. To corroborate this interpretation, I have correlated subjects' scale scores for this imfolding scale with their scores on a left-right self-placement scale. This correlation was 0.66. The European Demoaats for Progress (EDP, stimulus C) was not incorporated in the scale. This party group consists of the French Gaullists (RPR), the largest Irish party Fiaima Fail (FF). and the Danish Progress Party (FRP). This party group is not represented in many EC countries, so it is probably less well know than other party groups, and did not, therefore, receive high sympathy scores from respondents who might have been expected to be sympathetic, based on their positions on the scale. Concluding remarks The procedure described above for the analysis of 'pick k/n' data can be extended to apply to 'rank k/n' data. Such procedures have been independenUy proposed by Davison (1978) and by Van Schuur and Molenaar (1982). Using partial rank order information might provide more precise measurements for both the stimuli and the subjects. However, since in this procedure all six permutations of a triple of stimuli have their own observed and expected error patterns, the accuracy of estimation with the same data set decreases sixfold. As table 7 already shows, the H(ijk)-value of some triples is based on a comparison of rather small numbers, and such comparison will therefore be even more difficult in the 'rank'-case. Moreover, for small k the increase in measurement precision is minimal, and 1 have already expressed some doubts about the rehability of the k-th preference judgment, when k gets large. A computer program (MUDFOLD) has been devised to perform a multiple unidimensional unfolding aiialysis on complete or partial rank order data, 'pick k/n' or 'pick any/n' data, or on the usual attitudinal data, such as Likert items or thermometer scores. The program is interactive, self-explanatory, and very user-friendly. The user may define a startset rather than use the best elementary scale to find a larger unfolding scale, or test the unfoldability of a given set of stimuli in a given order. In either case, if a triple of stimuli in the user defined order has a negative H(ijk)-value, this triple will be flagged, along witii its E(o>-, E(e)-, and H(ijk)-values. The output not only consists of the H- and H(i)-values of the final scale, but also gives an overview of which stimuh at which places were candidates for selection at what step of enlargement, and the H- and H(i>-values of the stimuh in the scale at which step of enlargement Moreover, the output contains a variety of additional information which may help the researcher either find a better scale, or explain why certain stimuh did not fit in the unfolding scale. The computer program is available from the University of Groningen. The development of the unfolding model presented above, together with more than twenty applications, is described in more detail in my dissertation (Van Schuur. 1984).Q References: Bechtel, G.G. (1976). Multidimensional preference scaling . The Hague: Moutoa Bennett, J.F. and W.L Hays (1960). Multidimensional unfolding: determining the dimensionality of ranked preference Summer 1986 14 - iassist quarterly data. Psvchometrika. 25. 27-43. Bishop, Y.M.M.. S.E Fienberg. and P.W. Holland (1975). Discrete multivariate analysis: theory and practice . Cambridge, Mass.: MIT Press. Carroll, J.D. (1972). Individual differences and multidimensional scaling. In: R.N. Shepard et al (eds.). Multidimensional scaling. Vol. I: theory. New York: Seminar Press. Coombs. C.H. (1950). Psychological scaling without a unit of measurement Psychological review. 57, 148-158. Coombs, C.H. (1964). A theory of data. New York: WUey. Davison, M.L (1979). Testing a unidimensional, qualitative imfolding model for attitudinal or developmental data. Psvchometrika. 44, 179-194. Gold. EM. (1973). Metric unfolding: data requirements for unique solutions and clarifications of Schbnemann's algorithm. Psvchometrika. 38, 555-569. Heiser, W.J. (1981) Unfolding analysis of proximity data. University of Leyden: impublished dissertation. Jansen, P.G.W. (1983). Rasch analysis of attitudinal data . Catholic University of Nijmegen - Rijks Psychologische EKenst, Den Haag: unpublished dissertation. Kruskal, J.B., F.W. Young, and J.B. Seery (1973). How to use KYST. a very flexible program to do multidimensional scaling and unfolding. Murray Hill: Bell Labs., mimeo. Meerling (1981). Methoden en technieken van psvchologisch onderzoek. deel 2: data-analyse en psychometrie . Meppel: Boom. Mokken, R.J. (1971). A theory and procedure of scale analysis, with application in political research . The Hague: Mouton. Roskam, EE. (1968). Metric analysis of ordinal data. Voorschoten: VAM. SchOnemann, P.H. (1970). On metric multidimensional unfolding. Psvchometrika. 35, 349-366. Sixtl. F. (1973). ProbabiUstic unfolding. Psvchometrika . 38. 235-248. Tversky, A. (1972). Elimination by aspects: a theory of choice. Psychological review. 79. 281-299. Van Schuur. W.H. and I.W. Molenaar (1982). MUDFOLD. multiple stochastic unidimensional unfolding. In: H. Caussinus. P. Ettinger. and R. Thomassone (eds.) COMPSTAT 1982. Part I: proceedings in computational statistics . Vienna: Physica-Verlag. pp. 419-424. Van Schuur. H. (1984). Structure in poUtical beliefs, a new model for stochastic unfolding with application to European party activists. Amsterdam: CT Press. Young. F.W. (1972). A model for polynomial conjoint analysis algorithms. In: R.N. Shepard et al (eds.). Multidimensional scaling: theory and applications in the behavioral sciences, vol. I: theory . New York: Seminar Press. Zinnes. J.L and R.A. Griggs (1974). Probabihstic multidimensional unfolding analysis. Psyrhnmemka 39, 327-350. Summer 1986 iassist quarterly - 15 Table 1 Table 1: parallelogram analysis of perfect 'pick 3/11' data 1 : subject prefers stimulus O: subject does not prefer stimulus subjects : stimuli : 123456789 -I I I 1- ABCDEFGHIJK subject nr. response pattern: 1 11100000000 2 01110000000 3 111 4 111 5 ' 111 6 111 7 111 8 00000001110 9 00000000111 Table 2 Table 2: Two examples of response patterns that contain error Example 1: Example 2: A B C D Error in triples A B C D E F G Error in triples 1110 111 ADE ADF ADC BDE BDF BDG CDE CDF CD 110 1111 ACD ACE ACF ACG BCD BCE BCF BCG 10 11111 ABC ABD ABE ABF ABG 111111 none 10 10 ABC 10 1 BCD 10 1 ABD ACD 110 1 ACD BCD 10 11 ABC ABD Summer 1986 16 — iassist quarterly Table 3 Table 3: Observed data matrix and matrix with expected frequenc Observed data matrix: stimuli: SUM subjects: A B E ...i j k ...n 1 1110 3 2 110 1 3 N 10 10 1 p(i\) p(B) p(£:) p(D) p(i) p(j) p{k) p(n) Matrix with expected frequencies : stimuli: triples: A B C D ...i j k ...n (^«^) ^ABC ^ABG ^ABC ° a^^ (^^^ ^ABD ^ABD ° ^ABD a^^ (ijk) a. j^ a..^ a..^ (3) p(A) p(B)-'pTC) p(D) p(i) p(j) p(k) p(n) ijk : expected frequency of triple (ijk) = f .f (i) .f ( j) . f (k) (i.e., no interaction) "ijk The values for f and f(i) are found iteratively Summer 1986 iassist quarterly - 17 Table 4 Table 4: Dominance and adjacency matrix for a perfect 4-stimulus unfolding scale Data matrix A B C D frequency 1 p 1 q 1 r 1 s 1 1 t 1 1 u 1 1 V 1 1 1 w 1 1 1 X Dominance matrix: A B C D A P p+t p+t+w B q+u+x - q+t q+t+u+w C r+u+v+x r+v - r+u+w D s+v+x s+v s - Adjacency matrix A B C D A B t+w - C w u+w+x - D X v+x - Table 5 Table 5: Assignment of scale values to stimuli and subjects stimuli rank number subject nr . 1 2 3 4 5 6 7 A B C D E F G 12 3 4 5 6 7 1 1 10 10 10 10 10 1 10 10 1 1 1110 111 scale value of subject 3 3 3 2.5 5.67 4 - (missing datum) Summer 1986 18 — iassist quarterly Table 6 Table 6: Labeled H-matrix for 'pick 2/6' European party groups, Scale jik Scale i]k Scale1 ik3 E(o) E(e) H(ijk) E(o) E(e) H(ijk) E(o) E(e) H(ijk) ABC 106 88 -0.21 9 36 0.75 341 86 -2.97 ABD 262 177 -0.14 3 73 0.96 341 86 -2.97 ABE 86 226 0.62 2 93 0.98 341 86 -2.97 ABF 12 171 0.93 4 71 0.94 341 86 -2.97 ACD 124 75 -0.66 3 73 0.96 9 36 0.75 ACE 50 95 0.4 8 2 93 0.96 9 36 0.75 ACF 77 72 -0.07 4 71 0.94 9 36 0.75 ADE 217 192 -0.13 2 93 0.98 3 73 0.96 ADF 116 146 0.20 4 71 0.94 3 73 0.96 AEF 437 186 -1.35 4 71 0.94 2 93 0.98 BCD 124 75 -0.66 202 177 -0.14 106 88 -0.21 BCE 50 95 0.48 86 226 0.62 106 88 -0.21 BCF 77 72 -0.07 12 171 0.93 106 88 -0.21 BDE 217 192 -0.13 86 226 0.62 202 177 -0.14 BDF 116 146 0.20 12 171 0.93 202 177 -0.14 BEF 437 186 -1 .35 12 171 0.93 86 226 0.62 CDE 217 192 -0.13 50 95 0.48 124 75 -0.66 CDF 116 146 0.20 77 72 -0.07 124 75 -0.66 CEF 437 186 -1.35 77 72 -0.07 50 95 0.48 DEF 437 186 -1.35 116 146 0.20 217 192 -0.13 Table 7 Table 7: Final unfolding scale for 'pick 2/6' European party groups A Communists B Social Democrats D European Liberals and Democrats E Christian Democrats F Conservatives Dominance matrix Adjacency matrix p(i) H (i) 0.20 0..96 0.42 0,.85 0.37 0,.71 0.44 0..72 0.36 0,.79 N=1786 H = .79 A B D E F A - 1 19 19 19 B 17 - 25 31 35 D 30 19 - 18 24 E 41 37 29 - 17 F 32 31 25 7 - 17 - 11 - 5 12 1 6 Summer 1986