Microsoft Word - AJCEB Vol.2 No.1 compiled sized.doc MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 13 MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE Robby Soetanto and David G. Proverbs University of Wolverhampton INTRODUCTION Traditionally, the main participants of the construction project coalition (PC) are the client, the architect and the contractor. The interactions and interrelationships between these participants largely determine the overall performance of a construction pro- ject (Smith and Wilkins, 1996; Egan, 1998). The performance of these participants is also interdependent (Higgin and Jessop, 1965; Mohsini, 1989). Hence, in order to per- form effectively, a reciprocal requirement exists, whereby each participant requires the other participants to perform their du- ties effectively and in harmony with others. Notwithstanding this mutual dependency, the performance of individual participants remains important because overall project performance is a function of the performance of each participant (Liu and Walker, 1998). UK contractors have long been criticised for their failure to fulfil the needs of their cli- ents (Latham, 1994; Egan, 1998). In a broader sense, contractors should also per- form to the satisfaction of other PC partici- pants (e.g. architects) to maintain harmonious working relationships. This is because harmonious working relationships are essential if projects are to be successful (Baker et al., 1988; Smith and Wilkins, 1996; Egan, 1998). There is a need therefore, to investigate contractor performance from the viewpoint of other PC participants (espe- cially clients), from which models for pre- dicting levels of (client) satisfaction can be developed. The objective of this paper is to present and describe the development of such models which were developed using the multiple regression (MR) technique. The models could be used to identify attributes influencing satisfactory contractor perform- ance assessment. This would ultimately help to improve performance and enhance satisfaction for the betterment of overall project performance. CONCEPTUAL MODEL OF PERFORMANCE ASSESSMENT Satisfaction is regarded as an internal frame of mind, tied only to mental interpre- tations of performance levels (Oliver, 1997). That is, a performance assessor (e.g. client or architect) will have their own psychologi- cal interpretation of the performance of oth- ers (e.g. contractors). This psychological process is subjective and difficult to inter- pret. Based on this theorem, a conceptual model of performance assessment has been developed (refer to Figure 1). Figure 1: Conceptual performance assessment model Objective performance assessment Subjective performance assessment Performance assessment Participant performance attributes Project attributes Project performance Assessor and company assessor attributes Satisfaction Performance attributes Satisfaction attributes ROBBY SOETANTO AND DAVID G. PROVERBS 14 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 Conceptually, the outcomes of performance assessment (in terms of satisfaction levels) can be influenced by two major attributes; those of the performer (i.e. performance attributes) and those of the assessor (i.e. satisfaction attributes). Satisfaction attrib- utes are differentiable from performance attributes mainly due to their unique nature, being inherent within an individual (i.e. as- sessor). That is, performance attributes may reflect on both participants and projects, and will influence both participant and pro- ject performance. In contrast, satisfaction attributes reflect solely on the assessor and influence their performance assessment and as such are beyond the control of the performer. Performance attributes consist of partici- pant attributes and project attributes. Par- ticipant attributes represent the characteristics or nature of a particular par- ticipant or their organisation, such as com- pany age and turnover. Project attributes represent the characteristics/nature of a project, comprising attributes which may be outside the control of the participants. Con- trollable attributes are, for example, forms of contract, procurement route, and extent of design completed prior to work on site. Uncontrollable attributes include type of project, ground and weather conditions. Satisfaction attributes include the personal attributes of the individual assessor (e.g. experience, vocational background) and at- tributes of their employer (e.g. company as- sessor attributes). Company attributes are characteristics of the assessor’s company, which may influence their assessment (e.g. company age, turnover, number of employees). Figure 1 demonstrates the relationships between these variables. The performance attributes of a participant have a direct in- fluence on their own performance in the construction process. Project attributes in- directly influence the participant’s perform- ance since the attributes may enable/hamper the participant in executing their duties. Performance assessment in this respect is considered as ‘objective’ (i.e. tangible) in nature. For example, contractor performance may be assessed in terms of cost, time and quality performance (Holt, 1995). However, performance assessment goes beyond the objective aspects outlined above since it involves the feelings of the assessor, which in turn are dependent on their back- ground, i.e. frame of reference. This as- sessment is considered ‘subjective’ and at a higher level. This research embraces both ‘objective’ and ‘subjective’ (or higher level) performance assessment. In this case, sat- isfaction is measured using predetermined performance criteria, which are explained in research methodology section. A list of all performance and satisfaction attributes (as independent variables) identi- fied from the literature is presented in Table 1 (column 1). Using the correlation tech- nique, possible significant variables for modelling were selected and are shown in column 2. Some degree of multicollinearity was found in several groups of variables. To rectify this problem, those variables which were highly correlated were combined into a single indicator as suggested by Lewis-Beck (1993). The variables used for modelling are presented in column 3. MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 15 Table 1: List of independent variables of clients' assessment of contractor performance Identified variables Possible significant variables at 5% Variables for modelling Variable name Measure Satisfaction attributes Assessor RSEDU (1,2,3) respondent education nominal RSPRO involved in project years RSCOM working for company years RS5YR involved in similar projects within 5 years No. RSSATPR satisfaction on project performance likert 0–10 RSSATCO RSSATCO satisfaction on contractor performance likert 0–10 RSCON1 perception on contractor image likert 0–10 RSCON2 RSCO24 perception on contractor claims likert 0–10 RSCON3 perception on contractor on time likert 0–10 RSCON4 perception on contractor contractual likert 0–10 RSCON5 RSCO57 perception on contractor untidy likert 0–10 RSCON6 perception on contractor inefficient likert 0–10 RSCON7 perception on contractor technology likert 0–10 Company assessor CLNAT nature of client business nominal CL/AREST company establishment years CL/AREMP number of employees No. CL/ARATO company annual turn over Sterling (M) CL/ARABWNO no. annual building works No. CL/ARABWVA total value of annual building works Sterling (M) Performance attributes Project PRTPR (1,2) PRTPR (1,2) type of project nominal PRTBD (1,2,3,4) PRTBD (1,2,3,4) type of building nominal PRSTO PRSTO number of storeys No. PRGFA gross floor area area (m^2) PR5YR procured similar projects within 5 years No. PRROU (1,2,3) PRROU (1,2,3) procurement route nominal PRCTR (1, 2, 3) form of contract nominal PRCLA clarity and understanding of contract likert 0–10 PRDURPL PRDURPL planned duration time (months) PRDUROV PRDUROV overrun yes/no PRDURTI PRDURTI overrun duration time (months) PRBUDTE PRBUDTE tender sum Sterling (M) PRBUDOV PRBUDOV overbudget yes/no PRBUDMO PRBUDMO overbudget cost Sterling (M) PRVARSE PRVARSE severity of variations likert 0–10 PRVARFR frequency of variations likert 0–10 PRVARCL cause of variations by client likert 0–10 PRVARAR cause of variations by architect likert 0–10 PRVARCO PRVARCO cause of variations by contractor likert 0–10 PRVAROT cause of variations by others likert 0–10 PRCOMDE PRCOMDE design complexity likert 0–10 PRCOMCS construction complexity likert 0–10 PRDESCO design completed before work on site percentage PRCONGR PRCONGR constraint by ground conditions likert 0–10 PRCONWE PRCONWE constraint by weather conditions likert 0–10 PRCONGO constraint by government regulations likert 0–10 PRLOCAC PRLOCAC ease of access to project location likert 0–10 PRLOCCO remoteness from contractor office likert 0–10 PRINT PRINT interaction between contractor and architect likert 0–10 Contractor COSI (1,2,3,4) contractor size (catchment) ordinal COATO (1,2,3,4) company annual turn over ordinal COEMP (1,2,3,4) number of employees ordinal COEST company establishment years COWKDBF no. previous project undertaken by contractor No. COWL COWL architect work load likert 0–10 ROBBY SOETANTO AND DAVID G. PROVERBS 16 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 Table 1 (continued) Identified variables Possible significant variables at 5% Variables for modelling Variable name Measure COSELCO (1, 2) COSELCO (1, 2) method of contractor selection nominal COEVACL/AR COEVACL/AR contractor evaluation prior contract award likert 0–10 COPAYCO (1, 2) COPAYCO (1, 2) method of contractor payment nominal CODIFEST difference between estimate and contractor bid percentage CODIFSEC difference between contractor bid and sec- ond percentage COINFAP influence on appointment of site personnel likert 0–10 COPERCO COPERCO previous relationship with site personnel yes/no COATTFI COATFISI contractor attributes: financial soundness likert 0–10 COATTTY contractor attributes: experience in type of proj. likert 0–10 COATTSI contractor attributes: experience in size of proj. likert 0–10 COATTGE contractor attributes: exp. in geographical area likert 0–10 COATTRE COATTRE contractor attributes: references likert 0–10 COATTPP COATPPQU contractor attributes: past performance likert 0–10 COATTSC contractor attributes: time reputation likert 0–10 COATTBU contractor attributes: cost reputation likert 0–10 COATTQU contractor attributes: quality reputation likert 0–10 COATTLI COATLIIM contractor attributes: litigation reputation likert 0–10 COATTIM contractor attributes: claim reputation likert 0–10 COATTDI COATTDI contractor attributes: director likert 0–10 COATTSP COATTSP contractor attributes: site personnel likert 0–10 COATTHS COATTHS contractor attributes: health and safety likert 0–10 COATTTR COATTTR contractor attributes: training regime likert 0–10 COATTQC COATTQC contractor attributes: quality control likert 0–10 COATTSU COATSULA contractor attributes: subs and suppliers likert 0–10 COATTLA contractor attributes: labour likert 0–10 COATTPL contractor attributes: plant likert 0–10 COATTWR COATTWR contractor attributes: working relationship likert 0–10 COSCRTA contractor selection criteria: technical likert 0–10 COSCRPE contractor selection criteria: past experience likert 0–10 COSCRQP contractor selection criteria: quality and programme likert 0–10 COSCRRE contractor selection criteria: reference likert 0–10 COSCRTE contractor selection criteria: tender sum likert 0–10 COSCRPU contractor selection criteria: reputation likert 0–10 RESEARCH METHODOLOGY In the context of this paper, contractor per- formance criteria are defined as those used to measure the performance of contractors based on the views of clients. These criteria were determined through interviews with twelve experienced clients and supported by a literature review in the domain of (con- tractor) performance. These criteria were categorised under several main headings. A full list of the criteria identified is presented in Table 2. MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 17 Table 2: List of contractor performance criteria based on clients' opinion Contractor performance criteria Code Pre-construction stage ~ First interview and presentation P1 ~ Ability and willingness to help develop brief P2 ~ Contribution to design and buildability of project P3 ~ Plan of work and method statement P4 ~ Understanding of contract and specifications P5 Construction stage Site management ~ Site supervision and control S1 ~ Site organisation, tidiness and cleanliness S2 ~ Ability to plan and programme properly S3 ~ Health and safety performance / management S4 ~ Compliance to regulations (CDM, etc.) S5 Resource management ~ Material management R1 ~ Man power management (sufficient quantity and quality of craftsmen) R2 ~ Equipment and plant management R3 ~ Management and co-ordination of subcontractors and suppliers R4 ~ Payment to subcontractors and suppliers (on time) R5 ~ Strength of contractor site team (i.e. quantity) R6 ~ Concern/awareness of environmental issues R7 Site personnel ~ Co-operation with client (i.e. client representative) E1 ~ Individual performance and ability E2 ~ Project manager performance and adequacy of authority E3 ~ Site manner (i.e. no loud noises and swearing) E4 Variations and drawings ~ Processing variations (e.g. speed, flexibility) V1 ~ Preparation of shop drawings and as-built drawings V2 ~ Contribution to development of design drawings V3 Completion stage and ease of delivery ~ Completion of defects C1 ~ Smoothness of operation and hand-over C2 ~ Quality of hand-over document (O&M manual, H&S) C3 ~ Ease / speed of settlement of final account C4 ~ Ease of delivery (general feeling on how things went) C5 Principal ~ Adherence to schedule (time performance) M1 ~ Adherence to budget (cost performance) M2 ~ Quality of construction and workmanship M3 Quality of service ~ Handling of complaints (effectiveness) Q1 ~ Telephone inquiries and correspondence handled courteously and adequately Q2 ~ Speed and reliability of service Q3 ~ Responsiveness to client’s queries Q4 ~ Ability to make rapid decisions Q5 ~ Commitment of key person (active & continuous) Q6 ~ Corporate hospitality Q7 ~ Administration Q8 Attitude ~ Honesty and integrity A1 ~ Collaborative / spirit of co-operation / team work A2 ~ Customer focus / proactive to understand client A3 ~ Keep the client informed A4 ~ Communication (to coalition member & site person) A5 ~ Pro-active attitude toward problems A6 ~ Avoidance of claims (i.e. not claims conscious) A7 ~ Responsibility for their decision (understand the cost of their recommendations) A8 ROBBY SOETANTO AND DAVID G. PROVERBS 18 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 The questionnaire To provide the main modelling data, a ques- tionnaire was developed based on the at- tributes and performance criteria identified. Respondents (clients) were asked to identify a recent (within 2 years) UK building project in which they were involved (referred to as the ‘case project’). Respondents were asked to relate all their answers to the questions contained in the questionnaire to this one case project. This strategy was designed in order to capture a true and realistic reflec- tion of assessors’ satisfaction/ dissatisfaction feelings. To protect the con- fidentiality of the other parties involved in these case projects, respondents were not asked to identify projects, nor to name other participants. The survey Following the development of the question- naire and implementation of a pilot survey, a UK-wide questionnaire survey of clients was conducted. Distribution involved 536 experi- enced UK private and public clients, defined as those who regularly procure construction works from the industry. Private clients consisted of developers, retailers and finan- cial institutions. Retailers and financial in- stitutions were identified from the listing of Key British Enterprises (Dun and Brad- street, 1998) representing the top UK retail- ers and financial institutions. Developers were identified from the Estates Gazette (1999). Public clients, i.e. local authorities or City Councils, were identified from the Mu- nicipal Year Book (1999). Overall, 77 responses were received repre- senting a 14.4% response rate. This rela- tively low response rate is about the ‘norm’ for construction management research and in many ways can be associated with the ‘confidential’ nature of the questions and the comprehensive nature of the research instrument. DIMENSIONS OF CLIENT SATISFACTION In this research, satisfaction is measured using an interval scale (i.e. scale 0–10) which assumes that satisfaction is a matter of degree, not an all or none property. To measure an abstract concept such as satis- faction, the concept should be defined at an operational (i.e. lower) level, which is ob- servable and directly measurable (Johnson and Fornell, 1991). If the relationship between the abstract concept and the op- erational definition of satisfaction (i.e. per- formance criteria) is strong, the measurement instrument can be considered as valid and reliable to represent the ab- stract concept (Carmines and Zeller, 1979). To derive the dimensions of client satisfac- tion the factor analysis technique was ap- plied to the performance criteria of 50 responses (case projects). The Kaiser- Meyer-Olkin (KMO) measure of sampling adequacy (0.673) confirmed that the factor analysis technique could be meaningfully applied (Norusis, 1994: 52–53). This was fur- ther confirmed by Bartlett’s test of spheric- ity (chi-square = 3198.153, p < 0.0005). This technique has been previously used in construction research. For example, Sawa- cha et al. (1999) utilised the factor analysis technique to determine the group of factors affecting site safety performance. Langford et al. (2000) used factor analysis to identify factors that prompted the strongest effect upon attitudes to safety management. Chan et al. (2001) used factor analysis to catego- rise project success factors into smaller number of groups. The main purpose was to determine the number of common factors (i.e. satisfaction dimensions) that would satisfactorily pro- duce the correlations among the observed variables (Kim and Mueller, 1978a). The method of extraction was principal compo- nents analysis. This method allows for data reduction and is considered as a means of exploring interdependence of variables. The number of factors determined was based on the criterion that the eigenvalue for each factor should be greater than 1 (i.e. Kaiser’s criterion) (Torbica, 1997; Bryman and Cramer, 1999). This method is considered the most commonly used procedure to de- termine the number of initial factors to be extracted (Kim and Mueller, 1978b). To achieve the simplest possible factor struc- ture in order to obtain more interpretable factors/dimensions, promax oblique rotation with the power (Kappa) valued at 4 was util- ised. Oblique rotation (as opposed to or- thogonal rotation) was utilised since it allows the presence of correlations between factors/dimensions. In fact, this assumption concurs with the real life situation since one aspect of performance should be, to some extent, related to other aspects. Further, Norusis (1994) claimed that oblique rotations MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 19 have often been found to yield substantively meaningful factors since it is likely that in- fluences in nature are correlated. Promax has a reputation for demonstrable quality as evidenced in empirical studies (Gorsuch, 1983). Promax rotation raises the factor loading to a higher power in order that moderate and low loadings need to be lower while the high loadings remain rela- tively high (Gorsuch, 1983.). For example, the original loadings were 0.9 and 0.3. 0.3 is one-third as large as 0.9, but the squared loading for the second variable is 0.09 which is one-ninth as large as the squared loading for the first variable (0.81). By raising the power of factor loadings, the factor struc- ture becomes more interpretable. The power is known as the coefficient Kappa (k). Gorsuch recommended that the proper power is that which gives the simplest structure with the least correlation among factors. Furthermore, he claimed that a good solution is generally achieved by rais- ing the loadings to a power of four (SPSS default). In this research, Kappa = 2 and 6 were trialed, but these did not derive better solutions than Kappa = 4. Five dimensions of client satisfaction were extracted and altogether represent 76% of the variations in the variables (refer to Table 3). The scores of the performance criteria under each dimension were then averaged to obtain the satisfaction measure (i.e. fac- tor score). The factor score serves as an index of attitude towards a particular di- mension of satisfaction under investigation (Torbica, 1997). From the original 48 per- formance criteria, 28 were included in one of the five factors. The validity and reliability of the satisfaction measures were con- firmed. The validity assessment included the assessment of content, criterion-related and construct validity. The reliability of the measures (in terms of their internal consis- tency reliability) was assessed using coeffi- cient Cronbach’s alpha. For a full description of the validity and reliability of empirical measurement, readers may wish to consult Bohrnstedt (1970), Nunnally (1978), Carmines and Zeller (1979) and Lit- win (1995). In construction, Torbica (1997) used a similar method for testing the validity and reliability of satisfaction measures of home buyers. ROBBY SOETANTO AND DAVID G. PROVERBS 20 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 Table 3: Factor structure of contractor performance criteria based on clients' assessment Contractor performance criteria Code Factor loading Eigenvalues Percentage of variance explained Cumulative percentage of variance explained Satis1: 'Quality of service and attitude of contractor' ~ Quality of hand-over document (O&M manual, H&S) C3 0.827 28.873 60.151 60.151 ~ Telephone inquiries and correspondence handled courteously and adequately Q2 0.864 ~ Speed and reliability of service Q3 0.833 ~ Ability to make rapid decisions Q5 0.862 ~ Administration Q8 0.871 ~ Ability to keep the client informed A4 0.930 ~ Communication (to coalition member and site person) A5 0.903 ~ Responsibility for their decisions (understand the cost of their recommendations) A8 0.764 Satis2: 'Main performance criteria and completion' ~ Completion of defects C1 0.794 2.852 5.941 66.092 ~ Ease / speed of settlement of final account C4 0.804 ~ Ease of delivery (general feeling on how things went) C5 0.922 ~ Adherence to schedule (time performance) M1 0.808 ~ Adherence to budget (cost performance) M2 0.898 ~ Quality of construction and workmanship M3 0.861 Satis3: 'Performance in preliminary stage' ~ First interview and presentation P1 0.759 2.067 4.306 70.399 ~ Ability and willingness to help develop brief P2 0.839 ~ Contribution to design and buildability of project P3 0.727 ~ Plan of work and method statement P4 0.900 ~ Understanding of contract and specifications P5 0.779 Satis4: 'Performance of site personnel' ~ Co-operation with client (i.e. client representative) E1 0.893 1.374 2.862 73.260 ~ Individual performance and ability E2 0.849 ~ Project manager performance and adequacy of authority E3 0.870 ~ Collaborative / spirit of co-operation / team work A2 0.841 ~ Pro-active attitude toward problems A6 0.844 Satis5: 'Performance in resource management' ~ Material management R1 0.908 1.239 2.581 75.841 ~ Equipment and plant management R3 0.835 ~ Concern/awareness of environmental issues R7 0.824 ~ Site manner (i.e. no loud noises and swearing) E4 0.778 MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 21 Multiple regression technique As the purpose of the analysis was to de- velop models to predict levels of client satis- faction (a matter of degree, not an all or nothing property), the multiple regression (MR) technique was chosen as the model- ling tool. Moreover, preliminary data exami- nation showed a degree of linear relationship between dependent and inde- pendent variables. That is, MR represented an appropriate methodology for data of this nature (Lewis-Beck, 1993). The stepwise method for inclusion/exclusion of independ- ent variables was utilised. Stepwise multiple regression is the most commonly used method for model building (Everitt and Dunn, 1991; Norusis, 1995; Bryman and Cramer, 1999). Draper and Smith (1981) and Kinnear and Gray (2000) regarded step-wise as one of the best variable selection proce- dures. The procedure selects the independ- ent variables step by step. At each step variables already in the equation are evalu- ated according to the selection criteria for removal, and variables not in the equation are evaluated for entry. This process re- peats until no variable in the block is eligible for entry or removal (Norusis, 1995). F- statistics with probability of 5% and 10% were employed for entry and removal crite- ria as suggested by Draper and Smith (1981: 311). CLIENT SATISFACTION MODELS In total, seven models were developed to predict levels of client satisfaction based on contractor performance (refer to Table 4). Table 4: MR models of clients’ satisfaction Multiple regression models R2 satis1 = 0.01006 + 0.341(COATTHS) – 0.182(PRVARSE) + 0.338(COATPPQU) + 0.253(COATTQC) + 0.853(COPAYCO2) + 0.05308(PRDURPL) + 0.837(PRTBD3) 0.77 satis2 = 1.268 + 0.446(COATPPQU) + 0.317(COATFISI) – 0.175(PRVARSE) + 0.209(COATTSP) – 0.162(RSCO24) 0.73 satis3 = 1.404 + 0.524(COATPPQU) + 1.055(COSELCO2) + 0.292(COATTQC) – 0.141(PRCONWE) 0.60 satis4 = 2.411 + 0.491(COATPPQU) + 0.294(COATTSP) – 0.197(PRVARCO) – 0.135(PRBUDMO) 0.68 satis5 = -0.240 + 0.414(COATTTR) + 0.327(COATTQC) + 0.272(COATFISI) 0.67 avesat = 0.291 + 0.547(COATPPQU) + 0.368(COATTHS) – 0.156(PRVARCO) + 0.776(COPAYCO2) + 0.674(PRTBD3) + 0.09476(PRSTO) 0.80 totsat = 1.236 + 0.534(COATPPQU) + 0.330(COATTHS) – 0.219(PRVARCO) - 0.195(PRBUDMO) + 0.05465(PRDURPL) – 0.658(PRTBD0) 0.78 ROBBY SOETANTO AND DAVID G. PROVERBS 22 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 Summarisation of the MR models In multiple regression, standardised coeffi- cients (β) can be used to assess the relative importance among the independent vari- ables in determining the dependent variable within one model. Suppose, a simple model with two independent variables in standard- ised form (Lewis-Beck, 1993) is: *** 2211 XXY ββ += when YS YY Y − =* 1 11 1* XS XX X − = 2 22 2 * XS XX X − = Y X S S b 111 =β Y X S S b 222 =β where: Y is a dependent variable, X1 and X2 are in- dependent variables, Y X X*, *, *1 2 are standardised variables, Y X X, ,1 2 are the means of the variables, 21 ,, XXY SSS are the standard deviation of the variables, β β1 2, are beta coefficient or beta weight, b b1 2, are partial regression coefficients. Beta weight indicates the average standard deviation change in Y associated with a standard deviation change in X, when the other independent variables are held con- stant (Lewis-Beck, 1993). From the formula of beta weight, it is obvious that partial re- gression coefficients are corrected by the ratio of the standard deviation of the inde- pendent variable to the standard deviation of the dependent variable. In comparing the importance of an inde- pendent variable across several models, beta weights are determined by the standard deviation of the variable in the models. Therefore, the standard deviation must be held constant in each model. In the case of comparisons across samples (e.g. a com- parison of the importance of an independent variable in two models which were devel- oped from two different samples), unstan- dardised partial regression coefficients are preferred to beta weights (Lewis-Beck, 1993: 57–58). In this research, the standard deviation of any independent variable is con- stant in several models since these models use similar independent variables (i.e. from the same sample). However, the standard deviation of dependent variables is not con- stant across several models due to the use of several satisfaction measures in the models. This means that beta weights of an independent variable are not comparable across several models. This problem can be overcome by multiplying the beta weight by the standard deviation of the dependent variable. Based on this, importance weights (IWs) of the independent variables identified were established using the product of the stan- dardised coefficient (beta weight, β) of the independent variables in absolute terms and the standard deviation of the dependent variable (SY ). These weights were compara- ble across several models developed from the same sample. Then, the total impor- tance weight (TIW) of the independent vari- ables was obtained by adding the importance weights (IWs) of the variable in each relevant model. Table 5 shows the cal- culation of TIWs for independent variables identified as useful predictors of the satis- faction measures. For each satisfaction measure, an IW for each variable was pro- duced (Table 5, column 2 to 8). These weights were summed producing a TIW for each variable. These variables could then be ranked according to their TIWs in descend- ing order (column 10). In order to ease dis- cussion, based on their TIWs, the variables could be grouped into four categories, i.e. extremely important (TIW ≥ 2.0), highly im- portant (1.0 ≤ TIW < 2.0), medium impor- tance (0.1 ≤ TIW < 1.0) and some importance (TIW < 0.1) (last column). MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 23 Table 5: Summary of independent variables' total importance weights (TIWs) derived from clients' assessment of contractor performance Independent Satisfaction measures TIW Ranking Importance variables satis1 satis2 satis3 satis4 satis5 avesat totsat category COATPPQU 0.404 0.533 0.625 0.586 0.653 0.638 3.440 1 extremely important COATTHS 0.534 0.578 0.520 1.632 2 highly important COATTQC 0.376 0.434 0.485 1.295 3 highly important PRVARCO 0.415 0.328 0.462 1.206 4 highly important COATFISI 0.498 0.428 0.927 5 medium importance PRVARSE 0.468 0.448 0.915 6 medium importance COATTSP 0.375 0.526 0.900 7 medium importance PRBUDMO 0.309 0.446 0.755 8 medium importance COATTTR 0.724 0.724 9 medium importance PRDURPL 0.339 0.350 0.690 10 medium importance COPAYCO2 0.344 0.313 0.657 11 medium importance COSELCO2 0.488 0.488 12 medium importance PRTBD3 0.253 0.204 0.457 13 medium importance PRCONWE 0.328 0.328 14 medium importance RSCO24 0.314 0.314 15 medium importance PRTBD0 0.304 0.304 16 medium importance PRSTO 0.195 0.195 17 medium importance Discussion of the models The models identified seventeen independ- ent variables as useful predictors. One vari- able was classified as ‘extremely important’, namely past performance of contractor in terms of cost, time and quality (CLATPPQU). This suggests that contractors whose past performance is good are more likely to sat- isfy their clients. Numerous scholars (e.g. Russell et al., 1992; Assaf and Jannadi, 1994; Holt et al., 1994; Tam and Harris, 1996; Hatush and Skitmore, 1997; Ng and Skitmore, 1999) have reported that past per- formance is one of the most important attributes influencing contractor perform- ance. Therefore, this aspect should be care- fully considered in the contractor selection process in order to achieve higher client satisfaction levels. Three variables were classified as ‘highly important’: health and safety past performance and policy (COATTHS) quality control policy (COATTQC) the extent of variations caused by contrac- tor (PRVARCO). While COATTHS and COATTQC positively influence satisfaction, PRVARCO negatively influences satisfaction. This indicates that health and safety is a highly important factor for clients, even more so than quality. Varia- tions often hamper project performance (Thomas and Napolitan, 1995; Ibbs, 1997) and hence will impact on satisfaction levels. Contractors should maintain high levels of safety and quality, and attempt to reduce variations if they are to satisfy their clients. Variables classified as ‘medium importance’ comprised contractor, project and respon- dent attributes. Contractor attributes included: ROBBY SOETANTO AND DAVID G. PROVERBS 24 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 financial soundness and experience in type and size of project (COATFISI) qualification and experience of site per- sonnel (COATTSP) formal training regime of site personnel (COATTTR) cost reimbursement method of contractor payment (COPAYCO2) contractor selected through negotiation (COSELCO2). Financially sound contractors who have ex- perience in similar projects are more likely to satisfy their clients. Such contractors are more likely to provide an effective level of service. COATTSP and COATTTR highlight the importance of site personnel to contrac- tor performance and hence client satisfac- tion. That is, the site personnel represent a key resource in the production process. Contractors paid by cost reimbursement methods and selected through negotiation derive higher levels of client satisfaction. This suggests that less ‘confrontational’ methods of contractor procurement (rather than e.g. competitive tendering) are more likely to derive higher client satisfaction levels. Project attributes classified as ‘medium im- portance’ were severity of variations (PRVARSE) project overbudget cost (PRBUDMO) planned project duration (PRDURPL) residential projects (PRTBD3) the extent to which the project is con- strained by weather conditions (PRCONWE) public building projects (PRTBD0) number of storeys (PRSTO). It is no real surprise that clients become dissatisfied when projects are completed overbudget and incur many variations. In- terestingly, larger projects were found to raise satisfaction levels. This may be con- nected to the prestige associated with such projects, and the need to involve well re- sourced and experienced contractors whose performance may be superior to smaller firms. Clients were more satisfied on resi- dential projects than public building pro- jects. PRCONWE suggests adverse weather conditions may hamper contractor perform- ance and hence negate client satisfaction. One client attribute representing percep- tions of the assessor was found to be of ‘medium importance’, namely those who perceive contractors to be claim conscious, to fail to deliver projects on time, and to be contractual (RSCO24). Clients who have such perceptions are likely to suffer lower satisfaction levels. This suggests that some degree of subjectivity is prevalent in the cli- ents’ assessment of contractor performance. MODEL VALIDATION To confirm the robustness (in terms of ac- curacy and consistency) of the models in predicting satisfaction levels, the models were validated using a hold-back sample of 27 case projects. The predictive performance of the models was assessed by examining the residual (i.e. the difference between the actual and the models’ predicted satisfaction levels). These were measured using two prediction per- formance measures: mean absolute devia- tion (MAD) and mean absolute percentage error (MAPE) (Kvanli et al., 1996). While MAD indicates the mean of absolute devia- tion of the predicted levels from the actual levels, MAPE indicates the mean of absolute percentage of that deviation from the actual levels. Using these measures, it could be concluded that a model yields predicted val- ues with an average deviation of ± MAD, which is MAPE % from actual levels. For data of this nature, MAD of 1.5 to 2.0 and MAPE of 30 to 35% are considered accept- able. MAD of less than 1 and MAPE of less than 20% indicate good predictive perform- ance. The performance of the models was also tested using chi-square (χ2) analysis and Pearson’s correlation coefficient (Ed- wards, 1999). Results are summarised in Table 6. On av- erage, the deviation of the predicted satis- faction levels is 1.12 (MAD), which is 22.22% from the actual levels (MAPE). This is quite good given the subjective nature of satisfac- tion/dissatisfaction judgements. Pearson’s correlation tests confirmed that this level of accuracy is significant. Moreover, chi- square tests confirmed that the models have consistent predictive performance. These indicate that the MR models devel- oped are valid and robust. MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 25 Table 6: Summary of the validation of the MR models Chi-square test Correlation test Satisfaction measures MAD MAPE % D.F. Tab. Calc. r Sig. satis1 1.24 22.25 26 38.885 9.732 0.523 0.003 satis2 1.41 32.42 26 38.885 13.724 0.513 0.003 satis3 0.92 15.95 26 38.885 5.900 0.688 0.000 satis4 0.91 19.02 26 38.885 5.943 0.773 0.000 satis5 0.95 19.38 26 38.885 6.856 0.626 0.000 avesat 1.03 19.84 26 38.885 7.302 0.540 0.002 totsat 1.37 26.68 26 38.885 13.417 0.446 0.010 Average 1.12 22.22 38.885 8.982 0.587 0.003 CONCLUSION Based on a UK wide questionnaire survey of clients, multiple regression (MR) models have been developed and validated to pre- dict several dimensions of client satisfaction resulting from the performance of contrac- tors. For this research the MR technique was found to be appropriate, given the na- ture of the problem (i.e. satisfaction being a matter of degree) and results of preliminary data examination. The past performance of the contractor in terms of cost, time and quality was identi- fied as the most important independent variable. This suggests that contractors whose past performance is good are more likely to satisfy their clients. Moreover, health and safety, quality control, and the variations caused by contractors were also found to be of importance. Health and safety is emerging as a significant determinant of client satisfaction. The most important vari- ables indicate that client satisfaction levels are, to some extent, within the ‘control’ of contractor. The models also suggest that subjectivity is to some extent prevalent in clients’ performance assessment. In sum, contractors should focus on those attributes found to be significant in order to continu- ously improve performance and enhance client satisfaction levels. In summary, the models developed could be used, specifically by contractors, to improve performance and thereby improve levels of client satisfaction. This ultimately will help to create a performance-enhancing envi- ronment leading to harmonious working relationships between PC participants. This also ensures continuous performance im- provement for the betterment of all involved. REFERENCES Assaf, S. and Jannadi, M.O. (1994) A multi- criterion decision-making model for con- tractor prequalification selection. Building Research and Information, 22 (6), 332–335. Baker, B.N, Murphy, D.C. and Fisher, D. (1988). Factors affecting project success. In: Project Management Handbook, 2nd edition, Cleland, D.I. and King, W.R. (eds.). Van Nostrand Reinhold, New York. Bohrnstedt, G.W. (1970). Reliability and validity assessment in attitude measurement. In: Attitude measurement, Summers, G.F. (ed.). Rand McNally and Company, Chicago. Bryman, A. and Cramer, D. (1999) Quantita- tive data analysis with SPSS Release 8 for Windows: A guide for social scientists. Routledge, London. Carmines, E.G. and Zeller, R.A. (1979) Reli- ability and validity assessment. Quantitative applications in the social sciences series, No. 17, Sage Publications, London. Chan, A.P.C., Ho, D.C.K. and Tam, C.M. (2001) Design and build project success fac- tors: multivariate analysis. Journal of Con- struction Engineering and Management, ASCE, 127 (2), 93–100. Draper, N.R. and Smith H. (1981) Applied regression analysis, 2nd edition. John Wiley and Sons, New York. Dun and Bradstreet (1998) Key British en- terprises. Dun and Bradstreet, London. Edwards, D.J. (1999) A methodology for pre- dicting the total average hourly maintenance cost of tracked hydraulic excavators operat- ing in the UK opencast mining industry. Un- published Ph.D. thesis, University of Wolverhampton. ROBBY SOETANTO AND DAVID PROVERBS 26 THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 Egan, J. (1998) Rethinking construction. The report of the Construction Task Force on the scope for improving quality and efficiency in UK construction. Department of the Environment, Transport and the Regions, London. Estates Gazette (1999) Directory of property development and investment companies. August, 66–69. Everitt, B.S. and Dunn, G. (1991) Applied multivariate data analysis. Printed for Edward Arnold by the University Press, Cambridge. Gorsuch, R.L. (1983) Factor analysis, 2nd edition. Lawrence Erlbaum Associates, New Jersey. Hatush, Z. and Skitmore, M. (1997) Evaluat- ing contractor prequalification data: selec- tion criteria and project success factors. Construction Management and Economics, 15, 129–147. Higgin, G. and Jessop, N. (1965) Communi- cations in the building industry: the report of a pilot study. Tavistock Publications, London. Holt, G.D. (1995). A methodology for predict- ing the performance of construction con- tractors. Unpublished Ph.D. thesis, University of Wolverhampton. Holt, G.D., Olomolaiye, P.O. and Harris, F.C. (1994) Factors influencing UK construction clients’ choice of contractor. Building and Environment, 29 (2), 241–248. Ibbs, C.W. (1997) Quantitative impacts of project change: size issues. Journal of Con- struction Engineering and Management, ASCE, 123 (3), 308–311. Johnson, M.D. and Fornell, C. (1991) A framework for comparing customer satis- faction across individual and product cate- gories. Journal of Economic Psychology, 12, 267–286. Kim, J.O. and Mueller, C.W. (1978a). Factor analysis: Statistical methods and practical issues. Quantitative applications in the so- cial sciences series No. 14, Sage Publica- tions, London. Kim, J.O. and Mueller, C.W. (1978b) Intro- duction to factor analysis: what it is and how to do it. Quantitative applications in the so- cial sciences series No. 13, Sage Publica- tions, London. Kinnear, P.R. and Gray, C.D. (2000) SPSS for Windows made simple, Release 10. Psy- chology Press, Hove, UK. Kvanli, A.H., Guynes, C.S. and Pavur, R.J. (1996) Introduction to Business Statistics: A Computer Integrated Approach, 4th edition. West Publishing Company, St. Paul/Minneapolis. Langford, D., Rowlinson, S. and Sawacha, E. (2000) Safety behaviour and safety manage- ment: its influence on the attitudes of work- ers in the UK construction industry. Engineering, Construction and Architectural Management, 7 (2), 133–140. Latham, M. (1994) Constructing the team. Final report of the government/industry re- view of procurement and contractual ar- rangements in the United Kingdom construction industry. HMSO, Department of Environment, London. Lewis-Beck, M.S. (1993) Applied regression: An introduction. In: Regression analysis, Lewis-Beck, M.S. (ed.). International hand- books of quantitative applications in the so- cial sciences, Sage Publications, London. Litwin, M.S. (1995) How to measure survey reliability and validity. Sage Publications, Thousand Oaks, California. Liu, A.M.M. and Walker, A. (1998) Evaluation of project outcomes. Construction Manage- ment and Economics, 16, 209–219. Mohsini, R.A. (1989) Performance and build- ing: problems of evaluation. Journal of Per- formance of Constructed Facilities, 3 (4), 235–242. Municipal Year Book (1999) Municipal year book 1999 and public service directory, Vol. 1: Functions and Officers. (Lauren Hill, ed.). Newman Books, London. Ng, S.T. and Skitmore, R.M. (1999) Client and consultant perspectives of prequalifica- tion criteria. Building and Environment, 34 (5), 607–621. Norusis, M.J. (1994) SPSS Professional Sta- tistics 6.1. SPSS Inc., Chicago. Norusis, M.J. (1995) SPSS 6.1: Guide to data analysis. Prentice Hall, Englewood Cliffs, New Jersey. Nunnally, J.C. (1978) Psychometric theory, 2nd edition. McGraw-Hill, New York. MODELLING CLIENT SATISFACTION LEVELS: THE IMPACT OF CONTRACTOR PERFORMANCE THE AUSTRALIAN JOURNAL OF CONSTRUCTION ECONOMICS AND BUILDING VOL.2 NO.1 27 Oliver, R.L. (1997) Satisfaction: a behavioral perspective on the consumer. McGraw-Hill, New York. Russell, J.S., Hancher, D.E. and Skib- niewski, M.J. (1992) Contractor prequalifica- tion data for construction owners. Construction Management and Economics, 10, 117–135. Sawacha, E., Naoum, S. and Fong, D. (1999) Factors affecting safety performance on construction sites. International Journal of Project Management, 17 (5), 309–315. Smith, A. and Wilkins, B. (1996) Team rela- tionships and related critical factors in the successful procurement of health care fa- cilities. Journal of Construction Procure- ment, 2 (1), 30–40. Tam, C.M. and Harris, F.C. (1996) Model for assessing building contractors’ project per- formance. Engineering, Construction and Architectural Management, 3 (3), 187–203. Thomas, H.R. and Napolitan, C.L. (1995) Quantitative effects of construction changes on labor productivity. Journal of Construc- tion Engineering and Management, ASCE, 121 (3), 290–296. Torbica, Z.M. (1997) Total quality manage- ment and customer satisfaction in home building. Unpublished Ph.D. dissertation, University of Florida.