2Paterson.qxd In South Africa, testing was for a long time viewed as discriminatory and unjust, but this perception is slowly changing (Foxcroft, 1997). Practitioners are becoming aware of the advantages of sound assessment, and improvements in test development have lead to the implementation of cross-culturally fair tests. In the workplace tests, as part of assessment, are used to measure the performance and potential of current and future employees through selection and performance management respectively (Bartram, 2004). To arrive at an equitable and fair decision, psychological tests are usually not used in isolation, but as part of an extensive assessment battery (Fernandez- Ballesteros, 1999). It should be noted that for the purposes of this paper, the use of psychological tests is described in relation to psychological assessment in its broader definition. Fernandez-Ballesteros (2002) states that the most important difference between assessment and testing concerns the simplicity or complexity of assessment methods used. Assessment includes, but is not limited to testing. The distinction between ‘testing’ and ‘assessment’ can also be viewed in terms of the attributes required of the person applying the tests. Psychological assessment requires an extended set of skills and expertise from the assessor. The assessor should know which tests to select and how to integrate information derived from tests with other sources / types of information to attain a holistic assessment. This leads the discussion towards understanding the individual as a whole, and implies that decision-making should be based on more than test scores. In assessment, the practitioner uses information obtained through a variety of assessment methods, including a range of tests. Test use should thus inform this decision-making process. Now, in 2005, the South African business sector faces challenges in terms of psychological test use, adaptation and development. Practitioners are faced with a range of ethical and legal issues in psychological assessment (Bartram, 2004). Changes in legislation, i.e. South African employment Equity Act, 1998 (Act No. 55 of 1998), and a transforming labour market after 1994, emphasise the significance of appropriate psychometric properties for psychological tests, and the importance of ensuring validity and equity in assessment. A comprehensive overview of the tests currently in use in South Africa, as well as the needs of practitioners for future development in psychological assessment, is not available at present. Van der Merwe (2002) published a summative report on the findings of various exploratory studies that were conducted with regard to the practice of psychometric testing in the Eastern Cape. In the report he provides findings in terms of tests currently in use, information on test users and the level of satisfaction with regard to available tests as expressed by the participants. Culture-fairness of tests and applicability across different groups of people emerged as important themes from the findings (van der Merwe, 2002). Although this study has value, the scope is limited, and a national perspective on the use of tests is lacking. Consequently, a detailed comparison of testing practices between South Africa and other countries becomes problematic because of this lack of information. However, Oakland (2004) reported on an international survey of 29 countries (in which South Africa was included) that countries have different views in terms of their approach to testing, use of tests and attitudes towards test user qualifications. This paper aims to address the assessment concerns and issues relevant to South African psychological practitioners in the workplace. Oakland (2004) identified external and internal conditions which could influence test development and use. Some of these external conditions or critical issues include: a country’s social, political and economic conditions, attitudes and values towards science as well as prevailing social problems that could be addressed by test use. Examples of internal conditions are: availabilit y of suitable measures to assess a broad range of qualities, a positive attitude towards HEIDI PATERSON hpaterson@hsrc.ac.za Human Sciences Research Council Assessment Technology and Education Evaluation KOOS UYS Dept of Human Resource Management University of Johannesburg ABSTRACT Various factors in South African workplaces, ranging from unemployment and equity considerations to rapid occupational change driven by technological inputs, have placed greater emphasis on effective selection and development of human resources. This leads to greater demands on psychological assessment in the organisational setting with respect to ensuring responsible, ethical and equitable assessment. The study aims to investigate the extent and purposes of psychological test usage, the influence of the changing world of work on assessment, equity and cross-cultural applicability of tests and competence of test users in South Africa. The results are based on twenty two structured individual interviews, which were conducted with purposively sampled stakeholders from various economic sectors. OPSOMIING Verskeie faktore in Suid-Afrikaanse werkomgewing, wat wissel van werkloosheid en billikheidsoorwegings tot by vinnig veranderende beroepsomstandighede as gevolg van tegnologiese ontwikkeling, plaas toenemende druk op die doeltreffende keuring en ontwikkeling van menslike hulpbronne. Dit lei tot toenemende eise wat in die organisasie konteks aan sielkundige meting gestel word om verantwoordelike, etiese en billike assessering te verseker. Hierdie studie was daarop gerig om die doel met en omvang van die gebruik van sielkundige toetse, die invloed van die veranderende wêreld van werk op die assessering, billike en kruiskulturele toepaslikheid van toetse, asook die bevoegdheid van toetsgebruikers in Suid-Afrika te ondersoek. Die resultate is gebaseer op ’n doelgerigte steekproef van twee-en-twintig gestruktureerde onderhoude wat met belanghebbers uit verskillende ekonomiese sektore, gevoer is. CRITICAL ISSUES IN PSYCHOLOGICAL TEST USE IN THE SOUTH AFRICAN WORKPLACE Requests for copies should be addressed to: JS Uys, jsu@rau.ac.za 12 SA Journal of Industrial Psychology, 2005, 31 (3), 12-22 SA Tydskrif vir Bedryfsielkunde, 2005, 31 (3), 12-22 testing and adequate standards for appropriate test use. It was endeavoured during this study do identif y and describe critical issues that are relevant to the unique South African context that has an influence on psychological testing. As an external issue, the changing world of work has an impact on assessment practices. Bartram (2004) notes that international and local organisations are adopting common selection and recruitment practices to stay competitive in securing optimal human resources. In the “new” organisation the focus is on recruiting and developing employees with the ability to work flexibly and adaptively, due to rapid change inside and outside the organisation. Assessment can contribute to the identification of these employees. Furthermore, responsible assessment can benefit selection and development of candidates from previously disadvantaged backgrounds as part of a drive towards equity and representivity in the workplace (van der Merwe, 2002). Equitable and fair test usage requires a keen focus on cross- cultural applicability of tests. Cross-cultural use of tests can be discriminatory if the construct measured differs across cultures, especially if the tests are not standardised for use across all cultural groups through a representative sample (van de Vijver and Rothmann, 2004). Foxcroft (2004) further argues that very few tests have been developed that can be used across the diverse cultural and language groups in South Africa. Bartam (2004) calls for more cross-cultural studies to be undertaken, as the assumption cannot be made that all tests can be used with minority groups (van de Vijver, 2004). This will be extremely important in South Africa, since the possibility that a test does not measure what it claims to measure, calls into question all inferences drawn from the test results (Wallis, 2004). Extending fairness and equity to the assessment procedure as a whole can further enhance the perception of testing (Skarlicki, 2003). The test or method can be scientifically proven as fair, valid and reliable, but the perception of the method of assessment can still be perceived as negative by the testee. If the procedure is viewed as fair, and the practitioner effectively handled interpersonal relationships, there is a bigger chance that the employee/candidate will support assessment-based decisions (Skarlicki, 2003). The availability of suitable measures is one of the internal issues that practitioners and test developers have to address. In this regard, Oakland (2004, p. 168) states that “test adaptation is needed when tests are used in countries other than those in which they were developed, when tests are designed for use in two or more countries in which cross-cultural practices occur and when they are adapted for use with persons who differ in language, culture and other important qualities.” Local test developers are faced with the dilemma of either adapting international tests for use in South Africa, or developing new tests specifically designed with the unique societal composition in mind. Regardless of the chosen approach, test development, translation and adaptation should be governed by guidelines such as those published by the International Test Commission (Fernandez-Ballesteros, 1999), as this will ensure the use of high-quality, internationally comparable tests. Lastly, responsible test use requires sufficient training of users as part of initial qualification and continuous development. Practitioners should be sensitized to demands placed on assessment by the multicultural South African society though training in cross-cultural test use (Fernandez-Ballesteros, 1999). Training should further address access to tests, rights of practitioners and testees and the need for qualifications for the application of tests. This article does not engage specifically with hypothesis testing. Rather, it describes current assessment practices and circumstances as experienced by a selected group of participants. Consequently, this project was undertaken to address the following question: What critical issues are psychological assessment practitioners facing in the South African workplace? The aim was to uncover the issues impacting on appropriate, responsible and ethical assessment practices and test use. The article will proceed as follows. Firstly the methodology of the study will be described This will be followed by discussion of the findings from the study with reference to: the application of tests in the workplace; the effect of the changing world of work on the development and application of psychological tests; cross-cultural application of psychological tests which touches on aspects such as linguistic and cultural differences, norming and equity; the adaptation of existing tests versus developing indigenous instruments and lastly, the impact of training in ensuring competence in test use. RESEARCH DESIGN This study is part of a larger project: the Psychological Assessment Needs Analysis, conducted by the Human Sciences Research Council (HSRC), which took place in 2004 over a 5- month period. The overall project followed a multi-pronged approach to collecting data and included a national survey, focus group interviews and the series of interviews reported here. Although the main study addressed patterns in use and future trends across the entire scope of practitioners, the focus of this paper will only be on psychological assessment from the perspective of the industrial psychologist. Research approach The study was conducted from an interpretive paradigm in an attempt to understand lived, subjective experiences (Davis, Nakayama and Martin 2000). The interview can be described as topical, as the researcher and participant interacted on a defined issue, that is, psychological assessment (Tobin and Begley, 2004). The interview was structured and followed a standardised format, but it is important to note that the data gathered was qualitative in nature. The interview did not probe all issues in-depth, but provided the researcher with an overview of relevant issues pertaining to psychological assessment. The study meets the criteria for qualitative research, as the goal of this research by means of individual interviews was to develop an understanding of psychological assessment according to the meanings given by the participants (Stromquist, 2000). The methodology allowed the researcher to interpret participants’ social reality, attach meaning and then diagnose, rather than predict. An inductive process was followed, where the starting point was individual cases, incidents and experiences. This made it possible to progressively develop more abstract, conceptual categories to synthesise, explain and understand the data and the patterned relationships within (Riley and Love, 2000). Participants Participants were purposively sampled to obtain views from experts and practitioners, knowledgeable in the current trends in psychological testing. The participants practice different occupations, as well as in different institutional contexts. The sampling was not designed to achieve representivity, or to transfer the findings to a broader population, but to explore the perceptions and opinions of the particular participants across economic sectors and activities. It should be noted that the participants included only test users, and non-users were not approached. The views and experiences of non-users were thus not probed and the reasons why some people choose not to use tests, were not explored in detail. This is an area to be investigated by further research. ISSUES IN PSYCHOLOGICAL TEST USE 13 The participants included stakeholders working with people in the critical life phases of youth and adulthood. This reflects the broad areas of use of psychological tests, as summarized by Oakland (2004), namely: describing and predicting behavior, evaluating and diagnosing, screening and placement as well as assisting with guidance and counseling services. The focus was thus on gaining a perspective from practitioners involved in critical life events such as career guidance, selection and placement, development, etc through assessment. Certain crosscutting instances of test use were also addressed, for example: identification of psycho- pathology through the use of psychological instruments and the use of assessment tools for forensic purposes. These examples demonstrate that some psychological instruments can be used across different life phases. The Sector Education and Training Authorities (SETAs) served as an additional selection criterion. Because there are a large number of SETAs, a decision was made to group them together to cover broad areas of economic activity. At least one representative from each category was interviewed. The categorisation of the SETAs into sectoral groups is depicted below: TABLE 1 CATEGORISATION OF SETAS INTO SECTORAL GROUPS Category SETA Services FASSET (Financial and Accounting Services) BankSETA (Banking Sector) INSETA (Insurance Sector) SERVICES (Services Sector) TETA (Transport) THETA (Tourism and Hospitality) W&RSETA (Wholesale and Retail Sector) Public DIDTETA (Diplomacy, Intelligence, Defense and Trade and Industry Sector) LGWSETA (Local Government, Water and Related Services) PSETA (Public Services) Social Services ETDP SETA (Education, Training and Development Practices Sector) HWSETA (Health and Welfare Sector) POSLECSETA (Police, Private Security, Legal and Correctional Services) Manufacturing TEXTILES (Clothing, Textiles, Footwear and Leather Sector) CETA (Construction) FOODBEV (Food and Beverages Manufacturing Industry Sector) MERSETA (Manufacturing, Engineering and Related Services) CHIETA (Chemical Industries) Communication ISETT (Information Systems, Electronics and and Technology Telecommunications Sector) MAPP (Media, Advertising, Publishing, Printing, Packaging Sector) Resource-based ESETA (Energy Sector) FIETA(Forest Industries Sector) MQA (Mining)PAETA (Primary Agriculture) SETASA (Secondary Agriculture Sector) An additional consideration was to include large-scale users of psychological testing. Although the Department of Labour and Department of Education are classified as administering agents from the social services cluster, these departments influence training and development for employment throughout the individual’s life in terms of primary, secondary and tertiary education and training and lifelong learning. Other participants included Psychometrics lecturers from universities, as these stakeholders influence the training of future test users, i.e. Psychology students. Twent y-t wo interviews were conducted with 31 senior decision-makers and stakeholders. In some cases more than one person participated in a particular interview. All the interviews were conducted with registered psychologists across the registration categories (clinical, industrial, counseling, educational, and research), with the exception of two, where union representatives and decision-makers in the Department of Education were interviewed. Half of the participants were female, and the majority was white. The low representivity of black people in decision-making positions in the field of industrial psychological assessment is of concern. It should be noted that this report might not adequately reflect the voice of black people with regard to psychological testing. Table 2 provides information on the demographic distribution of the participants. TABLE 2 DEMOGRAPHIC INFORMATION OF PARTICPANTS Demographic information of participants Gender Ethnicity Area of work Male Female Black White Serv- Public Social Manu- Commu- Re- ices Serv- fact- nication source- ices uring and Tech- based nology 16 15 5 26 9 3 13 2 2 2 Data gathering instrument An interview schedule, which served as a guideline for the interviews, was developed and refined from the following pre- defined indicators: � Psychological tests currently in use. � Strengths and limitations of psychological tests currently in use � Suggestions and requirements for the development of new tests. � Importance of psychometric properties such as reliability, validity and norming. � Monitoring and management of the quality of psychological services and tests. The instrument was piloted and additional questions were added. Although a standard format was followed, the schedule wasn’t followed rigidly. Reliability As this is a qualitative study, the term dependability is better suited. The aim was not to ensure that the study could be replicated in another study, but whether the findings are reasonably based on the data (Pitney, 2004). A comprehensive register of data was kept and both theoretical and methodological memoranda were compiled. This took the form of an audit trail where all decision regarding the sources of data, collection techniques, and meanings interpreted, as well as assumptions and influences on the researcher are noted (Long & Johnson, 2000). The interviewer was assisted by other researchers in validating these notes and memoranda through cross-checks. Validity The term credibilit y should replace validit y, as in qualitative research this element of rigour demands that the researcher ensures that she adequately capt ures what she learnt in the research context (Pitney, 2004). Credibilit y as a criterion was imposed on the research procedure and findings. The procedures followed were credible in that the techniques and data-gathering instrument (interview schedule and framework) were designed to facilitate an analysis of the experiences in terms of the use of psychological tests and the PATERSON, UYS14 qualit y of psychological testing in general. Peer review was used to enhance credibilit y in that other researchers provided critical input during the design, execution and interpretation phases of the study. (Pitney, 2004) It was endeavored to maximize credibilit y in the findings in that only one interviewer conducted the interviews, with assistance from other researchers. The assisting researcher sat in on the interviews and read the transcription to confirm its accuracy. Peer debriefing entailed discussion the findings at intervals with knowledgeable colleagues (Long and Johnson, 2000). The interviews were conducted over a four-month time span. Data analysis and report writing occurred concurrently. Data Analysis Procedure Data analysis went through the following stages: i. Defining analysis in terms of research goals: The aim of the research was kept in mind and it was ensured that appropriate and sufficient data were collected. ii. Familiarisation and immersion: The researcher became familiar with the data by reading the material thoroughly. iii. Annotating: Themes were induced from the data and research memos were written. iv. Tagging data: Baptiste (2001) refers to tagging as the process of select ing from an amor phous body of materials, bits and pieces that satisf y the researcher’s curiosit y, and help support the purpose of the study. For the purpose of this study, tagging was also employed as a means of crosschecking and corroborat ing evidence. Tagging takes the form of open coding, where the focus is on potential meaning, and pre-existing codes are not used. v. Grouping tagged data into categories: The themes were elaborated by grouping the data together according to certain criteria. These categories were guided by the research question. The categories were mutually exclusive as far as possible. vi. Making connections: The data were interpreted through defining and redefining the themes. Certain patterns of relationships and concepts were identified. RESULTS Firstly, information gathered around the contextual back- ground of test use will be reported on. This provides a backdrop against which the purposes for which tests are used as well as the impact of tests on assessment will be discussed. Table 3 provides a summary of themes identified through data analysis. TABLE 3 THEMES IDENTIFIED THROUGH DATA ANALYSIS Key Theme Sub theme Contextual background of test use � Advantages of using psychological in the workplace tests· � Psychological test use in combination with other methods as part of assessment � Purposes of test use The influence of changes in legislation The influence of the changing � Changes in the labour market world of work � Use of international tests Cross-cultural use of tests � Influence of language and culture Norming/renorming of tests Adaptation of existing tests Development of new tests Training and competence of � Role of the Health Professions test users Council of South Africa (HPCSA) Contextual background of test usage Findings from the study suggest that psychological tests are commonly used in the workplace as a tool in decision-making in matters such as selection, job assignment, promotion, training and termination of employment. Participants stressed the advantages of using psychometric tests as part of an assessment battery: “Standardised tests act as equaliser, regardless of colour and background”. Participants reported that in most cases psychological tests are not used in isolation, but form part of an extensive assessment battery. “Such a combination of tests, simulations, inter views, and you name it, always work the best, but you obviously have to look at time and money.” Participants argue that the approach to selecting an assessment battery should be guided by the rationale of the assessment, and not according to the tests the organisation have in stock. There is no standardised assessment process that all the participants follow, but in general the procedure consists of the candidate being taken though various forms of assessment. The table below identifies the purposes of test use, ranging from most to least frequently cited in the interviews. TABLE 4 PURPOSES FOR WHICH PSYCHOLOGICAL TESTS ARE USED Most frequently cited Cited to an extent Least frequently cited � Selection and � Identification of � Optimal fit and recruitment potential placement � Training and � Research· � Retention· development· � Screening of large � Determining � Team development· numbers of people literacy levels � Interpersonal skills � Accreditation of and self-improve- qualifications· ment � Promotion � Career counselling � Identif ying beha- and development viour problems � Succession planning � Job analysis � Development of leadership � Bursaries and evaluation � Training of users The participants were probed on the reasons why they cease to use a particular test, and why this test is then in some cases replaced with another instrument. The reasons provided by the participants include: (i) the test has become outdated, (ii) the test is not culture friendly which hampers cross-cultural application, (iii) there is poor or no research on the application of the test, (iv) the user receives insufficient support from the distributor (v) perceived exorbitant prices of the instrument, and (vi) the test is no longer applicable in a changing population. Participants noted that the external environment, in the form of the world of work, has an impact on ethical and responsible test usage. The impact of changes in legislation on ethical application of tests Changes to legislation place new demands on psychological tests and practitioners. “… you must assess what you are supposed to be assessing, you are not allowed to discriminate [unfairly], you’ve got to be fair in your processes and ethical.” ISSUES IN PSYCHOLOGICAL TEST USE 15 Participants reported that in the past testing was seen in a negative light due to unfair and discriminatory application. This attitude is changing: “I think 10 years ago there was a very negative sentiment regarding testing, (and) I think that people started seeing that even an inter view can be unfair. They saw that there are alternatives, which also have inherent problems, and I think people started appreciating the objectivity of the kind of information you get [from tests]”. Participants argued that criticism should be levelled against practitioners instead of tests themselves. “The question mark drawn over psychometric testing was 5% about the content of the test. Ninety five percent was about the people that used it. People used it to keep the system in place. I think that still happens in some environments where people use the test or test material to keep their agendas in place. That can be political or financial agendas.” The influence of the changing world of work Some tests currently in the market were developed during the previous decade and cannot be applied without reserve in the new organisation, as the labour market has changed significantly. Participants state that applying outdated tests can lead to misuse and drawing incorrect conclusions. This has not only detrimental consequences for the individual, but also cost implications for the organisation. Although the participants cited some examples of inappropriate test use, in general they strive towards applying updated, relevant tests. “ [I think those things have to be updated every 5 years] in this fast changing world in which we live. Because at present you are only making inferences. You carry certain information over to other fields, because these fields are not being measured. And your population is changing.” The need to compete globally, had lead to practitioners using international tests imported to South Africa. These tests can be unsuitable in the South African context. “… you cannot always import things from other countries into your country. You must look at whether that system will be suitable for your environment as a developing country.” Cross-cultural use of tests When addressing the use of tests in a cross-cultural setting, the participants raised the issue of the influence of language and culture on test performance. The participants pointed out the need to overcome the language issue in assessment: “… where language and educational difficulties are a problem, we have to be creative in terms of how to measure it in a more tangible, concrete way.” Participants raised t wo very different arguments in proposals to overcome linguistic disadvantage in the workplace. Some participants felt that the test users should be able to respond to psychological tests written in English, regardless of the testee’s home language, as an inabilit y to deal with language barriers will negatively influence their performance at work. They argue that the major global business language is English and the participant should be able to converse fluently in this language. Other participants argued that test users who for whatever reason are not fluent in the language of the test might perform badly on, for example a cognitive test as a result of poor language skills rather than cognitive ability. If the test taker does not understand the words and prompts of the test, the instrument is incapable of delivering reliable information. Participants strongly argued that language differences should not prevent people from benefiting from psychological testing, as summarised by a participant: “But the most important thing is not to use it [test scores] as a barrier. Not to say that it keeps the person away from training, but that it is an indication to him or the organisation that he f irst improves his language ability, because if he improves his language ability, he will perform better work-wise. Because if the language is bad, language itself becomes your barrier.” This group of participants argue for the translation of existing instruments, the use of projective measures and/or non-verbal instruments or screening for language proficiency before psychological testing to avoid potential discrimination. Norming The participants stated that one way of addressing inequities in cross-cultural application of tests, is to ensure appropriate norming. “I don’t think you can go without norms. How will you compare? Our job in industry is to compare, not discriminate, but differentiate. Better, worse – that is our job. And you have to compare people to do this.” Moreover, participants are aware of the complexities of norming/renorming tests for use in the South African context. The consensus among participants was that the norming process should be scientifically approached with a representative, stratified and appropriately sized sample. This necessarily involves continuous research on all tests, local and international. A participant emphasised the role of research in test development and adaptation: “ … with all tests you have to do continuous research, the same for local and international tests. But to de velop a test is a long process. Our dilemma is that our cross-cultural situation is so different from other countries, that basic matters such as language cannot be addressed easily. So the changes required will be radical and you will have to standardise the things again to ensure that it still does what it is supposed to.” All the participants stressed the importance of developing appropriate norms, but presented different suggestions for making the process equitable and fair. Some argued for the use of different categories as reference groups, believing that it can be unfair to compare people from different educational backgrounds on the same norm. They felt that norms must be developed based on age, race, educational background and language because of past disparities in opportunities and access to education. Others felt that the development of different norms for different groups can be construed as discriminatory, paternalistic or can even be perceived as analogous to apartheid practices. Furthermore it was observed that using race and language as reference criteria in norming may also not be relevant any more on account of urbanisation, acculturation and social class mobility among previously disadvantage groups. “I think it will be unfair to have a disadvantaged norm. I also don’t think it will have practical value. I think you have to make distinctions based on more than language and race, because there is a lot of mixing. The groups are not as purely advantaged or disadvantaged as 10 years ago.” Other participants argued for a more qualitative, content-driven interpretation of tests results, or the comparative use of alternative norm groups where the test taker is compared to both the typical Western norm and a general mixed norm. Again it becomes clear that a normed result should not be taken on face value, but contextual factors should be considered. PATERSON, UYS16 “You should not develop a norm on those people for whom the test does not work. That is a prerequisite: you can only norm on groups where your test is reliable enough to use.” Furthermore, participants debated the issue between adapting existing tests versus developing new instruments. Adaptation of existing tests Participants stated that test use could be improved by adapting tests for the organisational setting. “ Testing should be adapted to be more focused on employers, and not only for a clinical set-up. We have to make these things practical. We say all this wonderful things: that testing is an aid for decision-making, but if you can’t use it, it can’t be an aid. And if you use it wrongly, it won’t be an aid.” Furthermore, participants indicated that for tests to add value to the decision-making process, test information should be triangulated against other forms of information (i.e. interviews and other forms of non-psychological assessment). In some cases other methods of assessment may be more effective, and testing is not the preferred recourse in all situations. Participants therefore observed that testing should only take place if this practice can add value to the decision- making process. “I think you can support testing if testing is one of the elements that is considered, … if it is adaptable and if it can’t be manipulated, if I have a clear indication of what is measured for which category of people.” The participants’ views on existing tests cover two broad areas: underlying principles of test usage (cross-cultural validity and reliability) and practical limitations (time and constraints and limited funding). This has to be taken into consideration when adapting existing tests in future. Their suggestions regarding the adaptation of existing test material included: � Test manuals should be explicit in defining the context, rationale, target group and applicability of the particular test. � Psychological tests should be adapted to integrate with National Qualification Framework (NQF) levels and should take into account qualifications not formally obtained. � Although there is still a need for paper and pencil tests, computer-based testing has many advantages. Participants expressed a need for existing tests to be computerised. Development of new tests Participants identified the following areas in which new tests can be developed: (i) identification of potential and developmental areas in people with lower educational qualifications, (ii) measuring cognitive potential with non-verbal instruments, (iii) leadership in the South African context, (iv) emotional intelligence, (v) integrity, (vi) language skills assessment, (vii) pre-selection screening test to indicate whether further in-depth testing is required, (viii) team performance, (ix) personality assessment for people with low literacy, (x) addressing the needs of illiterate adults and unemployed youth and (xi) meeting the assessment needs of people with disabilities. Participants felt that existing measures do not address these areas adequately. In terms of the development of new instruments, the participants emphasised that all instruments need to meet the following criteria: proven predictive validity, cross-cultural fairness, relevance and reliability. Use of unregistered tests Practitioners felt that one way of protecting the organisation against litigation and unethical practices is to only use instruments registered with the Health Professions Council of South Africa (HPCSA) and avoid instruments that any unqualified person can use. In addition, they feel that sensitive information generated by psychological tests should not be for public consumption, and advise against line managers and clients having direct access to it. The sensitivity towards the ethical use of tests is echoed by the labour union: “As a trade union we say it must be fair with reference to our members and equal. It has to be transparent. And the moment we see it is not equal, transparent and fair, we have a problem.” Sometimes the practitioner will need a type of test that is not included in the HPCSA list. The user is then forced to use available measures, which might not be registered by the HPCSA to assess these areas. In some cases, the users develop tests themselves to suit their unique needs and circumstances, and these tests are not tabled for registration. Some tests are in the process of being registered, e.g. a list of Neuropsychological tests covering a broad range of instruments, have been sent for registration, but a final decision is still pending. The majority of unregistered tests are international in origin. Practitioners sometimes utilise well-known international tests, which may have a solid base of research, without taking cognisance of the fact that the test is not standardised for use in South Africa. Practitioners are sometimes compelled to use international tests with international norms when a candidate has to be compared against a global standard, such as in the case of international appointments. The use of an international test is justified, according to the participants, if it has been standardised for use in South Africa. Situation-specific norms in the South African context are also meaningful and this is one of the reasons practitioners insist on South African standardisation. All tests have to be supported by local, empirical information. “The weaknesses here are not bad tests, but bad psychologists that use things incorrectly, or that use it exclusively, as if tests always give the complete picture. That is not true – it is only a tool.” The participants pointed out some difficulties with the registration process of psychological tests with HPCSA, which contributes to substantial use of non-registered tests. The registration process with the Psychometrics committee is viewed by the participants as very stringent, bureaucratic and in many ways confusing. “I can’t see that I’m getting any value from the HPCSA in terms of assessment… That they address the matter, have debates about it, do training, have workshops or develop guidelines [and] regulations...” Furthermore, it is expensive to submit tests for registration and the thoroughness of the process is questioned. Practitioners feel that there should be independent studies to validate results submitted by test developers as part of the registration process. In their view, the decision about which tests to table for registration should not be left to the test developer. Participants are also unclear about certain definitions in the classification of tests. There is no provision in the guidelines of psychological assessment for testing in a human resources (HR) capacity. “it is difficult to even find a person responsible in setting out the guidelines, but there is no active energy in the ball game. It is like: if they do or don’t, who cares? South Africa has the most regulated test environment in the world, but in practice it is difficult to implement.” Training and competence of test users Practitioners felt that all test users are not properly trained in the application and administration of psychological tests. “…they use it simplistically, mechanistically, without understanding the full context. It is irresponsibly used – it’s like using a medicine that you are not trained to use.” ISSUES IN PSYCHOLOGICAL TEST USE 17 Irresponsible test usage has detrimental effects for both the practitioner and organisation, especially in terms of South African legislation. “I think it is about the person’s integrity and training. If you work in court you expose yourself, and if you don’t use the stuff ethically and professionally, you would have exposed yourself.” Participants suggested that test-users should receive comprehensive training as part of their graduate work, and that further training should form part of continuous professional development. “… it must be a responsible test and it should be responsibly used. It must be relevant firstly and interpretation must be done responsibly by suitably trained people.” Currently, training at tertiary level includes areas such as test construction and psychometrics, as well as a course in test usage. It was observed that the training is not standardised across universities and there are some gaps in the training of psychologists and future test users. In this respect, the responsibility lies with universities and the HPCSA in terms of setting the accreditation examination, and regulating further training, to ensure that users are informed and skilled in the ethical use of tests. DISCUSSION In the discussion of the results of this study, information regarding the use of tests for a broad range of purposes and the use of psychological tests in combination with other assessment methods provides the contextual background to test use. Against this background, practitioners face many critical issues in ensuring appropriate and ethical test use, i.e. changes to legislation, the changing world of work and the use of international tests. Test use is also influenced by social and cultural factors such as the cross-cultural use of tests, and the inf luence of language and culture on ethical test use. Practitioners respond to these challenges by the appropriate norming/renorming, adaptation of existing tests, development of new tests and the use of unregistered tests. In general, ethical and responsible test use requires adequate training and competence of test users, and the active involvement of professional bodies. The above-mentioned critical issues are schematically depicted below in Table 5, which represents an illustration of the themes uncovered in the research as well as the relationship between the themes. TABLE 5 CRITICAL ISSUES AFFECTING PSYCHOLOGICAL TEST USE IN THE SOUTH AFRICAN ORGANISATIONAL CONTEXT Context of test use in Critical issiues and Responses to challenges the workplace challenges in terms posed to test usage of test usage � Advantages of using Environmental � Norming/renorming psychological tests variables: of tests � Purposes of test use � Changes in � Development of new � Psychological test legislation tests use in combination � The influence of with other methods the changing world as part of assessment of work � Training and competence of test users � Role of professional bodies Context of test use in the workplace Globally there has been a move towards a more positive perception of psychological tests and testing as an acceptable practice. Oakland (2004) reports that there are an estimated 5000 standardised tests in English, constructed mostly in the Western Europe and the USA. This shows that there are many tests available for a range if purposes. Psychological tests provide an indication of complex constructs such as knowledge, skills and psychological attributes which are difficult to measure with other methods. Existing tests cover a broad spectrum of measurable areas, and the users can choose between different instruments to meet specified needs. The findings of this study thus agree with Owen and Taljaard’s (1996) statement as cited in van der Merwe (2002) that psychological tests can contribute to the efficacy of selection and placement in industry, if applied appropriately in consideration of the context, testee characteristics and reason for assessment. Literature supports the advantages of test use as reported by the participants in this study. Appropriate testing can decrease the probabilit y of employing and training the wrong person. The organisation can thus strategically increase their effectiveness by recruiting and developing their human resources (Bartram, 2004). This is extremely important in an organisational setting, as inappropriate appointments and consequent training can lead to huge financial losses in the long term. Practitioners see psychological tests as a more ‘objective’ measure when compared to other methods such as interviews. Test results provide a wealth of information in a short time (Bredell, Van Eeden and Van Staden, 1999). This information forms a basis for comparison and the testee can be evaluated against his/her peers. The testee also benefits in that he/she is given the opportunity to increase his/her self-awareness and focus on his/her strengths. This can only be achieved when the testee is provided with extensive feedback. The finding that practitioners use psychological tests in combination with other methods like simulations, interviews and in-basket exercises corresponds with the findings from van der Merwe’s study (van der Merwe, 2002). Bartram (2004) found that the most effective test battery for selection purposes includes measures of ability and personality traits such as conscientiousness and integrity. Many factors are taken into consideration when deciding on an assessment battery, as the assessment procedure is seen as a reflection of an organisation’s values and culture (Bartram, 2004). From the above, it can be concluded that psychological tests are important tools, used frequently in the assessment decision-making process in both South Africa and inter- nationally. When applying psychological tests in the workplace practitioners face many critical issues and challenges, which will be discussed next. Critical issues and challenges in test use Psychological practitioners have to meet the demands of a changing workplace. South African legislation has shaped the practice of practitioners. The draft policy of the Professional Board of Psychology on the Classification of Psychometric Measuring Devices, Instruments, Methods and Techniques demands that scientific proof is provided of an instrument’s psychometric properties such as validit y, reliabilit y and absence of bias, to ensure that the instrument can be used equitably in the workplace. Practitioners therefore have to be more cautious and informed about the instruments they apply, and should take the responsibility to ensure that a test can be fairly applied. The letter and spirit of the Employment Equity Act, 1998 (Act No. 55 of 1998) prescribe the equitable use of psychological tests. Psychological tests can act as a disabling factor, if the test is inappropriately applied, or used in isolation without verif ying the results against other measures. The Skills Development Act (Republic of South Africa, 1998) through the Lev y Grant Scheme (Republic of south Africa, 1999) aims to improve human resources and promote equit y in the PATERSON, UYS18 workplace. Assessment can ensure appropriate targeting for training by identif ying candidates who might benefit most from further training and development. The workplace itself is changing. The new world of work requires speed, flexibility and an empowered work-force. Due to globalisation and rapid change, the workplace looks different than what it did a few decades ago. Work now requires a different set of skills and traits from the successful candidate, and psychological tests should be designed to measure this. Emerging technology and economic changes such as a greater reliance on outsourcing certain functions, have changed the organisational structure, as well as the kind of work that the employee is expected to perform. There is an increase in the demand for specialists, whereas the need for low level manual workers has decreased. The worker, in partnership with the organisation, has become an equal partner in determining and managing his/her career path. This requires resilience on the part of the worker and highlights the importance of life-long learning. From an international perspective, Fernandez-Ballesteros (1999) also note the following challenges that Industrial/Organisational psycho- logists have to face: a multi-ethnic, multiracial, multinational community, an aging society and a renewed interest in quality of life. Oakland (2004) found that in most countries foreign developed tests are used more frequently than nationally developed tests. From the responses of the participants it is clear that the use of tests not registered with the HPCSA is not simply a matter of wilful malpractice, but is symptomatic of a wider problem. This practice has its roots in a number of factors. In many cases there is a lack of awareness and knowledge among practitioners about which tests are registered and which are not. Organisations are sometimes not aware of the registration status of the tests they use. In some cases instruments are selected because it is familiar, well advertised or offer the “latest” technology. Organisations are constantly searching for cutting edge tools to enhance their competitive advantage. Many tests are new on the market and have not been through the registration process. Although there are many locally developed tests available in South Africa, many practitioners choose to use international instruments. Many test-users utilise tests not registered with HPCSA because these tests are more easily accessible. Furthermore they are not restricted by the requirement that a psychologist has to administer the test. Small organisations in general, do not have the specialist skills and resources to invest in assessment (Bartram, 2004). A wider range of people can thus administer these tests and the organisation can save money by not employing a psychologist. Practitioners without formal training in psychology are increasingly doing assessment, and personnel officers are expected to carry out assessment with psychological instruments in the course of their day-to-day work. Highhouse (2002) also identified this trend in an international study. Furthermore, practitioners have to keep social and cultural factors in mind of tests are to be used ethically. This will be addressed next. The impact of social and cultural factors on test usage: Cross- cultural use of tests From an international perspective, Bredell, et al. (1999) state that a recognition of cultural differences has lead to the development of instruments to measure potential, as well as a sensitivity towards the contexts in which the individual functions. Not all instruments used in South Africa have been thoroughly researched in terms of their cross-cult ural applicabilit y. Nevertheless, practitioners still engage in assessment practices using tests not standardised for South Africa. The difficulty of choosing between instruments is further exacerbated by the fact that information regarding the classification of tests according to performance-driven criteria is unavailable. When investigating the cross-cultural application of tests, attention should be given to factors such as language, race, gender, socio-economic status, educational background, etc. (Bedell, et al.1999 ). Specifically the impact of language and cultural differences on testing will be discussed in more detail. The most frequently cited hindrance to the administration of psychological tests as reported by interview participants is language. Language can be problematic on three levels: (i) the language in which the test is constructed, (ii) the difficulty level of the test language, especially if the test is administered in the testee’s second or third language, (van de Vijver and Rothmann, 2004) and (iii) the language competence of the testee. Language is closely tied to cult ure. Cult ural context particularly becomes a problem when doing personality assessment, as constructs have different meanings and are experienced differently across cultures. In general personality tests require a high level of language proficiency (van de Vijver and Rothmann, 2004). Bredell, et al. (1999) state that culture is an important moderator in test performance because it affects behaviour and consequently the psychological construct being measured. Other factors that can influence test performance include: cultural and/or environmental factors, language, level of familiarity with tests, rural or urban residence, acculturation, age, gender, etc. It is important to determine whether the performance on the test reflects the testee’s ability, and not her level of competence in the test language (Foxcroft, 2004). The test developer has to ensure that the same construct is measured across different groups when using different languages. This is in line with the stipulations of the International Guidelines for Test Use (Foxcroft, 2004). It is therefore necessary to examine the bias of the items as well as the validity of the test across cultures. It is important to recognise that translation is not a quick-fix solution to cross-cultural use of tests. Even changing some wording in an item can draw a question mark over the construct, score and predictive comparability (Bredell, et al., 1999). For these reasons some participants objected to the idea of translation, as this process might change the constructs being measured and will require additional expense for restandardisation. Another solution could be to eliminate language from testing by using non-verbal instruments. From an international perspective, Highhouse (2002) observes that a negative attitude towards psychometric tests and a lack of trust in judgmental prediction methods which characterises psychometrics, has lead to an increase in the use of projective techniques. In a multi- cultural context such as South Africa, this method presents problems because when using projective techniques the test taker’s response has to be translated and much of the nuances of the language can be lost through this process. Moreover, existing projective techniques are culture-bound and will be interpreted differently across cultures, which will not automatically solve the cross-cultural applicability debate. Thus in the case of projective techniques, the issue of language may be addressed, but cultural differences are not adequately accounted for. Van de Vijver and Rothmann (2004) suggests various approaches to eliminating discriminatory use of tests in cross-cultural application, i.e. the development of culture-specific norms (van de Vijver, 2004), introducing score adjustment procedures and the development of new instruments. The participants in the ISSUES IN PSYCHOLOGICAL TEST USE 19 current study echoed these sentiments. Norming/renorming new or existing tests, adapting existing tests using unregistered tests and lastly developing new tests will be discussed as responses to challenges posed to test usage Responses to challenges posed to test usage Norming/renorming Norming is a form of benchmarking where the performance of the individual is compared to a relevant reference group. The issue of norming or renorming psychological tests is a controversial issue, which requires careful consideration of all related factors i.e. age, education, gender and race. Norming is thus seen in conjunction with standardisation. Participants in the South African business world have followed the route of developing situation-specif ic norms, where candidates are benchmarked against specific job requirements. Factors such as age, gender and race are not taken into consideration and all candidates have to meet a uniform standard. The assumption is that predictive validity is facilitated through situation specific norming. This enables the psychologist to justif y decisions based on job requirements – all successful applicants have to meet particular requirements, regardless of demographic differences. The aim of the test is to reliably measure constructs present in the work situation with a proven link to job success. Although there is one norm group for each job, every different job will have a different norm group e.g. different norm groups for executive management and clerical staff. Some larger institutions and enterprises have the resources and data available to develop situation-specific norms and to do statistical analysis themselves. Private practitioners do not have such a facility or expertise at their disposal, and will clearly experience difficulty in generating norms tailored to their unique needs. This suggests that the clients of private practitioners are even less likely to be assessed with appropriate norms. To compensate for the problem, a facility could be created where private practitioners can contribute to a database and also have access to this source of information for norming purposes. Furthermore, it is imperative that tests meet the requirements of validity, reliability and cross-cultural applicability. Information on the aspects should be included in the test manual (van de Vijver and Rothmann, 2004). There are many different arguments about whether (i) new tests have to be developed from scratch, (ii) whether international tests can be adapted for use in South Africa and (iii) whether a strategic approach should be followed where certain tests are selected to be adapted while new local tests are also developed. This will be discussed in the next section. Adapting and developing tests Evaluating the adaptation of existing tests with the development of new tests calls for a decision-making process that compares advantages and disadvantages against cost implications. Foxcroft (1997) supports the adaptation of existing tests and the development of culturally appropriate tests and norms, but cautions that when developing tests, the practical limitations of the South African context should be taken into consideration. Cultural and linguistic differences can hamper the development and norming of tests and South African test developers and distributors are faced with an enormous challenge. Shuttleworth-Jordan (1996) argues in favour of adapting and standardising existing international measures, as increasing urbanisation, acculturation and literacy might hamper the development of culturally appropriate measures. The debate around adapting or developing thus needs to be addressed. Adapting existing tests A test can either be updated, or adapted or both. Updating refers to making a test more relevant and changing outmoded aspects of the tests, whereas adaptation can include this, but mainly refers to changing the test in terms of restandardisation and renorming. Participants did not refer to limitations of existing tests that could be addressed by adapting them. It is possible that the participants don’t give the adaptation of existing tests enough thought in their ordinary day-to-day functioning. Users might not be aware of how the limitations of existing tests can be addressed, or they might be content with the tests they are using. This corresponds with the findings from van der Merwe’s study, which showed that users in general are satisfied with the tests they use, although they do mention that some instruments and material are outdated (van der Merwe, 2002). Skarlicki (2003) mentions an interesting paradox in reviewing the literature on the effectiveness of HR practices: test takers tend to deem invalid tests (i.e. low predictive validity) as more fair, which stresses the point that the perception of fairness is as important as psychometric information. Developing new tests There is a view among the participants that in some instances existing tests should not be adapted, but new tests should be developed from scratch. This is especially the case in personality measures. When developing a new test, attention should be given to item format, construct validity, relevance of content and appropriate use of language within the South African context (Foxcroft, 1997). Van der Merwe in his study also found that the market would welcome tests developed in South Africa, where participants expressed a need for the development of culture-fair instruments. This approach could be enhanced by implementing competency-based assessment, which is job related rather than norm based (van der Merwe, 2002). Bartram (2004) states that competency-based assessment focuses on the essential behaviours required to perform a job though focussing on individual differences in terms of work-related constructs relevant to job performance. There is thus a move away from traits to potential in relation to work behaviour. Training and competence of test users and the responsibility of professional bodies Oakland (2004, p. 162) remarks that: “sound testing practice require suitable educated professionals to cor rectly select, administer, score and wisely interpret tests and other measures.” Adequate and continuous training thus enhance appropriate and ethical test use. Currently there is no standardised training programme for psychological assessment available across all training institutions in South Africa, and the ethical standards governing test development and use are embedded within the broader ethics code (Oakland, 2004). Bartram (2004) summarises many authors’ concerns about the indiscriminate use of tests without adequate training of practitioners. Current guidelines in terms of classification and application of psychological tests are not very clear. According to these guidelines some tests can be administered by “suitably trained” people, which do not necessarily entail a qualified psychologist or psychometrist. The question arises: What does “suitably trained” really mean? These and other specifications and guidelines require clarification. A policy document stating when, where and how psychological tests can be used should be available and easily accessible. The participants in this study observe that they urgently need to get more support from the HPCSA and other professional bodies with regard to assessment practices. Institutions like the HPCSA, Psychological Association of South Africa (PsySSA) and test developers could contribute toward standardising and providing training. PsySSA has a particularly important role to play in this regard. Oakland (2004) argues that conditions of test use are PATERSON, UYS20 more favourable if the professional association takes a positive attitude towards assessment, is involved in training and preparation and takes the initiative in setting out standards for test use and development. From the information received from the participants, it can be argued that the HPCSA is currently not in a position to efficiently regulate access to and use of tests. A possible strateg y is to strive towards self-regulation, where practitioners are empowered to make informed decisions through access to information and adequate training. Participants felt that if test users were adequately trained they would choose not to use unregistered tests, without being regulated into this decision. This argument is supported by comments from practitioners that the negative connotations around psychometric testing could be ascribed to unqualified, incompetent administrators, and not the tests themselves. Concluding remarks In the era after 1994, with changes in legislation and the structure of the work-force, psychological assessment has an important role to play in developing and maintaining an organisation’s competitive advantage, but also needs to be responsive to the demands and requirements of the organisation and worker. The study showed that although practitioners are faced with many critical issues, psychological tests are used in industry on a large scale and for a variety of purposes. The current study highlighted the tension in South Africa bet ween achieving equit y in the workplace and increasing productivit y. This debate has relevance when it comes to the application of psychological tests and selection of norms. The argument is raised that standards should not be lowered and all populat ion groups should meet the same criteria, especially when compet ing in a global economy. The counter argument is that psychological tests should take into consideration the inequalit y of the past and different norms and standards should be set for different population groups. Practitioners further remarked that the tests available in the market are mostly developed from a clinical model, where the focus is on deficiency and pathology. Practitioners have to adjust these measures to answer human resource related questions, as assessment in industry requires a focus on strengths and potential. A need for the development of tests that are specifically geared at industry and practical in nature was identified in the study. The cross-cultural applicability of tests is an important issue, which should receive attention through continuous research. Participants accept that language and culture influence test performance, and strategies should be put in place to ensure that tests are used equitably. Divergent arguments support the notions of developing local instruments, as well as adapting and standardising existing tests for use in South Africa. It is important that test developers and distributors, together with test users, take an active part in the governing and quality assurance of psychological tests. Industry should be made aware of the importance of using sound instruments applied by suitably trained and qualified administrators. Training is highlighted as a strategy to improve the quality of tests and testing practices in South Africa. St udents in industrial/organisational psychology as well as practising psychologists have to receive relevant and applicable training. Established psychologists have to be retrained and updated in new developments and practices in psychological assessment. The relevance and quality of current training of Industrial Psychologists was questioned, especially as there is not a standardised training programme in place. Are Industrial Psychologists adequately trained and prepared for assessment in 2005? It is not proposed that this study provide an adequate overview of all tests currently in use in South Africa. It is suggested that a database of all tests classified according to purpose (psychological and non-psychological) should be compiled. This requires focussed research on tests used in South Africa, which should not be limited to the tests registered with the HPCSA. Having access to this data will make it possible to study trends in current tests use as well as infer future needs. The information should be made available to all practitioners in an electronic format, i.e. Internet and should be easily accessible and updated regularly. A first step in addressing the confusion amongst practitioners around psychological assessment could be drafting a framework for guidelines and regulations addressing issues such as access to psychological tests, fairness in testing, ethics and professional qualification standards. Inter- nationally, the International Test Commission has been instrumental in providing a template on which such guidelines can be based (International Test Commission, 2000) – although the unique South African context should be taken into consideration. To ensure equitable, responsible and transparent assessment practice, all stakeholders should be involved in drawing up assessment guidelines. These include practitioners, test developers and distributors, universities and legislative bodies such as the HPCSA, as well as professional associations such as PsySSA and SIOPSA (Society of Industrial Psychology in South Africa). It would be wise to include representatives from all cultural and language groups in this stakeholder group. This will prove to be a challenging but rewarding endeavour. REFERENCES Baptiste, I. (2001). Qualitative data analysis: Common phases, strategic differences. FQS, 2 (3). 114 – 140. Bartram, D. (2004). Assessment in organisations. Applied Psychology: An International Review, 53 (2), 237 – 259. Bredell, B., Van Eeden, R. & Van Staden, F. (1999). Culture as a moderator variable in psychological test performance: Issues and trends in South Africa. Perspectives in Industrial Psychology, 25 (3), 1 – 7. Davis, O.I., Nakayama, T.K. & Martin, J.N. (2000). Current and future directions in ethnicit y and methodolog y. International Journal of Intercultural Relations, 24, 525 – 539. Employment Equity Act, 1998. (Act No. 55 of 1998). Pretoria Fernandez-Ballesteros, R. (1999). Psychological assessment: Future challenges and progress. European Psychologist, 4 (4), 248 – 262. Fernandez-Ballesteros, R. (2002). How should psychological assessment be considered? American Psychologist, February, 138 – 139. Foxcroft, C. (1997). Psychological testing in South Africa: Perspectives regarding ethical and fair practices. European Journal of Psychological Assessment, 13, 229 – 235. Foxcroft, C. (2004). Planning a psychological test in the multicultural South African context. SA Journal of Industrial Psychology, 30 (4), 8 – 15. Highhouse, S. (2002). Assessing the candidate as a whole: A historical and critical analysis of individual assessment for personnel decision making. Personnel Psychology, 55, 363 – 396. International Test Commission, (2000). International guidelines for test use. Louvain-la-Neuve, Belgium: Author. Long, T. & Johnson, M. (2000). Rigour, reliability and validity in qualitative research. Clinical Effectiveness in Nursing, 4, 30 – 37. ISSUES IN PSYCHOLOGICAL TEST USE 21 Oakland, T. (2004). Use of Educational and Psychological Tests Internationally. Applied Psychology: In International Review, 53 (2), 157 – 172. Pitney, W.A. (2004). Strategies for establishing trustworthiness in qualitative research. Human Kinetics, 9 (1), 26 – 28. Republic of South Africa. (1998). Skills Development Act. Pretoria. Republic of South Africa. (1999). Skills Development Levy’s Act. Pretoria. Riley, R.W. & Love, L.L. (2000). The state of qualitative tourism research. Annals of Tourism Research, 27 (1), 164 – 187. Shuttleworth-Jordan, A.B. (1996). On not reinventing the wheel: A clinical perspective on culturally relevant test usage in South Africa. South African Journal of Psychology, 26, 96 – 102. Skarlicki, D.P. (2003). HRM Special issue: Fairness and human resources management. Human Resources Review, 13, 1 – 5. Stromquist, N.P. (2000). On truth, voice and qualitative research. Qualitative Studies in Education, 13 (2), 139 – 152. Tobin, G.A. & Begley, C.M. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48 (4), 338 – 396 Van de Vijver, A.J.R. (2004). Assessment in Multicultural groups: The role of acculturation. Applied Psychology: An International Review, 53 (2), 215 – 236. Van de Vijver, A.J.R. & Rothmann, S. (2004). Assessment in multicultural groups: The South African case. SA Journal of Industrial Psychology, 30 (4), 1 – 7. Van der Merwe, R.P. (2002). Psychometric testing and human resource management. SA Journal of Industrial Psychology, 28 (2), 77 – 86. Wallis, T. (2004). Psychological tests do not measure what they claim to measure: A re-evaluation of the concept of construct validity. South African Journal of Psychology, 34 (1), 101 – 121. PATERSON, UYS22