12SpangenbergTheron.qxd


The objectives of Industrial Psychology are fourfold, namely: 

1. To formulate credible and valid psychological explanations of

the behaviour of people at work; and (flowing from that); 

2. Demonstratively affect efficient and equitable improvement

in the behaviour/performance of people at work through;

3. An integrated set of HR functions aligned with HR strategy,

which in turn is; 

4. Derived from and aligned with an appropriate business strategy.

The first objective, i.e. to explain work behaviour in terms of a

nomological network, is derived from the last three objectives. 

Knowledge provides the power and ability to improve the less

than perfect status quo. Knowledge on working man’s behaviour

is sought to improve organizational effectiveness by affecting

employee performance through theory based human resource

management interventions. The mission of Industrial

Psycholog y is, however, not fully explicated through a

description of what the discipline tries to achieve. It also requires

a description of the method through which the objectives are to

be achieved. The majority of I/O psychologists believe that

industrial psychology’s ideal of valid and credible psychological

explanations of the behaviour of working people can be best

achieved through the scientific method. This is based on

industrial psychology’s control mechanisms of objectivity and

rationality that are designed to compensate for the fallibility of

human decision-making. The scientific method is objective to

the extent to which it succeeds in reducing error which threatens

the validity of inferences (Babbie & Mouton, 2001). The

scientific method is rational to the extent to which it allows for

the evaluation of the evidence on which scientific verdicts are

based. Scientific rationality serves the epistemic ideal by

submitting new scientific claims, and the theoretical and

methodological argument on which it is based, to the scrutiny of

suitably qualified members of the scientific community.

Industrial Psychology is therefore in the long run more likely to

arrive at close approximations of the truth if it places a high

value on theoretical and methodological arguments being

honestly revealed, peer reviewed and openly debated.  Given that

scientific rationality serves the epistemic ideal of science, the

critical questions and concerns raised by Schroder (2004) on the

validation study by Spangenberg and Theron (2003) should be

welcomed as an opportunity to critically reevaluate the

theoretical and methodological grounds on which the

conclusions reached by Spangenberg et al. (2003) are based.

Schroder’s (2004, p. 1) main criticism seems to be aimed at the

use of the in-basket as a stand alone measure to validate the High

Performance Leadership Competencies:

The in-basket cannot be used as a complete measure of

leadership and therefore cannot be used alone as the basis for

investigating the validity of the HPLCs.

In addition Schroder (2004, p. 2) remarks:

... The design of the Spangenberg/Theron study was

inappropriate for investigating the validity of the HPLCs and

the title itself is inappropriate and misleading. More

accurately it was a study of the validity of in-basket based

measures of leadership dimensions.

Schroder (2004) is quite correct in objecting to the

appropriateness of the title of the Spangenberg and Theron

(2003) article. In addition he could have criticized the numerous

other references made in Spangenberg and Theron (2003) to the

validation of the High Performance Leadership Competencies.

In its most general sense validity should be interpreted as the

extent to which inferences are warranted or permissible (Guion,

2002). Measurement validity thus refers to the extent to which

the inferences made from test scores or the interpretation [i.e.

meaning] assigned to test scores is justified/supported (Guion,

2002). Strictly speaking, therefore, what is being validated is not

the measuring instrument, nor the measures obtained from the

instrument, but rather the inferences made from the measures.

Messick (1989, p. 13), in his monumental and definitive

treatment of the validity concept, states:

Validity is an integrated evaluative judgment of the degree to

which empirical evidence and theoretical rationales support

HH SPANGENBERG 
Centre for Leadership Studies, Graduate School of Business

University of Stellenbosch

CC THERON 
ccth@sun.ac.za

Department of Industrial Psychology

University of Stellenbosch

ABSTRACT
A response is given to the critical questions and concerns raised by Schroder (2004) on the validation study by

Spangenberg and Theron (2003). The importance of open debate as an expression of scientific rationally in service

of the epistemic ideal of science is acknowledged. Schroder’s (2004) concern essentially centers on the use of the in-

basket as a stand-alone measure and the appropriateness of the design of the Spangenberg and Theron (2003) study

for the investigation of the validity of the HPLC’s. The objectives of the Spangenberg and Theron (2003) study are

re-examined. The prudence of using and validating the in-basket as a stand-alone measure is discussed.

OPSOMMING
Repliek word gelewer op die kritise vrae en besorgdhede wat deur Schroder (2004) geopper word oor die validasiestudie

van Spangenberg en Theron (2003). Die belang van ’n oop gesprek as manifestasie van die rasionaliteit van wetenskap in

diens van die epistemiese ideal word erken. Schroder (2004) se kommer sentreer wesenlik rondom die gebruik van die

posmandjie as ‘n meetinstrument in eie reg en die toepaslikheid van die ontwerp van die Spangenberg en Theron (2003)

studie vir die validasie van die HPLC’s. Die doelstellings van die Spangenberg en Theron (2003) studie word weer in

oënskou geneem. Die wysheid om die posmandjie as ’n meetinstrument in eie reg te gebruik en te valideer word bespreek.

REJOINDER TO ARTICLE BY HAROLD M SHRODER: THE FALLACY

OF USING IN-BASKET BASED MEASURES FOR THE VALIDATION

OF LEADERSHIP DIMENSIONS

Requests for copies should be addressed to: CC Theron, Department of Industrial

Psychology, University of Stellenbosch, Private Bag X1, Matieland, 7602

107

SA Journal of Industrial Psychology, 2004, 30 (2), 107-108

SA Tydskrif vir Bedryfsielkunde, 2004, 30 (2), 107-108



the adequacy and appropriateness of inferences and actions

based on test scores or other modes of assessment. ... Broadly

speaking, then, validity is an inductive summary of both the

existing evidence for and the potential consequences of score

interpretation and use. Hence what is to be validated is not

the device as such but the inferences derived from test scores

or other indicators – inferences about score meaning or

interpretation, and about the implications for action that the

interpretation entails.

It thus clearly is inappropriate to speak of the validation of

HPLCs. Despite the title and the numerous other unfortunate

occurrences of this phrase in their article, this, however, never

really was the intent of Spangenberg and Theron (2003). The

High Performance Leadership Competencies (HPLCs) (Schroder,

1989) have been used by various South African organisations

since 1997 for strategic planning and the assessment and

development of senior and executive management. Typically the

HPLCs are assessed by means of multiple measures obtained

from several different exercises comprising an assessment

centre. The University of Stellenbosch Business School (USB) has

recently initiated research on the leadership patterns of MBA

students. Specifically the hypothesis is being explored that the

MBA program could induce improvement in the competency

profile of MBA students. Since the use of an assessment centre

for research purposes is expensive and time consuming, it was

decided to measure the HPLCs by means of an in-basket, a key

exercise of the assessment centre. Schroder was commissioned to

develop such an exercise specifically for the USB and specifically

for this purpose. In his commentary on the Spangenberg and

Theron (2003) article, Schroder affirms the meaningfulness of

the use of an in-basket for this purpose by stating:

(The in-basket) provides an efficient and effective vehicle to

offer “hands-on” experience for leadership development

programmes. Feedback on dimensions of leadership

measured by non-interactive methods like the in-basket can

be used to demonstrate the dimensions and as a basis for

coaching. They can be used on a pre-post basis to plot

developmental progress in non-interactive aspects of

leadership which will build the readiness of participants to

transfer this to the broader job of leading.

Given the implicit, if not explicit, objective of the MBA

programme to affect improvement in managerial performance

it, however, only makes sense to use the USB in-basket to

provide feedback, to coach and to plot developmental progress

as part of the MBA programme, if level of managerial

performance is systematically related to in-basket performance.

To aspire to affect improvement in the HPLC profile as

measured by the USB in-basket, and to use the USB in-basket for

feedback and coaching to assist in achieving such improvement

without any evidence that the in-basket derived profile is

systematically related to managerial performance seems

questionable both ethically and practically. To paraphrase

Schroder (2004, p. 1), one could have expected the architect of

the USB intervention to have selected the most valid measure of

the Schroder leadership dimensions to assist in the leadership

development of MBA students.

The real objective of the Spangenberg and Theron (2003) study

consequently was to examine whether the level of managerial

performance may permissibly be inferred from the measures

derived from the USB in-basket due to a systematic relationship

between the scores on the in-basket and construct valid measures

of managerial performance. Managers are responsible, and

should be held accountable for the performance of their work

units (Henning, Spangenberg & Theron, 2003).  Viewed from a

competency model perspective (SHL, 2000), the High

Performance Leadership Competencies are behavioural

expressions of a complex nomological network of person-

centered dispositions and attainments (meant to be affected by

MBA study) which result in superior unit performance,

especially in a complex and dynamic environment. Despite the

inappropriate wording of the title and the numerous other

unfortunate references to HPLC validation in their article,

Spangenberg and Theron (2003, p. 29) nonetheless managed to

provide the following more accurate formulation of their

research objective.

The purpose of the current study was to examine the validity

of unit performance related inferences made from the HPLCs

as measured by the USB in-basket.

We would agree with Schroder (2004) that to obtain a

comprehensive measure of leadership competence would

require multi-dimensional observations of managers in a broad

range of situations across several different exercises which

simulate the breadth of a leader’s work. The in-basket should

form part of array of assessment techniques to describe the

HPLCs of managers. Again Schroder (2004, p.2) seems to concur:

So does the in-basket have a place in management

development and research? Indeed, it does. While it is

inappropriate to use it alone when more accurate measures of

leadership are demanded in research and assessment, these

measures (1) are needed to round out our picture of

leadership based on multiple exercises

However, it only seems prudent to combine in-basket

information with information obtained from other assessment

techniques if (a) the in-basket information is relevant to the

inference being made, and (b) it does not duplicate the

information rendered by the other information sources. In the

context of managerial selection and/or managerial

development the use of the in-basket alongside an array of

other assessment techniques thus seems questionable if it fails

to significantly explain (unique) variance in managerial

(outcome) performance. 

Despite their rather disappointing findings, Spangenberg and

Theron (2003, p. 37) remain optimistic about the usefulness of

the in-basket in managerial selection and development, also with

regards to the assessment of the HPLCs.

In the South African context a major intervention will be

needed to ensure effective and fair application of the in-

basket as a single exercise. It should probably start with the

sensitisation of practitioners to the scoring problem

discussed above. The development of a scoring method that

provides a wider range of ratings is needed. This would

require experimenting with various In-basket scoring

methods, using the same set of in-basket protocols.

REFERENCE

Babbie, E. & Mouton J. (2001). The practice of social research.

Cape Town: Oxford University Press.

Guion, R.M. (2002). Validity and reliability. In S. G. Rogelberg

(ed.) Handbook of Research Methods in Industrial

Organizational Psychology. Malden, Mssachusetts: Blackwell

Publishers.

Schroder, H.M. (2004). The fallacy of using in-basket based

measures for the validation of leadership dimensions. South

African Journal of Industrial Psychology.

SHL (2000). Competency design: towards an integrated human

resource management system. SHL Newsline, March 2000, 7.

Spangenberg, H.H. & Theron, C.C. (2003). Validation of the high

performance leadership competencies as measured by an

assessment centre in-basket. South African Journal of

Industrial Psychology, 29 (2), 29-38.

SPANGENBERG, THERON108