Scanned Document

A Quality Picture: Perspectives
on a Distance Learning Programme

This paper reports part of a study which
examines the construct of quality in Massey
University's extramural, undergraduate,
business studies programme utilising the
perceptions of three groups of stakeholders -
students, teaching staff and senior managers.

The quest for quality has had a pervasive effect
on business activity in the 1980s and 1990s.
Boards of directors and managers believe that
it gives them a competitive advantage,
consumers demand it and employees want it.
There is a widely held belief that' quality does
it all. It saves. It sells. It satisfies' (Brown,
Gummesson, Edvardson, & Gustavsson, 1991,
xiii).

Quality control measures have long existed for
tangible goods. Definitions of quality centre
on the ability of a product to fulfill its purpose
(Juran's 'fitness for use') and to meet
predefined specifications (Crosby's
'conformance to specifications') . However, in
services, quality is more difficult to define and
measure. Although most services incorporate
the use of tangible goods, the service itself is
basically intangible. 'Services are timebound
and experiential, even though they may have
lasting consequences' (Lovelock, 1992, 6).
Moreover, service is usually dependent on the
interaction of at least two individuals - the
customer and the service provider, and it is
often a lengthy process to which the consumer
may make a considerable contribution, an
'input' often impossible to predetermine or
define. Gummesson calls this key concept
'interactive production' in which quality

ANDREA MciLROY

DEPARTMENT OF MANAGEMENT SYSTEMS

MASSEY UNIVERSITY

P ALMERSTON NORTH

contributions come from the concerted efforts
of both the customer and the service provider
(Gummesson, in Brown et aI, 1991,4). This is
true of all services, even those such as
education and health which have traditionally
not defined the users of their services as
customers.

Publically funded institutions, including local
bodies, health and education, have not
escaped the demand for quality. In New
Zealand this has largely been in response to
deregulation, increased competition, reduced
funding, the drive for efficiency and a greater
requirement for accountability from both the
government and the public. The result has
been massive changes in the structure and
culture of many public institutions. In the
tertiary education sector deregulation has
allowed many new entrants into a market
which was previously exclusive to the
universities - the teaching and granting of
degrees. While the universities have not been
at the vanguard of embracing quality
management, many worldwide have now
established their own quality monitoring
bodies . In New Zealand this is the New
Zealand Universities Academic Audit Unit.

There has also been a substantial reduction in
the level of government subsidy that students
receive to support their tertiary study. So as
the range of providers and programmes
expands and as students are expected to
contribute a greater proportion of their own
fees, they increasingly view themselves as
customers with consumer rights and use

Journal of Distance Lea ming, Vol 3, No I , 1997 (c) Distan ce Educa ti on Assoc iati on of New Zealand

quality as an important determinant of choice.

Quality measures in higher education have
traditionally focused very much on outputs
and objective performance indicators. In
distance education quality assessment has
centred on outputs such as good presentation
of course materials, integrated use of media,
and performance indicators such as retention
and pass rates. Until recently little attention
was paid to what quality means to the
consumers of the service, be they students or
prospective employers. While tertiary
education providers may have an increasing
awareness of stakeholders and the 'customer'
in education, there i.:; still a reluctance to take
cognisance of their expressed needs and
perceptions when deciding and designing
quality standards. However, in a market
which is becoming ever more competitive, it
is crucial that providers understand what
quality means to the consumers of the
'product' (distance education being a mix of
both goods, such as study guides and
computer software, and services) that they
offer.

Other important stakeholders in the quality
of urtiversity distance education courses
include the academic staff who design and
teach the programmes, the 'experts', and the
senior management of the institution who
have the overall responsibility for planning
and resource allocation.

An important advantage of the stakeholder
model is tha t in taking account of a
multiplicity of needs and perceptions, a
complex construct such as quality can be
illuminated. Our understanding of it is
enhanced and this highlights ways that it can
be measured and programmes improved. The
recognition of the importance of stakeholders
and a clear focus on defining and meeting
customer needs are central to many models
of quality management used in business. A
number of these models, including Total
Quality Management (TQM), have now been
applied to tertiary and distance education (see
for example Coate,1990; Lewis and Smith,

1994; McIlroy and Walker, 1993, 1996). There
are also some studies of quality in higher
education which have included student and
staff perceptions (for example Harvey,
Burrows and Green, 1992).

THE STUDY The aims of the study were
to:
• identify the major dimensions of quality as

perceived by students, teaching staff and
senior managers

• examine any gaps in perceptions among
the three groups

• reach conclusions il.bout how to improve
quality in the extramural, undergraduate,
business studies programme

In order to do this and to increase the validity
and reliability of the study, triangulation of
methods and data was used. Focus groups
were conducted with groups of students and
staff, individual interviews were conducted
with senior managers and a questionnaire was
sent to a large sample of students and staff.
From the qualitative data gathered in the focus
groups and interviews, seven dimensions of
quality were distilled. These became the
phenomenological scales upon which the
questionnaire was designed and are:

Scale A:
Scale B:
Scale C:
Scale D:
Scale E:

Scale F:
Scale G:

Course structure and content
Face-to-face contact
Assessment
Communication
Institutional and environmental
influences
Standards and evaluation
Programme integrity

The notion of service quality having multiple
dimensions is well documented (see, for
example, Sasser, Olsen & Wyckoff, 1978;
Garvin, 1988; Gronroos, 1990). One of the
most widely applied multi-dimensional
models of service quality is that proposed by
Parasuraman, Zeithaml and Berry (1985, 1990).
However, previous research on quality in
tertiary and distance education (for example
Bell and Shieff, 1990; Athiyaman and
O'Donnell, 1994) found that Parasuraman et

j OZlma/ of Distance Learning, Vol 3, N o I , 1997 (cl Distance Educati on Associati on of New Zealand

-
13

aI's generic SERVQUAL instrument was not
specifically suited for assessing service quality
in higher education.

The questionnaire developed from the seven
phenomenological scales contained 114 items
all of which were quantitative. It was
administered in the between semester break
in 1996, after students had completed their
Semester 1 examinations. A random sample
of students from the extramural,
undergraduate, business studies roll (n=584)
and the total population of teaching staff in
the Business Studies Faculty (n=145) were
surveyed which resulted in a response rate of
54 percent for students and 47percent for staff.

RESULTS AND DISCUSSION Basic
statistical analysis of the data indicated that
there were a number of differences between
staff and student responses. It had been
postulated during the focus group stage that
there would also be differences between the
perceptions of 'low experience' students,
defined as those who had passed five or fewer
extramural papers, and 'high experience'
students, defined as those who had passed ten

Item

or more extramural papers. The next stage of
analysis was therefore to examine the level of
statistical difference between the responses of
the three groups on all questions. The Mann-
Whitney U Test was used for this purpose and
was applied to the following four groups:

• staff and the total sample of students
• .staff and low experience students
• staff and high experience students
• low and high experience students

The greatest number of significant differences
were between staff and the total number of
students. Surprisingly, there were very few
significant differences between high and low
experience students, and only two that were
highly significant (p<.Ollevel).

As would be expected, the pattern of results,
including significant differences, for staff and
low experience students and staff and high
experience students reflected the pattern for
staff and all students.

Because of the large number of results, the
discussion that follows will focus on two

12 Aims, objectives and learning outcomes for all courses should be clearly stated

22 Course materials should be easy to use

36 All course texts should be available for the start of the course

39 All course materials should be clearly and simply written and jargon kept to a
minimum

50 The information in course materials should be accurate and error free

53 A range of readings that complement the set text should be included with the
course

54 The quality of courses is compromised if the quantity of readings is excessive

72 Course materials should be presented in such a way that they are durable and
robust

87 Course materials should be up-to-date/ current

94 Lecturers should anticipate areas of learning where students commonly have
difficulty and try to find ways to overcome them

Table 1: High Agreement, No Significant Differences Staff and all Students,
Scale A: Course Structure and Content

Journal of Distance Learning, Vol 3, N o 1. 1997 (c) Dis[ance Educa[i on Assoc ia[i on of New Zealand

scales: course structure and content and face-to-
face contact. Items showing significant
differences are grouped into those where the
results were congruent i.e. both groups either
agreed or disagreed, and those where the
results were incongruent that is, one group
agreed and the other disagreed.

Scale A: Course Structure and Content
There were 22 items on this scale which related
to how courses were structured and organised
and to issues of content. Some items looked at
support/ administrative issues such as the
extramural library service; others asked about
issues of course structure such as format and
organisation of the material; and others asked
about issues relating to content such as
relevance, and currency of material.

There were 10 items where there was high
agreement between staff and students and
there were no significant differences, see Table
1. These items relate to the structure, accuracy,
currency and usability of course materials, the

Item

availability of texts and the quantity of
additional readings. They also represent what
could be described as aspects of 'good
practice' in distance education courses and the
results confirm that both students and staff
concur with them. It is important that the
institution ensures that they are built into the
design of its courses. This is particularly so in
a market that is becoming increasingly
competitive with new providers entering the
field from both New Zealand and overseas.
As recent research by Wood (1996) found,
customer expectations are also continuing to
rise and identifying b est practice and
transferring it throughout the organisation is
a useful and effective strategy for improving
service quality.

There were 12 items where there were
significant differences between students and
staff opinion, see Table 2.

Items Showing Congruence
Three items, 60,63 and 97, refer to the need

Level of Sig

8·
The extramural library service is excellent .029*

.000**

.023*

.000**

.025*

.000**

60
63
64

68
74
84
92

Should be standard format for all courses

Course content should reflect learning outcomes
Courses should develop critical thinking abilities
Course materials should consistently refer to the text
Relative importance of all course reading should be indicated
University courses should be knowledge based rather than
skills based

No justification for purchase of multiple texts for a course
Courses should be practical and relevant to the business world

Texts should be written about or highly relevant to NZ
Courses should be based on contemporary international
literature and knowledge

A weekly time table should be included in all courses to
guide students

.. significant at th e p <. Ol leve l

* significant a t the p <.OS leve l

.000**

.002**

.028*

.000**

Table 2: Significant Differences Staff and all Students, Scale A:
Course Structure and Content

JOllmal o!DistdnC( Leaming. Vol 3. No 1. 1997 (el Di5[ance Edu ca tion Association of New Zealand

for learning to be structured. This includes
clear guidance from the course controller
about what should be read and when. On all
three items the level of student agreement was
stronger than for staff. Item 60 was also the
only item on this scale where there was a
significant difference (p <. 05 level) b e tween
low and high experience students, low
experience students agreeing with the item
more strongly than their more experienced
contemporaries. This could indicate that they
prefer even more direction and structure in
their learning. There was also extremely high
agreement from both the staff and all student
groups that course content should reflect
learning outcomes, a further indication of the
need for structure in the learning process. For
this item staff agreed more strongly than
students. Many courses already incorporate
these design features and for those that do not,
it would be a relatively easy matter for these
features to be included.

However, the need for structure does not
encompass standardising course formats.
Neither staff nor students felt that there should
be a standard format for all courses (item 8)
with staff disagreement being stronger. The
strength of staff opinion may reflect the
idiosyncratic approach to course design that
is part of Massey University's culture. This it
seems is more desirable than a template
approach which, it could be argued, makes
course design much easier. Many of the open
uni versities , including the UK Open
University, use a standard format for their
courses. While it may make courses more
acce s sible for users, it also has a very
important 'branding' function . In a market
which is becoming more and more
competitive, such a consideration may become
quite an important issue.

Although both staff and students agreed that
there was no justification for the purchase of
multiple texts (item 68), student opinion was
much stronger, resulting in a highly significant
difference between the two groups (p <.Ol).
This outcome is not unexpected, given that
students have to 'foot the bill' for their texts

and they are generally concerned about the
rising costs of study. It may also be related to
student concerns about excessive workload.

Both staff and student groups agreed that the
extramural library service is excellent but the
level of agreement was stronger from students
(item 4). However, it should also be noted that
21 students (7percent) and 6 staff (9percent)
commented on their questionnaires that they
either had never used the extramural library
service or lacked the know ledge necessary to
respond to the question. But it is obviously a
service which is highly valued by those who
do use it.

While all staff and a high proportion of
students agreed that courses should develop
students' critical thinking abilities, staff felt
more strongly about this than students did .
This same difference of opinion was reflected
in the responses to item 74 where students
agreed more strongly than staff that courses
should be practical and relevant to the
business world .

Items 84 and 92 relate to the content of course
texts. Students felt more strongly than staff
that course texts should be written about or
highly relevant to New Zealand. This suggests
that students want texts that reflect and can
be applied to their working environments .
Staff responses may reflect the common
difficulty of finding suitable texts about New
Zealand and the desire to place New Zealand
within the wider international context with
which they often strongly identify, a notion
supported by their strong agreement with item
92.

Items Showing Incongruence
There was only one item on this scale that
showed significant differences and
incongruence between the two groups. Item
64 stated that university courses should be
knowledge based rather than skills based.
Slightly more than half of staff, 58percent,
agreed with this statement. However, quite a
high proportion of students, 65p e r cent,
disagreed with the statement. While students

J ournal of D ista nce Learning, Vol 3, No 1, 1997 (c) Distan ce Educati on Assoc iati on of New Zeal an d

-
16

are indicating fairly clearly that they prefer
courses that are based on skills development,

it is interesting that staff, traditionally fairly
'academic' in their orientation to learning, are
rather equivocal in the ir responses. This may
reflect the applied nature of business studies
and it could well b e that staff in a more
traditional faculty, such as humanities, would
respond differently. However, it does suggest
that staff and students may well have different
expectations about the purpose of courses
offered by the Faculty of Business Studies and
that such a gap may n eed to b e addressed
through appropriate marketing.

As a means of cross validating and
summarising the concepts in the
questionnaire, an item (118) was included at
the end which asked participants to rate 12
dimensions of quality in business studies
extramural courses according to their
importance . There were four that related to
this scale. For 'course design' and 'course
content' both student and staff respondents
agreed that these were extremely important.
However, on 'relevance to the work situation'
there was a highly significant differenc~
(p <.Ol) with students seeing this as more
important than staff. The same difference was
found with 'intellectual challenge' but this
time staff saw this as more important than
students. These results confirm the findings
on Scale A.

This scale, course structu re and con ten t, ha s
highlighted three things:

1. There was a high level of agreement

Item

between students and staff about a number
of course attributes that could be said to
represen t good practice. These include
physica l, s tructural and content aspects of
course materials.

2. It is clear that students value a high level
of structure and guidance in their learning
and appreciate techniques such as keying
texts to the study guide that will facilitate
the learning process.

3. Students felt more strongly than staff that
courses should be practical, relevant to the
business world and based on texts that are
about or highly relevant to New Zealand .
They also felt that courses should be skills
based rather than knowledge based
reflecting the fact that many of them are
doing work-related study. Staff on the other
hand, placed a high e r value on the
intellectual challenge of courses than
students did.

Scale B: Face-to-Face Contact
At Massey University, most courses taught in
the distance mode, offer some opportunity for
face-to-face contact between students and the
course tutor. This is either at the Palmerston
North campus (campus course) or at a regional
venue (regional course) when course numbers
warrant it. Depending on cours'e
requirements, attendance may be voluntary,
strongly recommended or compulsory. This
scale,face-to-face contact, had 10 items relating
to the value of campus and regional courses .
Although there were only three items where
the differences between the two groups were
significant, the results from the other 7 items
also produced some interesting information.

Level of Sig

23 Face-to-face contact with lecturer enhances
overall quality of course

.037*

43 Weekend regional courses are of more value than on campus courses

108 The costs of attending an on campus course far outweigh the benefits

•• significa nt a t th e p <.Ol level
• significant a t the p <.OS level

.000**

Table 3: Significant Differences Staff and all Students, Scale B, Face-to-Face Contact

JOllrllal of Distance Learning. Vol 3. No 1. 1997 (e) Distance Educa,ion Assoeia,ion of New Zealand

There were no items on this scale where there
were significant differences between high and
low experience students.

Items Showing Congruence
Both students and staff supported the
contention that face-to-face contact with the
lecturer enhances the overall quality of a
course (item 23) but staff agreement was
stronger. However, item 108 stated that the
costs of attending a campus course far
outweigh the benefits and produced
significantly different results for students and
staff at the p<.Ollevel. 86 percent of staff but
a much lower 58 percent of students disagreed
with the statement. While staff may base their
responses on pedagogical concerns, student
opinion may be mediated by other factors such
as the considerable costs of accessing face-to-
face contact as well as the perceived value to
them. This is supported by comments made
by 11 students on their questionnaires to
elucidate their response, for example "depends
on the lecturer"; "the personal cost of losing work
to attend these courses is prohibitive"; "includes
time off work"; "sometimes yes, sometimes no".

Items Showing Incongruence
Item 43, weekend regional courses are of more
value than campus courses, produced a result
that was both incongruent and highly
significant. 60 percent of students agreed with
this statement while 64 percent of staff
disagreed . The cost factor again probably
influenced student opinions . This is
supported by a number of comments added
by students to questionnaires for example "less
cost for the South Island";" only because of
accessibility"; "easier to attend" . It is interesting
to compare the results of this item, which looks
at value to students, with item 26 which stated
that students prefer to attend regional courses
than campus courses. Students both value
regional courses more and prefer them. On
the other hand, staff attribute considerably less
value to regional courses but agree that
students prefer them. This probably reflects
staffs' own preference for campus courses
which do not involve them in travelling away
from home and teaching in strange venues .

item 26 (above), did not yield significantly
different results but nevertheless they were
interesting. For example, in response to the
statement that mixing with students at campus
courses is more valuable than meeting
lecturers, 61 percent students disagreed but 51
percent staff agreed. This finding suggests that
the value that students place on networking
with fellow students is not as important as staff
think it is.

Both students and staff disagreed that campus
courses were of more value at 300 level than
100 and 200 level suggesting that both student
and staff groups also value campus courses at
the 100 and 200 level.

A higher percentage of staff (71 percent) than
students (59 percent) agreed with the
statement that often it is only at the campus
course that the course is put in perspective
which is another indication that staff value
campus courses more highly than students.

There was high agreement from students and
staff that a variety of teaching techniques add
value to the learning experience at campus and
regional courses and that being able to use the
library and access additional resources was an
important aspect of campus courses.

Finally, the results to the statement that there
is no necessity for compulsory campus courses
were equivocal for both students and staff. 55
percent students and 50 percent staff agreed.

In response to items on this scale, there were a
number of comments made by students on
their questionnaires indicating that they had
never attended either a regional or a campus
course. They therefore felt unqualified to
respond. There could be a significant number
of students who seldom or never attend a
campus or regional course during their time
as an extramural student. Furthermore, with
over 40 percent of students indicating that the
costs of attending a campus course do
outweigh the benefits this could well be an
issue that the university has to seriously
reconsider particularly in the light of ever

Journa l of Distance Learning. Vol 3. No 1. 199 7 (c) Distance Educa ti o n Associati on of New Zealand

-
18

increasing and accessible technology options.
It is interesting to note that anecdotal evidence
suggests that attendance at both regional and
campus courses is declining.

As would be expected from the results on this
scale, on the 'face-to-face' dimension in
summary item 118, there was a highly
significant difference (p<.Ol). Staff rated this
dimension as very much more important than
students did.

SUMMARY The results of the survey have
identified areas of congruence and
incongruence in student and staff perceptions
of quality on the' two scales: course content and
structure and face-to face contact. Many of the
items where agreement and congruence were
high represent aspects of good practice and
are already incorporated into many courses.
However, the perceived need for structure on
the part of students is an area where the faculty
may need to put more effort.

In order to improve student perceptions of
quality, areas of incongruence or the 'gaps' in
perception need to be addressed. There are a
number of areas where this could be done
without compromising the integrity or overall
standards of courses. The 'necessity' for face-
to-face contact may well be a case in point. This
year, Massey University'S College of
Education is offering New Zealand's first fully
external education degree . There is no face-
to-face component but there is extensive use
of email. Student response to this so far has
been extremely positive.

The other scales representing dimensions of
quality sampled in this survey have also
produced interesting results and highlight
areas where the faculty could direct effort in
order to continuously improve the quality of
its extramural programme.

REFERENCES

ATHIYAMANS, A. & O ' DONNELL, B. 1994:
Exploring graduates perceptions of quality in
higher education, journal of In stitutional Resea rch
in Australasia, 3, 1, 1-7.

BELL, D.R. & SHIEFF, D.s. 1990: Managing service
quality for improved competitive performance,
New Zealand Journal of Business, 12,1-12.

BROWN, SW., GUMMESSON, E., EDVARDSSON,
B. & GUSTAVSSON, B. (Eds) 1991: Service quality
multidisciplinary and multinational perspectives.
Lexington Books, Lexington.

COATE, L. E. 1990: TQM at Oregon State University.
journal for Quality and Participation, 90-10l.

GARVIN, D.A. 1988: Managing quality. The Free
Press, New York.

GRONROOS, C. 1990: Service managem en t and
marketing. Lexington Books, Lexington.

HARVEY, L., BURROWS, A., & GREEN, D. 1992:
Criteria of quality . The QHE Project. The
University of Central England in Birmingham,
Birmington.

LEWIS, R.G. & SMITH, D.H. 1994: Total quality in
higher education. St Lucie Press, Delray Beach.

LOVELOCK, C. H . 1992: Managing services
marketing, operations and human resources. (2nd
ed). Prentice Hall International, Englewood
Cliffs .

MCILROY, A. & WALKER, R.J. 1993: Total quality
management: some implications for distance
education. Distance Education, 14, I, 40-54.

MCILROY, A. & WALKER, R.J . 1996: Total quality
management: policy imp lications for distance
education. In Evans. T. & Nation, D. (Eds) .
Opening education policy and practices from open
and distance education. Routledge, London.

P ARASURAMAN, A., ZEITHAML, V.A., &
BERRY, L. L. 1985: A conceptual model of service
quality and its implications for future research.
journal of Marketing, 49, 41-50.

SASSER, W.E. JR., OLSEN, p.R., & WYCKOFF, D.D.
1978: Management of service operations: Text and
cases . Allyn and Bacon, Boston.

WOOD, D. 1996: Service with soul: Trends in service
quality strategies. MRL Research Group,
Wellington.

ZEITHAML, VA., PARASURAMAN, A. & BERRY,
L.L. 1990: Delivering quality service. The Free
Press, New York.

Andrea McIlroy is Senior Lecturer, Department
of Management Systems, Massey University,
Private Bag 11 222, Palmerston North.

Journal of Distance Learning, Vol 3, No I , 1997 (c) Distance Education Association of New Zealand