Quality Assurance in Tertiary Education: A Discussion of Current Developments and Practices INTRODUCTION The concept of Quality and Quality Assurance (QA) in tertiary education has taken on increasing significance over the last two decades. Practices of teaching and research in institutions of higher learning have been criticised both inside and outside the sector as lacking claimed and espoused excellence and quality. Governments demand greater accountability and transparency with regard to public expenditure, whilst industry spokespersons tend to criticise the quality of recent graduates. The growing QA movement, partly legislated, partly self-imposed, is the response of the sector to the disquiet in the community. This article cites some of the criticisms made of the tertiary education secto r, briefly describes the history of the quality movement in industry and provides a snapshot of the evolving tertiary education QA systems in Britain, the US, Australia and New Zealand. It concludes with a more detailed example of QA practice at institutional level in the fastest growing sector of tertiary education, that of Open and Distance Learning (ODL). It is suggested that the introduction of QA has been beneficial but that it presents an ongoing challenge to many individuals in regulatory government agencies and in the tertiary education sector itself. BA CKG R OU N D Until r e c e n t l y, t h e accepted view that universities would always provide quality services was rarely called into question. Three pillars of quality maintenance in universities, consisting of external peer BILL GROTE THE OPEN PoLYTECHNIC oF NEw ZEALAND review of research publications and theses, external accreditation or endorsement of professional programmes and stringent admission requirements for new students, were deemed to be sufficient. There were no explicit standards to judge teaching performance, and individual behaviours that are now questioned were accepted as quaint eccentricity. It was up to the (elite) student to succeed, sometimes against the odds. As the sector moved from an elitist to a mass education s y s t e m (Wagner, 1998), new universities (for example in the UK and Australia) or university 'equivalent' institutions (for example the New Zealand Polytechnics) were created, straining the existing order. Concurrently, respected reports were published which highlighted perceived problems in the sector. For example, the Boyer Commission (1998) in the US found that many students graduate from university and colleges unable to think logically, write clearly or speak coherently. A report released by the Australian Minister of Education (Eunson, 2000) on employer satisfaction with graduate skills, found that three quarters of Australian university and TAFE (Technical And Further Education colleges) graduates are not suited for the jobs for which they apply, with the greatest problem being lack of communication and generic skills. In addition (as discussed further below), critical review of the research publication output of universities, a key performance indicator that is subject to peer review, has contributed to a more detached and questioning view, not only of universities Journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 5 but also of the entire tertiary education sector. These issues taken together with questions a b o u t high student drop-out rates 'and demands for greater1 public accountability (McGettrick and Mansor, 1999, p. 131) have provided fertile ground for continued political concern and intervention. This scenario of growth, change and challenge in the tertiary education sector set the scene for a reappraisal of quality, and of how it might be achieved a n d maintained. It is now accepted that this new quality consciousness in higher education has tended to follow developments in the manufacturing and service industries. As little as two decades ago, manufacturing in Western countries concentrated on quantity. For example in the US, one third of product was scrapped as a normal operation during production (Schargel, 1994, p. 15). Customers n o t happy w i t h products learned from experience and could resort (in the West anyway) to finding another supplier. A typical example is provided by Morrison (1994, p. 45) who cited a report by the (British) Motor Industry Research Unit, which quoted average warranty figures of £100 per car for Austin Rover, compared with £17 for Nissan. The technologically advanced Western countries found themselves suddenly overtaken by the newcomer, Japan, which not only produced goods more cheaply but also to unsurpassed l e v e l s of quality. In the context of the automotive, optical, machine-tool, electronics and shipbuilding industries of the time, this meant that products performed in the way they were meant to, that they did so without breaking down, and that they lasted longer. The US manufacturing industries bearing the brunt of this quality onslaught realised that many of the theories that guaranteed the Japanese success had been supplied to the Japanese by some their own experts, notably W. Edwards Deming and later J. M. Juran, (Deming, 1986; Juran, 1992). What followed in the US and the rest of the Western world is the well-known period during the 1980s of learning, understanding and apply ing quality concepts t o manufacturing and subsequently t o service industries. Saylor (1996, p. 21) writes: "Total Quality Management has transformed some of America's competitor nations into economic powers. In addit ion, m a n y American institutions have already used TQM to pursue victory". The slogans of the time include the now well­ known acronyms of Quality Assurance (QA), Total Quality Management (TQM), Just In Time (JIT) production, Quality Circles, Kaizen, Total Quality (TQ) and Zero Defects. · Deming, Juran, and Crosby (see for example Kogan, (Ed.h1989; Cornesky, 1992; Sallis, Hingly, et al 1992; Sallis, 1993; Kaufman & Zolm, 1993; Sims & Sims, 1995; Lewis & Smith, 1994; Liston, 1999) are possibly the best known gurus of this quality reawakening. They and others in the field were giving essentially the same message, expressed succinctly by Crosby (1984), as the a b s o l u t e s of quality management: 1. Quality is defined by conformance to requirements. 2. There is no such thing as a quality problem. 3. It is always cheaper to do the right thing the first time. 4. The only performance measure is the cost of quality. 5. The only performance standard is zero defects. The requisite various schemes and processes involved include the e s tablishment of standards and self-correcting mechanisms (an example is The Deming Cy cle) to ensure maintenance and improvement of quality. Two common definitions of quality also used in educational contexts are 'fitness for a purpose' (after Juran) and 'meeting specifications' or 'c onformance to requirements' (after Crosby). The second of these is probabfj' the less subjective and the Journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand more accepted by industry. The principles of Quality Assurance (QA), used here as a global term rather than one to describe another specific theory, were subsequently adopted by service industries with great success. The victory of the quality move ment in technologically advanced Western countries was dramatic, effective and complete. In 1994, Morrison, with regard to British industry, wrote: "Now we are in the 1990s it is good to be able to record that the downward slide in quality appears to have halted and there are encouraging signs of industry beginning to respond to all the efforts that have been made to recover lost ground" (Morrison, 1994, p. 41). Quality is now expected, is built into processes and barely needs to be mentioned in the new, much more competitive and critical, global consumer culture of the late 90s. One concept from this period that has stayed in the public view is adoption of, and accreditation with, the ISO 9000 family of international standards. These standards assure potential clients of a manufacturing or service industry that the organization will do what it says it will do, and that there are mechanisms in place to detect and correct non­ conformance. The tertiary education sector was slow to take up the new quality message and this resulted in the call for grea ter accountability and transparency from funding and regulatory agencies. The following arguments were put forth (with some justification) in support of the status quo by tertiary institutions: • Quality is already assured through networks o f peer review and the performance of graduates in the public and private sectors. • Higher education requires much professional judgment that cannot be constrained by or contained within regulations. The best quality assurance is highly qualified, competent staff who put their research work out for peer review through regular publication in learned journals. • Even if quality control measures were imposed, they could not be enforced because of issues related to academic freedom, tenure of position and autonomy of the institution. • H o w can one c o m b i n e adherence t o standards, with excellence? Institutions of higher learning are in the pursuit of excellence in teaching and research. " F o c u s i n g on f i t n e s s - f o r - p u r p o s e or conforming to r e q u i r e m e n t s could diminish the mission of the institution to be the critic and conscience of society. • The ultimate aim of adopting QA (or TQM) in a service industry is to satisfy and/ or delight the client. Many students spend 3 years or more in tertiary study driven by necessity, to attain a desired position possihly against their strong preference if they had a choice. Compared to students who are intrinsically motivated and who enjoy learning, the former might at times feel antagonism and little satisfaction with their study. How can the QA approach deal with this potentially large, negative group of dissatisfied clients, although later in life these same students might come to value and appreciate the experience? It w a s (and i s ) sugg e s t e d that a key performance indicator for universities is the number of research papers published by staff. Individual output is also r e cognised for promotion purposes; thus, vast numbers of papers in increasing numbers of journals have been and are being published. Such a system, where professors with a pool of eager and t a l e n t e d postgraduate s tudents are particularly blessed, has caused growing d i s s a t isfaction w i t h the q u a l i t y of this indicator. Commentators in the US have been particularly blunt. Morrison (1996) writes: "The vast majority of what passes for research publication in the majority of universities in A m e r i c a is mediocre, e xpensive and unnecessary; it is dispiriting and it depresses the whole scholarly enterprise". Finn and Manno (1996) state: "among 833,000 faculty members [in US institutions] it's a fair guess that no more than 50,000 will ever produce journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association ofNew Zealand 7 new knowledge of any significance"; and there is " ... constant pressure from faculty for less teaching and more time for research; and tons of research that primarily serves the career needs of t h e p r o f e s s oriate r a t h e r than significantly enlarging human knowledge ... ". Similar sentiments were expressed by the New Scientist (May 2, 1998): "Forty-seven percent of articles are never referred to again in the scientific literature, even by the researchers who did the work. Another 33% were cited 10 times or fewer." Critical evaluations were also provided by Alexander (1999) who reviewed publications on innovative teaching research projects in Australia. Another damming investigation of research related to open and distance learning is contained in "A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education" (Phipps & Merisotis, 1999). Evidently there are unresolved issues with the use of this indicator in ranking universities. But the measure is easy and cheap to use, the selection having been done free by peer reviewers. Additionally it may be argued that papers that do not report new knowledge at least enable their writers to maintain the skills, the networks and the conversations amongst the research community. In addition, attempts at using citation counts, or the prestige of the publishing journal, have had limited success in ranking academics or in demonstrating the quality of the university, as shown by the continuing disquiet amongst some of the stakeholders (Johnstone, 1992; Daniel, 1997). West (1997, p. 1) wrote in the introduction of his report about higher education in Australia: "There is a feeling of unease in the universities. Many believe that traditional intellectual values and sound scholarship associated with higher education - and in particular, the pursuit of knowledge for its own sake - are under threat." And: "There is also a feeling of unease about universities in terms of their capacity to meet the needs of business and industry." Hence, it seems reasonable that there will be questions about effectiveness or cost-benefit, in the light of ever-increasing expenditure of public funds, at a time of critique from within and outside the sector (Livingstone, 1999). This contrasts with the overwhelming success in improving quality and service, and in cutting costs, by some private and public organisations. Excluding the United States, the first segments of the tertiary education sector to become subject to new methods of external scrutiny were the former polytechnics and colleges of higher education in the UK and Australia respectively. These institutions were given the opportunity to e n t e r the realm of the universities by offering degree courses. To enable this to happen, accreditation processes were institute� (and repeated in N ew Zealand with regard to the polytechnics) that gave the public some confidence and these institutions the necessary seal of approval to proceed. This early growing c o n f idence in the then polytechnics and institutes of technology in the UK and Australia expressed itself through, for example, industry preference in hiring graduates in many disciplines from these institutions (unpublished report Grote, 1980). Since these beginnings this 'new' form of Quality Assurance in the entire tertiary education sector (in the four countries considered here) has, after many false starts, taken on some momentum. It is evolving and is likely to encompass the entire sector including the old universities in the nations of the English-speaking world. The above discussion attempts to provide reasons that, if taken together with growing budgetary pressures in many advanced countries, could account for the interventionist and regulatory stance taken by many ministries and ministers of education. As discussed in the next section, this critical and regulatory stance expresses itself, particularly in UK, Australia and NZ, in visiting and revisiting the area of Quality Assurance in the tertiary education sector. journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 8 , QuALITY AssURANCE IN TERTIARY EDUCATION IN AusTRALI�, THE us, THE UK AND NEW ZEALAND AUSTRALIA For Australian universities the Federal G overnment establishes the accrediting agencies and receives and evaluates the reports relating to QA activities. Recent education policy development in Australia is informed by the West Report (West, 1997), a culmination of other white papers and reports. Together with the ''Higher Education Report (HER) for the 1999 to 2001 Triennium", it was prepared for the Minister of Education, and both are references for the following section. The Australian Commit t e e for Quality Assurance in Higher Education (CQAHE) was established for 1993, 1994 and 1995. The Committee considered general issues in 1993, teaching and learning in 1994 and research in 1995 (Liston, 1999, p. 44). The terms of reference for the committee expired at the end of 1995 and the general assessment was that "The quality review process had been successful in changing the culture within the whole university system" (HER). The Higher Education Report lists a number of changes that resulted from the CQAHE activities, such as the creation of senior posts within universities to take responsibility for teaching quality; the creation of teaching awards; the use of a wider range of assessment procedures; the incorporation of teaching criteria in promotion; and encouragement of innovation in teaching (see also Alexander, 1999 on this topic). However, it became apparent that the CQAHE' s work carried a high administrative cost and therefore the review programme was not made permanent. Another organisation, the Higher Education Council (HEC) will continue consolidation o f the "quality framework". W hilst the Australian universities will be responsible for quality assurance in their institutions, they are required to provide some minimum data to H EC on, for example, graduate attributes, feedback from employers about the quality of graduates, and graduate destination s.urveys. It was evident that individual universities' approaches to quality vary widely. They have variously decided, for example, that there will be reporting requirements, benchmarking projects, institutional and faculty -based quality assurance and improvement guide­ lines. The system will include participation of external assessors, the gathering of external feedback and periodic completion of surveys. At the �o urse a n d p r o g r a m m e l e v e l , Australian universities a r e self-accrediting through such structures as their Academic Board. Government approval is required for funding purposes and allocation of Effective Full Time Student Unit (EFTSU) places (see also Quality management in Universities, Piper, 1993). In summary, Quality Assurance in Australian universities involves: • t h e r o l e of p r o f e s s i o n a l bodies in accrediting professional courses • the sector-wide indicators published in "The Characteristics and Performance of Higher Education Institutions" • the use of external examiners for most research degrees and s o m e h o n o u r s degrees • the encouragement of innovation and good teaching through t h e Committee for University Teaching and Staff Develop­ ment and specific initiatives through the Higher Education Innovation Programme. It is of interest to note that, according to a number of articles in the Higher Education Section of The Australian, the Federal Minister of Education intends to re-establish a permanent body along the lines of the short­ lived CQAHE. Attempts to define the sector are ongoinlf For example it is now proposed journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 9 (Illing, 2000) to measure how Australian academics spend their time, by carrying out an audit of teaching costs in universities: The exercise is intended to provide data for allocating future university funding. The Technical and Further Education (TAPE) sector is under the control of the respective state minister of education and subject to centralised Quality Assurance measures with regard to curriculum and delivery methods. UNI T E D STATES The US has a long­ stand i n g sy stem of n o n -g o v e r nmental regulating organisations, whose task it is to establish norms for the methods and structure of teaching programmes, the qualifications of teaching f a c u l ty, and t h e facilities and equipment necessary to support instruction (World Guide to Higher Education, 1996). Four components make up the US Institutional Evaluation System (Kells, 1989). First, there is State controlled licensing or granting of a charter. Second, there are over 50 national accrediting organisations established within the framework of various professions and academic fields, which lay down standards and guidelines in their subjects thereby influencing and sometimes determining curriculum and assessment. Third, individual institutions may initiate and maintain audit, review and assessment activities comple­ menting or going above those of the other agencies. Fourth, there are six regional accrediting organisations, founded by the colleges and universities themselves, that on a voluntary basis certify institutions as meeting the standards in their region. These accrediting organisations then certify both college and university departments and specialised professional and occupational schools. As an example, a community college due for re-accreditation (which occurs every three to five y ears) would write a self-evaluation report against the standards determined by the board of the local accrediting organisation. The college would then be visited by a group of peers who might spend two to three days at the colleg e . Their r e p o r t and recommendations would be taken into account i n a decision to grant further accreditation or a request to remedy certain problems. It seems fair to say that the US system has not led to the sustained controversy (related to ranking) generated in Australia through the operation of CQAHE, or more recently, the work of QAA in the UK (excluding Scotland). However, there is criticism of the quality of graduates (see for example the Boyer Report, 1998). The US system seems to be simple and cost-effective and has had the support of stakeholders (Kells, 1989, p. 96). However, the US accreditation system has been under some criticism receatly through the accreditation of The University of Phoenix (Padilla, 1999) and the accreditation by the North Central Association of Colleges and Schools of Jones International University (Crow, 1999). Both are unique institutions of higher learning that do not fit the m o u l d of the " s tandard university ". It is debatable at this point whether this highlights the flexibility or the looseness of this accreditation system. UNITED KINGDOM Universities in the UK are self-accrediting, whilst polytechnics have been quality assured by the Council for National Academic Awards (CNAA). The United Kingdom Education Reform Act of 1988, the 1992 Education Bill and a number of reports and white papers set the scene for reform of the sector. This recent history of change saw t h e poly technics become universities and the CNAA disestablished in 1992. Since 1998 the enlarged university sector has, as a recommendation of the Dearing Report (Dearing, 1997) in an obvious parallel development with Australia, the Quality Assurance Agency (QAA) (Barnett, 1999; Newton, 1999), which is drawing up codes of practice. For example, the "QAA Subject Review Handbook, October 1998 to September 2000", has over 53 pages of detailed guidelines journal of Distance Learning, Vol5, No 1, 2000 ©Distance Educationfusociation of New Zealand - 10 and instructions for reviewers. "The Times Higher Education Supplement" (THES July 23 and August 20, 1999) describes some of the controversial aspects of QAA operation with regard to funding, consultation and lack of scrutiny of it by a third agent. As a further quality measure, UK universities are ranked on research performance, where the ranking i n f l u e n c e s future funding allocations. However, Utley (1999) reports that the Higher Education Funding Council for England acknowledges that "the standing of teaching has been damaged by the prestige and cash attached to research excellence". Hence, a shift in funding priorities is being proposed. Quality Assurance of the tertiary education sector has been taken up by the UK Open University (OU) as one of its key functions. The UK, like the other countries reviewed, has about 30 professional organisations that lay down stringent requirements or carry out their own examinations before registration or admission of candidates to the profession. In Britain and other countries inspired by British customs and processes, there exist Committees of Vice-Chancellors or of Vice­ Chancellors and Principals who may have some tacit QA function. In the case of New Zealand such QA functions are defined in law. Related to these functions is the ancient institution of "the visitor", a person who might visit a university as an independent arbitrator and provide recommendations to the Council and Vice-Chancellor on matters that could include QA concerns. In addition to peer networking and reviewing, UK universities invite an external examiner to sit in Committee when final marks and grades are determined. The current state of affairs in the UK is neatly summed up in the opinion section of the THES (July 23, 1999, 14): "The QAA, keen to implement the government's agenda by securing uniform arrangements, has once more run up against the autonomy of its masters, the institutions. Universities with visitors are apparently unwilling to give up a remote, slow and secretive arrangement devised by the medieval Church to check heresy, despite the evident drawbacks." The article goes on to suggest that a reasonable compromise, could be made resulting in the modernisation of the role of the visitor. NEW ZEALAND New Zealand established and defined the operation of QA through the Education Amendment Act of 1990. The Act created the New Zealand Qualifications Authority (NZQA), giving it a number of functions, which included the setting of standards, developing a framework for national qualifications, and establishing policies and criteria for QA. However, this legislation also ensured that the autonomy of universities with regard to QA remained intact: "The Authority (NZQA) shall, in the case of criteria or policies in respect o f universities, consult the Vice-Chancellors Committee (VCC)" (Education Amendment Act 1990, s. 253 (2) (a)). Nearly ten years down the track, one can discern some similarities in NZ developments with the current role of the QAA in the UK in 1999. The role of NZQA was under scrutiny in 1999, with an expectation that it would be r e o rganised and its functions c h a n g e d . However, the n e w NZ Government, elected in November, 1999, has since indicated that it does not wish to make changes at present and instead organized the formation of a committee to guide future tertiary education developments. The Polytechnic sector has, under delegated authority from NZQA, established the Poly technics Programme Committee (NZPPC), which accredits all programmes other than degrees. The NZQA still retains the task of accrediting non­ university degree and higher d e g r e e programmes through a peer committee and against standards that take industry support, academic quality, merit and coherence, financial resources and student support issues into account. After accreditation, ongoing monitoring and moderation regimes must be journal of Distance Learning, Vol 5, No 1, 2000 ©Distance Education Association of New Zealand - II instituted and maintained. The V CC has set up its own QA body, the Academic Audit Un�t (AAU), which, on its own initiative or by invitation, reviews and assesses academic quality at the whole university o r faculty I department level (Woodhouse, 1995). The AAU audited all seven NZ universities between 1995 and 1998. Reports of the AAU are available to the public. A summary report outlining examples of good practice (Williams and Woodhouse (Eds.), 1999) has subsequently been published. In addition, the Committee on University Academic Programmes (CUAP) is involved in accredi tation and approval of new programmes. CUAP, like the NZPPC, uses criteria published by NZQA as the framework in accreditation processes. QA IN AcnoN: AN OPEN AND DISTANCE LEARNING EXAMPLE The availability of Open and Distance Learning (ODL) courses and programmes is growing rapidly world wide. It is noteworthy that much of this growth occurs in traditional campus based face-to-face institutions, and the Internet seems to be the favourite delivery option chosen. The literature on Quality issues in ODL has grown significantly over recent years (see for example Tait, 1993a, 1997; Mcilroy & Walker, 1993, 1996; Mcilroy, 1997; Rowntree, 1998; Calder, 1994; Fage & Mills, 1999, Prebble, 1999). The 1995 conference of the International Council For Distance Education was dedicated to quality issues (see conference Proceedings edited by D. Sewart). One benchmark for successful application of QA in practice, in open and distance learning, is the UK Open University (OU). Thorpe (2nd ed, 1993) describes in detail the reasons for and practices of evaluation in the OU' s open and distance learning programmes, and provides a detailed working guide for practitioners. Tait (1993b) summarises OU quality assurance practices and activities as occurring through: • the course team, where collaborative non­ hierarchical teams work and rework draft materials • developmental testing of course materials before they become generally available • monitoring of correspondence teaching • m o n i t o r ing ,of s t u d e n t assignment turnaround times by tutors • inspection and support of tutorial and counselling staff's face-to-face activities. Course development at OU may take three years and courses may, with modifications, stay in production for eight years. This may raise other quality issues such as currency of materials and relevance. In addition, the OU participates in those QA­ related activities applying to all universities in the UK, such as use of external examiners or research ranking (although QAA processes have been modified to apply to an ODL provider). An example of QA practice in ODL outside of the university sector is provided by the Open Polytechnic of New Zealand (TOPNZ). The Open Polytechnic is involved in a wide range of integr ated QA activities, w h i c h are distributed throughout the organization (TOPNZ Quality Policies, internal documents 1999). As mentioned before, NZQA accredits then monitors all degree and postgraduate programmes. NZPPC fulfils this function for non-degree level programmes. There are also professional bodies that register or admit graduates of relevant programmes, and the entire Open Polytechnic organization has attained ISO 9001 certification. In addition the Open Polytechnic degree programmes are about to be approved by the Open University for Open Polytechnic candidates who ha,ve completed their degrees to receive also the OU d e g r e e credential, and for the Open Polytechnic to be able to deliver OU courses. The Open Polytechnic collects and publishes the results of student and stakeholder surveys in its annual reports. In addition, individual staff members engage in ODL- related research journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 12 projects that are p ublished in conference proceedings and in journals. Students and representatives f r o m industry \and the professions are c.bnsulted for the purposes of new programme development, accreditations and review processes. Staff upgrading of qualifications, professional development, and consulting and research outputs are encouraged and supported. Internal academic quality assurance rests with the Academic Board, which debates and determines academic issues and regulations and recommends approval of courses, programmes and student results to t h e Council o f the institution. Like the UK OU, the course design process is an example of applied quality assurance. At the Open Polytechnic, the wider course design team, comprising project manager, discipline specialist (content writer), instructional designer, technical editor, moderator, and editor, includes (for degree level courses) normally at least two external individuals. The discipline specialist or technical editor and the moderator are u s u a l l y drawn from the university sector o r are practising professionals. External moderators are also consulted in the writing of exams and at times during reviews or consultations on academic issues. This external input contributes to the currency and overall quality of the material. EX TENDING THE ODL EXAMPLE: QA IssuE s IN I N T E R NE T PROVISION ODL programmes entirely delivered via the internet are in the main still in prototype and testing stages (Farrell, 1999), although a number of post-graduate programmes, for example the MBA, show growing maturity. The literature in this new field is dominated by advice books such as Internet Based Learning: A n Introduction and Framework for Higher Education and Business, (French et al., 1999) or The Digital University, Reinventing the Academy (Hazemi et al (Eds.), 1998), providing typical examples. The debate about QA issues in Internet ODL occurs at present mainly in journals such as The Chronicle of Higher Education (for example, Carnevale, 2000 "Assessing the Quality of O n l i n e Courses Remains a Challenge, Educators Agree"; or Hara & Kling, 2000, "Students' Distress with a Web-based Distance Education Course"). A recent monograph p u b l i s h e d ,un d e r the auspices of New Zealand's Universities Academic Audit Unit (AUU) by Butterfield et al (1999) titled: "External Quality Assurance For The Virtual Institution" gives a more in-depth treatment of the topic. The shift to provision via internet of tutorial and student support and the production of interactive multimedia courses, delivered via the net or on carriers such as CD-ROM, will pose a new set of quality challenges. In many instance� existing methods will be adequate but in some cases new protocols will need to be developed. For example, some publishers who were happy to give copyright clearance for print-based material are reluctant to do the same for material that is to be included in net­ based c o u r s e s . Universally acceptable protocols to regulate such activities are required. Furthermore, the validity and reliability of assessment over the net, if implemented, will require particular attention in order to retain the confidence of those who participate in the assessment process and/ or depend on these results. In summary, some of the key issues are: • lack of access by many students, raising questions of equity and fairness • institutions providing dual mode, that is, contact and distance (net) provision, where one, usually net, provision is under­ resourced • in their 'me too' rush to be on the internet, institutions scanning vast amounts of lecture notes and readings into the web­ site, with no benefit to the students who have to print off material that may not have been designed for instruction • q u e s t ions related to ownership of intellectual property and to security of information • the misjudgement by many that (well­ designea) telelearning material may be journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 13 cheaper to produce and maintain " a lack of instructional designers who know how to use these new technologies to enhance deep learning " the unavailability of trained individuals, standards and processes to effectively judge the quality of net-based courseware and student support. The entire enterprise is in transition. It is a seemingly mad rush driven by competition. Involved a r e s o f t w a r e and hardware producers, various vendors, t e r t i a r y institutions, enthusiastic o r Luddite lecturers, n e r v o u s m i n i s t r i e s of education and increasingly c o m p u t e r - and net-literate students. It is clear that an institution or group of institutions providing internet-based learning opportunities to an international audience will not be able to let QA standards slip f o r v e r y l o n g . N o t only e t hical considerations but also pure self-interest when wishing to attract fee-paying international students will enforce the setting up and the maintenance of high standards guaranteeing quality. CONCLUSION The examples of national QA approaches provided indicate a desire to achieve g r e a t e r t r a n s parency and accountability through operation of external audit and accreditation agencies. Concurrent with these activities are the ongoing actions by governments to rein in the burgeoning costs of mass education and to shift more of the costs away from the public purse on to students, as has happened o v e r the last decade i n Australia, NZ, and the UK (in that order). It seems that the UK, NZ and Australia have moved in the direction of the 'voluntary' US accreditation s y st e m , despite the many reviews, changes in direction and examples of political opportunism, in the former countries. However, it seems that QA, often part of a TQM approach, has had some successes. For example as the previously cited reports indicate, there is now much greater emphasis on teaching quality by having teaching­ learning included in QA assessments and ranking, and more serious efforts are made to reduce the drop-out of students (Brindley, 1989; 1995). Traditional ODL with team based production systems can provide a good example of QA in action. But paradoxically at present, ODL' s latest offshoot, internet ODL provision, suffers from a lack of agreed standards of QA,, where the simple 'scanning in' of large text based learning resources is but one common example of bad practice. The application of QA principles in tertiary education provision is, on balance, a positive development, especially as it applies to a publish-or-perish syndrome to the detriment of good teaching philosophy, and to the newer often private-for-profit institutions. A well constructed and balanced external review of the tertiary education sector may propose many positiv€ developments. However, the many white papers, reports and legislative changes in the UK, Australia and New Zealand could have the effect of stifling reforms, as a result of constantly changing requirements, confusion, resistance and uncertainty. This ongoing fine-tuning may in fact have the effect of reducing quality. This happens because the service of tertiary education is provided to students over many years, 3 to 4 years for undergraduates, much longer for those studying part-time, requiring long- t e r m planning and s ignificant investment Thus, providers are able to react to changing market forces only relatively slowly. Whilst ongoing change is necessary, too often changes are imposed on changes in progress, preventing a required minimum degree of stability to be maintained. However, the real danger is that governments' hidden agendas are primarily cost containment and redistribution of resources in the light of expanding mass education, whilst paying lip service to academic freedom and the independence of universities. On review of the literature it seems that in the debate about quality, students, the principal clients, who pay increasing amounts of money for these learning opportunities, have been relegated to play a minor and passive role, essentially that of filling in questionnaires. journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association ofNew Zealand - 14 I I This treatment, as well as the often, token membership on academic boards or councils (students suffering financial hardship and dependence tend not have a strong input into such committees) must become a focus of future deliberations on QA issues. In this international move towards national institutionalised QA systems, operating amongst competing institutions of higher learning, there exists now an opportunity to establish and publish internationally comprehensive best practice and benchmarking data (see Vroeijenstijn, 1999; Liston, 1999, pp. 4-6). This information will benefit national and international students and communities without needing to diminish or intrude upon academic freedom and institutional autonomy. But more importantly, it is suggested that the time may be right to agree on a number of standards. The US sy stem provides a good example. An interna t i onally recognised academic validating and/ or accrediting agency, above and beyond ISO 9000 or the Global Alliance for Transnational Education (GATE) (Liston, 1999, p. 5), could then be established. Such an agency could bring some order and uniformity in QA approaches, and most urgently, ensure maintenance of highest possible standards, including internet provision, across the world's tertiary educational institutions. However, whatever system is adopted, if it is to be used widely, must be free from ongoing government interference, be affordable, inspire confidence and have the requisite simplicity and stability to allow long term planning. REFERENCES A L EXAN DER, S. ( 1 9 9 9 ) . An evaluat i o n of innovative projects involving communication and information technology in higher education. HERDSA, 18, 2. BARNETT, R. (1999). The coming of the global village: A tale of two inquiries. Oxford Review of Education, 25, 3. Boyer Commission on Educating Undergraduates, Reinventing Undergraduate Education: A Blueprint for America's Research Universities. Strum Kenny, S. Chair (1998) http:/ /notes.cc.sunysb.edu/ Pres/boyer. n s f/w e b f o r m/ i m a g e s/$F i le/ boyer.txt BRINDLEY, J. E. (1989). Retention strategies: A pre­ admission p r o g r a m m e f o r a d u l t d i s t a n c e learners. I n Eastcott, D., Farmer, B . and Lantz, B. (Eds.) Aspects of Educational and Training Technology,.Yol23, London: Kogan Page. BRINDLEY , J. E. (1995). Learners and learner services: The key to the future in open and distance learning. In Roberts, J. M. and Keough, E. M. (Eds.) Why the information highway: Lessons from open and distance learning, Toronto: Trifolium Books. BUTTERFIELD, S., CHAMBERS, M.,MOSELEY, B., PREBBLE, T., UYS, P. and WOODHOUSE, D. (1999). External quality assurance for the virtual institution. Wel l i n g t o n : New Z e a l a n d Universities Academic Audit Unit, AAU Series on Quality No 4. CALDER,J. A. (1994). Programme evaluation and quality: A comprehensive guide to setting up an evaluation system. London: Kogan Page. CARNEVALE, D. (2000, February 18). Assessing the quality of online courses remains a challenge, e d u c a t o r s agree. T he Chronicle of H i g h e r Education, A59. CORNESKY, R. A. (1992). Using Deming to improve quality in colleges and universities. Madison, WI: Magna Publications. Inc. CROSBY, P. B. (1984). Quality without tears. New York: McGraw-Hill. CROW, S. (1999). Virtual universities can meet high standards. T he Chronicle of Higher Education, h t t p : / I c h r o n i c l e . c o m/w e e k l y /v 4 6/i 1 0 / 10b00501.htm DANIEL, J . S . (1997). W h y universities need technology strategies. Change, 4, 29. DEARING, Sir R. (1997, July). Summary report: Higher education in the learning society. National Committee of Inquiry into Higher Education. 'The Dearing Report'. NCIHE, London: Crown Copyright. Also at http:/ /www.leeds.ac.uk/ educol/ncihe DEMING, W. E. (1982). Quality, productivity and competitive position. Cambridge, MA: MIT Press. DEMING, W. E. (1986). Out of the crisis. Cambridge, MA: MIT Press. Education Amendment Act (1990). Sections: 253 (2) (a), 257 (5) and 260. Wellington: Published under the authority of the New Zealand Government. EUNSON, B. (2000, February 16). 'Soft' de-skilling costs grad jobs. The Higher Education Section of The AusJ-yalian. . FAGE, J. B. and MILLS, A. R. (1999). Qualzty journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 15 Assurance: Internally embedded or externally exposed. Pan Commonwealth Forum on Open Learning, Brunei Darussalam. Vancouvet;: The Commonwealth of Ifarning. FARRELL, G. M. (Ed.)/ (1999). The development of virtual education: A global perspective. Vancouver: The Commonwealth of Learning, and http:/ I www.col.org FINN, C.E., and MANNO, B.V. (1996, April). American higher education: Behind the emerald city's curtain. Hudson Briefing Paper No. 188. FRENCH, D., HALE, C., JOHNSON, C. and FARR, G. (1999). Internet based learning: An introduction and framework for higher education and business. Sterling, VA: Stylus Publishing. HARA, N. and KLING, R. (2000, January). Students' distress with a web-based distance education course. The Chronicle Of Higher Education. HAZEMI, R., HAILES, S. and WILBUR, S. (Eds.) (1998). T he digital university, Reinventing the academy. New York: Springer. Higher Education Report for the 1999 to 2001 Triennium, prepared by t h e A u s t r alian Department of Education, Training and Youth Affairs (DETYA). www.deet.gov.au/highered/ otherput/he-report/report.htm ILLING, D. (2000, February 23). Audit to measure academic efforts. Higher Education Section of The Australian. JONSTONE, D.B. (1992). Learning productivity: A new imperative for American higher education. In Studies in public higher education, New York: State University of New York. Also, www. e d u c o m . e du/ w e b I p u b s I p u b H o m e Frame.html JURAN, J. M. (1992). Juran on quality by design: The new steps for planning quality into goods and services. New York: The Free Press. KAUFMAN, R. and ZOLM, D. (1993). Quality management plus: The continuous improvement of education. Newbury Park, CA: Corwin Press Inc. KELLS, H. R. (1989). The second irony: The system of institutional evaluation of higher education in t h e U n i t e d S t a t e s . In K o g a n . M. ( E d . ) Evaluating higher education. London: Jessica Kingsley Publishers. K O G A N , M. (Ed.) ( 1 9 8 9 ) . Evaluating higher education. London: Jessica Kingsley Pub. LEWIS, R.G., and SMITH, D.H. (1994). Total quality in higher education. Delray Beach, Florida: St Lucie Press. LISTON, C. (1999). Managing quality and standards. Buckingham: Open University Press. LIVINGSTONE, G. (1999). The quest for the holy grail or hunting the snark? Bridging the quality- resources divide on effectiveness and efficiency. Assessment & Evaluation in Higher Education, 24, 2. M C G E T T R I C K , A. and MANS OR, N. ( 1999). Standards and levels - a case study. Assessment & Evaluation in Higher Education, 24, 2, p. 131. MCILROY, A. and WALKER, R. (1993). Total quality m a n a g e m e n t : •S o m e i m p l i c a t i o n s f o r t h e management of distance education. Distance Education, 14, 1. MCILROY, A. and WALKER, R. (1996). Total quality management, Policy implications for distance education. In Evans, T. and Nation, D. (Eds.) Opening education. London: Routledge. MCILROY,A. (1997). Aquality picture: Perspectives on a distance learning programme. Journal Of Distance Learning, 3, 1. MORRISON, S. J. (1994). Managing quality: An historical review. In Gale, B. D. (Ed.) Managing quality (2nd ed.). New York: Prentice Hall. M O R R I S ON ,\J . L . ( 1 9 9 6 , M a y/J u n e ) . A new Carnegie Commission on higher education may well be overdue. On the Horizon. New Scientist (1998,May 2). Newswire section, 158, 2132, p. 21. NEWTON, J. (1999). An evaluation of the impact of external quality monitoring on a higher education college (1993-98). Assessment & Evaluation, 24, 2. OAKLAND, J.S. (1989). Total quality management. Oxford: Heineman Publishing. PADILLA, A. (1999 July I August). The University of Phoenix, Inc. On the Horizon, 7, 4. PHIPPS,R. and MERISOTIS,J. (1999,April). What's the difference? A review of contemporary research on the effectiveness of distance learning in higher education. The Institute of Higher Education Policy and US National Education Association (NEA), www.IHEP.com P I P E R , D. W. (1993). Quality management in u n i v e r s i t i e s , v o l .l. C a n b e r r a : A u s t ralian Government Publishing Service. PREBBLE, T. K. (1999). Quality assurance in dual m o d e i n s t i t u t i o n s . In W i l l i a m s , P. and Woodhouse, D. (Eds.) Achieving quality: Examples of good practice in New Zealand universities. Wellington: Academic Audit Unit, AAU Series on Quality, No. 3. Quality Assurance Agency for Higher Education QAA (1997, December). Subject review handbook, October 1998 to September 2000. Reference No: QAA 1/97. ROWNTREE, D. (1998, June). Assessing the quality of materials-ba�ed teaching and learning. Open Learning. journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 16 SALLIS, E. and BINGLEY, P. et al (1992). Total quality management. Coombe Lodge Report. T he Staff College, 23, 1. SALLIS, E. (1993). Total quality management in education. Philadelphia: Kogan Page. SAY LOR, J. H. (1996). Total quality management simplified: A practical guide (2nd ed.). New York: McGraw Hill. SCHARGEL, F. P. (1994). Transforming education through total quality management: A practitioner's Guide. Princeton Junction, NJ: Eye On Education Inc. SEW ART, D. (Ed.) (1995) Conference proceedings: One world many voices, Quality in open and distance learning. 17th World Conference for Distance Education, Birmingham 25-30 June. SIMS, S. J., and SIMS, R. R. (Eds.). (1995). Total quality management in higher education. London: Praeger. TAIT, A. (1993a). Systems, values and dissent: Quality Assurance for o p e n and distance learning. Distance Education, 14, 2. TAIT, A. (1993b). Systems, value and dissent: Quality a s s u r a n c e for open and distance learning. In Tait A. (Ed.), Quality assurance in open and distance learning: European and international perspectives, Conference Papers, 28- 30 September. Cambridge: Downing College. TAIT, A. (1997). Quality assurance in higher education: selected case studies. In Tait, A. (Ed.) Perspectives on Distance Education. Vancouver: Commonwealth of Learning. The Times Higher Education Supplement, THES (1999, July 23 pl4, & August 20). THORPE, M. (1993). Evaluating open and distance learning (2nd ed.). Harlow, UK: Longman. UNESCO (1996). World guide to higher education, (3rd ed.). Paris: author. UTLEY, A. (1999, September 3). Jury out on bid to fix university teaching. Times Higher Education Supplement, THES. V RO E IJ E NS T IJ N , A. I . ( 1 9 9 9 , June). The international dimension in quality assessment and quality assurance. Assessment and evaluation in higher education. Bath: Carfax Publishing Company. WAGNER, A. (1998, November). Redefining tertiary education. The OECD Observer, 214. WEST, R. (1997). Learning for Life (The West Report). Canberra: Australian Government Publishing Service and www.deetya.gov.au WILLIAMS, P. and WOODHOUSE, D. (Eds.) (1999). Achieving quality: Examples of good practice in New Zealand universities. Wellington: New Zealand Universities Academic Audit Unit, AAU Series on Quality No.3. WOODHOUSE, D. (1995). Audit manual: Handbook for institutions and members of audit panels (2nd ed.). Wellington: New Zealand Universities Academic Audit Unit. Dr Bill Grote is Senior Research Fellow, The Open Polytechnic of New Zealand, Private Bag 31914, Lower H u t t , New Z e a l a n d . E m a i l : GroBil@topnz.ac.nz journal of Distance Learning, Vol5, No 1, 2000 ©Distance Education Association of New Zealand - 17