National Archives and Electronic Records: Where Are We Going? by Sue Gavrel^ Electronic Records: The Challenge to Archives Introduction The information society has had a major impact on the activities of traditional archives. The prediction of the "paperless office" in the eighties has not materialized as yet and in fact the amount of paper has increased sub- stantially over the past decade. Rather than reduce paper, the introduction of computer technology has increased the number of products and copies of those products. The work of the archivist in the identification of the archivally valuable records has increased due to the paper burden. One must sift through far more records to identify those of historical value. Added to the increase in the number of records created is the pressure of the research community to retain more rather than fewer records. Prior to the mid-seventies, the major factor in the appraisal of records was that of evidential value - the evidence the records contain of the organization and functions of agencies. Archives (and I restrict my comments mostly to North American and particularly Canadian Archives) have acquired many more records based on their informational and research value over the past fifteen years. In Canada, this coin- cides with the growth of social and economic programs of the federal government. The computerization of many government programs began in the early sixties and has increased ever since. The centralization of edp expertise was very evident during this time and continued until the arrival of the micro computer. The large database systems were, in most cases, built by the EDP experts and used to service the program managers' needs. Archival Programsfor Machine Readable Records The use and importance of computers was recognized by the large National Archival repositories in many coun- tries in the establishment of machine readable record programs. Over the years standards were developed for the appraisal, acquisition, processing, conservation and servicing of machine readable records. Due to the small number of archivists involved in these programs, a great deal of co-operation and sharing of information lead to the develqjment of procedures to handle these new records. Only a minimum amount of success was achieved in the identification and subsequent transfer of computer records of archival value from government agencies. The control of these records was in the hands of the edp area and outside the normal channels of the control of paper records (Records Managers). Efforts to mimic the systems in place for the control of paper records met with limited success, mostly due to the lack of familiarity with the archives by those in charge of the development of the systems. Machine readable programs, in traditional archival settings, although recognized as important, lacked the focus and strength required to affect the overall organization of records. Trends In recent years, this trend is changing due to a variety of reasons. In the discussion which follows, I will outline some of the changes and trends which will have a profound impact on archival repositories. These changes range from new understandings of the importance of information; technological change; and major changes in the ways data are created, stored and used. The next decade will require archives to focus on electronic records or risk losing the electronic cultural heritage. Technological Change It is not my intention to provide an overview or history of the changes we have experienced in technology in the past decades. It is, however, important to review some of these changes in light of how records are created, why, and how archives must adapt to these changes. The major trend which has affected the way records are created results from the rapid penetration of microcom- puters into the market in the last five years and, in particular, into government departments and agencies. The centralization of edp services is disappearing with the use of microcomputers in the office environment. Managers, officers and clerks have now as much comput- ing power on their desks as the mainframes of the seventies provided. The ability to create, manipulate, access and disseminate data has been decentralized. The trend to purchase "off the shelf' software has taken away from the centralized edp shops in the creation of in-house software and database systems. Linked to the penetration of microcomputers is the development of local area networks. The linking of staff provides for the creation 32 lASSIST Quarleriy and revision of documents which can be done on-line with only the final version being available in either hard copy or electronic format The ability to provide for the development of documents relating the evolution of the policy, to changes in administration, or to data is now in the hands of the creators of those documents. In most cases, LAN's do not provide for the traditional records management approach to control of records. Individual workers make decisions regarding the disposi- tion of the recOTds. Such a system existed for the development of large database systems under the control of edp professionals. The difficulties experienced in gaining control over what information is being created and destroyed can be magnified as all employees become responsible for their records. The program of economic restraint experienced in most western countries is also leading to the use of microcomputers, as it is seen as a way of increasing productivity and decreasing personnel costs. A major effort can be seen in the development and interest in communication standards. The lack of compatibility between hardware has been, and continues to be, a major problem to the increased usage of data. The efforts now seen in the development of International Standards is encouraging. The trend to the Open Sys- tems Interconnection standard protocols provide for the possibility of connecting systems with different hard- ware. Other standards will have an impact of the in- creased sharing of data. Map and Chart Data Interchange Format, or MACDIF, data is an attempt to provide a standard format for the transfer of chart and graph data from and to a variety of systems. Office Document Architecture/Office Document Interchange Format (ODA/ODIF) provides for similar transferability of text. The work towards developing and implementing such standards must be followed closely by archivists, as it is through such efforts that some of the technical issues such as making valuable data accessible in the future may be resolved. New techniques for software development such as fourth generation languages and expert systems techniques are becoming important tools providing faster and more flexible software development and more user interfaces. No longer must the design and development of databases be the sole responsibility of the edp professional. Data- bases can be created and used by those who have access to D-Base or other such software. Expert systems potentially pose major problems for the archivist In the past the acquisition of data tried to steer away from software dependent systems. Expert systems which can be defined as " an intelligent computer program that uses knowledge and inference procedures to solve problems that are difficult enough to require significant human expertise for their solution" are only in the early stages of practical application. Their potential to assist managers with complex planning and scheduling tasks, diagnose diseases, etc. is great The impact of such systems on the documentation of the decision making process is evident How archivists will respond to such systems is a major challenge. Types ofData Part of the technological changes, but one which should be highlighted, is the trend towards integrated systems and appUcations. Two specific types will be discussed: The Geographic Information System (GIS) GeograjAic Information Systems are beginning to play a major role in the information society. Today's systems only superficially resemble the automated mapping systems of the sixties. GIS are increasingly being used to conserve and manage a wide variety of data from natural resources to environmental pollution as well as in the planning and management of cities - such land informa- tion systems cross organizational and sectorial bounda- ries and represent an opportunity to develop new infor- mation based products and services. Compound Documents The move to more integrated systems is seen in the "compound document". The integration of voice, data documents and graphics oversteps the traditional media boundaries. All are reduced to the common language of binary code. Not only is it feasible to create the com- pound document, but it may also have been created from information which was only accessible on the screen for a brief period. The source of that information cannot be traced. The accessibility of data from other systems through local area networks and the merging of data from a variety of data bases will create documentation prob- lems for the archivist. The ease with which such infor- mation becomes available and usable will be reflected in the move to adopt communication standards, more user oriented software, and more computing power. Ir^ormation As A Resource The information society has created a new awareness of information as resource. The major expenditures on hardware and software development of the seventies has created a new awareness of the value of the information which these systems manipulate and store. In Canada, Access to Information and Privacy legislation led to the acceptance of a computer based record as a record. In the definition of a record for the purpose of the legisla- tion, machine readable is included. The requirement to account for information regardless of the machine on which it was stored was an important step in the recognition of computer records. The new National Archives Act passed in 1987 also uses the same definition of record. The Act stipulates that no records of Spring 1991 the Government of Canada can be destroyed without the consent of the National Archivist It further stipulates that those records deemed to have archival value must be transferred to the National Archives. The definition of record to include machine readable records ensures that electronic records are part of the National Archives' responsibility. More recently two new policies have added to the importance given to information: the Information Holdups Policy (just recenUy approved) which will direct government dejjartments to manage their information in a holistic manner and account for that information through the development of directories to it; and the Information Management Plan which ouUines to departments the importance of the planning of information management rather than the justification of new equipment purchases. All of these policies, acts, and planning strategies are the result of an awakening to the importance of information as a resource. Another interesting trend in the field of information is the move by the public and private sectors to develop co- operative databases. The Geographic Information Systems are an example of this type of co-operative effort. Similar to the compound document such co- operative databases will have an impact on the archival organization of records. Much more could be said on the trends and changes which technology will initiate. It is important to look briefiy at the impact such changes will have on the traditional functions of archives. The Challenge to Archives "The digitization of information through the common language of the binary code is bringing about the conver- gence of voice, usage and data - and of the telecommuni- cations, electronics and computing industries based upon them". Over the last decade, Archives have tended to expand acccH-ding to media-based responsibilities; textual records, cartographic, film and television, photographic and machine readable. Practices and procedures were developed to acquire, process, store and service the different forms of information as each had its special requirements. The fundamentals of archival theory were common to all media. Appraisal criteria, were based on the principles of evidential, infcxmational and legal value. It was in practices for arrangement and descrip- tion where the differences became more evident. Proce- dures and practices for the long term preservation of machine readable records were developed in the seven- ties. The procedures were based on large mainframe systems and proved successful ioc the conservation of data in systems. Technology is now the driving force behind the integra- tion of the different types of records. Just as media divided Archives into specific units, it now will play a large role in integrating these units. With the use of electronic technology in the creation of all types of records, the media on which the information resides becomes the common element. Information is created and transmitted in so many forms that archivists must now look at program activities as a whole and identify those records which have archival value as well as the most appropriate form in which they should be stored. Electronic records provide many research possibilities. As more types of records are created in electronic form - more such records are likely to be of archival value. The major obstacle is, of course, the long term accessibility of the information in elec- tronic form. For other records-paper, photos, maps estabhshed techniques have been developed which preserve the records for future use. Technology did not have a major effect on long term preservation except for improving the techniques. With electronic records, the media, the software and hardware are constantly chang- ing - Evidence of this can be seen in the experience gained to date. Electronic records created in the seven- ties are different from what is now being created. Proce- dures, valid for data in systems, must be modified and reevaluated to cope with compound documents and GIS. The technological requirements put pressure on estab- lished procedures. The focus of any archival program for electronic records must focus its resources on resolving the technical issues of how best to transfer records to the archives, how to process these records to ensure their accessibility. Archivists will be required to support efforts to ensure standards; to become involved with systems as they are being created; and to keep abreast and knowledgeable about the changing technology and how it affects the creation and use of records. These are major changes for institutions which have traditionally dealt with the past. Finally, new methods in records creation may have fundamental effects on traditional archival theory and principles. Archivists must become active participants in the creation of information, in many instances identifying elements of archival value before they are created, in order to ensure the preservation of the historical record. Archives have, to date, been concerned with document- ing the activities of an organization or business. How does this new role affect the documentation of activities when the archivist has participated in the creation stage? As more organizations undertake co-operative efforts in information creation, how do we determine which records originate with which organization? Who has the ultimate responsibility or control of the records? Such lASSIST Quarterly systems break down the barriers between public and private sectors; federal, provincial and municipal govern- ments. The clear lines of origin become blurred. Conclusion Today's presentation can only briefly mention the issues and resulting challenges to traditional archives. Issues such as these are of utmost importance to the archival community. The "paperless office" is not yet a reality but signs of its existence are much more evident today than they were two to five years ago. EffcMts to resolve the problems are imperative if records documenting the nineties are to be available for future generations. Presented at the IFDO/IASSIST 89 Conference held in Jerusalem, Israel, May 15-18, 1989 Forester, Tom. High-Tech Society. MIT Press, 1988. p.l Spring 1991