vol233


4 IASSIST Quarterly Winter 1999

Automated Preservation of Electronic
Records: A Case Study of the Archival

Preservation System

by Fynnette Eaton*

The National Archives and Records
Administration has had a program for
accessioning, describing, preserving, and
providing reference service to the
electronic records (Machine Readable)
records) created by Federal agencies and
transferred to the National Archives for
almost thirty years. Although there have
been many changes in the name of the
office, its basic mission has remained the same: to preserve
and make available those records created by Federal
agencies in electronic format that the National Archives has
determined to have value beyond the short-term need of the
originating agency.

Most people think of the National Archives as the keeper of
the Constitution and the Declaration of Independence.
Even the most experienced researchers are largely unaware
of the growing number of files in electronic format.  Since
the creation of the Center for Electronic Records in October
1988, the number of files transferred has literally
skyrocketed. In 1988 the Archives received 150 files from
Federal agencies.  In fiscal year 1991, the number was
1500, ten times as many in three years.  The numbers
jumped again in 1992 to 8730 files. Unfortunately, the
Center became involved in a resource-draining court case,
which forced Tom Brown and his staff to reduce their
efforts in accessioning new files, with a resultant decrease
in accessions in FY94 and FY95 to 843 files and 1590 files
respectively.   Nevertheless this is a vast increase compared
to earlier years.

Currently, the Center has accessioned about 23,000 files
produced by over 100 bureaus, departments, and other
components of executive branch agencies and their
contractors.  These files range from the American Soldier
surveys of World War II to records of the 1980 and 1990
Decennial Censuses.  These files include education data
illustrating the variety of education programs of the Federal
government; health and social science data incorporating
both biomedical and sociological information and efforts to
measure the effectiveness of a variety of social programs;
international data including import-export statistics and
USIA-sponsored surveys.  The represented military data
ranges from Prisoner of War records for World War II and
the Korean conflict, and casualty records for the Korean
and Vietnam conflicts, to a large collection of data files

resulting from the use of computers for
military operations, management, and
research dating from the 1960’s
especially during combat in Southeast
Asia.

Clearly, as the size of our holdings grew,
with the tremendous increase in transfers
of files, the Center recognized the need to

develop new methods for accessioning and preserving these
new files.  My paper will discuss the development and
implementation of the Archival Preservation System
(which I will refer to as APS).  Another system, the
Archival Electronic Records Inspection and Control system
or AERIC was also developed in the early 1990’s to
provide automated validation of electronic files and to build
a base of descriptive data drawn from the data elements of
these files.  But that is another paper, by a different author.

As I stated at the beginning of this paper, the Center has
been involved with the various archival activities
associated with electronic records for more than twenty
years.  But it was only in the late 1980’s that the number of
files being transferred overwhelmed the staff, requiring
reexamination of the methods used to process these files.
Permit me to give you a brief overview of how the staff
used the resources that were available at the time to
perform the basic preservation work at the National
Archives.

During the 1970’s all computer processing required by the
Machine Readable Archives Division was performed at
service bureaus.  The division had an IBM 029 keypunch,
which the staff used to punch cards for the programs to
copy or dump tapes.  The program card deck was wrapped
in a rubber band along with a sheet of instructions to the
service bureau indicating which tape volumes were to be
used for input and output and any other special instructions.
The card decks, instructions, input tapes, and blank output
tapes were boxed and sent by courier to the service bureau.

The service bureau staffs usually ran the jobs at night with
a twenty-four to forty-eight hour turnaround.  The division
preservation and reference staff checked the jobs, labeled
the tapes, assigned the output tapes location numbers and
took the tapes to the storage area in the Washington
National Records Center in Suitland, Maryland.



IASSIST Quarterly Winter 1999 5

During this period the original agency tapes was kept as the
master tape and the NARS-created tape became the
reference copy to be used as input tapes to make copies for
researchers.

In 1975/1976 the transfer of DOD files written in NIPS
(National Military Command System Information
Processing System) software required special handling.
GSA made available to NARS a copy of the NIPS software
at the DC Share computer facility.  Generally the staff
transported the tapes and card decks to DC Share.  Because
of the faster turnaround time, staff began to use DC Share
to run other jobs as well.

In 1981 the Machine Readable Archives Division acquired
on loan a dumb terminal so that staff could run some jobs
for the FBI Appraisal Task Force (another court case that
severely strained NARS resources) at the National
Institutes of Health (NIH).  The Division took advantage of
the access to this computer center by performing some of
its normal copy and dump jobs at NIH.

Because of organizational changes within the National
Archives and Records Service, the Machine Readable
Archives Division became the Machine Readable Branch
of the Special Archives Division in 1982.  Plans to procure
a minicomputer for its own use did not materialize and
the Branch lost the terminal with access to the NIH
computer center.  By October 1982 all requests for
preservation and reference work for computer files was
submitted to another office for processing.  Although
reference work continued, only a few preservation jobs
were successfully completed after this transfer.

In 1984 the Branch acquired a DECWriter terminal that the
Motion Pictures Branch was surplusing to use for its work
on the Catalog of holdings. Beginning in January 1985, the
Branch received authorization to use the terminal to access
the National Institutes of Health computer center, as it
assumed responsibility for preservation copying of its
accessioned files.  Within a few months the Branch
obtained additional terminals for submitting jobs to the
NIH computer center.  The computer center at NIH served
as the computer resources for all of the Branch’s
requirements.

This acknowledgment that the Machine Readable Branch
should perform the work is borne out by the few statistics I
could locate in the files. During the period Fiscal Year 82
through 84 less than 100 tapes were copied.  The Machine
Readable Branch staff copied over 200 tapes in six months
of 1985.  Another report indicates that in FY 87 104 reels
had been copied by May. Yet these numbers were far too
low, once the Center began its accelerated accessioning
program.

The preservation work, which required making two copies

of each file offered by a Federal agency, was performed
using the mainframe computers at the National Institutes of
Health Computer Center in Bethesda, Maryland.  This
mainframe computer center used IBM machines, so the
types of outputs that we could produce were limited to the
options that were available to us at that Center.  In addition,
the file formats that we could accept for transfer were
limited to those formats that we could process at this site.
Our requirements, which were published in the Code of
Federal Regulations, stated that agencies were to transfer
permanent computer files in a hardware and software
independent format. Specifically the files were to be
written on half-inch magnetic tape in EBCDIC or ASCII,
without internal control characters on 7 or 9 track
open-reel magnetic tape recorded at 800, 1600 or 6250
bytes per inch or on 3480 cartridges and blocked not higher
than 32,000 bytes.

These requirements clearly reflect the use of a mainframe
computer in creation of our preservation copies of these
files.  Even though we had control over the preservation
copying of our files, the Center was at a disadvantage.  We
had to relinquish physical control of both the agency tapes
and the blank tapes or cartridges that we would use to make
the preservation copies on when they were sent to the NIH
computer center to be mounted on tape drives there, but
that was the only real choice we had. We were able to
streamline some of our copying procedures, but we
often had to wait in line for access to tape drives because
we were dependent on modems and phone lines to connect
to the NIH Computer Center which had thousands of users.
We determined that although the staff time required to copy
and compare files created at NIH was between 2 1/4 and 4
hours, the time that elapsed from when we prepared the
tapes to be transported to the computer center and their
return after successful copying, was literally one week.

One of the first priorities enunciated by the Director, Ken
Thibodeau, when he joined the Center in the winter of
1988/89, was to reengineer this process by developing an
in-house capability for making preservation copies of
electronic files sent to the Center by Federal agencies.
By examining the processes associated with producing
preservation copies of electronic files, the staff developed a
statement of work for prospective contractors that defined
the basic requirements and outlined additional features we
would like to see developed over the life of the
contract. Although we defined the processes based upon
our current practice, we also recognized the opportunity to
streamline some of the work that had almost developed a
life of its own.  The development of the statement of
work took about a year.  The National Archives issued a
Request for Procurement in March 1992 and selected as the
successful bidder, Muller Media Conversions of New York
City in May 1992.  Although in the statement of work we
defined what the processes should be, we did not define
how they should be accomplished.



6 IASSIST Quarterly Winter 1999

There were four objectives with this contract.  First, we
wanted to retain control of the media and perform the work
in-house, streamlining and saving valuable time and
therefore increasing productivity.  Second, we wanted to be
able to handle a wider variety of file formats from
agencies and produce standardized output.  Third, we
wanted to capture automatically from the processing of the
files the technical attributes of the file, such as the logical
record length, blocksize, character code and media.  In
effect, we wanted to automate the collection of
technical description during the processing, rather than
entering this information into a separate database (TAPES).
Fourth, we sought to increase the types of media we could
accept from agencies and, as well, increase the types
of media on which we could output records.

Muller Media began developing the software which was the
largest component of the contract.  The estimated costs for
the Archival Preservation System (APS) included the CPU
unit, a 66 MHZ 486 IBM Value Point running on OS2,
with two 9 track Overland tape drives, 2 Overland cartridge
tape drives, cables, and bar code apparatus at a cost of
$203,165.  The software development, $129,500, was more
than 63% of the contract cost.  It had been our intention to
develop a system, operate it for a certain period of time,
and if it performed as we expected, to purchase additional
systems as money permitted to increase our efficiency in
copying files.  We had one system, but I had six
programmers; so we anticipated purchasing additional
systems.  However, we did not anticipate purchasing
additional systems even before the software was developed,
but life overtook plans.

On January 19, 1989--the last day of the Reagan
Administration--Scott Armstrong, among others, filled
Freedom of Information Act (FOIA) requests for
information stored on the computer system in the offices of
the President from its date of installation in 1985 until the
end of the Reagan Administration.  They sued the
Government, including the National Archives, asking for
the court to declare many of the materials on the system to
be Federal and Presidential records.  Until the issues could
be resolved, the court ordered the Government not to
destroy or alter any of the systems’ backup computer tapes
since they contained the only extant copies of some of the
information on the system.  The lawsuit carried
on throughout the Bush Administration.  On the day after
the election, when Bush was defeated, the plaintiffs
extended the lawsuit to include materials residing on the
computer systems in the Bush White House as well.  On
January 6, 1993 -- two weeks before Bush formally left
office -- the court ruled that some materials on the White
House computer systems were Federal records and
Presidential records and the court directed the Government,
specifically, the Archivist of the United States, to “take
all necessary steps to preserve, without erasure, all
electronic Federal Records generated” by the White House

agencies.  This court order was not to be taken lightly as
events showed.  The Archives took physical custody of all
of the tapes from the White House as the Bush
Administration was leaving and the Clinton Administration
took office.  In late May, the court ruled that the Archives
had not complied with his order to preserve the tapes,
found the Archives in a state of civil contempt, and
levied fines which would amount to $2.5 million if the
Archives did not come into compliance within thirty days.
When the ruling also referred to ”increases in . . . sanctions
reserved . . . for any further noncompliance. . .” it meant the
threat of possible jail time was real. While the fines were
lifted, upon appeal, it was apparent that the threat of fines
and possible imprisonment was very real.

Since the National Archives had acquired custody of the
computers files and since the records in question were
electronic, it was inevitable that the Center for Electronic
Records would become involved in this case.  This in fact
happened in March 1993 when the Acting Archivist, Trudy
Huskamp Peterson, transferred the responsibility for
preserving these computer backup tapes from the Office of
Presidential Libraries, which had had physical custody of
these materials from the time they left the White House
until late May, to the Center for Electronic Records within
the Office of Special and Regional Archives.

While the APS had been conceptualized to expedite routine
preservation processing, we had to use it first in the very
non-routine processing of the materials from the White
House.  Responding to the crisis situation, the APS
contractor Muller Media Conversions compiled enough
software so that the staff with the requisite clearances were
able to make duplicate copies of files in most cases.  We
encountered problems in copying some of the files and
noted any problems.  But we complied with the court
orders and have successfully avoided legal sanctions.  If the
Center for Electronic Records had not previously
developed the concept of the Archival Preservation System
and did not have the APS on order, NARA would have
been unable to comply with the court orders.

Unfortunately the work required by what became known as
the Armstrong v. Executive Office of the President case
overtook the development of the APS system for the next
full year (1994).  Refinements to the software were made
so that problems that we initially encountered were either
alleviated or at least better documented by the system.
Over the next two years, more than 5900 tapes and/or
cartridges were copied, using one or more versions of the
Archival Preservation System software. I can state with
pride that the Center successfully copied more
than 99.998% of the media transferred from the White
House.  Out of 5906 items, only twenty nine unique items
had a data error.

Although it had been anticipated that the full development



IASSIST Quarterly Winter 1999 7

of the APS would take approximately 150 days, the
requirements of the court case overshadowed the
development of the full system.  Nonetheless, the
staff devoted many hours to developing the data elements
for the catalog database, which was the only part of the
system that had not been well defined in the contract.
Basing the database on a preexisting system that was used
to track the technical attributes of electronic files, known
as TAPES, the staff sought to include the essential elements
from the TAPES database, to capture preservation activities
that were previously recorded in a second database
(PRESLOG) and to capture information about
the individual files as the APS processed the new files.
These long staff meetings paid off because the database
reflected the needs of the Branch in capturing the
information necessary for tracking new media, the progress
of preservation work, and the technical attributes of
files processed on this system.

As I said, the contract was not completed as soon as we had
anticipated, because we had to make adjustments to the
APS system to enable the Center to meet the requirements
of the Court order for preserving the backup files.  There
were problems that had to be overcome in processing
these backup tapes from a variety of computer systems at
the White House. In many cases the tapes had been written
over, since they were used for weekly backups; so when we
attempted to copy the files, there was information beyond
the tape mark which meant we did not get a normal end of
file mark to cease the copying operation. We were not able
to determine what some of these problems were until we
obtained greater functionality with the APS system.  With
the Bush cartridges our greatest problem was the fact that
the backup utility had also used compression, so the
APS system had to be modified so it could make duplicate
copies of compressed files.  Under our normal operating
procedures, data in compressed formats do not conform to
our transfer requirements.

Without APS the National Archives could not possibly
have met the requirements set by the court, duplicating all
of the backup tapes, thus ensuring preservation of whatever
is found on these backup tapes.  But the cost was the delay
in full implementation of the APS for the
“normal” processing for which this system was designed
and purchased.  In fact, the Center only accepted the system
as meeting the basic requirements as outlined in the
statement of work in Spring 1994.  We are still
working with a system that clearly is evolving.  Currently
we are moving the catalog database to a network, which
will make the information available to the Center staff and
increase the functionality of the system by being able to use
any number of drives for performing copy jobs.
Unfortunately for us most of the additional development in
the APS system up until last September had been in
refinements to address additional problems encountered in
copying the White House system backup tapes.

Does that mean that APS is not meeting the needs of the
Center in making preservation copies of electronic
records?  Absolutely not.  We have been processing
“normal” files on APS since the fall of 1994, whenever we
were not processing files associated with the PROFS case.
We have copied over 1,375 accessioned files using this
system.  It has greatly increased our ability to handle a
wide variety of file formats, which we could not handle
previously.  One example is the 1990 Decennial Census
Public Use Sample files that the Bureau of the Census has
been transferring to us for more than two years.  Although
they are in a format that is hardware and software
independent, user labels and the blocking factor used with
these records, prevent us from being able to process these
tapes at the NIH computer center.

Perhaps even more importantly, the APS provides the
Center with a mechanism to accept files on wider variety
of media.   In using the NIH mainframe, we had two
choices: records on 9 track tape or 3480-class cartridge.
We have both of those options with APS, but we have
ordered a CD-ROM drive to be installed, so that we can
begin to copy scheduled electronic files transferred to us
on CD-ROMs.  We also want to have the ability to copy
files from diskettes which was never possible
previously. And, there are other forms of media which we
might want to be able to access, which will be possible, by
attaching drives to the system.  For the classified PROFS
files, for example, we had to install both a 4mm and 8mm
drive, because some of the files were recorded on that type
of media.

Perhaps even more importantly, the Center anticipates
using the Archival Preservation System to make copies of
files to fill reference requests. We have always used the
National Institutes of Health Computer Center to make
reference copies of electronic files.  We have had to limit
the choices of output to 9 track tape or 3480 cartridge.  But
most users now use personal computers or are attached to
networks.  In many cases researchers do not have access to
tape drives.  So, one of our goals is to use the APS to make
copies of reference requests and to output some files on
diskettes when appropriate, and possibly other media such
as CD-ROM as well.  We have just received the funding to
purchase the system with a CD-R drive attached to the
system. Again, it will mean that we will not lose physical
control over our records.  The tapes and/or cartridges
will not be exposed to poor environmental conditions,
while they are in transit between our vaults and the
external computer center, and we can tailor the output to
meet the needs of our customer base.

Has the Archival Preservation System been a success?  It
literally saved us from a contempt ruling in a contentious
court case.  We are employing it to make preservation
copies of files.  Our current objective is to improve the
functionality of the catalog database, by moving it from



8 IASSIST Quarterly Winter 1999

a Faircom server to ORACLE on a RISC6000 computer
and integrating the two systems that are currently used to
make copies of accessioned files.  And, recognizing the
utility of this system, we want to secure another system to
make copies of files, and possibly extract of files, to fill
reference requests.  Do I regret that its development was
delayed by the court case? Yes, but there is a silver lining.
The APS system was able to deal with nonconforming
media and software dependent files.  The success we had
in overcoming the problems posed by the backup computer
tapes has given us greater confidence in being able to use
the APS system beyond the narrow confines for which it
was developed.  The Center will be able to modify this
system to meet the demands posed by the newer media
being employed by Federal agencies.  The APS is a viable
system for preserving information into the twenty-first
century.

* Fynnette Eaton Smithsonian, Institution Archives. [This
paper was presented at the May 1996 IASSIST meeting in
Minneapolis, MN and reflects the situation at the National
Archives through 1997.  The author left NARA in May
1997 to join the Smithsonian Institution Archives, where
she is currently employed.]



IASSIST Quarterly Winter 1999 9