New technology for interactive CAL:

the Origami project

Robert Harding * Steve Lay* Peter Robinson** Dan Sheppard** and
Richard Watts**

*Department of Applied Mathematics and Theoretical Physics, University of
Cambridge
**Computer Laboratory, University of Cambridge

Origami is a three-year EPSRC project that forms part of a general research programme on human-
computer interaction. The goal of this research is to investigate and implement new methods for
human-computer interaction, and to apply and evaluate their use. The research centres on the
DigitalDesk, an ordinary desk augmented with a computer display using projection television and a
video camera to monitor inputs. The DigitalDesk allows electronic and printed documents to be
combined to give richer presentation and interaction possibilities than are possible with either separate
medium. This paper examines the implications of such a system for CAL, and presents two prototype
applications that demonstrate the possibilities.

Introduction
For many years, almost since the dawn of computing, people have speculated about the
paperless society. Each new technology is heralded as the one that will finally take us to
the fully electronic Utopia. However, the use of computers as document preparation tools
has actually increased the number of paper documents. The latest mass-market
development, the Internet, is unlikely to buck this trend even though the tools used seem
to encourage on-screen browsing. The problems inherent with screen presentation have
not gone away, while the amount of information available to the average computer user is
increasing all the time.

Education is no exception; students too are encouraged to create documents as part of
more active learning programmes, building an awareness of the importance of presenting
their own ideas to others as well as helping to meet targets for the integration of
technology into teaching. But CAL is about more than just having better tools to create
more attractive documents.

Air-J Volume 5 Number I

The arrival of multimedia-capable computers has provided an opportunity for the
integration of new media types into documents. These media types range from sound,
video and typed text to fully interactive computational tools such as virtual worlds.
However, multimedia technology has its problems:

(i) video frame rates and resolution are often low;

(ii) audio quality is often poor;

(iii) text is hard to read from computer displays;

(iv) the document must be viewed with a suitable computer.

These are problems of perception, rooted in our experience of existing single-medium
documents. We have got used to broadcast video, CD audio, good-quality colour imaging
and high-resolution text rendering. It seems likely that the advances being made in
software and hardware will fairly quickly remove the barriers presented by (i) and (ii)
through better compression and more bandwidth. The typical display and speaker system
of a modern multimedia PC are already good enough to deliver the fruits of these
advances. On the other hand, solutions to (iii) and (iv) do not appear to be just around
the corner. This threatens to divide multimedia documents into two types: those that are
delivered in a virtual world (by computer) and those that are delivered in the real world
(by paper).

The typical solution to this division is to create a metaphor which reduces the gap
between the real and virtual worlds - hence the desktop metaphor. Many CAL
experiments have involved placing material which is available to the student in printed
form online in an attempt to take advantage of the added features that a computer text
can have - for example hypertext links and search engines for navigation - as well as
using this as the starting point for adding multimedia content to the original publication
and linking it to computer-based instructional resources such as self-assessment engines.
See Rush (1995) for a recent example.

DigitalDesk
The philosophy of the Origami project is that a significant improvement to this situation
can be made by using a human-computer interaction technique known as computer-
augmented environments (Robinson, 1995) to allow publications which sit uneasily in the
computer domain to have the best of both worlds. Rather than create virtual books, we
seek to unite the value-added functionality of computer-based texts to real paper
documents and ultimately to the student's working environment itself via the DigitalDesk
(Wellner, 1991; Wellner, 1993).

In the 1970s, Xerox PARC developed the desktop metaphor which made computers
easier for many people to use by making them look and act like ordinary physical desks.
Electronic documents could be manipulated like paper documents, but the computer
added powerful new capabilities. Recently, a joint project between the Computer
Laboratory at the University of Cambridge and Rank Xerox EuroPARC resulted in the
development of BrightBoard (Stafford-Fraser and Robinson, 1996), a computer-aug-
mented whiteboard, and prototype DigitalDesks (Wellner, 1994). Instead of making the

Robert Harding et a\ New technology for interactive C A L the Origami project

Computer

Active paper

Desk surface

figure /: The DigitalDesk is built
around an ordinary physical desk

figure 2: The anatomy of trie DigitalDesk

computer act like a desk, the DigitalDesk makes the desk act like a computer. With a
DigitalDesk, the user can do things with ordinary paper that normally can be done only
with electronic documents on a screen.

The DigitalDesk is built around an ordinary physical desk and can be used as such, but it
has extra capabilities. A video camera is mounted above the desk, pointing down at the
work surface. This camera's output is fed through a system that can detect where the user
is pointing (using an LED-tipped pen) and it can recognize the documents that are placed
on it. The more advanced version also has a computer-driven projector mounted above
the desk enabling electronic objects to be projected onto real paper documents, thus
removing the burden of having to switch attention between screen and paper and
allowing additional user-interaction techniques. It will be some time before such systems
become widely available, but workstations with built-in support for video input have been
available for many years and have now even found their way into the personal computer
market. •

The techniques employed by the DigitalDesk take advantage of the fact that most written
materials are already created in electronic form. Sadly, much of the richness of this form
is thrown away when the document is printed, but the DigitalDesk attempts to utilize this
extra information to augment the paper copy.

Educational applications
The DigitalDesk is an ideal way to advance the computer-illustrated text (CIT) concept
(Harding and Quinney, 1990) which sought to combine the strengths of the traditional
printed textbook with those of the computer as an educational medium. The earliest CITs
were printed books with software, the use of which was an integral part of the learning
strategy of the package as a whole. They covered subjects in mathematics, computing and

Aa-j Volume 5 Number I

physics, areas where screen presentation is particularly difficult but where interactive
graphic displays and dynamic computation can considerably enhance the learning
activities. Of course, computers have come a long way since that series was published. The
Renaissance project (Harding et al, 1995) experimented with electronic delivery but more
recently Calculus Connections (Harding and Quinney, 1995), which involves video-clips,
sounds and sophisticated computer animation, is still centred on the printed course book.

When graphical user-interfaces were first introduced to desktop computers they were
principally aimed at document generation. In order to help the user transfer existing skills
in handling paper documents to the computer, the desktop metaphor was invented. An
integral part of the resulting systems was the printer which allowed the user to transfer the
finished document back to the real world, throwing away properties (such as the ability to
edit) which were no longer needed.

Although document generation is an important part of educational computing, the real
explosion in computer use has come from the ability to interact with multiple media
types, to gain access to information resources that exist outside the user's immediate
'desktop', and to participate in new styles of communication. As before, real-world
metaphors have been applied in order to assist the use of these new applications.
However, the problem with these new media types is that printing them causes them to
lose properties which are still useful. The DigitalDesk is effective for these applications
because its purpose is to retain these useful properties and, where possible, to add them to
documents which were not even created on the computer.

Working prototypes
To illustrate the DigitalDesk's contribution, two illustrative scenarios are presented here.
These scenarios represent possible uses of software that has already been developed for
the DigitalDesk as part of the current Origami project. The first covers a broad range of
computing applications and is used to set the scene for the second, an example aimed
specifically at education.

Augmenting paper documents
Consider a user at the DigitalDesk browsing the World Wide Web, with the current page
projected onto the desk surface in front of her. She selects links to follow by pointing with
a special 'active' pen and pressing a button located near the tip. Having discovered a page
of interest, she uses the projected print button to create a hard-copy with all the familiar
and desirable qualities of paper: readability, tactile sensation and portability. She takes
this paper copy away with her to read on the way home. The next day she returns to her
desk with the paper copy of the page she printed the day before, marked up with her own
notes. When she places it on the DigitalDesk it is recognized as the page that was printed
the day before, suitable browsing software is launched and the same set of buttons that
are available when browsing the electronic version are projected onto the header of the
paper version and the links automatically become active. She can then continue to work
with the document.

Computer-aided assessment
Now consider a mathematics student working with his course book at the DigitalDesk. The
software which accompanies the course book is automatically launched when the book is

Robert Harding et al New technology for interactive CAL the Origami project

The expresses ax* * tx* e. where a, b mi e ere conitntt. u caOed • q
function of x- The bigheit power of the vtriifcle x ii the tecond. Iffuch m ezpreoic
if plotted tgiiiut x. the resulting curve wil hare «ne of the * tupet shown below:

•.positive a.fiegtfm

The tign of a determtnei which w>? «p the carve i*.

Considerthe eqa«ion y = x 1 - 3x + 1 Titnlttiat Ac » * » « for J»for »ehiei of x
between 0 and 4 gives the following Titoe*:

xO 1 2 3 4
y20 0 2 6

Wh«t bvpeni between z - l«rf x "2? CAMl*iftgj> for J - 1H » r » « - W. We cm
now me thwe *4uef to ntke • sketch of the carte.

Note that the vatuet of x m which the curve crone* the x-«xii we unptj die potati
which y = 0. They »re therefore fce tohitioiu of the quadrtbc eqntfioa:

L . *" v : • • •

figure 3:7?ie D/g/to/Desk brings paper publications to life

first placed on the desk. During a section on curve-sketching, he reads about polynomials.
The generic equation of a quadratic polynomial is given with spaces for the values of the
coefficients and an empty box underneath for plotting the graph. The software projects
default values and draws the graph into the box. However, it also projects controls
alongside the coefficients to allow him to change these values while observing the
corresponding change in the graph (perhaps in response to a rhetorical question like: What
happens if you change c?) On the next page of his course book, he is faced with an
assessment exercise. This time the polynomials are fixed and he must draw the curve into
the box (the active pen does of course have a real nib for writing). When he has finished, he
clicks a projected button to ask the computer to assess his sketches. The sketches are
analysed for features such as maxima, minima and axis crossings, and marked accordingly.

Future developments
Hardware and software have already been developed to meet the requirements of the
above scenarios. For the future, the DigitalDesk provides exciting possibilities for the
development of educational computing projects. However, it is a research project; it will
be some time before computer-augmented environments make their way onto the average
laboratory desktop. Ten years ago, the same could have been said for desktop video.
In the meantime, when designing applications, it is useful to break down the technology
into several levels of increasing technical sophistication:

• printed materials only;

• printed materials augmented by traditional tools such as pen and paper, hand
calculator, etc.;

ALT-J Volume 5 Number )

• printed materials augmented by a computer system (with keyboard and pointing
device);

• printed materials augmented by a computer system fitted with a page-recognition
device (possibly driven by a hand-held device such as a simple bar-code reader);

• printed materials augmented by a computer system with page recognition and a
pen/finger-driven locator;

• printed materials used in the context of the fully featured DigitalDesk with page
recognition, locator tracking and projection.

Materials that are flexible enough to be used at any of these levels would be useful even if
everyone had access to a fully functional DigitalDesk, since there are bound to be times
when the complete system is not available. By designing computer-aided software
packages with this in mind, it is hoped that a greater continuity can be provided between
the student's experience in the computer laboratory and his or her own private time spent
revising and reflecting on the subject matter.

Acknowledgements
The original DigitalDesk was built by Pierre Wellner, and BrightBoard by Quentin
Stafford-Fraser (1996), both research students sponsored by Rank Xerox in the
Computer Laboratory at the University of Cambridge. Current work on animated paper
documents is sponsored by the EPSRC under grant GR/J65969.

References
Harding, R.D. and Quinney, D.A. (1990), 'Computer illustrated texts (CITs) for teaching
numerical analysis', Computers and Education, 15 (1-3), 205-11.

Harding, R.D., Lay, S.W., Moule, H. and Quinney, D.A. (1995), 'Multimedia interactive
mathematics courseware', Computers and Education, 24 (1), 1-23.

Harding, R.D. and Quinney, D.A. (1995), Calculus Connections, Chichester: Wiley

Rush, D.G. (1995), 'Do we need hypermedia and books in engineering education?',
Proceedings of Hypermedia in Sheffield '95, University of Sheffield, 284-90.

Robinson, P. (1995), 'Virtual offices', Proceedings of Royal Society Discussion Meeting on
Virtual Reality in Society, Science and Engineering, BT Publication SRD/R5/1, July.

Stafford-Fraser, Q. and Robinson, P., (1996) 'BrightBoard - a video augmented
environment', Proceedings of the ACM Conference on Human Factors in Computing
Systems, Vancouver, April.

Stafford-Fraser, Q. (1996), 'Video augmented environments', Ph.D. dissertation,
University of Cambridge Computer Laboratory.

Wellner, P. (1991), 'The DigitalDesk calculator - tangible manipulation on a desk-top
display', Proceedings of the ACM Symposium on User Interface Software and Technology,
Hilton Head, November.

Robert Harding et al New technology for interactive CAL the Origami project

Wellner, P. (1993), 'Interacting with paper on the DigitalDesk', Communications of the
ACM, 36 (7), July 1993.

Wellner, P. (1994), 'Interacting with Paper on the DigitalDesk, Technical Report 330'
(Ph.D. dissertation), University of Cambridge Computer Laboratory.