lindgaard Human factors in telecommunications research Gitte Lindgaard Telecom Australia Research Laboratories Back in the days when 'telecommunication' meant that you could still get an operator to connect you to another person on the telephone network, some aspects of life were easy and straight-forward. You didn't have to be an electronic wizard to use the available technology correctly and efficiently. In those days, when computers filled entire rooms and were tended to by bald little, white-clad men, mumbling incomprehensible formulae to themselves, the average citizen did not need to bother about computers, the silicon microchip had not yet made its digital debut into the electronic world. Things are changing though. Computer technology and telecommunications are rapidly merging in ways that will eventually affect all of us. This merging was perhaps most clearly demonstrated in the recent launching of the Telecom ComputerPhone, an all-in-one desktop electronic tool which, at present, is probably most relevant to business managers. In due time however, electronic shopping and banking facilities, train and bus timetables, and perhaps even computerised tertiary education will be available, although probably not equally accessible, to the entire population. Telecom Australia, as a provider of services, is concerned about making its services readily accessible to the whole Australian community. The range of services provided by Telecom is increasing, as is their complexity. No longer do you get an operator to complete the link-up for you; more and more you will have to rely on your ability to access and interpret information that allows you to explore and exploit the available services. It is important, therefore, that services are presented so that customers can use them efficiently and to their satisfaction. How to achieve this is the 64 dollar question concerning a small research group in Telecom Australia Research Laboratories. We know that people inexperienced with computers have difficulties interacting with them. We also know that telecommunications users will need to interact with a computer in many future services. Factors that may make this interaction difficult for novice computer users thus need to be thoroughly investigated. Knowing what makes the interaction difficult 4 Australian Journal of Educational Technology, 1985, 1(2) enables us to design communications systems that are easy to use. If systems are easy to use, one would assume they are also more satisfactory than systems which cannot communicate effectively with their users. Interest in user-computer interaction is spreading at an almost exponential rate. Whereas only a few years ago research papers dealing with the interactive process were scattered throughout the professional literature in computer science and cognitive psychology, several large international conferences are now held annually; societies are being formed across the world, journals are initiated, and books appear on the commercial shelves, all based on a common interest in people's interactions with computers. This intense research and activity testifies to the recognition of the urgent need for the kind of research that will enable the generation of guidelines for the design of user- computer interfaces. Despite this surging interest and despite the large amount of work needed to understand the psychology of computer interaction, very little experimental work has so far been carried out in Australia. This may be due partly to the lack of sufficient computer facilities on university campuses and partly to the fact that very few graduates, and hence, very few university lecturers, have a background of combined computer science and cognitive psychology. Within Telecom Australia's Research Laboratories a small interdisciplinary team of engineers, technical officers and psychologists is currently working in the user-computer interaction area concerned with what is called 'Human Factors'. The term 'Human Factors' refers to the behavioural manifestations of human capabilities and limitations based on anatomical, physiological and cognitive characteristics. In our research project, we are not primarily concerned with the purely ergonomic aspects of the interactive process between user and computer, such as screen brightness, angle of the screen relative to the user's eyes, seating, lighting or environmental noise levels. Rather, we are interested in exploring factors that may make the interaction conceptually difficult for novice users. So far, a series of experiments has been carried out, designed to investigate the importance of vocabulary selected to communicate with users. The research has been conducted within the framework of an electronic mail systems (EMS) which is under development on a worldwide basis. The concept of electronic mail offers one great advantage - one can assume that most people have a working model of how to send and receive messages from their knowledge of the current postal system and from common business practice. The availability of a conceptual model reduces the number of factors that must be considered in the interpretation of experimental results. It is necessary to limit the number of factors investigated in any one experiment, since the more factors that interact and influence people's performance, the more ambiguous and uninterpretable are the resulting data. Lindgaard 5 Our initial experiments adopted a pen/paper format. We wanted to know to what extent pen/paper studies can be of predictive value for the user performance on a computer. If performance in the two types of studies is totally dissimilar, one would be wasting one's time trying to use pen/paper studies as a first approximation for the effectiveness of a computer system. On the other hand, pen/paper experiments are quick and easy to administer and are considerably less expensive to conduct than designing, programming and testing a computer system. Another purpose of the pen/paper studies was to find out whether people clearly prefer some short labels describing services typically available in a MHS over other synonymous labels. The use of words in computer languages has been extensively debated in the current literature. Scientists agree that so-called 'natural language' is preferable, since it requires no special skills on behalf of the user. Unfortunately, computers that are able to communicate effectively with users in everyday language are quite some time away. Meanwhile, computers are with us anyway and so we must seek to answer questions of immediacy, while also trying to gain a more global understanding of communications between human users and computers. The language debate has centred on people's preference for certain words, often comparing preferences of highly experienced users with those of novices. The link between preferences and performance has generally been inferred from tests of recall in which subjects learn certain commands and are then required to recall them later. It is thereby assumed that performance can be predicted on the basis of recall of commands. By analogy this is tantamount to saying that one's performance using the telephone network is dependent upon recalling particular phone numbers from memory. Instead of inferring one type of performance, such as performance on the computer, from another type, such as recall of labels, our aim was to measure performance directly on the basis of preferences. That is, we asked whether words that are better liked also lead to a better performance than less liked words. If there is no link between preference and performance, one may as well forget about conducting preference studies. Our pen/paper studies indicated that people clearly prefer some descriptive labels over others. Technical terminology was generally disliked, subjects preferred more general terms. Three sets of labels were generated from the preference studies. One set contained the most preferred, one the least preferred, and one the labels that were rated in between those two extremes by subjects in the pen/paper study. Using a form-filling approach, subjects were exposed to only one set of labels. Acting as both sender and recipient, subjects were required to fill in forms sufficient for supplying a computer with the details necessary to forward a message in the sender's tasks. In the recipient's tasks, subjects read information appearing on the screen, said to be the details of an incoming message, and answered questions about this information. All tasks were thus concerned with the so-called 'electronic 6 Australian Journal of Educational Technology, 1985, 1(2) envelope' which is comparable to, although more complex than, a normal post envelope. The next experiment was carried out both in a pen/paper format and on a computer, simulating a MHS. Subjects completed either the computer tasks or the pen/paper tasks. Pen/paper and computer performance turned out to be comparable: recipient's tasks were completed faster than sender's tasks and vocabulary did affect performance to some degree. That is, subjects exposed to the labels previously assessed to be least preferred, performed significantly worse than those exposed to better liked labels - at least in some of the tasks. For this situation, pen/paper studies would appear to be reasonably accurate indicators for performance on the computer, at least when the experiments are very similar in layout. It was evident, particularly from the computer experiment, that subjects' performance improved dramatically during the course of the experiment. Their first set of tasks took much longer to complete than later tasks. It seemed that knowledge of the task requirements was an important factor determining performance. This was then investigated. It was argued that if knowledge of the task is very important, then pre- experimental training, presenting tasks as later seen on the computer, should reduce the initial task completion time on the computer when compared with that of subjects who had no such prior training. If exposure to vocabulary is important, pre-experimental training should facilitate initial performance on the computer only when the same vocabulary is seen in both the training task and on the computer: exposure to a different vocabulary should not affect initial computer performance. If however, the improvement in performance observed in the previous experiment was due to subjects becoming confident in their interaction with the computer as such, a pen/paper pre-experimental training task should not affect initial computer performance. We found that training did help a great deal: subjects who had received training performed significantly better than subjects who had not, regardless of whether or not they saw the same vocabulary in all tasks. This confirmed the predication that knowledge of the task requirements is very important for performance using a computer. It also showed that actual interaction with the computer is perhaps not quite as important as one may assume, provided subjects know what to do. An interesting question arising from the results was whether the elimination of effects of vocabulary was really due to subjects having learned the meaning of the various labels. All the forms used had been presented several times and it was likely that subjects were learning where to enter what information on the form, rather than learning the meaning of the labels. That is, if one notes where information belonged on a form, they need not recall, or even interpret, what the actual label describing the relevant service said. To test this, we simply randomised the order in which the labels appeared in the screen between tasks, so that subjects never saw labels in the same order twice. If subjects rely on the position of information on the screen, performance should improve less during the Lindgaard 7 experiment than noted before and, more importantly, vocabulary should again affect performance. Both of these suggestions were confirmed: performance improved less than observed previously and it was severely affected by the vocabulary. Subjects did, it would seem, rely quite heavily on the position in which to enter information. Improvements in performance during the earlier experiment could thus be attributed partly to subjects learning what the various labels meant, and partly to a consistency across screen displays. The question then was, for how long do subjects rely on consistency in screen displays? One may assume that sooner or later enough will have been learned about the task and about the vocabulary that the position of the items on the screen should lose its importance. The next experiment provided both pre-experimental training and a randomised screen display to test this assumption. Performance here improved to such an extent that it was no longer affected by screen vocabulary. The reliance of subjects on constancy between screen displays was, it would seem, very short lived: adding only one set of tasks in the form of pre-experimental training was sufficient for subjects to improve their performance independently of the position of items on the screen. Overall, our research so far indicated that subjects learn to use the computer very quickly. Whereas both vocabulary and constancy across screen displays are very important factors initially, once subjects know what is expected of them, they are no longer affected by these factors. The next stage in our research project is to look at ways in which we can help users form a workable model of a given computer system. We know that such 'user models' are notoriously insufficient for complete understanding of a system, or they are entirely wrong. Given our consistent finding that knowledge of the task requirements is perhaps the most important aspect of efficient user-computer interaction, it seems that this area is one that urgently requires attention. At present, we are developing a system that is more truly interactive, and also complex, than the one used in the previous experiments. To enable Telecom to provide efficient and satisfying customer tailored computer based services requires a substantial and concerted research effort. As you will probably appreciate, the questions yet to be answered are many and the answers can be provided only at a painstakingly slow pace. One day we will get there though and meanwhile we press on with the good work. The permission of the Director, Research, Telecom Australia, to publish this paper is gratefully acknowledged. Please cite as: Lindgaard, G. (1985). Human factors in telecommunications research. Australian Journal of Educational Technology, 1(2), 3-7. http://www.ascilite.org.au/ajet/ajet1/lindgaard.html