Student reaction to parallel hypertext and menu-based interfaces C.K. Ramaiah, Mubarak Sulaiman and A.J. Meadows Department of Information and Library Studies, Loughborough University, Lough- borough LE11 3TU Abstract It is often necessary to consider the question of what sort of interface is most useful for retrieving information from a particular land of database. A small database of text-based but multi-faceted items is used here in order to compare ease and speed of retrieval from two commonly used combinations of interface — HyperCard on an Apple Macintosh and dBase III+ on a PC. For the restricted range of tasks employed here, the latter combination appears to be more acceptable to students with limited computer experience. However, in more general terms, the acceptability of an interface for information retrieval depends on what particular aspect of information retrieval is being emphasized, and what conceptual frameworks students bring to their tasks. Introduction Many types of interface to electronic databases now exist. However, they can readily be grouped into a small number of categories according to their style of approach. Thus Shneiderman (1991) has recently suggested a taxonomy which distinguishes five styles of interaction. These consist of: menu selection, form fill-in, command language, natural language, and direct manipulation. They can be used as a convenient basis for considering and comparing the properties of different types of interface. Interfaces aimed at beginners in electronic information retrieval typically rely heavily on computer-initiated interaction with users. Two of the commonest such interfaces currently are (1) menu-based database management systems (DBMS) and (2) hypertext systems. The differences between these interfaces correspond to two differing interaction styles in Shneiderman's classification. The former come within the 'menu selection' group, while the latter are classified under 'direct manipulation'. The styles, and correspondingly the interfaces, 17 C.K. Ramaiah et al Style Menu selection Direct manipulation Advantages Little training required. Easy structure for decision-making. Easy to remember, once learnt. High subjective satisfaction rating. Student reaction to hypertext and menus Disadvantages Access via a sequence of menus may be irritating. Lack of flexibility. May be cumbersome to initiate some actions. Navigation may be difficult. Table 1: Advantages/disadvantages of different interaction styles differ in their advantages and disadvantages, as indicated below in Table 1 (where only those relevant to the present study have been included). These particular two interaction styles are especially interesting from the viewpoint of teaching requirements, because they relate to a fundamental distinction (which is, perhaps, understressed in Shneiderman's formulation). This concerns the difference between ease of learning (the ability of a novice user to become proficient in using a system with minimal training) and ease of use (the ability of knowledgeable users to perform tasks with minimum impediment). Our main purpose here is to use Shneiderman's differentiation of styles, with this slight change of emphasis, to compare the introductory presentation of two popular database interfaces to university students. The present study is concerned with evaluating the comparative usability of a menu-based interface (dBase m + and PC) and a hypertext interface (HyperCard and Macintosh). For this study of 'usability', the specific set of users comprised student volunteers from the Department of Information and Library Studies at Loughborough University. They were observed while using the two computer combinations in an isolated laboratory as a means of retrieving a series of items from a common database. The study concentrated on examining two parameters: performance in answering a set of test questions, and user ratings. The two commercial packages investigated here - dBase m + and HyperCard - have different overall purposes. The dBase III+ package is a relational database management software program. HyperCard is a programming system which uses a graphical user-interface to relate on-screen objects to an underlying database. However, both obviously provide information- retrieval facilities which can be studied in parallel. It is hoped that the results will cast some light on the wider question of the differences between menu-based and hypertext-based information retrieval. Both packages are widely used for educational purposes. There have been several general studies of ways in which HyperCard can be employed in teaching (Eckols and Rossett, 1989; Muir, 1992) and training (Flynn, 1988). There has also been a more limited number of 18 ALTJ VOLUME 1 NUMBER 2 subject-specific studies. The most interesting of these for the present work have been in psychology (Giesler-Brenstein and Brenstein, 1989) and in teaching information systems (Fritz, 1991). There have been considerably fewer studies of the use of dBase HI+ in teaching. Some attention has been paid to it in terms of evaluating different DBMS for teaching (Miller, 1986), but there has been little in generally available sources on its use for teaching information and library studies (Burrough, 1987). Method The database used in this study was a catalogue of teaching software prepared by the Computers in Training Initiative Centre for Library and Information Studies, which is situated in the department at Loughborough. This database has the advantage of being of direct interest to library and information students. It is a small catalogue (containing 40 entries), but each entry may need to be retrieved under a range of headings. Altogether there are 11 aspects of each software entry in terms of which retrieval might be necessary (i.e. who uses the software for teaching, what sort of teaching is involved, how much does the software cost, etc.). All retrieval headings are available simultaneously on the screen. They comprise: software designation, card number, type of software, price, author, author address, known users, hardware requirements, availability, supplier, teaching applications. The HyperCard and dBase IH+ interfaces to the database were prepared in parallel by the authors of this paper. The intention in each case was to provide an interface which was acceptable to relative beginners in the art of information retrieval, but which also exploited the particular advantages of each type of interface. Both versions were pre-tested with students and staff from Loughborough and elsewhere. Modifications were made, where necessary, in the light of this pre-testing. It should be emphasized that although two interfaces were used, the catalogue content remained the same for both. Volunteers were obtained from among undergraduate and postgraduate students in the Department of Information and Library Studies. Altogether 40 students participated, 9 of whom were postgraduate research students, 22 postgraduate Master's course students and 9 ' undergraduate students. They were asked to answer a series of questions concerning the contents of the database using successively the HyperCard/Mac and dBase/PC combinations. To allow for transfer effects (i.e. experience from the first trial affecting the results of the second), the participants were divided into two groups - which will be referred to from here on as Group A and Group B - each consisting of 20 members. Members of Group A used the HyperCard/Mac combination first, while members of Group B used the dBase/PC combination first. Group A consisted of 14 females and 6 males, whereas Group B contained 6 females and 14 males. The spread of ages of the two groups was similar, with averages of 31.3 years for Group A and 33.0 years for Group B (with standard deviations of +/- 7.7 and +/- 5.9, respectively). In each group, 18 members had had previous training on computers (mainly microcomputers), while two had not. Previous training on computers had mainly been on PCs, with word-processing as the most popular activity. Of Group A members, 13 had previous experience with the Macintosh as compared with 9 members of Group B. Previous acquaintance with HyperCard and dBase III+ was similar between the two groups, as indicated in Table 2. 19 C.K. Ramaidh et al Student reaction to hypertext and menus Previous experience with HyperCard Previous experience with dBase m + Group A Yes No 3 15 8 10 Group B Yes No 5 13 11 7 Table 2: Participants' prior acquaintance with HyperCard and dBase 7ZT+ The participants consisted of a mix of 20 overseas students and 20 UK-based students, evenly divided (10 overseas and 10 home-based students) in each group. This division formed an important part of the experiment, since the overseas students differed systematically in terms of background from home-based students. In general, overseas students were more experienced in terms of the use of catalogues (i.e. their acquaintance with the domain under test was better), but they had had appreciably less experience with computers (i.e. their system knowledge was less extensive). The interesting question, therefore, is whether overseas students have differing training needs from other students. The evaluation exercises were carried out with one member of Group A and one of Group B participating simultaneously. As far as possible, each pair of participants was matched (for example if the Group A participant was an older overseas student, so would be the Group B participant). This was intended to compensate for any changes in the environment (time of day, external noise, etc.) as the evaluation exercises proceeded. Students were set a series of 12 questions concerning the contents of the catalogue. Finding the answers entailed accessing different parts of the database, starting each time from the initial screen. The questions were graded so that the routes by which the answers could be retrieved varied from the obvious (in terms of the initial menu) and quick (in terms of the number of steps required) to the less obvious and more roundabout. In terms of this grading, the questions were asked in random sequence (to allow for any learning experience that might accrue during the experiment itself). The time taken to find an answer to each question was recorded by an observer, together with the number of steps required to answer the question. (A 'step' here is defined as any input by the user to the system, for example a command to move from one screenful of information to another.) The most efficient routes for answering the questions had been worked out beforehand by the authors. The number of steps taken by a participant was compared with this predetermined figure: any excess was recorded under the heading 'extra steps'. The intention here was to examine whether differences in domain knowledge and systems knowledge were reflected in the number of steps required to answer a question. Results from the performance tests Table 3 shows the times taken (in seconds) averaged over all the participants' answers, along with the number of extra steps similarly averaged. There is no significant difference (at the 1% level) in times and extra steps taken between Groups A and B. (The implication is that 20 ALT-J VOLUME 1 NUMBER 2 Group A Group B Number in group 20 20 Time taken for HyperCard 136 126 Time taken for dBase m + 104 110 Extra steps for HyperCard 6.4 5.9 Extra steps for dBase m + 1.8 2.0 Table 3: A comparison of retrieval (Group A v. Group B) any transfer effect works to the same extent in either direction.) However, there are differences in both time and steps taken between HyperCard and dBase m+, with the former offering a less rapid and less obvious passage for the user. (Only the difference in the number of steps is significant at the 1% level.) An alternative breakdown of the data provides additional illumination. Table 4 shows the time and extra steps taken separately for overseas and home-based students. The significant difference between HyperCard and dBase III+ times remains for both groups, but the overseas Overseas Home-based Number in group 20 20 Time taken for HyperCard 165 98 Time taken for dBase m + 137 77 Extra steps for HyperCard 6.7 5.7 •Extra steps for dBase m + 1.5 1.5 Table 4: A comparison for retrieval (overseas students v. home-based students) students clearly have greater problems in rapid retrieval than home-based students (t = 4.075 for HyperCard and 3.760 for dBase HI+, as compared with a two-tailed probability of 2.704 at the 1% level.) There was, however, no significant difference in the number of extra steps taken. The value of prior acquaintance with the system for rapid retrieval can be examined by looking separately at the results for students who had used a Mac previously. (Since most of the students had a prior acquaintance with PCs, this factor could not be examined separately.) As Table 5 shows, prior experience of a Mac appreciably affected speed of extracting 21 C.K. Ramaiah et al Student reaction to hypertext and menus Used Mac previously Not used Mac previously Number in group 22 18 Time taken for HyperCard 105 164 Extra steps for HyperCard 5.4 7.1 Table 5: A comparison of retrieval (Mac users v. non-users) information via HyperCard (t = 3.20 at the 1% level), but not the number of extra steps taken. It will be noted that the average time recorded for experienced Mac users retrieving information via HyperCard is similar to the average time recorded in Table 3 for retrieval by all users via dBase EH+. This may be compared with the effect of prior knowledge of the HyperCard software. As Table 6 shows, there is no significant difference in retrieval times between those who had Previous experience of HyperCard No previous experience Number in group 8 32 Time taken for HyperCard 124 133 Extra steps for HyperCard 6.1 6.3 Table 6: A comparison of retrieval (HyperCard users v. non-users) Previous experience of dBase m + No previous experience Number in group 19 21 Time taken for HyperCard 97 116 Extra steps for HyperCard 1.6 2.1 Table 7: A comparison of retrieval (dBase III+ users v. non-users) 22 ALT-J ' VOLUME 1 NUMBER 2 used HyperCard before, and those who had not (though the number of students in the former category was small). Almost half of the students had used dBase DI+ before. Table 7 suggests some difference in retrieval speed by this group, but it is not significant at the 1% level. Results from the user ratings At the end of the exercises on Mac/HyperCard and PC/dBase m + , participants were asked to complete a questionnaire recording their subjective assessment of the two interfaces. In the first place, they were asked with which interface they found it easier to retrieve information rapidly. The responses (contained in Table 8) indicate a strong preference among both overseas and home-based students for dBase III+. This accords with the picture derived from the timings in Table 3 - a point worth noting, since some evidence exists for supposing that users are not always able to recognize which is the fastest interface (Hayes, 1989). There was much less agreement concerning which was the more flexible interface for seeking information. Table 9 suggests a fairly even division of opinion, but that conceals an appreciable difference between overseas and home-based students. The latter were appreciably more in favour of HyperCard, and the former in favour of dBase III+. The participants were next asked about the appearance of the information on the screen and how well they could absorb it. As Table 10 indicates, both interfaces proved to be generally HyperCard dBase m + Equal Total 8 28 4 Overseas 5 13 2 Home-based 3 15 2 Table 8: Which systems permitted more rapid retrieval? HyperCard dBase ffl+ Total 22 18 • Overseas 9 11 Home-based 13 7 Table 9: Which system was more flexible? 23 C.K. Ramaiah et al Student reaction to hypertext and menus User grading Poor Acceptable Good HyperCard 2 15 23 dBase m + 0 11 29 Table 10: Relative assessment of presentation User grading Hard Fairly hard Fairly easy Easy HyperCard 3 10 14 13 dBase m+ 0 2 16 22 Table 11: Relative assessment of movement between screens User grading Poor Acceptable Good HyperCard 8 19 13 dBase m+ 0 11 29 Table 12: Relative assessment of organization for rapid retrieval acceptable in this respect. The main difference was that some users found the HyperCard screen too full of different types of information for a beginner to disentangle easily. A typical comment was that dBase III+ was easier to read, but not so attractive as HyperCard. (Overseas and home-based students agreed in these comments.) 24 ALT-J VOLUME 1 NUMBER 2 Participants were next questioned concerning ease of movement from one screenful of information to another. Their responses (Table 11) • show that greater difficulty was encountered with HyperCard. This point was taken up in a further question, which asked participants to assess how well-organized the information was to allow rapid retrieval via the two interfaces. Their responses (Table 12) clearly favour dBase m + , in correspondence both with observation of their retrieval activities and of the measured times. Conclusion Along with the hands-on tests and the questionnaires, the student participants were asked to write short essays on their opinions of the two systems they had used. Data from all three sources has been used here in assessing the results: agreement between them seems to be good. It is clear, in the first place, that dBase 111+ and HyperCard provide good examples of Shneiderman's 'menu selection' and 'direct manipulation' styles, respectively. As he suggests, menu selection is easier; navigation, for the latter, is more difficult; and so on. In practice, the relative importance of the advantages and disadvantages he lists depends on the form of the interface as much as on the style. In this case, the initial problems of handling a Mac compounded the problems of handling a hypertext interface for the first time. A typical comment was: 'HyperCard looks more user-friendly, but the use of a mouse is not as familiar to me as using a keyboard; so I prefer dBase m + ' . However, ultimate judgement of an interface must depend on its suitability for carrying out the kind of task that immediately concerns the user. For library and information students, the typical requirement for this kind of database is rapid retrieval. As the results show, experienced Mac users were retrieving information via the HyperCard interface at only about the same rate as average users of dBase III+. This disadvantage outweighed the advantages of HyperCard for many students. As one of them commented: 'I thought that PC/dBase m + was clearly laid out, uncluttered and could get the required information without resorting to "gimmicks". The Mac/HyperCard combination is more fun to use, but if information finding is the main prerequisite, then PC/dBase III+ wins.' Overseas students were even more strongly in favour of dBase HI+ than home-based students. For example, home-based students agreed with the implication of Shneiderman's classification that HyperCard was a more flexible interface; this was less accepted by overseas students. Here, it seems from the responses that their greater domain knowledge over other students is at work. The dBase III+ interface resembles more closely the cataloguing approach with which most overseas students are well acquainted, and they therefore find it easier to accept. The implications for teaching are twofold. First, new forms of interface (hi this case, hypertext) may best be presented initially in terms of their special capabilities, rather than as an alternative for standard activities which may fit better - from a student's viewpoint - into another framework (and so a different interface). Secondly, the greater the knowledge students have of a particular domain, the more likely they are to prefer an interface that fits their conceptual framework. '25 C.K. Ramaiah et al Student reaction to hypertext and menus References Burrough, H.C. (1987), Exploring dBase IH+: The Development of a Database for Use in the Teaching of Library and Information Studies (M.Sc. Dissertation, Loughborough University). Eckols, S.L. and Rossett, A. (1989), 'HyperCard for the design, development and delivery of instruction', Performance Improvement Quarterly, 2, 2-20. Flynn, L. (1988), 'More firms using HyperCard for training as a front end', InfoWorld, 10, 6. Fritz, J.M. (1991), 'HyperCard application for teaching information systems', SIGCSE Bulletin, 23, 55-61. Geisler-Brenstein, E. and Brenstein, R. (1989) 'The potential of HyperCard for psychological research and instruction', Behavior Research Methods, Instruments and Computers, 2 1 , 307-311. Hayes, B. (1989), Designing and Implementing User Interfaces for Microcomputer Programs, ESRC Programme on Taxation, Incentives and the Distribution of Income, LSE, London. Miller, R.P. (1986), 'Using geocoded databases in teaching urban historical geography', Journal of Geography, 85, 148-153. Muir, M. (1992), 'How do you run HyperCard projects?', Computing Teacher, 20, 10-12. Shneiderman, B. (1991), 'A taxonomy and rule base for the selection of interaction styles' in Shackel B. and Richardson, S.J. (eds), Human Factors for Informatics Usability, Cambridge, CUP, 325-342. 26