Usability Testing Of Web Mapping Portals Petr Voldán Department of Mapping and Cartography, petr.voldan@fsv.cvut.cz Keywords: usability, web-design, map sites interface, user experience Abstract This study presents a usability testing as method, which can be used to improve controlling of web map sites. Study refers to the basic principles of this method and describes particular usability tests of mapping sites. In this paper are identified potential usability problems of web sites: Amapy.cz, Google maps and Mapy.cz. The usability testing was focused on problems related with user interfaces, addresses searching and route planning of the map sites. Introduction Many people today use map portals for finding addresses, travel planning or finding points of interest (POI). Their main purpose is not only the provision of maps, but also providing the various mapping tools or services. Unfortunately a lot of people have problems to control map or to use the offered services. The reason can be the diversity of people attending the map page [6]. Web portals also do not always offer the services that users want or there is time pressure as a factor that may lead to problems (Situation when traffic or weather is rapidly changing) [10]. These all factors have a negative effect for each side. User can get information that looks differently and generally it could lead to dissatisfaction [8];[9]. This situation is not suitable for site owner. The web is the ultimate customer-empowering environment. It’s the user how decides everything, because it is so easy to go elsewhere. Usability testing can help developers create high-quality (usability) user interface. Quality interface provides users ”easy-to-use” system and users achieve their goals more effectively [7]. The purpose of this study is to present usability testing, and show the main problems of some Czech mapping portals. Usability testing People, who do not know anything about usability testing, often ask: What is the aim of testing? Here are some benefits: � Make sure that the product is easily understandable for users. Geinformatics FCE CTU 2010 57 Voldán P.: Usability Testing Of Web Mapping Portals � People will be effective and satisfied with your product. � Usability testing helps eliminate design problems. � ”Testing one user is 100 Better then testing none.” [3] Tested portals Three web map sited were evaluated in the study: � Amapy.cz abbreviated in this paper as AM, available at http://www.amapy.cz. Amapy was the first Czech professional web map portal [1]. � Google maps abbreviated in this paper as GM, available at http://maps.google.com [2] � Mapy.cz abbreviated in this paper as SM, available at http://www.mapy.cz. Mapy.cz is one of the greatest Czech map sites (240,000 entries per day) [4]. These sites offer standard an interactive 2D map with zooming and panning functions. Ad- ditionally, there are features as finding location, addresses, businesses, route planning and many more. Task scenario It was created task scenario same for all tested site which focused on the following areas: map controlling, searching places in an unknown location, route planning and on maps themselves. Task scenario was a coherent story of a weekend visit. The aim is to create familiar situations with realistic reason for performing the task. The closer, that the scenario represents reality, the more reliable the test results [8]. The story itself had 4 parts and assumes that user live in or near Prague. The text of the scenario as it was presented to participants: 1. Your friend Paul invited you to a weekend visit to his new apartment. In addition to invitations you were asked to pick up a carton of wine in Paul favorite wine shop. � Users got address of new apartment and name of wine shop without address. � This section was focused on finding unknown company and route planning 2. Plan a walk trip to the pond ”Velké Dářko” � Velké Dářko is a pond placed 8 km north of Paul’s apartment � The aim was to plan trip with using another maps (touristic, terrain, ortophoto) 3. Determine whether there is any pool in Paul’s city and note the route. 4. Plan your route home including a visit of castle Lipnice nad Sázavou. � Lipnice nad Sázavou is castle 30 km from Paul’s apartment � This section focused on route planning over transit point and finding POI. Procedure Geinformatics FCE CTU 2010 58 http://www.amapy.cz http://maps.google.com http://www.mapy.cz Voldán P.: Usability Testing Of Web Mapping Portals All tests were performed in normal office either a desktop PC or laptop. When author selected respondents, he tried to find wide range of users – in terms of age, gender or education (the words participant or user in the following text mean test users). Testing all mapping sites participated in sixteen users (eight in testing of SM site and eight in testing of AM and GM sites). About half of the participants were university students, both technical and nontechnical subjects and they were from twenty to thirty years old. The second part of participant was consisted of people older than thirty years until retirement age. These included a office-girl, a mechanical engineer, nurse or schoolmistress. It should be emphasized that none of the participants had cartographic or geodetic background. Figure 1. Scheme of the test room The operating system was Windows XP, user could choose the web browser and users have used a mouse. Task scenario was presented to each participant before start of the test. Mod- erator assured of each participant that the user is not tested, but mapping sites. Participants were asked to think aloud during the test and then the moderator read the tasks (think aloud is a technique where users are asked to speak about their thoughts, feelings). Each session was about sixty minutes. After the execution of all tasks, moderator or observer discussed with the participant about test conduct, problems or feelings about the test. At the end of the session there was a brief discussion between the observer and moderator about the main problems. Usability problems The list of positive or negative situations was obtained by analysis of observer’s notes or citations of the users. Geinformatics FCE CTU 2010 59 Voldán P.: Usability Testing Of Web Mapping Portals Controlling Tested portal allows controlling of map windows (zooming, panning) using either graphics elements or using mouse gestures. In generally it is possible to say that one half of users used the full mouse gestures. The second group of users used a combination of a graphical interface and mouse gestures or only the graphical interface. Method Use [%] drag & drop 67 GUI – crosshair 33 Table 1a. Control method – map pan Method Use [%] Double click left mouse button 42 GUI – slider 33 Scroll wheel 25 Table 1b. Control method – zoom in Method Use [%] Double click right mouse button 17 GUI – slider 33 Scroll wheel 50 Table 1c. Control method – zoom out AM and SM allow zoom-in into rectangular region using combination ctrl key + left mouse button. This feature was not used during testing (GM also allows zoom-in into rectangular area, but still it is a laboratory function). Searching A large number of queries in full-text search are addresses or locations. Company location and even geographic places people are not searching on mapping sites but on the classic search engines (Google, Seznam.cz). There are map elements that the users know for example from the classic maps (restaurants, railway stations, bus stops, pools). These elements users usually search visually on the map (SM). Big problem are two search fields on AM site (Figure 2.). Users do not know which field to use and its leads to user confusion. In the case of AM there is a problem enhanced by the fact that the main search box (the top one) is visually separated from the map window, and users can easily overlook it. Jacob Nielsen, guru on the field user experience, says the system should always keep users informed about what is going on [5]. Unfortunately this rule is broken on AM. There is big time delay before the system notifies the state ”Searching”. This delay causes that users think that there is no search results and tray search again. Search engine also has to handle different formats of addresses. AM returned no results where the user entered an address in format for system nonstandard (street, postal code, city). Geinformatics FCE CTU 2010 60 Voldán P.: Usability Testing Of Web Mapping Portals Figure 2. The user doesn’t know what field have to use for searching (AM) Autocomplete feature has been positively evaluated by users (GM). ”It offers me . . . nice”, user citation about address autocompleting. Next good feature for users were additional information in the search results (GM) – see Figure 3. These images quickly help to decide about the relevance of search results. Figure 3. additional information in the search results (GM) Appropriate issue for further research is the level of zoom after searching addresses, cities and etc. From the tests generally follows that users behave differently in a situation where they search the points of interests in known location or in unknown location. In known location users zoom in after showing result. Conversely in unknown location users zoom out after showing result. In both cases, the reason is checking of results of user’s search. Geinformatics FCE CTU 2010 61 Voldán P.: Usability Testing Of Web Mapping Portals Direction, route planning According to task scenario the participants carried out route planning during the test. The evaluation shows that when the users are planning the route, they are interesting in the following facts: 1. The users want to information about the part of the route, they don’t know. User citation: ”I catch the highway” 2. Users are interested in the names of major cities on the route, which city they will turn and name of the next town on the route. Important are the numbers of roads and numbers of highway exits. 3. Detailing listing of streets and intersections is relevant to the users when they reach destination city/POI. From these points implies that the current itinerary of all portals are too detailed, users lose character information and in real situations are difficult to use itinerary for navigation. Most of the users do not use the route planner, where the starting or destination point should be any POI. Route planner is primarily used for getting direction between a particular city and address. In spite of the previous fact, mapping portals should allow to enter in search field as start or destination POI. This fact did not meet at AM site. In the case of GM, there is no way how to end the route planner. After searching the direction, searched route remains displayed and the user can not cancel it. User citation: ”I don’t know how to cancel it”. On the other hand, there was no problem to add new transit point into the route for users (GM). Surprisingly, participants also used the route planner in a small (for them unknown) town to short directions (around one kilometer distance). Map types All mapping sites offer different types of maps – base map, orthophoto map, tourist map, etc. Users generally prefer default (base) type map and they switch to other map type in a few cases. Even in the situation, if the map switching could lead to faster execution theirs aims. Generally, base maps of SM and AM were positive evaluated by participants. The reason is probably a better cartographic language of SM and AM sites. The added value of the SM and AM are also the tourist maps – including hiking and biking paths (GM has not the tourist map – only the terrain map). One point of the task scenario directly encouraged to use tourist maps (walk trip to the pond Velké Dářko). Anyhow for some users it was easier to work only with a basic map, because it is more synoptic. User citation: ”The touristic map has a lot of lines”. Those users, who used the tourist map, were confused in some situations: Tourist maps include map symbols which are not interactive – it is not possible click on them with aim to get more information (SM, AM). Users know that it is possible click on the map symbol on the basic map and thus they want also to click on the map symbol on the touristic map, but there are the map symbols only parts of the static images (Figure 4.). User citation: ”I expect when I am placing the mouse cursor over a map symbol, it tells me something about symbol.” There was another issue with AM site. AM do not zoom-out to the level where can be seen all state. Geinformatics FCE CTU 2010 62 Voldán P.: Usability Testing Of Web Mapping Portals Figure 4. Clickable and not clickable symbol look same The system should correspond to relationships in the real world [5]. On the SM site was disabled by default the tourist paths (Figure 5.), but classical (printed) tourist map has highlighted tourist paths. This situation has led to unnecessary confusion for the user (this time is this feature enabled). Figure 5. Match between system and the real world Other expierences This section describes other situations which were not included in any of the preceding chap- ters. Positively assessed are the info windows that appear after clicking on the marker of the POI. These info windows serve as a good source of additional information. AM does not allow the browser’s function ”back” – that was one of the major shortcomings of the AM site. Next issue is improper placement of application windows which can cause major problems. AM shows settings panel so unsuitable that panel overlaps another buttons. Figure 6 shown two states of AM site. First one shows normal state and the second one the state where are the control buttons hidden under another window. User citation: ”Also, it does not have a biking path”. User wanted to show the biking path, but the button for turning on of the biking path was hidden by another window. Geinformatics FCE CTU 2010 63 Voldán P.: Usability Testing Of Web Mapping Portals Figure 6. Settings panel overlaps GUI buttons Conclusion In this study was presented the results of usability testing three map sites. For testing all sites was developed same task scenario. These tasks tried to respect the user’s real-life situations (finding addresses, route planning). This paper shows that through usability testing, you can find a critical point or, conversely, quality points, and work with these parts in further development. Although some findings of fact may seem like small things, each such problem can lead to loss of users and may thus have a financial impact. Author hope that described problems could serve as a guide in development of other web map services. Identified problem also enables us to make a brief comparison of test sites – in terms of usability. Table 2 shows summary of positives or negatives aspects of evaluated map sites. Map site Negative Positive Google maps 7 4 Mapy.cz 10 2 Amapy 16 2 Table 2. Summary of positives or negative reaction According to the number of problems have the best ratings SM and GM sites. The main advantages of the GM site are autocomplete function or images tips in results. On the contrary, the weakness of the GM is informational value of basic map. It is the clearness of base maps and lots of POIs that have been positively evaluated in the case of SM site. The worst-rated is AM site, which contains a lot of critical comments. The improving of usability of AM site doesn’t have to be great changes. It should be a gradual evolution. In first step remove most significant problems (two search fields, function back). Then conduct a new usability testing and repeat all process. Paper also offers possible directions for further research, such as: level of zoom of the results after searching or simplified of itinerary of route planner. Author thanks the company Seznam.cz for the aid in testing. Geinformatics FCE CTU 2010 64 Voldán P.: Usability Testing Of Web Mapping Portals References 1. Amapy (2010), online at: http://www.amapy.cz 2. Google Maps (2010), online at: http://maps.google.com 3. Krug, S., Don’t Make Me Think: A Common Sense Approach to Web Usability, 2nd Edition, New Riders Press, ISBN 978-0321344755 4. Mapy.cz (2010), online at: http://www.mapy.cz 5. Nielsen, J., Ten Usability Heuristics (2010), online at: http://www.useit.com/papers/heuristic/heuristic list.html 6. Nivala, A.-M., Brewster, S. and L.T. Sarjakoski (2008) Usability Evaluation of Web Mapping Sites, The Cartographic Journal Vol. 45 No. 2 7. Nivala, A.-M., Brewster, S. and L.T. Sarjakoski (2007). Usability Problems of Web Map Sites. Paper presented at the International Cartographic Conference, Moscow 8. Rubin, J., Chisnell D., Handbook of Usability Testing: Howto Plan, Design, and Con- duct Effective Tests, 2nd Edition, Wiley, ISBN 978-0470185483 9. Wachowicz, M., Vullings, W., Bulens, J., Groot, H. d., & Broek, M. v. d. (2005). Uncovering the Main Elements of Geo-Web Usability. Paper presented at the AGILE 2005 – 8th Conference on Geographic Information Science, Lisboa, Portugal. 10. Wilkening, J. (2010). Maps users’ preferences and performance under time pressure. GIScience 2010: sixth International Conference on Geographic Information Science. Zurich. Geinformatics FCE CTU 2010 65 http://www.amapy.cz http://maps.google.com http://www.mapy.cz http://www.useit.com/papers/heuristic/heuristic_list.html