e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 156 Florian J. Zach Virginia Tech Yufeng Ma Virginia Tech Edward A. Fox Virginia Tech A Preliminary Analysis of Images in Online Hotel Reviews Following the idiom “A picture is worth a thousand words”, recent tourism and hospitality research has adopted deep learning techniques to better understand the content and effect of photographs. This exploratory study delves into image content, to learn which sets of images relate to positive and negative reviews. Several image classes appear in both positive and negative reviews. However, the distribution of images across classes differs. Of interest were images that did not represent core hotel services, which suggests that users review the surrounding area, attractions, and activities as well as the hotel property. This finding is relevant for managers to learn which areas (to engage with) could improve guest experiences. Key words: online hotel reviews, image classification, image Florian J. Zach Howard Feiertag Department of Hospitality and Tourism Management Pamplin College of Business Virginia Tech Wallace Hall 355 Blacksburg, VA 24061 USA Phone: +1 (540) 231-8425 Email: florian@vt.edu Edward Fox Department of Computer Science College of Engineering Virginia Tech 114 McBryde Hall Blacksburg, VA 24061 USA Phone: +1 (540) 231 5113 Email: fox@vt.edu Yufeng Ma Department of Computer Science College of Engineering Virginia Tech http://ertr.tamu.edu/ mailto:florian@vt.edu mailto:fox@vt.edu e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 157 114 McBryde Hall Blacksburg, VA 24061 USA Phone: +1 (540) 449 2944 Email: yufengma@vt.edu Florian J. Zach is an Assistant Professor of innovation and entrepreneurship in tourism and hospitality at Virginia Tech with an interest in new technologies. His current research focuses on the implementation of technological advancements to create tourism experiences. Yufeng Ma is a Ph.D. candidate in the Department of Computer Science at Virginia Tech, where he is co-advised by Dr. Patrick Fan and Dr. Edward Fox. He is mainly interested in the interaction between Computer Vision and the combination of Natural Language Processing, Deep Learning, and Machine Learning. Edward Fox is a Professor of Computer Science and of Electrical and Computer Engineering (by courtesy), as well as Director of the Digital Library Research Laboratory, at Virginia Tech. His research interests include digital libraries, information retrieval, natural language processing, machine learning, multimedia, and related areas. http://ertr.tamu.edu/ mailto:yufengma@vt.edu e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 158 Introduction User-generated content plays a critical role in travel planning. It allows travellers to engage with likeminded individuals and share experiences, providing service providers with an opportunity to leverage these interactions to strengthen customer relationships (Wang and Fesenmaier, 2004). Online hotel reviews affect attitude toward hotels (Vermeulen and Seegers, 2009) and booking intentions (Tsao, Hsieh, Shih, and Lin, 2015). It is often assumed that hotel reviews are limited to a specific property. However, hotel guests post pictures not only of hotel properties, but also of attractions, land- or city-scapes of the destination, and activities in which they engage (Xiang, Du, Ma, and Fan, 2018). The goal of this study is to extend our knowledge of images posted in hotel reviews, specifically in off-property images. Literature review Past research on the text of online hotel reviews investigated heuristic and systematic cues in reviews, to evaluate the usefulness of reviews (Liu and Park, 2015; Chung, Le, Koo, and Chung, 2017), effects of language on review text valence (García-Pablos, Duca, Cuadros, Linaza, and Marchetti, 2016), intention to book (Tsao et al., 2015), and reliability of online reviews (Xiang et al., 2018). In addition to evaluating text reviews from multiple perspectives, the continuous development in the field of computer science enabled scholars to gain deeper insights. While initial studies focused on word count frequencies to glean topics of interest to reviewers, advanced machine learning algorithms can extract meaning using topic modelling (Xiang et al., 2018). These insights allow decision-makers to better connect with their potential customers. Besides text, images hold valuable information. Indeed, metainformation about an image can be useful. For example, Liu and Park (2015) found that user profile images increase the perceived usefulness of travel online reviews. Similarly, Ma, Xiang, Du, and Fan (2018) http://ertr.tamu.edu/ e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 159 found that the helpfulness of online reviews increased when images were included. As images “can become a primary source of data for understanding the form, meaning, and the process of photographic representation in tourism” (Albers and James, 1988, 135) it is necessary to understand image content. Indeed, taking photographs empowers the photographer to take ownership, to interpret the image, and to use it for storytelling (Urry, 1990). An image taken and uploaded thus has value to the photographer. In the context of reviews, we can assume that the uploading user felt that the images are useful for potential hotel guests. Ma et al. (2018) found do images not represent hotel features when analysing review images. Such non-hotel images (e.g., land- or city-scapes,) enable potential guests to form an opinion of the destination. The goal of this exploratory study, thus, is to understand the relationship between image content (hotel, non-hotel, or both) and review rating. Method Data for this study was collected in 2016 from TripAdvisor using Python and Java web crawlers that mimic user access. The destination of Orlando, Florida was chosen, as it is a travel hotspot and had many reviews with photos. A total of 86,227 images were collected. For each image, the corresponding rating score (1-5 stars) of the reviews was also collected. Based on the rating score, reviews were categorized as positive (4-5 stars), neutral (3 stars), or negative (1-2 stars). Using the popular Residual Network (ResNet) model, images were analysed to extract the main image content. The ResNet algorithm (He, Zhang, Ren and Sun, 2016) by default has 1,000 built-in classes of images, stretching from animals to buildings to household items, and can predict ImageNet image classes (Russakovsky et al., 2015) with a top-5 accuracy of 94.06%. The network is extremely deep with 152 layers, which enables it to learn abstract hierarchical features. Each class was evaluated as adequate of representing core hotel (e.g., http://ertr.tamu.edu/ e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 160 bed) and non-hotel aspects (e.g., buildings), or both (e.g., kitchen and household items that are present in restaurants). Findings The spread of positive, neutral, and negative ratings was 78-12-10 for reviews, and 84- 9-7 for images. Hence, travellers are keen to provide visual evidence of positive rather than negative experiences. Next, 114 image classes the algorithm can identify were not found in the data (e.g., digital watch, beagle). Across the remaining 886 image classes only 8% had a higher negative than positive frequency count. Examples from the 886 image classes are shown in Table 1. Images of geography/landscape and buildings were nearly always in positive reviews: 92% and 89%, respectively. Images of spiders, other insects, and tools were the only images that appeared more often in negative reviews: 89%, 71%, and 61%, respectively. This suggests that reviewers either discovered these animals or that repairs took place. A chi-square test revealed that the relationship between image classes and review ratings was significant X2 (1768) = 1, p < 0.05. Table 1: Image class examples in negative/neutral/positive reviews (frequency counts) Image Class Negative Neutral Positive band aid 41 7 19 barn spider 4 1 0 corn 0 1 9 day bed 177 412 3,341 diaper 20 4 51 gown 3 0 17 paper towel 79 15 90 pig/hog 2 0 2 radiator 70 22 22 sleeping bag 9 3 14 steel arch bridge 2 3 78 totem pole 7 13 218 http://ertr.tamu.edu/ e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 161 Table 2: Image distribution (absolute numbers) Hotel Non-Hotel Both Positive 14,918 39,091 18,333 Neutral 1,945 3,478 1,839 Negative 1,779 2,948 1,896 Most images (52.8%) posted were non-hotel. A chi-square test revealed that uploaded images differ across property aspects to categorize hotel, non-hotel or both, X2 (4) = 1, p < 0.5 (Table 2). Tables 3 and 4 display the top 5 image classes for positive and negative reviews. For positive reviews, the top 5 image classes represent a higher share of each list (hotel/non- hotel/both) compared to negative reviews. Four of the top “hotel” items are listed in both positive and negative reviews. Similarly, there is a repetition of two classes among “non-hotel” (lakeshore and medicine) and “both” (sliding door and patio). However, the frequency among negative reviews is lower than in positive reviews. This suggests that these items are of particular interest to travellers and thus need to be tended to by tourism stakeholders. Table 3. Top 5 image classes in positive reviews (in % of hotel/non-hotel/both lists) Hotel % Non-Hotel % Both % quilt comforter 23.45% lakeshore 18.87% patio terrace 20.62% day bed 22.40% palace 4.08% fountain 12.89% four-poster 11.85% seashore coast 3.72% dining table 8.72% washbasin 8.95% restaurant 3.57% microwave 8.44% shower curtain 5.25% medicine cabinet 3.13% sliding door 6.95% Sub-total 71.89% Sub-total 33.37% Sub-total 57.63% “Non-hotel” image classes in positive reviews include landscapes and buildings/shops, whereas in negative reviews we find rather detailed items such as doormat and bannister. This is similar for image classes in positive reviews that can represent both hotel and non-hotel images, where the positive reviews include larger furniture items (e.g., dining table) versus http://ertr.tamu.edu/ e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 162 detailed images (e.g., electrical switches) in negative reviews. Across both the top 5 positive and negative image classes several can be described as furniture (e.g., day bed, four-poster, dining table, medicine cabinet, bannister). Negative reviews include images of items that might have been used to fix a problem (e.g., toilet paper, paper towel). Table 4. Top 5 image classes in negative reviews (in % of hotel/non-hotel/both lists) Hotel % Non-Hotel % Both % washbasin 13.66% doormat 5.19% electrical switch 10.60% quilt comforter 13.32% lakeshore 4.21% sliding door 5.17% day bed 9.95% medicine cabinet 3.63% patio terrace 5.17% toilet paper 9.67% bannister 2.75% paper towel 4.17% shower curtain' 9.27% dishwasher 2.10% lampshade 4.11% Sub-total 55.87% Sub-total 17.88% Sub-total 29.22% Discussion and conclusions The importance of images for travel and tourism is generally acknowledged. The advent of user-generated content websites and camera-enabled smartphones enables guests to share visual insights in areas and travel aspects previously not possible. This exploratory study provides first insights into the classes of images uploaded to TripAdvisor. Specifically, we found that images of non-hotel related travel aspects account for a large share of images in positive reviews. These image classes are essentially a vote for travel aspects that the reviewers consider relevant for other travellers to know or learn about. This is relevant as tourism stakeholders depend and rely on each other to provide travellers with a satisfying travel experience. Hotel managers can leverage this information to draw attention to activities and attractions deemed relevant by past visitors, as consumption of such activities or visitation to such attractions seems to positively affect the overall guest experience at a hotel. http://ertr.tamu.edu/ e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 163 While the share of same image classes differs among positive and negative reviews, it is unclear if these image classes are motivation or hygiene factors. Future research should investigate this issue; both factors require different managerial responses. Lastly, the 1,000 ResNet algorithm image classes are a good start; however, they exclude critical hotel aspects such as check-in counter, hostess desk, or mini-bar, which could be added as specialized hotel image classes. A limitation of this study is that no specific image classes for destination or hotel management were identified. Specific classes such as “check-in area” could be beneficial for hotels to identify property specific areas of improvement. Classes of area-specific attractions could allow identifying which specific places beyond image classes visitors recommend or discourage as sights. Another limit is the use of only one data source (TripAdvisor) for one destination. References Albers, PC. and James, WR. (1988) Travel photography: A methodological approach. Annals of Tourism Research 15: 134-158. Chung, HC., Lee, H., Koo, C. and Chung, N. (2017) Which Is More Important in Online Review Usefulness, Heuristic or Systematic Cue? In: Schegg R, Stangl B (eds) Information and Communication Technologies in Tourism 2017. Springer International Publishing, Switzerland, 581-594. García-Pablos, A., Duca, AL., Cuadros, M., Linaza, MT. and Marchetti, A. (2016) Correlating languages and sentiment analysis on the basis of text-based reviews. In: Inversini A, Schegg R (eds) Information and Communication Technologies in Tourism 2016. Springer International Publishing, Switzerland, 565-577. He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition, 770- 778. Liu, Z. and Park, S. (2015) What makes a useful online review? Implication for travel product websites. Tourism Management 47: 140-151. Ma, Y., Xiang, Z., Du, Q., and Fan, W. (2018) Effects of user-provided photos on hotel review helpfulness: An analytical approach with deep leaning. International Journal of Hospitality Management 71: 120-131. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, AC., and Fei-Fei, L. (2015) ImageNet large scale visual recognition challenge. International Journal of Computer Vision 115: 211-252. http://ertr.tamu.edu/ e-Review of Tourism Research (eRTR), Vol. 16, No. 2/3, 2019 http://ertr.tamu.edu 164 Tsao, WC., Hsieh, MT., Shih, LW., and Lin, TM. (2015) Compliance with eWOM: The influence of hotel reviews on booking intention from the perspective of consumer conformity. International Journal of Hospitality Management 46: 99-111. Urry, J. (1990) The Tourist Gaze. SAGE, London. Vermeulen, IE., and Seegers, D. (2009) Tried and tested: The impact of online hotel reviews on consumer consideration. Tourism Management 30: 123-127. Wang, Y., and Fesenmaier, DR. (2004) Towards understanding members’ general participation in and active contribution to an online travel community. Tourism Management 25: 709-722. Xiang, Z., Du, Q., Ma, Y., and Fan, W. (2018) Assessing reliability of social media data: lessons from mining TripAdvisor hotel reviews. Information Technology & Tourism 18: 43-59. http://ertr.tamu.edu/