International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 13, No. 11, 2019 Paper—Online User Reviews as a Design Strategy for Global Communities: Online User Reviews as a Design Strategy for Global Communities: Contributions of the Open Device Labs Case https://doi.org/10.3991/ijim.v13i11.11042 Raquel Paiva-Godinho Federal Institute of Education, Science & Technology (IFSul), Pelotus, Brazil Ruth S. Contreras-Espinosa (*) University of Vic-Central University of Catalonia (UVic-UCC), Barcelona, Spain ruth.contreras@uvic.cat Abstract—Grass-roots community movements related to design practices and open spaces have emerged to address different issues. This study is part of a comprehensive research, which aims to explain the Open Device Labs (ODLs) ecosystem. The ODLs are a grass-roots community movement that aims to de- mocratize cross-platform tests and evaluation on real devices. As a global community with 152 laboratories located in 35 countries, online user reviews play an essential role in helping the long-term prospects of the movement. From the Design perspective, this paper aims to answer the question: what can be learned about the ODL ecosystem from online user reviews? To answer this re- search question, we conducted a qualitative inductive analysis of n=217 user re- views posted on the community website, from 65 labs located in 12 countries. The results and categories presented here are a key contribution to understand- ing the ODL ecosystem, and ultimately to other global service communities. Keywords—global grass-roots community movement, Open Device Labs, online user review, service user experience evaluation 1 Introduction The Web, since near its beginning was intended to be universally accessible. Web- based interactive content based on specific design standards has become and remains possible because of global collaboration. The World Wide Web Consortium (W3C) founded in 1994 aims to develop protocols and guidelines for an open and accessible Web for everyone. All the aspects of the Web 2.0 paradigm, from its foundation through its evolution, the openness, the global reach, the architecture of participation, the harnessing of collective intelligence, led to a breadth of new interface design pos- sibilities and challenges. Fragmentation issues and concerns about it have been running side by side with the evolution of the Internet and the Web. The diversity of hardware and software makes 70 http://www.i-jim.org https://doi.org/10.3991/ijim.v13i11.11042 https://doi.org/10.3991/ijim.v13i11.11042 https://doi.org/10.3991/ijim.v13i11.11042 Paper—Online User Reviews as a Design Strategy for Global Communities: people experience and visualize content differently. The development of smart mobile devices, and the mobile software industry, extended the challenges to the software development life-cycle. All these issues are addressed by different actors and commu- nities working together in one way or another across the globe. Our professional interest in the evaluation of mobile user interaction led us to ex- plore gaps in this topic, related to the practices of Design community. Lab-based versus field-based evaluation is an issue already addressed by the Human-computer Interaction (HCI) research community. On the other hand, other topics such as the emergence of open lab grass-roots communities are recent and underexplored. These communities act locally and connect globally, presenting challenges from multiple approaches. Identifying and gathering data about these complex and multifaceted phenomena is as important as it is difficult. In 2011, the demand for a way of facing browser fragmentation challenges led to the emergence of the Open Device Labs (ODLs). The ODLs are a movement to pro- mote free access to physical labs equipped with a pool of mobile devices connected to the internet to Web, app, and game testing. This idea arose from Web developers contact at international Web conferences. After the opening of a few physical open labs in different countries, their online presence began to be established, and connec- tions were made between them. Firstly, a list of ODLs was made. After that a Google Groups was set up, and then a community website was established, and this was the beginning of the online global network. We have been conducting case study research on the Open Device Lab global community movement since 2015 [1]. The comprehensive study aims to explore, describe and explain the ODLs ecosystem. The first phase of analysis was conducted on the online documents that were available, the second on interviews with ODL managers and collaborators and the third phase on user reviews. The previous phases of our research collected data on the ODL hosts perspective. This paper presents the first results of the ODLs guests perspective through online user reviews a key design strategy for helping in the running of the Open Device Lab community movement, and aims to explore what can be learned from the ODLs eco- system. 2 Literature Review The digitalisation of communications has gone through two phases a) the spread of the World Wide Web in the 1990s and b) the transformation of the Web itself passing from web 1.0 to web 2.0 in the 2000s; and the big bang of new mobile devices, from smartphones to tablets [2]. The gradual introduction of mobile phones in the 80's, its evolution and the Web 2.0 principles converge to create the mobile Web 2.0 phenom- enon [3]. iJIM ‒ Vol. 13, No. 11, 2019 71 Paper—Online User Reviews as a Design Strategy for Global Communities: 2.1 The web (2.0) The change in the perspective of understanding the Web as a service platform was the key aspect of the Web 2. The term Web 2.0 was coined by Darcy DiNucci [4], and Tim O’Reilly published the first initiative to define it and understand its implications for future generations of software summarising the core competencies of Web 2.0 companies as follows [5]: • Services, not packaged software, with cost-effective scalability, • Control over unique, hard-to-recreate data sources that get richer as more people use them, • Trusting users as co-developers, • Harnessing collective intelligence, • Leveraging the long tail through customer self-service, • Software above the level of a single device, • Lightweight user interfaces, development models, AND business models. The Web 2.0 architecture of participation is a term coined by Tim O’Reilly to de- scribe the nature of systems that are designed to encourage user contribution [6]. In this context, collective intelligence refers to Web-based software which help people share, collaborate and interact [7]. Systems designed to find out what people think about a product or service led to the increase of user-generated content phenomenon [8]. Many different disciplines comprise user engagement, recommender systems and related terms. Recommendation systems have been studied in diverse areas such as social-based recommender systems [9], user expertise through online reviews [10], expert opinion [11], customer reviews and automated recommendation systems [12], which are also deployed in industry for product and service recommendation [9]. Online customer reviews is defined by Ref. [12] as “peer-generated product evalua- tions posted on company or third-party websites” and the authors explain how retail websites use it to offer consumers the opportunity to post product reviews in the form of numerical star ratings (usually ranging from 1 to 5 stars) and open-ended customer- authored comments. Therefore, referring to future work, the study recommended an analysis of text reviews compared to the start rating to determine how the stars match the review content. We considered the authors recommendations and applied it to our research, which analyses service reviews, from current design perspectives. Moreover, this study presents the global community running challenges using user experience and service design concepts. 2.2 Service Design and User Experience Interaction Design is the umbrella term for concepts like user interface design, us- er-centred design, and experience design, which designs experiences to support peo- ple communication and interaction in their everyday and working life [13]. User experience (UX) has different meanings and definitions and, in this study, we adopted the term as a measurement concept. UX goals are divided by Ref. [13] in 72 http://www.i-jim.org Paper—Online User Reviews as a Design Strategy for Global Communities: desirable and undesirable aspects according to user feelings. UX is concerned about the user feeling, pleasure and satisfaction when interacting with a product or using a service related to the overall impression [13]. According to Ref. [14] “In user experi- ence design multiple components must be designed: visuals, features and commands, copywriting, information architecture, and more. Not only should each component must be designed correctly, but they also be integrated to create a total user experi- ence. Service design follows the same basic idea”. In the context of online reviews, experience “is some quality that users gain over time, as they consume, rate, and re- view additional products” [10]. Service Design is concerned with making services better suit the user and customer needs examining all activities, infrastructure, communication, people, and material components, to improve the quality of service and the interactions between the pro- vider of the service and its customers. It aims to suit the customer need while seeking to be sustainable for the service provider, and ideally being user-friendly and competi- tive within their market [15]. Service Design components According to Ref. [14] there are three main compo- nents of service design: Props – physical or digital artefacts, including products, used to perform the ser- vice successfully. It considers: • Physical space (storefront, teller window, conference room), • Digital environment through which the service is delivered (webpages, blogs, so- cial media) • Objects and collateral (digital files and physical products) People – anyone who creates, uses, or is indirectly affected by the service (em- ployees, customers, fellow customers encountered throughout the service, and part- ners). Processes – workflows, procedures, or rules needed to perform the service success- fully. These components are divided into frontstage (channels, products, touchpoints, in- terfaces) and backstage (policies, technology, infrastructure, systems) depending on the customer perspective. Service Design is concerned to “meet the customer’s needs in the most relevant way whilst remaining economic (or sustainable) for the service provider” [15]. Considering all the characteristics presented above Service Design was identified as an approach that meets the aspects of the Open Device Labs free service aspects and needs. 2.3 Open Device Labs (ODL) The ODLs are a grass-roots community movement. They are a global voluntary initiative, which establishes physical device labs as a free service for mobile testing and evaluation on real devices. "In result, ODLs lead to an ultimate improvement of the web & app experience both for developers and for consumers" [16]. iJIM ‒ Vol. 13, No. 11, 2019 73 Paper—Online User Reviews as a Design Strategy for Global Communities: In April 2012, the initiative took-off with the first ODL in Brighton, UK. In August 2012, when there were eight ODLs, all in Europe, the first online list of ODLs was published [1]. In January 2013, the opendevicelab.com, a directory of a global move- ment, was published with 37 ODLs across 18 countries with over 500 devices acces- sible [17]. The high number of labs spread across so many countries brought up challenges in terms of global aspects for the people, located in Germany, who were running the global community movement. Users were reaching out to the ODLs runners and com- plaining about device labs, which were listed on the server, were not available or were not there anymore, for example. Thus, one of these key challenges was a) keeping one eye on all the labs and validate if they were set up according to the information listed on the community website b) validate if it was a proper service linked to the basic principles of the community c) to get an idea of how many people were using ODLs services. Therefore, the ODLs team implemented the online user reviews as a strategy to address this problem, namely, if it gave an opportunity to everybody who uses an open device lab to comment, rate, and share if the lab is found as open, if it is a nice place to go and if they provide what they propose to do. It is important to mention the ODLs do not offer a testing service. This means they do not test a product for others. An ODL is a tool for the owner testing purpose, who shares it with the local community as an "open practice". They offer free access to device labs as a type of "self-service". The hosts use the device lab to test and evalu- ate their own products. Additionally, external people are invited to use the lab for their own purposes by appointment. It is these aspects that frequently cause hosts to consider that the ODLs are not a service, although the evidences presented below shows that for guests it is clear that ODLs are a service. Thus, in this study, in which the guests evaluate their overall satisfaction, we will use the term service as a central category. The OpenDeviceLab.com The online directory serves to three major goals (ODL 2013): • Help people to locate the right Open Device Lab for the job • Explain and promote the Open Device Lab movement, and • Attract contributors and sponsors to help and donate to ODLs. The fast growth of ODLs across the globe resulted in 157 ODLs located in 35 countries being registered in March 2016. The online user reviews form has 4 areas: 1 - user identification 2 - ODL status situation 3 - rate overall satisfaction (1-5 stars) 4 - Comments (see Figure 1.and Figure 2) 74 http://www.i-jim.org Paper—Online User Reviews as a Design Strategy for Global Communities: Fig. 1. Screenshot of the ODLs online user review Fig. 2. Screenshot of the online user review online form iJIM ‒ Vol. 13, No. 11, 2019 75 Paper—Online User Reviews as a Design Strategy for Global Communities: 3 Research Method Based on the global research approach, this study was designed to address the main research question: what can be learned about the ODLs ecosystem from the online user reviews? 3.1 Sampling and study population At the time we collected the data, there were 152 labs registered on the webpage community. We worked on a homogeneous sample without concern for statistical generalisability [18]. We classified the ODL’s users in two main categories: hosts and guests. The hosts are the organisations and / or institutions (companies, co-working or education institutes) which host the lab. The guests are external people who make use of the lab for free (individuals or companies such as freelance designers and develop- ers, students, and web agencies). This study is based on the Open Device Lab global movement guest users. 3.2 Data collection From the 152 labs, we have selected and collected data from the labs that had user reviews published. We collected all the data available online from 7th to 9th May 2017 and have been updating this until 8th January 2018. The collected sample represents a total of n=217 reviews from 65 labs located in 12 countries: Australia, Belgium, Bra- zil, Finland, Germany, India, Poland, Spain, Thailand, United Kingdom, United States, and Uruguay. All data was captured with the original layout as selectable text in pdf files (Figure 3). Fig. 3. screenshot of ODLs user reviews 3.3 Data analysis We conducted a general inductive approach for qualitative analysis [19] using At- las.ti. This approach provides a systematic set of procedures to: • Condense raw textual data into a brief, summary format 76 http://www.i-jim.org Paper—Online User Reviews as a Design Strategy for Global Communities: • Establish clear links between the evaluation or research objectives and the sum- mary findings derived from the raw data • Develop a framework of the underlying structure of experiences or processes that are evident in the raw data. In the analysis processes, first, we performed a pilot study analysing the data per lab reviews, but there were labs with 20 reviews and others with only one. Therefore, for the final analysis, we decided to consider each review as a document and not each lab. We imported the 217 pdf documents, user reviews, to the software. We searched for data not written in English, translated and reviewed it. We conducted the primary-cycles coding using descriptive and in vivo methods [20] in which we first gathered “what’s going here” [21] and defined the specific research questions to answer the main RQ: RQ1: What is the ODLs guest users’ rate and comment about? RQ2: What is being tested by guest users? RQ3: What is the overall satisfaction of the ODLs guest user experience? RQ4: What are the ODLs guest users’ profiles? RQ5: According to the ODLs guest user, what are the good and bad practices? In addition to the first-cycle coding, we created a first model which was later re- fined in the second-cycle coding based on the disciplinary and theoretical concepts [21]; Design lens presented in the previous section. The In vivo method was essential to identify similar technical concepts described differently by participants like web, web-based, web app. 4 Results The results are explained in the following sections according to each research ques- tion, and the hierarchical category system in Figure 6. 4.1 RQ1: What is the ODLs guest users’ rate and comment about? In terms of numbers, the global results of the user rating about overall satisfaction show that the majority of them had a pleasant and satisfactory experience; rating it with five stars, n=186. A few labs received four stars, n=18. There was no lab rated with three stars, n=0, one lab received two stars, n=1, and a few received one star, n=12 (see Table 1). These numbers are related to 65 ODLs located in 12 countries, Australia, Belgium, Brazil, Finland, Germany, India, Poland, Spain, Thailand, United Kingdom, United States, and Uruguay. The UK and Germany have the higher number of user reviews published. iJIM ‒ Vol. 13, No. 11, 2019 77 Paper—Online User Reviews as a Design Strategy for Global Communities: Table 1. Results of the rating and status categories per user reviews n=217 and per ODLs n=65 Category Code Per user review per ODL Rate ● URW - R-1S 12 11 ● URW - R-2S 1 1 ● URW - R-4S 18 14 ● URW - R-5S 186 52 Status ● Open 204 54 ● Closed 13 12 Totals 217 65 URW means user reviews, R means rating, and S means stars The results of the qualitative analysis related to these numbers are presented below, according to each category. There was no ODL rated with three stars, and considering the inductive analysis approach, there was no coding for this case. The final discus- sion is presented at the end of the results sections. URW – R-1S The labs rated with one star, n=12, are all almost found as closed, n=11, a proper and justified rate; e.g. "It has closed". Only one lab, n=1, was rated with one star and found open. The user’s comment about his/her experience started with the positive aspects of the space, but the primary purpose of the ODLs, the devices were considered negative: "Great digital workspace: Nice and quiet with free vending machine coffee, but the device lab has a couple of odd chinese knockoffs and a cheap htc all with more securi- ty cables than an iPhone 6 on the outside of a phone shop on a council estate. Don't waste your time trying to find this space... That'll teach me trying to do some work whilst on a weekend away!" URW – R-2S The lab, n=1, which received two stars, was found open, in terms of status, and the user comment was not much help; the guest just wrote: "Good'. URW – R4S The labs which received four stars, n=18, were all found to be open and had com- ments about positive experiences, detailed in the following paragraphs; e.g. "All de- vices I pre-booked were available, apart from one Samsung which was broken". URW – 5S The labs rated with five stars, n=186, were almost all found as open but there were two reported as closed. In one of these cases, it seems the user selected the wrong choice in terms of status in the form. First, because it was rated with five stars and second because the comment says "Thanks". The other case was about a user who was in an ODL, which was moving to another place, so even if this user had a five-star experience, the lab would be closed. After explaining the relationship between the ODLs rating and their status briefly, we present an example of the final qualitative categories according to its data analysis process cycles (see Figure 4 and Figure 5). 78 http://www.i-jim.org Paper—Online User Reviews as a Design Strategy for Global Communities: Fig. 4. Results of the main themes from the first-cycle coding analysis, Atlas.ti Fig. 5. Results of the main themes from the second-cycle coding analysis, Atlas.ti STATUS – Refers to open or closed labs. • Open – Refers to the ODLs found open. • Closed - Refers to the ODLs found closed. Although reporting an ODL as closed depends of different situations: Fact – Refers to an unquestionable source like directly from the ODL. Based on conclusion – Refers to different reasons (see Figure 6). SERVICE – Refers to the guest user comments about the ODL service. • Props - Is about the venue, the devices, the lab as space, software, Wi-Fi, cable and extra facilities like drinks (coffee, tea, water etc.). • People – Refers to the comments about the ODLs teams, which are usually posi- tive and more about being friendly, helpful and experts on testing. Therefore, it is not about going there and just using the infrastructure. It is also about knowledge exchange, which was classified as a benefit in a subcategory. • Processes – Refers to booking system, communication aspects and devices ar- rangement. • User experience - Refers to the guest user impression about the ODLs service experience. • Testing experience – Refers to the guest comment specifically about the testing experience at the ODL. • Benefits – Refers to the guest user comments about the service benefits. TESTING – Refers to technical issues about the testing field in general, not relat- ed to the ODLs services such as methods, purposes, and tested products. The main themes presented above and all the lower-level categories, which answer all the specific research questions, are presented in the hierarchical category system (Figure 6). iJIM ‒ Vol. 13, No. 11, 2019 79 Paper—Online User Reviews as a Design Strategy for Global Communities: Fig. 6. ODLs online user reviews hierarchical categories system 80 http://www.i-jim.org Paper—Online User Reviews as a Design Strategy for Global Communities: 5 Discussion Status is simple and handy information for both hosts and guests. First, there is the fact that when a guest decides to go to the ODLs website and register their opinion, they are primarily contributing to their local community, helping to verify if an ODL is open or closed. It is key information for the locals and the people running the global movement. Validating if an ODL is open or closed around the world would be invalu- able for a small team or an individual. The closed labs are more challenging to verify because there are cases where the ODL is listed but is not available anymore. There are also the ones who are working but not updating their online presence or the ones who are in standby or in a similar situation. Therefore, a lab reported as closed by a user depends on many different conclusions. Validating the ODLs status as open or closed is the primary contribution of the online user reviews, which allows the guest to complain about inconsistences with the information published on the community website. The rating star system is an objec- tive evidence about the guest overall satisfaction with the service. Besides specific exceptions, ODLs rated with one star are clearly related to closed labs and labs rated with two to five stars are related to open labs. Considering the ODLs are a voluntary movement and offer a free service, it is re- markable to observe in the results of the open-ended comments how demanding the guest users are. The comments help to understand the guest user experience; the de- sirable and undesirable aspects; what was positive and what was negative about it to guide recommendation about good, bad and key practices. Moreover, the comments help to understand what guest value more about an ODL, what kind of products they are testing and in which way, what are their profile, and issues needing to be im- proved. Service helped us to understand, for example, the user preferences about the devic- es organisation. There are some labs, which do not have the devices on a stand and ready to go; the devices sometimes are in boxes. When we were seeking for good practices on device management, it was not clear what the best options were. The user review evidence shows that to the guests it is high valued to get the requested devices charged and ready to go when they get to the lab, even if it is a free service. Considering all the themes evaluated by the guests, the overall satisfaction is posi- tive. The findings confirm that a representative pool of devices is fundamental. At the same time, the people, a friendly and knowledgeable team, provide high added value to the ODLs service. The booking and structure arrangement for the testing perfor- mance are reported as very satisfactory. The ODL hosts could also share information about ODLs processes. As a community movement, we believe examples of a service design plan would help the entire community. The help of the ODL team and the opportunity of knowledge exchange and exper- tise sharing related to testing issues during the time spent at the lab is not something promoted online, but the user reviews findings show the hosts are of service and this is something highly valued by the guests. There were cases of helping first visitors and first time testing on multiple devices. The evidence shows the guests value getting help from the hosts and at the same time, they value their privacy to conduct the tests. iJIM ‒ Vol. 13, No. 11, 2019 81 Paper—Online User Reviews as a Design Strategy for Global Communities: We previously knew for what purpose the ODLs implemented the online user re- views. Thus, analysing these data, we could learn key aspects of the ODL ecosystem from its global users. The ODLs guest user overall satisfaction is related to open sta- tus, free access to a representative pool of mobile devices ready to go, in a place which combines privacy with knowledge and expertise sharing. To sum up, we could gather global information about: • ODLs status – the foreseen categories open or closed and unexpected ones like temporarily closed. • The overall guest level of satisfaction about the ODLs, and with what is related to, both common and special cases. • Indications about testing approaches and tested products. • Hints on guest profile. • Information about positive and negative service, testing practices, and ODLs bene- fits. In conclusion, we may say the user review is essential both to the long-term of the local and global community. The results can be used as the base to create another version for the user reviews rate and comments in a semi-structured way based on the categories we found; not to replace the current open-ended text field but as an option for users who would be willing to contribute through a more informative and struc- tured form. It would be a way to gather more and comparable data from the ODLs and improve the quality of service, the interactions between the hosts and the guests, and ultimately the user experience. 6 Limitations and Conclusions This study is part of global research, which aims to explain the Open Device Labs ecosystem through a qualitative case study. This paper presents the main themes from the ODLs online user reviews (rating and open-ended comments) related to an evalua- tion of service user experience. An inductive analysis was conducted based on 217 reviews from 65 labs located in 12 countries. Online user reviews are a useful strategy both for the hosts and for the guests. The sample is limited in number, but considering the nature of the study and the open-ended and spontaneous comments, the findings are crucial to the primary research. It is the first step to understand the ODLs guest experience; without it, we would only have the hosts’ perspective, not the guests. The user reviews added a clear perspective of the ODLs as a service, a different view to previous data analysis, which led to select the Service Design as a proper approach to improve ODLs service user experience, for employees and guests. As a final contribution, the categories and qualitative data may help to build service design strategies for labs and communities in general. 82 http://www.i-jim.org Paper—Online User Reviews as a Design Strategy for Global Communities: 7 Future Work Primarily, we will use this study results to present the data triangulation of the comprehensive research on the Open Device Lab community ecosystem. Second, the results will be used to work on semi-structured user reviews and service design strate- gy. Additionally, we still have much to learn about the ODLs due to its openness nature, global reach and there are few studies about this. The identified gaps in this study provide opportunities for further research focused on service design and user experience. 8 Acknowledgement We would like to thank the Open Device Lab community for allowing us to con- duct the comprehensive research, which this study is part of and partially supported by the IFSul, the UVic-UCC, the Erasmus+ program, and by the BBVA bank group and Antiga Caixa Manlleu Foundation. 9 References [1] Author. 2015. [2] C. A. Scolari, J. M. Aguado, and C. Feijóo, “Mobile Media: Towards a Definition and Taxonomy of Contents and Applications,” Int. J. Interact. Mob. Technol., vol. 6, no. 2, pp. 29–38, 2012. https://doi.org/10.3991/ijim.v6i2.1880 [3] P. Kuklinski and J. Pablo, “Mobile Web 2.0. theoretical-technical framework and develop- ing trends,” Int. J. Interact. Mob. Technol., vol. 2, no. 4, 2008. [4] Darcy DiNucci, “Fragmented Future,” Print Mag., vol. 53, no. 4, pp. 221–222, 1999. [5] T. O’Reilly, “What Is Web 2.0: Design Patterns and Business Models for the Next Genera- tion of Software,” Commun. Strateg., vol. 65, no. 1st quarter, p. 17, 2007. [6] T. O’Reilly, “What Is Web 2.0,” O’Reilly Media, 2005. [7] D. Bollier, “The rise of collective intelligence - decentralized co-creation of value as a new paradigm of commerce and culture,” 2007. [8] E. Platzer and O. Petrovic, “Learning Mobile App Design from User Review Analysis,” Int. J. Interact. Mob. Technol., vol. 5, no. 3, Jul. 2011. https://doi.org/10.3991/ijim.v5i3.1 673 [9] H. Ma, D. Zhou, C. Liu, M. R. Lyu, and I. King, “Recommender Systems with Social Regularization,” in WSDM, 2011, pp. 9–12. [10] J. Mcauley and J. Leskovec, “From Amateurs to Connoisseurs: Modeling the Evolution of User Expertise through Online Reviews,” in WWW, 2013, pp. 13–17. https://doi.org/10.11 45/2488388.2488466 [11] X. Amatriain, N. Lathia, J. M. Pujol, H. Kwak KAIST, and N. Oliver, “The Wisdom of the Few A Collaborative Filtering Approach Based on Expert Opinions from the Web,” in SIGIR, 2009, pp. 19–23. https://doi.org/10.1145/1571941.1572033 [12] S. M. Mudambi and D. Schuff, “What makes a helpful online review? a study of customer reviews on amazon.com,” MIS Q., vol. 34, no. 1, pp. 185–200, 2010. https://doi.org/10.2 307/20721420 iJIM ‒ Vol. 13, No. 11, 2019 83 https://doi.org/10.3991/ijim.v6i2.1880 https://doi.org/10.3991/ijim.v6i2.1880 https://doi.org/10.3991/ijim.v5i3.1673 https://doi.org/10.3991/ijim.v5i3.1673 https://doi.org/10.1145/2488388.2488466 https://doi.org/10.1145/2488388.2488466 https://doi.org/10.1145/1571941.1572033 https://doi.org/10.1145/1571941.1572033 https://doi.org/10.2307/20721420 Paper—Online User Reviews as a Design Strategy for Global Communities: [13] Preece, Rogers, and Sharp, Interaction Design: beyond human-computer interaction, 4th ed. Wiley, 2015. [14] S. Gibbons, “Service Design 101,” NN/g Nielsen Norman Group, 2017. [Online]. Availa- ble: https://www.nngroup.com/articles/service-design-101/. [Accessed: 25-May-2019]. [15] Interaction Design Foundation, “What is Service Design?,” Interaction Design Foundation, 2018. [Online]. Available: https://www.interaction-design.org/literature/topics/service- design. [Accessed: 14-Oct-2018]. https://doi.org/10.7551/mitpress/9780262037518.003.0 005 [16] ODL, “OpenDeviceLab.com - Locate, Contribute to and Sponsor an Open Device Lab (ODL),” 2018. [Online]. Available: http://www.opendevicelab.com. [17] Andre Jay Meissner, “OpenDeviceLab.com – a directory to a global movement,” klick- ass.com, 2013. [Online]. Available: http://klick-ass.com/awesomeness/opendevicelab-com- a-directory-to-a-global-movement/. [Accessed: 08-Jun-2019]. [18] A. Crossman, “Understanding Purposive Sampling: an overview of the method and its ap- plications.,” ThoughtCo., 2018. [19] D. R. Thomas, “A General Inductive Approach for Analyzing Qualitative Evaluation Da- ta,” Am. J. Eval., vol. 27, no. 2, pp. 237–246, 2006. [20] J. Saldaña, The Coding Manual for Qualitative Researchers. London: Sage, 2009. [21] S. J. Tracy, Qualitative Research Methods -collecting evidence, crafting analysis, com- municating impact. Wiley-Blackwell, 2013. 10 Authors Raquel Paiva Godinho is a tenured lecturer and researcher at the Design Depart- ment, Federal Institute of Education, Science and Technology Sul-rio-grandense (IFSul), Brazil. Currently, she is a member of the Balmes Foundation team involved in a H2020 project, and a Ph.D. candidate in Experimental Science and Technology at the University of Vic-Central of Catalonia (UVic-UCC), Spain. Her current research interests include design and collaborative practices. raquelpg@pelotas.ifsul.edu.br Dr. Ruth Sofia Contreras Espinosa is a Professor at the Faculty of Business and Communication, University of Vic-Central of Catalonia (UVic-UCC), Spain and a Project Manager in H2020 European projects. She is also coordinator of Observatory of Communication, Video Games and Entertainment (OCVE), InCom-UAB-UVIC. Her current research interests include game studies, game user experience and games user research. Article submitted 2019-06-16. Resubmitted 2019-07-23. Final acceptance 2019-07-26. Final version published as submitted by the authors. 84 http://www.i-jim.org https://doi.org/10.2307/20721420 https://doi.org/10.7551/mitpress/9780262037518.003.0005 https://doi.org/10.7551/mitpress/9780262037518.003.0005