Journal of Software Engineering Research and Development, 2019, 7:8, doi: 10.5753/jserd.2019.155  This work is licensed under a Creative Commons Attribution 4.0 International License.. On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Eduardo Pinheiro  [ Universidade Federal de São Carlos | edu.g.pinheiro@gmail.com ] Larissa Lopes  [ Universidade Federal de São Carlos | larii.albano@gmail.com ] Tayana Conte  [ Universidade Federal do Amazonas | tayana@icomp.ufam.edu.br ] Luciana Zaina  [ Universidade Federal de São Carlos | lzaina@ufscar.br ] Abstract Context: Requirements elicitation phase in software development investigates both requirements, functional and user experience (UX). Proto-persona is a technique that encourages attention on the needs of a group of users. Usu- ally, the elaboration of proto-personas is done by software specialists and technical stakeholders without the par- ticipation of non-technical stakeholders. However, non-technical stakeholders often have a well-knowledge about target users. Objective: This work aims to investigate the contribution that non-technical stakeholders bring to the specification of UX requirements when they use the proto-persona+ technique to this end. To achieve our objec- tive, we extend the original proposal of proto-persona technique creating the proto-persona+. We also explored the construction of proto-persona+ artifacts and their use to the prototyping of solutions. Method: We conducted an empirical study in two rounds, wherein we analyzed and compared the contributions of technical and non-technical stakeholders on the specification of UX requirements. In the first round, 8 non-technical and 5 technical stakehold- ers built the proto-personas+. In the second round, 36 software developers worked in pairs to create low fidelity prototypes using the information provided by the proto-persona+ artifacts. For the two rounds, we conducted a qualitative analysis exploring which UX requirements were described and used. Results: The results revealed that although both types of stakeholders had written the details of UX requirements on the artifact, they did in different and complementary perspectives. We could also observe that the proto-persona+ artifacts that were produced by both stakeholders were used on the prototyping activity. Conclusion: Our study indicates that non-technical stake- holders are able to contribute to the specification of UX requirements and that proto-persona+ is a suitable artifact to promote such activity. The details described by non-technical stakeholders brought new and different contributions when compared to the ones described by the technical stakeholders. From the results of the first round, we con- cluded that the non-technical stakeholders elicited requirements which impact on accessibility and fun issues. By considering the findings of the second round, we concluded that the UX requirements provided by both stakeholders allowed the developers to build more comprehensive and minimalist user interface prototypes. Keywords: Non-technical stakeholder, Proto-personas, Requirement Engineering, UX Requeriments 1 Introduction Requirements elicitation is widely discussed in software en- gineering. The challenges of this important area of software development include issues that range from technical aspects (e.g. use of appropriate tools) to human aspects (e.g. type of stakeholders involved in the process), Sharma and Pandey (2014); Aranda et al. (2016); Abelein et al. (2013); Hadar et al. (2014). Some works have highlighted that the involvement of end- users in the elicitation process can bring important contri- butions to software construction and that consequently, this affect user satisfaction on the software, Berti et al. (2004); Maceli and Atwood (2011). Additionally, the authors stated that the process of requirements elicitation can be enriched not only by the participation of end-users but also by in- cluding different stakeholders in this process. Non-technical stakeholders are recognized as those who are not a part of the software team, Hadar et al. (2014). These stakeholders can be professionals that have close contact with the end-users, for instance. Nevertheless, they often have much knowledge about the audience and the domain of the application, Aranda et al. (2016). During the elicitation process, a diversity of types of re- quirements can arise. Non-functional software requirements, such as usability and user experience, are linked to quality- related requirements; therefore they can impact software ac- ceptance by end-users, de la Vara et al. (2011); Palomares et al. (2017). Nielsen and Norman (2013) state a definition of user ex- perience (UX) in a holistic perspective: “User experience encompasses all aspects of the end-users interaction with the company, its services, and its products”. Differently, in a more pragmatic definition, Garrett (2010) states that for a product provide a good user experience, the software de- velopers have to pay attention to what the product does and how it does it. Considering both definitions above, we can affirm that the elicitation of UX requirements involve the gathering of aspects and characteristics of the end-user and the product. These requirements should assist the technical stakeholders (i.e. software experts) in designing and devel- oping software that has good acceptance and brings value to end-users, Brown et al. (2011); Kashfi et al. (2017). Technical stakeholders can be supported by several tech- niques and methods to the eliciting UX requirements. For this purpose, questionnaires, interviews, as well as tech- niques and methods from the human-computer interaction (HCI) area can be applied, Garcia et al. (2017); Brown et al. https://orcid.org/0000-0001-5719-4852 mailto:edu.g.pinheiro@gmail.com https://orcid.org/0000-0001-5189-0291 mailto:larii.albano@gmail.com https://orcid.org/0000-0001-6436-3773 mailto:tayana@icomp.ufam.edu.br https://orcid.org/0000-0002-1736-544X mailto:lzaina@ufscar.br On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 (2011). Personas are artifacts that have applied to support software teams in both activities, elicitation and use of UX re- quirements, Ferreira et al. (2015). The technique to create the personas follows a process that analyzes end-users data. The persona artifact generated from the technique consists of a fictional character that represents a group of real users of the system and their relevant characteristics within a given soft- ware domain, Gothelf (2012); Grudin (2006); Cooper et al. (2014). Additionally, personas are useful for establishing an em- pathy relationship between technical stakeholders and end- users, Grudin and Pruitt (2002); Billestrup et al. (2014). How- ever, the application of personas can be onerous and costly to the team. By the classical definition, a persona is cre- ated by analyzing a significant amount of data regarding end-users that requires extensive research and data collec- tion, Billestrup et al. (2014). Gothelf proposes a new approach to elaborating personas called proto-persona1, Gothelf (2012); Gothelf and Seiden (2013). Rather than using the classical technique to creat- ing personas, Gothelf’s proposal considers the prior knowl- edge of stakeholders about end-users and the software do- main in question. The technique to constuct proto-persona recognizes that these stakeholders are able to build a sketch of a persona with their assumptions based on their knowl- edge about a given domain. The technique of constructing proto-personas provides a practical way to gather the knowl- edge that the stakeholders have about end-users. However, the author recommends that the proto-persona artifact should be validated later by conducting end-user research, Gothelf (2012). Usually, technical stakeholders work on the development of a diversity of software, which can bring difficulties in ob- taining in-depth way the knowledge about different software domains. Furthermore, non-technical stakeholders (i.e. who are not a part of the software team) are those who have knowl- edge about a given domain and can provide relevant informa- tion about end-users and the aspects of their interaction with the software. Considering the aforementioned discussion, we decided to study the use of proto-personas to elicit UX requirements in the perspective of non-technical stakeholders. Our study was focused on investigating how non-technical stakehold- ers contribute to the requirements elicitation activity. To do this, we selected the proto-persona technique, that produces a lean artifact which can be easily used by this type of stake- holders. The intention of this study is not focus on the com- parison of different personas techniques, but to collect evi- dence about the potential of use proto-persona technique to the purpose of elicitating requirements. To support our study, we extended the proto-persona tech- nique proposed by Gothelf (2012) and Gothelf and Seiden (2013) creating the proto-persona+. Developers frequently report that they struggle on how to arrange information of per- sona, Billestrup et al. (2014). Considering the difficulties that the participants would have to handle with the proto-persona technique, in our extension we included a new template and 1Proto-persona is also known as Lean Persona, Gothelf and Seiden (2013). guide questions which support the individuals that will use the proto-persona+. The construction of the proto-personas is supported by the template and the questions that guides the participants to the writing of the personas. Taking into ac- count the basis of the Gothelf’s proposal, Gothelf (2012), the template outlines the important points that individuals should have during the design of the proto-personas. To conduct our study, we defined three research ques- tions (RQs): (RQ1) Which UX requirements do non-technical stakeholders describe while using the proto-persona tech- nique?; (RQ2) How is the acceptance of the use of the proto-persona+ technique by these stakeholders?; and (RQ3) Which UX requirements presented in the proto-personas+ can support the prototyping of user interfaces?. We con- ducted two rounds of experimental studies. Firstly, we ex- plored the use of the technique to construct the proto- persona+ artifact with the participation of 8 non-technical stakeholders and 5 technical stakeholders; and consequently, we could answer the RQ1 and RQ2. To respond to RQ3, we invited 36 software developers to design user interface pro- totypes by using the proto-personas+ artifacts that were pre- viously created in the first round. The participants worked in pairs and produced 18 user interface prototypes in total. From this second round, we were able to examine the use of the proto-personas+ that was previous developed by the different stakeholders (i.e. technical and non-technical). This paper presents these two rounds in details and discusses the results. In this paper, we extend our previous result presented in the Brazilian Symposium on Software Engineering in 2018. In the earlier version, we have discussed the results regarding the RQ1 and RQ2. In this version, we added a new perspec- tive of analysis (related to RQ3) that enriched our findings re- garding the contributions of non-technical stakeholders and the potential of proto-persona to support the elicitation of UX requirements. The process of selecting of individuals to participate in the study as non-technical stakeholders was directly related to the domain that our research focused on. Our domain was defined as: Applications to support e-learning, and con- sequently, pedagogues2 were the non-technical stakehold- ers. As our research group have experience in the devel- opment of applications in the e-learning domain, we cre- ated a network of contacts with pedagogues, (i.e., potential non-technical stakeholders) which was the key factor to our choice. The study allowed us to observe how the stakeholders described UX requirements by applying the proto-persona+ technique and how these artifacts were used to design soft- ware solutions. Our main contribution is the discussion of the feasibility in introducing the non-technical stakeholder as an active agent in the specification of UX requirements through the use of the proto-persona technique. Our study not only examines the construction of the artifacts but also their use in the elaboration of software. The rest of the paper is organized as follows: Section 2 presents the fundamentals and related work; the proto- 2In Brazil, pedagogues are professionals who are responsible for the education of children in elementary schools; they obtain their degrees by attending a pedagogy course. On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 persona+ artifact is presented in Section 3; the domain se- lected to the study and scenario we applied in are explained in Section 4; the details of the first round of investigation are discussed in Section 5 and its results are in follow Section 6; the second round of investigation and its results are presented in Section 7 and Section 8, respectively; in Section 9 we re- turn to our research questions to point out the important re- sults and make a comparison with the literature; the main lim- itations of our study is pointed out in Section 10; and finally Section 11 presents the conclusion and future work. 2 Fundamentals and Related Work Requirements elicitation can be considered a complex task that often requires the participation of different stakehold- ers. These stakeholders contribute with different knowledge in this process, Fernandez and Wagner (2015). Recently, the identification of UX requirements during requirements elici- tation has become a trend, Castro et al. (2008); Ferreira et al. (2015, 2018a); Choma et al. (2016a,b). Personas allow the production of artifacts in which UX related issues, such as personal characteristics, needs, and restrictions of end-users, are described, Cooper et al. (2014). Personas are recognized as important artifacts by both of professionals, academics and practitioners, Billestrup et al. (2014). It can support teams during the software devel- opment by providing important insights about end-users, Fer- reira et al. (2018b). Another benefit of this technique is to place the user at the center of the development process, which keeps the teams informed about end-users’ require- ments. Frequently, software teams have personal assump- tions about end-users’ characteristics that may differ from the users’ needs in real life, Jansen et al. (2017). The team can predict user behavior in a more pragmatic perspective by using persona in their activities. Therefore, persona plays the role of developing the empathy of developers toward end- users, Cooper et al. (2014); Grudin (2006). Alves and Ali (2018) applied goal-oriented require- ments engineering (GORE) together with the personas tech- nique to enrich the specification of human factor require- ments. GORE is focused on fulfilling the demands regarding business goals. The authors stated that by including personas in the process, they could improve the specification of the users’ needs in the software with more assertive and specific details. Consequently, they could satisfy the needs of groups of real end-users. Gothelf (2012) proposes proto-personas as a technique in which the domain-specific knowledge that specialists have about the audience is used to describe personas. The tech- nique run from a series of brainstorming sessions, Osborn (1979), wherein each participant (i.e. specialists) proposes the personas individually. In the next step, these initial pro- posals are refined by all the participants in the session un- til they produce a maximum of four personas that represent the target audience. Afterwards, the software teams apply these sketches of personas during the software development. These sketches can be validated in future development cy- cles. The proto-persona technique has the main goal of cap- turing the knowledge of the experts and uses it in the writing of the proto-persona artifact. This artifact can aid the teams in kicking-off a discussion about the user in the early devel- opment phases(e.g. design phase). In the work of Anvari et al. (2015), the traditional persona technique was used to hold the emotional characteristics of users. The authors’ intention was to verify whether the de- velopers could see the differences among the characteristics of the personas and whether these differences caused some influence during the software design. Results revealed that most participants noticed the variance on the details of those personas and reported that the artifacts helped them in de- signing the software. Ferreira et al. (2015) proposed the PATHY, a technique that adapts the empathy map to the construction of per- sonas. The empathy map provides a different form to the building of personas, wherein the focus is on establishing an empathy relationship between end-users and developers. PA- THY provides a set of questions that drives the software en- gineer to the artifact elaboration . The technique includes the specification of the user characteristics as well as of other software features. Subsequently, Ferreira et al. (2018b) in- vestigated the feasibility of combining the PATHY technique with user stories to support software development. The re- sults suggested that PATHY helps the team in understand- ing the context of use, identifying potential software require- ments, and integrating personas into the design and develop- ment process. Bhattarai et al. (2016) applied the proto-persona technique in the construction of user profiles. The experience was con- ducted in several sessions with the participation of different developers. The findings showed that proto-persona supports the teams in aligning their point of view about the software to a set of testable hypotheses about consumers or end-users. Kortbeek (2016) presented an experience of using the Gothelf technique to build and communicate the hypothesis of a user in industry context. Later, in order to verify whether the hypothesis reflected information about the end-users, in- terviews were conducted with users that have the same char- acteristics found in the proto-persona. Unlike previous works, this paper presents not only the application of proto-persona+ technique but also discusses the contribution that the non-technical stakeholder brings to the specification of UX requirements while using the proto- persona technique. To the best of our knowledge, no prior research investigated the contribution of non-technical stake- holders in elaborating personas. However, there are other works regarding the participation of non-technical stake- holders in different requirements engineering tasks, mainly in End-user Development or End-user Software Engineer- ing context. Berti et al. (2004) discuss how scenarios and sketches can be used to capture informal input from end- user developer stakeholders. Faily (2008) presents a case study where end-user developers obtained practical bene- fit by adopting professional Requirements Engineering prac- tices. Maceli and Atwood (2011) claim that people need to be involved in the software design, not just as workers, but as someone who brings their entire life experience into the design. They identified some principles for participatory co- design, and they described guidelines to help to achieve these principles. On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 3 Proto-persona+ We chose the Gothelf’s proto-persona technique, Gothelf (2012) and Gothelf and Seiden (2013), to conduct our study. However, in this work we made some improvements to the original proposal of Gothelf that resulted in a new version of proto-personas, which we named proto-persona+. Proto- persona+ extends the original proto-persona by adding a set of guideline questions that aid the stakeholders to produce the proto-personas artifacts. We considered that this adapta- tion was fundamental to support non-technical stakeholders. The main difference between the traditional per- sona, Cooper et al. (2014), and the proto-persona is in the order that the steps to its construction are performed. The building of traditional persona begins with wide demo- graphic research about end-users. On the contrary, the proto-persona elaboration is not driven by collecting data from direct users but it is constructed based on the knowledge that specialists have of the domain, Gothelf and Seiden (2013). According to Gothelf and Seiden (2013), the design of proto-personas starts with assumptions of potential personas and the validation of assumptions are performed afterwards. Additionally, the whole team contributes to the process of proto-persona creation by providing their premises about the end-users. As the team members participate actively, this process becomes an effective way to create a shared understanding of the end-users needs and characteristics. As a consequence, the feeling of empathy to the end-users is evolved by the team members. Proto-persona produces a lean artifact that is seen as one of the advantages of the technique. The artifact focuses on delivering only the relevant information about end-users, Gothelf and Seiden (2013). After examining different proto-persona’s templates, we concluded that by joining different parts we could provide a better way to use the artifact. The mix of templates aids the stakeholders to describe UX requirements whereas keeping the concise format of the proto-personas. We considered two templates proposed by Gothelf, wherein the information is reported in four quadrants. Pro- posal (A) has two quadrants in which demographic informa- tion and characterization of users (e.g. how user looks like, individual’s name, and attributes that defined the users) are described. The other two quadrants refer to attitudes (e.g. life history, routine, habits) and needs (e.g. what motivates them, what they do daily), Gothelf (2012). In proposal (B), the first quadrant outlines of the persona’s name and its role in the software, the second describes the basic demographic infor- mation, the third informs the needs and frustrations of users about a product, and the fourth reports potential solutions that can fulfill the needs of the users, Gothelf and Seiden (2013). After analyzing the similarities and differences in both Gothelf’s proposals, we rearranged the quadrants to give a new shape to proto-persona+. Table 1 presents its objectives and a relationship with the Gothelf’s models. Different from others’ templates, proto-persona+ provides a set of guideline questions to aid the stakeholder during its elaboration. We de- cided to add guideline questions because professionals claim that persona is a difficult technique of handling, Billestrup et al. (2014). Those responsible for the creation of proto- Figure 1. Proto-persona+: template and guideline questions personas+ fill the template by answering the guideline ques- tions. However, it is not mandatory to answer all the ques- tions to use our proposal. Finally, the proto-persona+ proposal is flexible by allow- ing to extend the guideline adding other questions in future research. Some questions could be more or less related to the domain in which the study is running. The flexibility for adapting the set of questions can improve the potential of proto-persona+ to catch relevant knowledge of the different types of stakeholders in a particular domain. Table 1. Proto-persona+: purpose of the quadrants Q Objective Relation to Gothelf proposals (Q1) Provides persona characterization and relevant information about the individuals that impact on the software development. Joint of the two demographic quadrants of proposal A and quadrants 1 and 2 of proposal B. (Q2) Provides details of what users need to reach their objective while using the software. Based on the quadrant about the user needs presented in pro- posal A and in parts of the quad- rant 3 of proposal B. (Q3) Points how users like to accomplish the steps to fulfill their objectives, description that focuses on the content, and in- teraction types that they prefer. Based on the quadrant about at- titude from proposal A and in some parts of the quadrant 4 of proposal B. (Q4) Describes the difficul- ties faced by the user while interacting with the software and iden- tifies the potential frus- trations that could arise during software use. Refined from quadrant 3 of pro- posal B. 4 Study context Before starting our study, we decided to focus on in a partic- ular domain area. Our research group has worked on the soft- ware development to support e-learning area. Consequently, we have several contacts with non-technical stakeholders in this field. e-Learning is the term that defines the use of elec- tronic systems in the context of learning being applied in both situation in-class and distance courses, Clark and Mayer On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 (2007). In our study, we took m-leaning area which is a sub- set of e-learning domain. M-learning applications allow the interaction of students and teachers in a learning environment through the use of mo- bile devices and the Internet, Dodero et al. (2014). Although several companies in the world have demonstrated their in- terest in the development of applications for educational pur- poses, the software teams often face difficulties in the m- learning domain, Filho and Barbosa (2013); Chimalakonda and Nori (2013). In addition to the common issues that arise during the software development for mobile devices, Filho and Barbosa (2013), m-learning domain demands for close work with different domain stakeholders (e.g. teachers, gov- ernment regulations, designers of learning contents) to cap- ture the knowledge they have, Dodero et al. (2014). For our study, we used a scenario of an application within the m-learning domain. We chose an application of virtual museum that would aid in the learning of history and arts. It was a part of a project that the research group was devel- oping. The scenario is described as follows: “An interactive museum is adopted by an elementary school to support the learning of students aged 9 to 11 in history and arts. The mu- seum’s collection comprises of several galleries that deliver the artworks in different formats (e.g. games, videos, images, texts). Access to the museum will be facilitated through a mo- bile application that should provide a variety of options for student interaction (e.g. speech recognition, touchscreen, and recognition of gestures) with the aim of being comprehensive to the public.”. 5 First Round: Using Proto-persona+ 5.1 Planning The first round of our study had the goal of answering RQ1 and RQ2. Therefore, we analyzed whether non-technical stakeholders could describe UX requirements by using the proto-persona technique (i.e., proto-persona+). Additionally, we verified the acceptance of this technique. To do this, we compared the artifacts produced by technical and non- technical stakeholders, i.e. software engineers and peda- gogues, respectively, looking for evidence of UX require- ments. Our analysis focused on exploring qualitative data by examining the descriptions presented in the proto-persona+ artifacts. Quantitative descriptive data were used only to il- lustrate the acceptance of the artifacts from the perspective of the participants. The first round was conducted in five steps. Before the conduction, the participants filled (i) a profile questionnaire. Then, we carried out (ii) a training session presenting the key concepts of the study to level the participants’ knowledge before performing the activity. To complement the training, (iii) a hands-on exercise was applied using an m-learning scenario which was different from the scenario of the study. The activity of (iv) elaboration of proto-personas+ was per- formed. Finally, the participants (v) completed the question- naire on the acceptance of using the proto-persona+. A set of artifacts to support the steps above was pre- pared. Besides demographic information, the profile ques- tionnaire (i) had questions to capture the participants’ prior knowledge about m-learning applications. A consent form, wherein the participants should agree about the use of their data for academic purposes, was also prepared. A set of slides to present concepts about persona and m-learning was de- signed to be used in the 15-minute training session (ii). For the hands-on exercise (iii), a scenario of a m-learning appli- cation was used. From this exercise, the participants could have contact with the proto-persona+ template. After per- forming these steps, the experiment to construct the proto- persona+ artifact was conducted within a period of 40 min- utes (iv). Upon completion, the participants answered the ac- ceptance questionnaire (v) on the proposal of proto-persona+, indicating their opinions and suggestions. 5.2 Execution The experiment was performed in two different days for the groups of technical and non-technical stakeholders. The study followed the steps that were planned and was con- ducted in the same physical space of a classroom at UFSCar - Sorocaba. All the participants signed the consent form and declared to have experienced e-learning software at least. A total of thirteen stakeholders participated, wherein eight un- dergraduate students of a pedagogy course who represented the non-technical stakeholders (i.e., Pedagogues - Ped), and five students of computer science courses: four Bachelor’s students and one postgraduate, who represented the techni- cal stakeholders (i.e., Software Engineers - Eng). Participants built the proto-personas+ individually. Each participant generated at least one and at most four artifacts. In total, 22 proto-personas+ were designed, being 11 created by pedagogues and 11 by software engineers. The participants did not receive any recommendations or restriction about the number of proto-personas they should produce. Participants were encouraged to construct as many proto-personas+ they considered appropriate to provide the characterization of the end-users in the virtual museum scenario. 5.3 Analysis A qualitative analysis was performed in two stages on the 22 artifacts produced by participants. First, the proto-personas+ were evaluated to identify if they reported UX require- ments. Then, we conducted an analysis on the results of the first step to find out the focus of these requirements. As UX has several definitions in the literature, the re- searchers could have different interpretations regarding what was a UX description. To avoid the different interpretations, the authors of this article decided to create an instrument to guide the data analysis. The instrument was based on a compilation of a set of UX dimensions. The works of Winck- ler et al. (2013) and Ardito et al. (2006) gave us the ground to elect and compile the UX dimensions. We selected these works as the basis of our dimensions because they discuss UX in the two areas our study focused on, mobile with the work of Winckler et al. (2013) and e-learning applications with the work of Ardito et al. (2006). To define the dimensions, we examined the similarities be- tween the dimensions described in the two works and those On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 that attended the particularities of our study domain. The di- mensions of stimulus and value were selected from Winck- ler et al. (2013). The work of Ardito et al. (2006) presents a set of heuristics for evaluating e-learning applications and a methodology for using such heuristics. From this work, four dimensions were chosen: access, media, organization, and interaction. As a result, six dimensions were outlined and considered in our analysis. These dimensions focused on the type of UX requirements that we had to search for in the proto-personas. To keep the researchers’ attention to the same UX defini- tions, we wrote in the guide the meaning of each dimension in details. Access dimension covers the aspects of technol- ogy and its quality for use; media specifies what media sup- port the communication considering the e-learning context; organization focuses on how learning contents and naviga- tion are arranged; stimulus examines the motivations that lead the participants to engage in the interaction, and encom- passes impressions and opportunities for use; value explores what the use of that product brings to the students’ learning; and interaction focuses on the results that each type of inter- action deliver to the student. Considering the dimensions, first, four researchers searched for evidence of UX requirements on each proto- persona+. This first step was carried out for three Master’s students in software engineering (SE) and human-computer interaction (HCI) and an undergraduate student in computer science with experience in HCI. After examining an artifact, the researcher had to assign labels on it. The labels indicated in which degree the UX dimensions were being fulfilled or not considering the description found on the artifact. These degrees were classified into three levels (fulfilled com- pletely, fulfilled widely and fulfilled partially). Besides, the researchers took notes to justify their rationale to have assigned one or another classification for each dimension. In case of the researchers did not assign any degree they did not make notes. The researchers examined the artifact from a whole perspective because the information of one quadrant of proto-persona+ was complementary to the other. Each researcher analyzed 11 artifacts: 2 researchers evaluated 5 proto-personas+ of pedagogues and 6 of software engineers; and 2 others evaluated 6 artifacts of pedagogues and 5 of engineers. In the second round, two senior researchers in SE and HCI revisited the data and refined the results. Taking into account the results of the first step, the first au- thor of this article performed a new qualitative analysis. For this, the open coding technique was used, Strauss and Corbin (1998). Open coding relates codes to chunks of text. These codes receive denominations that give certain significance to the chunks of texts they refer to, Strauss and Corbin (1998). Subsequently, these codifications were revisited and they are grouped when patterns of information were identi- fied. For instance, the code interface could be assigned to chunks of texts that report information on user interface. During the coding process, codes were assigned to parts of the notes written by the researchers. Then, this set of codes was re-analyzed to search for patterns of information. The results of these two steps were verified by two senior re- searchers in the areas of SE and HCI. The coding was per- formed using the NVivo 113 tool and a total of 26 codes re- lated to the UX dimensions were generated in this process. 5.4 Threats to validity Internal threat could be refereed to the tiredness of the par- ticipants. This could happen due to participants spend a long time concentrating on the activity of the experiment. To mit- igate this, we scheduled a break between the hands-on train- ing and the activity of proto-persona+ elaboration. External threat refereed to the use of students as participants. How- ever, Salman et al. (2015) provide evidence that there are few differences in the performance of students and practitioners when they performed an activity they have not previously knowledge. Even with greater practical expertise, the fact that professionals do not know a new technique such as proto- persona+ allows us to compare them to students. Salman et al. results allow us to state that our findings getting from students using proto-persona+ can be extended to more ex- perienced professionals who have never used proto-persona technique. The threat of construct was mitigated by the training and hands-on exercise when the participants had the opportunity to request clarification about the technique and the template. Consequently, we considered our sample of artifacts have good quality. Additionally, all participants were prior users of e-learning applications. We handled the threat of conclu- sion by using a common definition of UX based on dimen- sions. All the researchers inspected the artifacts using this guide avoiding different interpretations about the UX mean- ing. A bias on the conclusion could be introduced in the study by the fact that there were no limits on how many personas each participant could create. As a consequence, a partici- pant could produce more personas than others, and therefore, s/he could become more representative within his/her group. However, our goal was not to verify how much information each participant offered individually. Rather, our focus was to see the contributions that arose from the different types of stakeholders. Besides, this study analyzes two groups with the same number of artifacts in both, which mitigates the problem of comparing unbalanced groups. Anyway, we con- sidered this is an issue that other researchers should be aware of if they decide to run a similar study. 6 Findings of the first round The profile questionnaire showed that out of the 13 partici- pants, 84.5% used mobile devices 5 or more days in a week, 61.5% preferred to access the Internet through their mobile phones, and 77% had participated in an online course in the last two years. The findings of the first round aided us to answer the RQ1 and RQ2. We will present the results in the follow sections. 3http://www.qsrinternational.com/nvivo On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 6.1 UX requirements To respond the (RQ1) Which UX requirements do non- technical stakeholders describe while using the proto- persona technique?, we observed the codes generated from the open coding process. Figure 2 presents the codes associated with the artifacts of each type of stakeholder. Our analysis did not have the purpose of quantifying the occurrence of a code. Rather, the qualitative analysis concentrated on exploring the evidence of UX issues that arose from the data. In this analysis, we ob- served the convergence or divergence of the codes assigned to the artifacts of the different stakeholders. We see the codes that are common and different considering the artifacts built by pedagogues and software engineers. We will concentrate our discussion on the codes in bold that represents more rel- evant findings. Both types of stakeholders described characteristics of en- joyment, stimulus, and satisfaction to highlight the impor- tance of building an enjoyable experience that holds the stu- dent attention during the learning process. However, by ob- serving the codes, we can see that this objective was ex- pressed in different ways. The pedagogue described a learn- ing process that should be fun (see the code in bold), thereby showing the intention of organizing lessons from this per- spective. On the other hand, the software engineers pointed out requirements regarding the focus on use and easiness of use with the intention of avoiding users frustration. These examples are shown in Figure 3. The examples highlight the parts where we see how each type of stakeholder describes a way of maintaining students’ interest in learning. The fol- lowing examples show the different contributions provided by the stakeholders. In addition to focusing on distinct user information and user characteristics, each type of stakeholder provided spe- cific user profile details. Two non-technical stakeholders specified requirements for visual impairment or attention deficit that can attend users with special needs. Two technical stakeholders delineated the characteristics of users who like to learn by participating in interactive spaces where they can interact with their colleagues. These examples can be seen in the two artifacts in Figure 4. Figure 5 shows the codes that were found in common or not from the proto-personas+ per dimension and per type of stakeholders. From the access dimension, we see that peda- gogues’ artifacts had codes related to accessibility that refers to the availability of hardware that would meet the special needs of each user, as well as the forms of interaction that could attend this audience. Only the artifact of the technical stakeholders provided dif- ferent codes in the media dimension. While reporting the use of different types of media (e.g. Video), the software engineer showed concern about media that could provide interactions; therefore the Interaction Mode code was assigned to the ar- tifact of this stakeholder. An example of this is interaction with text on a small screen of mobile device that can intro- duce barriers for users to perform their actions. In the Organization dimension, the pedagogues focused on how structuring the learning path for a given student profile. The codes assigned related tto this dimension were Application Complexity, Focus on Learning Process, User Restrictions, and Student Objective. Additionally, from the proto-personas+ created by the software engineers, we could see the focus on building applications that could motivate the students to the interaction by providing different medias. Both types of stakeholders were concerned about stimu- lating the students by offering an enjoyable application (see Figure 3). It can be seen from the Stimulus dimension that had the codes Satisfaction and Fun related to the proto-personas+ which were created by the pedagogues. On the contrary, the software engineers considered that the care in aspects that bring Frustration would encourage the student to continue using the application. In the Value dimension, the codes Me- dia, Device, and User Restrictions demonstrated the matter of enriching the user experience during the learning process. Finally, in the Interaction dimension, we could identify the contributions that the pedagogues did from observing their artifacts. The Accessibility and Application Complex- ity showed that these stakeholders concentrated their atten- tion on delivering a more personalized interaction in accor- dance with users’ profile. Consequently, these issues can bring Stimulus and Value to the UX. 6.2 Acceptance of proto-persona+ To answer (RQ2) How is the acceptance of the use of the proto-persona+ technique by these stakeholders?, three dif- ferent analyses were performed: (i) the importance that the stakeholders perceived on the template’s quadrants to per- form the activity; (ii) the usage and relevance the stakehold- ers saw in the guideline questions to complete the quad- rants; and (iii) the perception of usefulness and ease-of- use regarding the proto-persona+. The participants answered the questions after finishing the elaboration of the proto- personas+. Given the small size of our sample, we analyzed the data from a descriptive perspective. The results will be presented in detail in the following subsections. 6.2.1 Importance of quadrants We explored the importance of the quadrants (Figure 1) in re- lation to the description of the proto-persona+ in the perspec- tive of the participants. For this, the participants should clas- sify each quadrant in one of the following categories: Very Important (VI), Important (Imp), Unimportant (UI) or Ir- relevant (Irr). Table 2 presents these classifications in two complementary representations: the sum of classifications for each quadrant in brackets and the percentage of the par- ticipants that chose that classification. In Table 2, it can be seen that all the quadrants were almost solely classified as Very Important or Important. The quad- rant (Q2): Objectives and Necessities was considered Very Important for all the stakeholders. Although all the quad- rants seemed to have similar importance to the stakeholders, an exception was observed for quadrant (Q1) Demographic Data: only one software engineer (i.e. technical stakeholders) pointed out the Q1 as Unimportant. Comparing the classifications for the Q1, it could be seen that the software engineers mostly pointed this quadrant as On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 Figure 2. Codes assigned to the artifacts of each type of stakeholders Figure 3. Two ways of working on student engagement: examples of tech- nical and non-technical stakeholders Figure 4. Definitions of different end-user profiles: examples of technical and non-technical stakeholders Important, while the pedagogues (i.e., non-technical stake- holders) indicated it as Very Important. The personas tech- nique has the focus on developing the empathy between users and developers; therefore, we can conclude that the non- technical stakeholders can contribute to characterizing the end-users. On the contrary, technical stakeholders were not concerned with these aspects. Table 2. Degree of importance of the quadrants Q1 Q2 Q3 Q4 VI 20% (1) 100% (5) 60% (3) 60% (3) Eng (5) Imp 60% (3) 0% (0) 40% (2) 40% (2) UI 20% (1) 0% (0) 0% (0) 0% (0) Irr 0% (0) 0% (0) 0% (0) 0% (0) VI 75% (6) 100% (8) 62% (5) 62% (5) Ped (8) Imp 25% (2) 0% (0) 38% (3) 38% (3) UI 0% (0) 0% (0) 0% (0) 0% (0) Irr 0% (0) 0% (0) 0% (0) 0% (0) VI 53.8% (7) 100% (13) 61.5% (8) 61.5% (8) Total (13) Imp 38.5% (5) 0% (0) 38.5% (5) 38.5% (5) UI 7.7% (1) 0% (0) 0% (0) 0% (0) Irr 0% (0) 0% (0) 0% (0) 0% (0) 6.2.2 Usage and relevance of the guideline questions We examined the participants’ answers about the perception of the stakeholders regarding the relevance and the use of the guideline questions. An open question asked the participants for suggestions to improve the proto-persona+ template. Ta- ble 3 presents the results in percentage and in absolute num- bers of the “yes” answers. This double representation of the results provides a more real overview considering that we had a small sample of technical stakeholders. Therefore, the percentages might not clearly indicate the differences and similarities between the two types of stakeholders. In Table 3, we can see that the software engineers used and considered relevant the question Q3 - What are they better at doing?. On the contrary, most the pedagogues did not show the same results. An inversion was observed from the ques- tion Q3 - How do they like to do it?, which was not used by the software engineers but had great application to the peda- gogues. Finally, the question Q4 - What frustrates them? pre- sented a considerable difference in the responses; while all the software engineers used and found it relevant, the peda- gogues used it very little, although they found it to be a rele- vant question. These differences from the perceptions of both types of stakeholders restate that both stakeholders have the potential to give different contributions. By exploring the stakeholders’ written notes for the Q3, it can be observed that the questions motivate different points of view. The pedagogues focused on encouraging students to overcome their barriers. They reported the need of perform- ing activities that helped students in developing new skills and not only on improving something in which they were con- sidered good. On the contrary, the software engineer showed emphasis on what the student already knows in a tentative On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 Figure 5. Codes per UX dimensions and per type of stakeholders Table 3. Usage and perception of relevance of the guideline questions Uses Relevance Guideline questions Eng (5) Ped (8) Eng (5) Ped (8) Who are they? 100% (5) 100% (8) 80% (4) 100% (8) Q1 What are their ages? 100% (5) 100% (8) 100% (5) 100% (8) What are their school levels? 100% (5) 100% (8) 100% (5) 100% (8) Q2 What do they want to accomplish? 100% (5) 100% (8) 100% (5) 100% (8) What do they need to reach their objective? 80% (4) 88% (7) 100% (5) 88% (7) What do they like? 100% (5) 75% (6) 100% (5) 100% (8) Q3 What are they better at doing? 80% (4) 38% (3) 80% (4) 50% (4) How do they like to do it? 40% (2) 75% (6) 80% (4) 100% (8) What are the difficulties they can face? 80% (4) 100% (8) 100% (5) 100% (8) Q4 What frustrates them? 100% (5) 63% (5) 100% (5) 88% (7) What are the known issues that affect their interaction? 80% (5) 100% (8) 80% (4) 100% (8) of stimulating such student behavior during the use of the application. Among 13 participants, only one pedagogue and one soft- ware engineer gave suggestions through the open question; both were for the Q1 - Demographic Data. One pedagogue suggested the addition of the question: “Do users have any deficiencies or restrictions?” that focuses on the individual characterization of users. On the contrary, one software en- gineer suggested a more technological question: (“Do users have access to mobile devices?”). Table 4. Preferences for each guideline question Guideline questions Uses Relevance Who are they? - Ped Q1 What are their ages? - - What are their school levels? - - Q2 What do they want to accomplish? - - What do they need to reach their objec- tive? Ped Eng What do they like? Eng - Q3 What are they better at doing? Eng Eng How do they like to do it? Ped Ped What are the difficulties they can face? Ped - Q4 What frustrates them? Eng Eng What are the known issues that affect their interaction? Ped Ped We examined the number of times that each question was answered by the participants and identified the questions that were more important for the different stakeholders. The Relevance column in Table 4 indicates which stakeholder presented more answers for each question. Software engi- neers demonstrated greater interest in the use of the Q3 ques- tions. On the contrary, the Q4 was the most used for the peda- gogues. The quadrants Q1 and Q2 were answered in a similar manner by the two types of stakeholders. 6.2.3 Perception of usefulness and ease-of-use This analysis was based on the responses of the Technol- ogy Acceptance Model (TAM) questionnaire, conceived by Davis (1989), that aims to analyze the acceptance of certain technology by a group of participants, Dias et al. (2011). We included a question regarding the ease of mem- orizing the technique that was based on the work of Stein- macher et al. (2015). Table 5 lists the questions. For each question, the participants chose the option that best represented their degree of agreement. The options avail- able were “Fully Agree”, “Largely Agree”, “Partially Agree”, “Partially Agree”, “Largely Disagree”, and “Fully Disagree”. By observing the percentages of both types of stake- On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 Figure 6. Perception of usefulness and ease-of-use Table 5. Questions used from the TAM questionnaire Dimension Question Usefulness U1 By using the persona technique, I was able to describe the user characteristics more quickly. U2 By using the persona technique, I was able to enhance my ability to describe the user charac- teristics. U3 By using the persona technique, I was able to en- hance my efficiency during user characteristics description. U4 By using the persona technique, I was able to more effectively describe the user characteris- tics. U5 By using the persona technique I was able to improve my perception about the good practices for describing user characteristics. U6 I consider the persona technique useful in de- scribing the user characteristics. Ease-of-use F1 It was easy to learn to use the persona technique. F2 I was able to use the technique in the way I in- tended to. F3 The orientations of use for the persona tech- nique were easy to understand. F4 I understand what happened during my interac- tion with the persona technique. F5 It was easy to gain ability to use the persona technique. F6 The persona technique allows flexibility to de- scribe the user profile using the quadrants. F7 It is easy for me to remember how to use the persona technique. holders, it can be seen that a great number of questions was answer as “Largely Agree”. Few exceptions could be found. The difference in agreement perceptions in ques- tion F5 about “the easiness to gain ability to use the tech- nique” was high. The group of software engineers answered 60% (3 of 5) with “Partially Agree” and had 20% (1 of 5) that “Partially Disagree” on the questioning that the tech- nique was easy to gain ability. Revisiting the notes in the proto-persona+ artifact, we found out that the software engi- neers struggled in describing the proto-persona+, which can explain the low “easy to gain ability” perception on the above question. On the contrary, the pedagogues showed a lower percep- tion that the technique improved their efficiency to describ- ing the audience. The majority of the pedagogues answers was “Partially Agree” in the question U5 (50%, 4 of 8). How- ever, 60% (3 of 5) of the software engineers indicated that they “Largely Agree” that the proto-persona+ improved their efficiency to describing the users (question U3). Overall, only the software engineers pointed out some degree of disagreement (“Partially disagree”). Moreover, “Fully Agree” prevailed in the pedagogues’ responses, which can reiterate the fact that they had the perception that the tech- nique was useful to describe end-users. 7 Second round: using the proto- personas+ in design 7.1 Planning After exploring the creation of the proto-persona+ artifacts we decided to investigate whether these artifacts could sup- port developers during the prototyping of solutions. The re- sults of this investigation aided to answer the (RQ3) Which UX requirements presented in the proto-personas+ can sup- port the prototyping of user interfaces?. The objective of this second round was to analyse whether the information from proto-persona+ artifacts contributed to the design of the low fidelity prototypes. The participants constructed low fidelity prototypes by using storyboards technique. Storyboard is a technique in which the people’s interaction with an application is shown. Often it delivery a complementary view of the static draw- ings of user interfaces. Storyboard simulates the flow that users can follow from one part of the interface to an- other, Rogers et al. (2015). In this round, our subjects were software developers. In our study, the storyboard artifacts were drawn on paper. The participants could enrich their proposal by adding stick- ers around of interface elements. These stickers contained supplementary textual information such as actions associ- ated with buttons, navigation flow between the screens, and so on. Additionally, in the stickers, the developers also re- ported which part of the proto-personas+ and the scenario have aided them to make their choices regarding the de- On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 sign. Through these textual justifications, we could analyze what information they used and from which proto-persona+ it originated. This second round was conducted in four steps. First, we performed a (i) pre-analysis of the participants knowledge about HCI techniques that were used in the activity; (ii) a training session on HCI techniques and mobile development; (iii) a hands-on exercise to prototyping a user interface by using a scenario; and (iv) the construction of the storyboards considering the proto-personas+. Artifacts to support the steps were prepared. The pre- questionnaire of participants’ profiles (i) contained as much personal information as information about their knowledge about personas and prototyping techniques, and about the Nielsen heuristics, Nielsen (1995). The training session (ii) consisted of a two-hour class that presented techniques that were required for the development of the storyboards. For the hands-on exercise (iii) a two-hour activity was planned, wherein some proto-personas+ and an example of scenario were made available, from which the participants experi- enced the same artifact that they would use in the study. For the step of the construction of the storyboards, (iv) a con- sent form was distributed to the participants to indicate their agreement on the use of their data for the purpose of aca- demic research; here, the same scenario used in the first round was applied. 7.2 Execution Thirty-six undergraduate students in computer science at UF- SCar participated in the study, known as developers hence- forth. They answered the pre-questionnaire, and in the pre- analysis we were able to fathom their knowledge about HCI techniques (see Figure 7). We noticed that 78% of develop- ers “did not know the technique of persona”; 67% “did not know the Nielsen heuristics”; and about prototyping tech- nique: 22% “did not know” and 47% “knew, but had never used”. From the questionnaire results, we separated the devel- opers into 18 pairs. We balanced the pair composition based on their knowledge on the techniques. As noticed, the participants did not have practical knowl- edge about the techniques we planned to use (i.e., personas and storyboards). To mitigate this, we conducted the train- ing session in two steps to leverage the participants’ knowl- edge. First, a senior professor in SE and HCI carried out two- hour class covering topics about personas, storyboard, and how Nielsen heuristics could help them to the application of design good practices. Later, on the same day, two Master’s students conducted a two-hour hands-on in which the par- ticipants built a storyboard based on a new scenario and ex- amples of proto-personas+ (the artifacts were different from those used later). A week later, the study was conducted in a 3-hour session, wherein 18 pairs of developers constructed storyboards by us- ing the scenario of the study (i.e., the same that was used to construct the proto-personas). We also requested the pairs to select only two proto-personas+ to support their work. This decision to limit the choice into two artifacts was taken so that the participants did not have to deal with a large diversity of user profiles. First, the pairs received 22 proto-personas+ Figure 7. Participants’ knowledge about HCI techniques in a random order to prevent that the same artifact was always placed in the same position of the order of presentation. To avoid the participants selected always the same artifacts, we shuffled the proto-personas+ before presenting them to the participants. The participants received a set of these artifacts arranged from different orders. By doing this, we avoided that the order of presentation could cause biases on the selec- tion of the proto-persona+. The pairs built the storyboards and also fixed the post-it stickers to report their decisions on the design. The partici- pants were instructed to explain through the stickers, which parts of the scenario and proto-personas+ they used to gain the insights into the design. Each pair generated five user in- terfaces (three to nine user interfaces) on average. 7.3 Analysis We performed the analysis in two phases. The first one ex- amined which proto-personas+ were selected and applied to the construction of the storyboards by the developers. In the second step, through a more in-depth analysis, we explored which parts of the proto-persona+ were used. From this sec- ond analysis, we intended to understand how the information found in the proto-personas+ aided the developers’ work on building the solutions. We first identified the most chosen proto-personas. Fur- ther, by considering the developers’ notes about the use of the proto-persona, we could identify which parts of the proto- personas+ were used most. The first phase followed the same procedure as in the first round, wherein UX dimensions were applied (see the def- initions in Section 5.3). Different from the first round, in this phase, the storyboards were the targets of the evalua- tion. Twelve software engineers with different profiles at- tended this session: two undergraduate students in computer science from UFSCar (Campus Sorocaba), five Master’s stu- dents, of which four were from the graduate program at UF- SCar (Campus Sorocaba) and one from the graduate pro- gram at UNICAMP, two graduates working for more than On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 three years in software companies; and three masters in Com- puter Science. All had experience in HCI and background in Computer Science. None of these evaluators participated in the previous evaluation of the proto-persona+ (i.e., the first round described in this work). The storyboards were distributed among the evaluators. Each storyboard produced by a developer had five low fi- delity prototypes on average; therefore, the division of what each evaluator would explore considered the following fac- tors: (i) we made a uniform distribution so that each partici- pant received at the same number of prototypes to evaluate, and (ii) each storyboard was evaluated by two participants; this redundancy in the evaluation intended to enrich the ana- lyzes. However, the same pair did not analyze the same set of storyboards. Considering the UX dimensions, each evalua- tor examined fifteen low fidelity prototypes. As a result, the evaluator took notes to justify whether a given UX dimen- sion was being applied or not in that prototype. None of the participants had seen the proto-personas+ used to create the prototypes they were evaluating. After, we proceeded to the second step, wherein the open coding process happened in two iterations. The first author of this article inspected the notes that the evaluators took in the first step based on 24 codes that were previously generated. Later, the fourth author refined the findings and 23 new codes were generated at the end of this round. This generated a total of 49 codes in the two experimental rounds combined. 7.4 Threats to validity To deal with the bias on the preference of proto-persona+ se- lected by the developers, which is an internal threat, we pre- sented the 22 artifacts in a random order to the participants. In our arrangement, the proto-persona+ did not appear more than twice in the same ordinal position of the list. With the order changed for each group, the threat of a possible false preference was mitigated and the results became more reli- able for the inferences and support. Another threat to the in- ternal validity refers to the motivation of the participants dur- ing the experiment because the workshop was applied during a compulsory course in computer science. We collected the participants’ opinion about the activity at the end of the study. The participants’ feedback showed that they considered the activity important, e.g., “I found the activity very interest- ing”; and opinions like the “[proto-persona] was useful to the achiement of my goal...”. The feedback showed that the participants felt motivated to participate in the study. A threat to the external validity was the fact that the story- boards were constructed by participants that had no prior con- tact with proto-persona+ and storyboards. To mitigate this threat, we conducted a training about the proto-persona+ and storyboard techniques and a hands-on exercise using them. On this validity, we arranged the developers in homogeneous pairs that had complementary knowledge. Similar to the first round, the subjects here were also students. Salman et al. (2015) in their work provide evidence that students and expe- rienced professionals have equal performances in new activi- ties. Although storyboard and prototyping are largely applied techniques, in our case, we changed the traditional applica- tion of both. By using a scenario and the proto-personas, we provided a method to mitigate the lack of experience of the developers because it is different from the usual prototyping. 8 Findings of the second round Using the results of the second round, we answered (RQ3): Which UX requirements presented in the proto-personas+ can support the prototyping of user interfaces. The details are presented in the following two subsections. 8.1 Developers’ preferences Firstly, we identified the proto-personas+ that the developers chose and used on considering that they should select only 2 from the 22 proto-personas+ that are available. We orga- nized this result in three groups of proto-personas. Group (I) presents the proto-personas+ that were widely used, being the most chosen; group (II) comprises the proto-personas+ that were chosen by the groups in an amount equal to the av- erage relative to the distribution of the choices; and group (III) comprises the proto-personas+ that were chosen at least once by the pairs (i.e. developers). Table 6 summarizes these groups and indicates the id of the proto-persona, which stakeholder was the creator of the proto-persona, and some features of the artifacts. One of the goals of proto-personas+ is to promote the empathy between developers and users; therefore, the use of an image to repre- sent the persona could be important. We also obtained direct and indirect findings about the use of the artifacts. The direct analysis comprises the absolute number of ref- erences that each proto-persona+ received by the develop- ers. On the contrary, the indirect one comprises the results from the perspective of the authors analysis regarding the preference between the two proto-personas+ chosen by the pair. Considering the two artifacts, it was analyzed which of the two proto-personas+ was most emphasized during the construction of the storyboard while counting the number of references to the parts of each artifact. The indirect analy- sis resulted in two cases: (1) of equal interest, wherein both the artifacts obtained the same amount of references and (2) different interests among the proto-personas+ (classifying ar- tifacts into primary or secondary personas). The classification in primary and secondary personas hap- pens when there are more than one user profiles that will use the application; however, one of them should be considered with higher priority owing to being the primary user of the application, Cooper et al. (2014). A Primary Persona is de- fined as a profile that represents the user’s focus of the appli- cation; therefore, it will have its prioritized needs met. Sec- ondary Persona refers to a user profile that will use the ap- plication; however, to fulfill its needs is not a priority for the application. Based on these definitions, a classification of the proto-persona+ that fits in case (2) was performed. The proto- persona+ with the highest number of parts referenced was classified as primary, whereas the lower one was classified as secondary. In Table 6 were found some relevant results. All the proto- personas+ that had an image in (Q1) (i.e. Demographic Data) On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 Table 6. Proto-personas selected to the construction of the storyboards Direct Indirect Id Proto- Primary Secundary persona+ Stakeholder Image References Persona Persona Equal I 9 Pedagogue X 10 5 2 3 22 Engineer X 8 4 0 4 8 Pedagogue X 3 1 1 1 II 21 Engineer X 3 0 1 2 14 Engineer 3 0 3 0 2 Pedagogue 2 0 1 1 20 Engineer 2 1 1 0 18 Engineer 1 1 0 0 III 11 Pedagogue X 1 0 0 1 1 Pedagogue 1 0 1 0 7 Pedagogue X 1 0 1 0 19 Engineer 1 0 1 0 were chosen at least once by the pairs of developers. Addi- tionally, from the five most chosen artifacts (i.e., groups I and II), four had an image associated to the proto-persona. This fact reinforces the idea that persona is a technique that stimu- lates empathy in the developer. The use of image to represent the target audience is a method to instigate the developers to think and develop the association of their ideas with those of the user represented in the persona, Grudin (2006). It can be seen that two artifacts, i.e. id 9 and 22 obtained the highest numbers of references (group I) in both indirect and direct analysis. They were designed by the pedagogue and the software engineer, respectively. During direct anal- ysis, we observed that proto-persona+ 9 was chosen by 10 of the 18 developers, whereas the 22 was chosen by 8 of the 18. Considering that only 10 of the 22 proto-personas+ got indications of at most 3 groups and that 10 others were not chosen by any group, we can see a clear preference for arti- facts 9 and 22 to support the construction of the storyboards. Additionally, the proto-personas+ 9 and 22 were classified as primary personas in most cases. Examining the data, we could observe that proto-persona+ 9 was used 5 times as pri- mary, 22 was used 4 times as primary, and all other artifacts obtained only one emphasis as a primary persona. These re- state our results found out in the direct analysis. To explain the preference for proto-personas+ 9 and 22, the 4 authors of this article conducted a qualitative analysis on the content of the quadrants of these proto-personas. The results demonstrated that both artifacts had a more clear defi- nition on the users they represented. They provided informa- tion in rich details of who the end-user is, being these details evident in Quadrant 2 (Objectives and Needs). Fisher’s exact test Fisher (1922) was taken to analyze the existence of a statistical significance between the proto- personas+ produced by the pedagogues and software engi- neers. By running the same testing, we also checked the influ- ence that an image have on the choice of an artifact. Fisher’s exact test is recommended either for small samples of cat- egorical data and for calculating the exact significance of the deviation from a null-hypothesis using the p-value. The statistical analysis was conducted with certain scenarios and proto-personas+ groups, with their respective null (H0) and alternative (H1) hypotheses. To conduct the testings, we defined null and alternative hypotheses considering that the characteristic C1 could in- fluence the results C2 (see Table 7). Taking into account these assumptions, we could represent the null and the al- ternative hypotheses respectively as (H0) There is no influ- ence of on the and (H1) There is an influence of on the . Table 7. Fisher exact tests results C1 C2 p-value stakeholder that create the proto-persona+ classification of the artifact as a primary persona 1 stakeholder that create the proto-persona+ classification of the artifact as a secondary persona 1 stakeholder that create the proto-persona+ classification of the artifact as “equal interest” 1 stakeholder that create the proto-persona+ number of references of the artifact in the prototypes be equal or greater than 3 1 the presence of a rep- resentative image number of references of the artifact in the prototypes be equal or greater than 3 0.2424242 We run the testings using R software environment4. It was assumed a p-value with significance 0.05 in the analysis. In Table 7, the final p-value got after performing the Fisher ex- act test. The p-value of the analysis do not indicate any sta- tistical significance to refute the null hypothesis in any one of the analyzed pairs of elements. Statistically, the proto- persona+ creator (i.e. pedagogue or engineer) could not be related to how the proto-persona+ was used. Similarly, we could see that the fact of a proto-persona+ presenting an im- age did not affect the number of times that artifact was re- ferred in prototypes. Finally, we explored which proto-personas+ were chosen in the perspective of who created them. Table 8 presents a mapping of the storyboard and the type of stakeholder who was the author of the artifact used in the construction of the solution. Only four groups used the proto-personas+ created only by the pedagogues. The same could be seen for the appli- cation of those built by the software engineers. From this, it was confirmed that the developers mostly opted to build their solutions considering the proto-personas+ of the two special- ties. We need to restate that the set of artifacts was delivered in a random order and without indication of which of the two stakeholders elaborated them. The results showed that 4https://www.r-project.org/ On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 Table 8. Type of proto-persona selected vs storyboards Proto-persona+ used Storyboard id Number of times Only pedagogue S1, S9, S12, S16 4 Only software engineer S4, S10, S11, S13 4 Mix of both S2, S3, S5, S6, S7, S8, S14, S15, S17, S18 10 the combination of the artifacts from different stakeholders aided the developers in most cases. 8.2 Application of UX requirements The codes that emerged in the analysis of the storyboards were related to the codes found in the analysis of the proto- personas’ descriptions (see Sub-section 6). To support our presentation of the results, we discuss the codes of the story- boards comparatively with the codes used in the first round of the analysis and presented in Figure 5. To illustrate our discussion, we will show figures in which the codes was split into three groups. Group A represents the five most recurrent codes for that dimension. Group C rep- resents the codes that appear only once related to that dimen- sion. And finally, Group B represents the codes that arose more than once associated with a dimension but not in an amount that justified to be one of the top five codes (group A). Figure 8. Codes of access dimension found in the storyboards In Figure 8, it can be seen that the questions regarding the physical devices and infrastructure to access were the main focus of the participants. This was noted by the recurring codes of Hardware (A), Internet (A), and the characteristics of Device (A). While comparing the codes found out in this analysis with the ones uncovered in the proto-personas+ anal- ysis, we got the Interaction Mode (A) code as one of the most presented for this dimension. This code was identified in the proto-personas+ produced by the software engineers and ap- peared in several UX dimensions in the previous analysis (Figure 5). This result demonstrates the concern with these forms of interaction were presented in the prototypes of the storyboards to meet users’ needs. It is also seen that the codes Universal Accessibility (C) and Social interaction (C) that re- fer to the two profiles built by the pedagogues and the soft- ware engineers, respectively, as shown in Figure 4. This find- ing illustrates how the knowledge of different stakeholders contributed in enriching the description of end-user details. In the media dimension (see Figure 9), the Image (A) and Game (A) media of interaction were the major codes men- tioned. The code of Interaction Mode has been found several Figure 9. Codes of media dimensions found in the storyboards times in the analysis of proto-personas+ created by the soft- ware engineers. In this context, the focus reiterates the results found in the first round that took the concerning on which me- dia could affect the users’ learning process and consequently their user experience. Considering the common points be- tween the pedagogue and the software engineer stakeholders (see Figure 5), we noticed that they concentrated on Focus on Learning Process (B), Student Preferences (B), and Student Objective (B). Finally, the concerning on a Misleading (B) of how a media works or what it stands for has also emerged as a code, thereby demonstrating how App Overall Organi- zation problems (A) and Frustration (C) can affect student learning process. Figure 10. Codes of organization dimension found in the storyboards Simplicity (A), Easiness of use (A), Navigation (A), App Overall Organization (A), and Confusion (A) were the codes that arose in the organization dimension (see Fig- ure 10). These codes indicate that applications in this domain On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 should not introduce complex ways of interaction providing a simple manner of use. To promote the Stimulus (B) to the users engagement into the software and a Pleasant interac- tion should be the goals of the applications. These codes in- dicate that applications in this domain should provide a sim- ple and not complex manner of use. The applications should have the Stimulus (B) and a Pleasant (C) experience as their goals when the user learns and uses this application. By ob- serving the previous results (see Figure 5), we noticed that the artifacts developed by the pedagogues had the codes: User Restrictions (B), Application Complexity (B), and Fo- cus on Learning Process (A) assigned to them; this demon- strates the importance of providing a learning application in which users can have an easy journey. Figure 11. Codes of stimulus dimension found in the storyboards In the stimulus dimension (see Figure 11), the focus on Me- dia (A), Stimulus (A), and Focus on Learning Process (A) was presented. By looking at Figure 5, it can be seen that both pedagogues and software engineers focused on the same points during the construction of the proto-persona+. Regard- ing the Fun and Satisfaction codes that were assigned to the proto-personas+ of pedagogues, we noticed that the low fi- delity prototypes had similar codes associated to them (i.e., Fun (B), Curiosity (B), and Enjoyment (B)); this is an ev- idence that the developers have tried to keep an exciting experience for the students. Considering the codes related to the proto-personas+ of the software engineers, Frustra- tion (B) and Student Objective (B) were the codes identified in the prototypes which demonstrated the concern that these stakeholders had on encouraging students to use the applica- tion. Lastly, focusing on the App Overall Organization (A), the prototypes provided a method in which students can cus- tomize their learning process and consequently improve their experience. A prevailing occurrence of the codes: Media (A), Focus on Learning Process (A), and Game (A) could be found in the value dimension (see Figure 12). These three code have already been found out from both types of the stakeholders (i.e., pedagogues and software engineers) in the results of the proto-personas’ analysis (see Figure 5). While explor- Figure 12. Codes of value dimension found in the storyboards ing the proto-personas+ of both stakeholders we saw their concerns on the learning process, user experience, and the use of suitable channels of interaction. Codes such as Stimu- lus (B), User eXperience (B), Satisfaction (B), Fun (B) and Pleasant (C) demonstrate that developers who constructed the storyboards were able to catch such claim. Considering the code App Overall Organization (A), we noticed that only the proto-personas+ of software engineers had such code as- signed. This code provides evidence that this stakeholder worried on the integration of different resources and features of the application. Figure 13. Codes of interaction dimension found in the storyboards Finally, in the Interaction dimension the Focus on Learn- ing Process was the main common code. Accessibility (B) and Social interaction (C) are codes that were pointed out respectively from pedagogues and software engineers proto- personas+ and that appeared again in the analysis of the story- boards. These codes allowed to reaffirm the different contri- butions that both types of stakeholders provide to the design of solutions. By observing the differences between the two types of stakeholders we see that Interaction Mode (A) and Stimulus (B) were found in the proto-personas+ of software engineers and pedagogues respectively. These codes clearly demonstrated that software engineers concerned more on technical aspects of interaction whereas the pedagogues were worried on keeping students motivated to the learning. On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 9 Discussion This study investigated the effect of the non-technical stake- holders’ participation on UX requirement specification. It is different from other works, wherein the non-technical stake- holders provide information in a passive participation. In our investigation, we considered these stakeholders as active members during the elicitation of UX requirements by using proto-personas+. The findings showed that non-technical stakeholders brought important contributions to the UX requirements elaboration. They could point out the requirements that re- port UX in different perspective from that provided by the technical stakeholder. UX requirements are strongly context-dependent, and given that this context suffers con- stant changes over time, Kashfi et al. (2017). Non-technical stakeholders are the ones that have the knowledge about the context. From the findings, it could be noticed that although the technical stakeholders had experience in the domain, the non-technical ones demonstrated to check the aspects that can directly influence the acceptance of the software, Hadar et al. (2014). By looking at the steps that the technical and non-technical stakeholders followed, we can summarize the findings be- low. The main points in the first round were the preparation of the stakeholders to be able to apply the proto-persona+ tech- nique correctly. Firstly, the non-technical stakeholders took part in the training regarding the presentation of the proto- persona+ technique, its benefits, and purposes of use it. Ad- ditionally, a scenario about the domain of the application was presented to these stakeholders in order they had a clear view about the scope of the application. Subsequently, a hands-on exercise about the use of the technique in practice was run. This step allowed the participants to clarify their doubts and consequently it avoids misunderstandings and misusing of proto-persona+. Finally, the proto-personas+ artifacts were constructed by using the template with the guideline ques- tions and supported by the information presented in the pre- vious steps. In the second round, the focus moved on to the use of the information described in the proto-personas+. The artifacts produced in the previous round were explored, and the infor- mation on it supported the construction of the user interface prototypes. To make suitable the usage of the information provided by the non-technical stakeholders, we conducted some actions to the participants (i.e. developers) got the ex- pertise to use the artifact. First, the developers took a part of training session about the concepts of proto-personas+ and how to use these artifacts in the practice. The scenario used in the previous round was presented to keep the same scope of the application. After, a hands-on exercise was run with the purpose of the participants become acquainted with proto- personas+ artifacts. This hands-on focused on the reading of the details available on the proto-persona+ for then extracted the information the developers considered relevant. An arti- fact example was delivered to the developers that should read and explore it as well as ask questions for clarifying their doubts. Afterwards, all the proto-personas+ produced in the first round were offered to the developers. They could select the ones they considered that provided useful information to their activity of prototyping the user interfaces. We must mention that the UX requirements that were raised are relative to a more minimalist application in the m-learning area. This enables them to be reused within the same scope. However, the exploration of results in other e- learning applications should be done to verify these require- ments reuse. It is also relevant to discuss the scope of the answers to our research questions. Since this a first study about the con- tribution that non-technical stakeholders bring to the specifi- cation of UX requirements, we tried to understand this phe- nomenon, and we asked exploratory questions, Easterbrook et al. (2008), aiming at characterizing the non-technical stakeholder contributions. However, the answers to our re- search questions are context-dependent. Different stakehold- ers would describe different UX requirements. Neverthe- less, the answers to these questions result in a clearer under- standing of the phenomenon, since they show that the non- technical stakeholders bring a valid contribution to the speci- fication of UX requirements. Considering (RQ1): Which UX requirements do non-technical stakeholders describe while using the proto-persona technique?, we could answer that the non-technical stakeholder has elicited different UX require- ments when compared to the technical stakeholders. On ex- ploring the artifacts that both types of stakeholders produced, we could affirm that they contributed in different perspec- tives. The first round showed that both types of stakeholders described the UX requirements differently e.g., the differ- ent actors described in their proto-personas’ characteristics of how to keep the student using the application. While the pedagogues pointed out that students would be encouraged by the enjoyable features that would give rise to the inter- action fun, the software engineers preferred to mention the student motivation focused on dealing with student frustra- tion. These approaches are a reflection of the requirements that the e-learning application should have to delivery fun in a learning space, Gomes et al. (2018). Another evidence of the different contributions that both types of stakeholders brought was seen in the user profiles described by them. The pedagogues suggested profiles in which accessibility issues were at the center, whereas the software engineer provided the description of profiles associated with developing work in groups. Therefore, it can be inferred that the knowledge of both are complementary. Our findings restate the need of an interdisciplinary participation of various stakeholders, Fer- nandez and Wagner (2015). Concerning the (RQ2): How is the acceptance of the use of the proto-persona+ technique by these stakeholders?, we concluded that the technique proved to be suitable to be used by both types of stakeholders. We can point out some dif- ferent perspectives in using the technique. The pedagogues showed greater degrees of importance in the use of the de- mographic quadrant. This represents an important result on the description of end-users. Therefore, this quadrant reports an individual’s personal information that can contribute to the construction of a picture of the end-users; it can conse- quently boost the development of empathy between the de- velopers and the audience, Billestrup et al. (2014); Ferreira On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 et al. (2018b). In the guideline questions, it was noticed that the two stakeholders demonstrated different perceptions for each questions. As a result, it was seen that these different perceptions can provide complementary viewpoints on the audience, thereby enriching the details about the end-user. By observing the perceptions of ease-of-use and usefulness of use, the findings showed that the non-technical stakeholders presented easiness in using the technique. These results re- vealed that the proto-persona+ is a suitable technique to be handle non-technical stakeholders with the purpose of elicit- ing UX requirement. Other techniques could be taken into account to elicit- ing UX requirements. However, often personas are artifacts that stimulate the discussion about the end-user needs in- depth. Therefore the proto-persona+ provided an adaptation of proto-persona which the aim of being easier to the use by the stakeholders. By answering the RQ2, we could verify that the proto-persona+ was suitable to capture the particu- lar knowledge of the different type of stakeholders. This re- vealed that the different types of stakeholders can contribute to describing different UX requirements. By answering (RQ3): Which UX requirements presented in the proto-personas+ can support the prototyping of user in- terfaces?, we noticed what were the sets of UX requirements presented in the proto-personas+ that supported the develop- ers on the prototyping of solutions. By comparing which UX requirements are presented in the storyboards we saw that the proto-personas+ of both types of stakeholders (i.e., ped- agogues and software engineers) provide information that these artifacts supported the developers in the design of so- lutions. Additionally, the findings from storyboards analysis reaffirmed that both stakeholders provided complementary information Fernandez and Wagner (2015). 10 Study Limitations Considering all the steps of our study, we can highlight some limitations which we discuss follow. Proto-persona is an approach that focuses on providing a sketch of the representative group of people in a specific do- main. From conducting workshops the proto-persona tech- nique allows the participants (i.e. stakeholders) to achieve a shared understanding about the audience. One of the advan- tages is that the technique offers a practical way to gather the specialists’ knowledge and discuss their inputs about the end- users. However, as the proto-persona is built from assump- tions about the end-users it presents some limitations regard- ing their validation. Differently from proto-persona, the tra- ditional persona is constructed by using data gathering from the audience. To mitigate the problem of not collecting data from real end-users, Gothelf proposes that proto-persona val- idation should be carried out later. We did not performed this validation, this could be conducted in another study. We can point out as another limitation, the fact that this study was conducted with a specific group of stakeholders in a specific city in Brazil. Further studies are necessary to re- iterate the proposed methodology as a generalized approach to capture non-technical stakeholder knowledge in other con- texts. Up to now, our research did not compare the results of the proto-personas+ with different approaches that use tradi- tional personas; or even no personas to elicit requirements from non-technical stakeholders. Therefore, we do not claim that applying proto-personas+ leads to a better result than us- ing traditional personas approaches. We also do not claim that the proto-personas+ results are better than not apply- ing any persona approach at all. Further comparative studies are needed to fully understand the effectiveness of the proto- persona approach. Our results must not be generalized to all scenarios and the particularities of our study must be considered. Proto- persona construction should be seen as a tool to encourage the sharing and discussion of stakeholder knowledge. This study investigated whether the proto-persona is suitable to be used by both technical and non-technical stakeholders to support the UX requirement elicitation. 11 Conclusions and Future Work This paper presents an experimental study that aimed to ex- plore whether a non-technical stakeholder contributes to the description of UX requirements. To conducted the study, we applied the proto-persona+ technique. The results showed that the non-technical stakeholder contributed by giving de- tails about the end-users in a complementary view of the tech- nical stakeholder. Considering the types of UX requirements the participants described, we noticed that the non-technical stakeholders raised different ones. Fun and accessibility issues were found exclusively in the proto-personas+ created by these stake- holders. Accessibility issues are fundamental to meet the needs of a wide range of end-users in the domain we explored in this study. In addition, by taking into account fun issues these stakeholders demonstrated their concerns on motivat- ing the users to keep engaged in the application. We could conclude that by describing these type of UX requirements the non-technical stakeholders had an important contribution on eliciting requirements which have a great impact on the experience of the end-users. The results of our second round revealed that the user inter- face prototypes produced by the developers encompassed dif- ferent UX requirements in a complementary way. We could see that the prototypes presented a diversity of details about UX. We could conclude that for the design of the proto- personas+ of the different stakeholders allow the developers to build more comprehensive prototypes at the same time that provided minimalist solutions. To sum up, we could point out that our study provided two important contributions. First, our investigation brought the discussion of how a non-technical stakeholder can con- tribute to the elicitation of requirements that are linked to the end-users characteristics. Our findings revealed that the non-technical stakeholder can be a co-participant in the elic- itation process and not just a provider of information. In ad- dition, we extended the proto-persona technique by creating the proto-persona+ and showing that our proposal is suitable for the purpose of including the non-technical stakeholder in the process of eliciting UX requirements. Our work also pre- On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 sented as contribution the structuring of a qualitative analysis that can be replicated in other studies on UX requirements. As future work, we intend to carry out studies on the qual- ity of the low fidelity prototypes by conducting an usability inspection on these. We also intend to evaluate the quality of the storyboards on the perspective of domain experts that in our case are the pedagogues. 12 Acknowledgements We thank the financial support of the Coordenação de Aper- feiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. We also thank the grant #2013/25572-7 São Paulo Research Foundation (FAPESP) and the support of CNPq (311494/2017-0). References Abelein, U., Sharp, H., and Paech, B. (2013). Does involv- ing users in software development really influence system success? IEEE Software, 30(6):17–23. Alves, C. and Ali, R. (2018). A persona-based modelling for contextual requirements. In Requirements Engineer- ing: Foundation for Software Quality: 24th International Working Conference, REFSQ 2018, Utrecht, The Nether- lands, March 19-22, 2018, Proceedings, volume 10753, page 352. Springer. Anvari, F., Richards, D., Hitchens, M., and Babar, M. A. (2015). Effectiveness of persona with personality traits on conceptual design. In Proceedings of the 37th Inter- national Conference on Software Engineering-Volume 2, pages 263–272, Florença, Itália. IEEE Press. Aranda, A. M., Dieste, O., and Juristo, N. (2016). Effect of domain knowledge on elicitation effectiveness: An inter- nally replicated controlled experiment. IEEE Transactions on Software Engineering, 42(5):427–451. Ardito, C., Costabile, M. F., Marsico, M. D., Lanzilotti, R., Levialdi, S., Roselli, T., and Rossano, V. (2006). An ap- proach to usability evaluation of e-learning applications. Universal access in the information society, 4(3):270–283. Berti, S., Paterno, F., and Santoro, C. (2004). Natural devel- opment of ubiquitous interfaces. Communications of the ACM, 47(9):63–64. Bhattarai, R., Joyce, G., and Dutta, S. (2016). Information se- curity application design: Understanding your users. In In- ternational Conference on Human Aspects of Information Security, Privacy, and Trust, pages 103–113. Springer. Billestrup, J., Stage, J., Nielsen, L., and Hansen, K. S. (2014). Persona usage in software development: advantages and obstacles. In The Seventh International Conference on Advances in Computer-Human Interactions, ACHI, pages 359–364, Barcelona, Espanha. Citeseer. Brown, J. M., Lindgaard, G., and Biddle, R. (2011). Collab- orative events and shared artefacts: Agile interaction de- signers and developers working toward common aims. In 2011 Agile Conference, pages 87–96. Castro, J. W., Acuña, S. T., and Juristo, N. (2008). Integrat- ing the personas technique into the requirements analysis activity. In 2008 Mexican International Conference on Computer Science, pages 104–112. Chimalakonda, S. and Nori, K. V. (2013). What makes it hard to apply software product lines to educational tech- nologies? In 4th International Workshop on Product LinE Approaches in Software Engineering. Choma, J., Zaina, L. A. M., and Beraldo, D. (2016a). Userx story: Incorporating UX aspects into user stories elabora- tion. In Human-Computer Interaction. Theory, Design, Development and Practice - 18th International Confer- ence, HCI International 2016, Toronto, ON, Canada, July 17-22, 2016. Proceedings, Part I, pages 131–140. Choma, J., Zaina, L. A. M., and da Silva, T. S. (2016b). Softcoder approach: promoting software engineering academia-industry partnership using cmd, DSR and ESE. J. Software Eng. R&D, 4:8. Clark, R. C. and Mayer, R. E. (2007). e-Learning and the Sci- ence of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. Pfeiffer, 2nd edition edition. Cooper, A., Reimann, R., and Cronin, D. (2014). About Face 2.0 The Essentials of Interaction Design. John Wiley & Sons Wiley. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technol- ogy. Management Information Systems Research Center, 13(3):319–340. de la Vara, J. L., Wnuk, K., Svensson, R. B., Sanchez, J., and Regnell, B. (2011). An empirical study on the importance of quality requirements in industry. In 23rd International Conference Software Engineering and Knowledge Engi- neering, pages 438–443. SEKE. Dias, G. A., da Silva, P. M., no Junior, J. B. D., and de Almeida, J. R. (2011). Technology acceptance model (tam): Avaliando a aceitação tecnológica do open journal systems (ojs). Informação & Sociedade, 21(2):133–149. Dodero, J. M., García-Peñalvo, F.-J., González, C., Moreno- Ger, P., Redondo, M.-A., Sarasa-Cabezuelo, A., and Sierra, J.-L. (2014). Development of e-learning solu- tions: Different approaches, a common mission. IEEE Revista Iberoamericana De Tecnologias Del Aprendizaje, 9(5):72–80. Easterbrook, S., Singer, J., Storey, M.-A., and Damian, D. (2008). Selecting empirical methods for software engi- neering research. In Guide to Advanced Empirical Soft- ware Engineering, chapter 11, pages 285–311. Springer. Faily, S. (2008). Towards requirements engineering practice for professional end user developers: A case study. In Proceedings of the 2008 Requirements Engineering Edu- cation and Training, pages 38–44. IEEE. Fernandez, D. M. and Wagner, S. (2015). Naming the pain in requirements engineering: A design for a global family of surveys and first results from germany. Information and Software Technology, 57(1):616–643. Ferreira, B., Barbosa, S., and Conte, T. (2018a). Creating personas focused on representing potential requirements to support the design of applications. In Proceedings of the 17th Brazilian Symposium on Human Factors in Com- puting Systems, page 15. ACM. On the contributions of non-technical stakeholders to describing UX requirements by using proto-persona+ Pinheiro et al. 2019 Ferreira, B., Silva, W., Barbosa, S. D. J., and Conte, T. (2018b). Technique for representing requirements us- ing personas: a controlled experiment. IET Software, 12(3):280–290. Ferreira, B., Silva, W., Jr., E. A. O., and Conte, T. (2015). Designing personas with empathy map. In The 27th Inter- national Conference on Software Engineering and Knowl- edge Engineering, SEKE 2015, Wyndham Pittsburgh Uni- versity Center, Pittsburgh, PA, USA, July 6-8, 2015, pages 501–505. Filho, N. F. D. and Barbosa, E. F. (2013). A requirements cat- alog for mobile learning environments. In Proceedings of the 28th Annual ACM Symposium on Applied Computing, pages 1266–1271. ACM. Fisher, R. A. (1922). On the interpretation of χ 2 from con- tingency tables, and the calculation of p. Journal of the Royal Statistical Society, 85(1):87–94. Garcia, A., Silva da Silva, T., and Selbach Silveira, M. (2017). Artifacts for agile user-centered design: A system- atic mapping. In Proceedings of the 50th Hawaii Interna- tional Conference on System Sciences (2017). Garrett, J. J. (2010). The Elements of User Experience: User- Centered Design for the Web and Beyond. New Riders Publishing, Thousand Oaks, CA, USA, 2nd edition. Gomes, T. C. S., Falcão, T. P., and de Azevedo Restelli Tedesco, P. C. (2018). Exploring an ap- proach based on digital games for teaching programming concepts to young children. International Journal of Child-Computer Interaction, 16:77 – 84. Gothelf, J. (2012). Using proto-personas for executive align- ment. UXMagazine. Gothelf, J. and Seiden, J. (2013). Lean UX: Applying Lean Principles to Improve User Experience. O’Reilly Media. Grudin, J. (2006). Why personas work: The psychological evidence. In The Persona Lifecycle, chapter 12, pages 642–663. Elsevier Inc. Grudin, J. and Pruitt, J. (2002). Personas participatory de- sign and product development: An infrastructure for en- gagement. In PDC’02, pages 144–152. Hadar, I., Soffer, P., and Kenzi, K. (2014). The role of domain knowledge in requirements elicitation via interviews: An exploratory study. Requirements Engineering, 19(2):143– 159. Jansen, A., Van Mechelen, M., and Slegers, K. (2017). Per- sonas and behavioral theories: A case study using self- determination theory to construct overweight personas. In Proceedings of the 2017 CHI Conference on Human Fac- tors in Computing Systems, pages 2127–2136. ACM. Kashfi, P., Nilsson, A., and Feldt, R. (2017). Integrating user experience practices into software development processes: implications of the ux characteristics. PeerJ Computer Sci- ence, 3:e130. Kortbeek, C. (2016). Interaction design for internal corporate tools. Maceli, M. and Atwood, M. (2011). From human crafters to human factors to human actors and back again: Bridging the design time – use time divide. In End-User Develop- ment. IS-EUD 2011. Lecture Notes in Computer Science, volume 6654, pages 76–91. Springer. Nielsen, J. (1995). 10 usability heuristics for user inter- face design. https://www.nngroup.com/articles/ ten-usability-heuristics/. Online; acessado em 12 de agosto de 2016. Nielsen, J. and Norman, D. (2013). The Definition of User Experience. Osborn, A. F. (1979). Applied imagination. NewYork: Scrib- ner. Palomares, C., Quer, C., and Franch, X. (2017). Require- ments reuse and requirement patterns: a state of the prac- tice survey. Empirical Software Engineering, 22(6):2719– 2762. Rogers, Y., Sharp, H., and Preece, J. (2015). Interaction de- sign: Beyond human-computer interaction. John Wiley & Sons, United States, 4th edition. Salman, I., Misirli, A. T., and Juristo, N. (2015). Are stu- dents representatives of professionals in software engi- neering experiments? In Proceedings of the 37th Inter- national Conference on Software Engineering-Volume 1, pages 666–676. IEEE Press. Sharma, S. and Pandey, S. K. (2014). Requirements elicita- tion: Issues and challenges. In 2014 International Confer- ence on Computing for Sustainable Global Development (INDIACom), pages 151–155. Steinmacher, I., Conte, T. U., Treude, C., and Gerosa, M. A. (2015). Overcoming open source project entry barriers with a portal for newcomers. In ICSE ’16 Proceedings of the 38th International Conference on Software Engineer- ing, pages 273–284, Austin, Estados Unidos. Strauss, A. and Corbin, J. (1998). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, volume 4. Thousand Oaks, CA: Sage, 2 edition. Winckler, M., Bach, C., and Bernhaupt, R. (2013). Identify- ing user experience dimensions for mobile incident report- ing in urban contexts. IEEE Transactions on Professional Communication, 56(2):97–119. https://www.nngroup.com/articles/ten-usability-heuristics/ https://www.nngroup.com/articles/ten-usability-heuristics/ Introduction Fundamentals and Related Work Proto-persona+ Study context First Round: Using Proto-persona+ Planning Execution Analysis Threats to validity Findings of the first round UX requirements Acceptance of proto-persona+ Importance of quadrants Usage and relevance of the guideline questions Perception of usefulness and ease-of-use Second round: using the proto-personas+ in design Planning Execution Analysis Threats to validity Findings of the second round Developers' preferences Application of UX requirements Discussion Study Limitations Conclusions and Future Work Acknowledgements