Journal of Software Engineering Research and Development, 2023, 11:7, DOI: 10.5753/jserd.2023.2657 This work is licensed under a Creative Commons Attribution 4.0 International License. Education, Innovation and Software Production: the contributions of the Reflective Practice in a Software Studio Aline Andrade [ Pontifícia Universidade Católica do Paraná | alinesf.andrade@gmail.com ] Alessandro Maciel Schmidt [ Pontifícia Universidade Católica do Paraná | alessandromacielschmidt@hotmail.com ] Tania Mara Dors [ Pontifícia Universidade Católica do Paraná | taniadors@ppgia.pucpr.br ] Regina Albuquerque [ Pontifícia Universidade Católica do Paraná | regina.fabia@pucpr.br ] Fabio Binder [ Pontifícia Universidade Católica do Paraná | fabio.binder@pucpr.br ] Dilmeire Vosgerau [ Pontifícia Universidade Católica do Paraná | dilmeire.vosgerau@pucpr.br ] Andreia Malucelli [ Pontifícia Universidade Católica do Paraná | malu@ppgia.pucpr.br ] Sheila Reinehr [ Pontifícia Universidade Católica do Paraná | sheila.reinehr@pucpr.br Abstract The growth of the mobile phone market has been generating a great demand for professionals qualified for applications (APPs) development. The required profile includes technical skills, also known as hard skills, and behavioral or soft skills. The training of these professionals in speed, quantity, and quality demanded by the market poses a significant challenge for educational institutions. Apple and PUCPR have established a partnership to build a software studio to develop such talents using the Challenge Based Learning (CBL) method and associated practices whose effects need to be studied. This research aims to analyze the contributions of reflective practice in a software studio to teach the main professional competencies regarding app development, including hard and soft skills. The research method was the case study, based on semi-structured interviews with 28 participants in three cycles. The collected data were analyzed with open and axial coding from Grounded Theory and Atlas.ti tool. The results demonstrate that reflective practice, applied in a software studio environment that uses CBL was able to help students to map new ideas and acquire valuable hard and soft skills. The study pointed out that reflective practice is an effective instrument for developing the skills required by the app market, which demands innovation and quality at high speed. Keywords: Reflective Practice, Software Studio, Challenge Based Learning, Software Quality Education, App Development 1 Introduction The demand for technological products has been growing in recent years, which requires better training of computing professionals, especially for developing applications for mobile devices. Among the most requested competencies are technical knowledge (hard skills) and behavioral skills (soft skills), such as teamwork, collaboration, and communication. These abilities are crucial once information technology (IT) professionals tend to be more introspective. Although students perceive the development of soft skills as relevant, studies show that there is not the same degree of concern about acquiring these skills compared to more technical skills (Lima and Porto 2019). The Apple Developer Academy, or simply Academy, is a technological innovation project run in partnership between university environments and Apple through a course that offers a complete education to students, allowing them to learn how to code, test, and publish applications based on their ideas. Academy is a software studio (Bull et al. 2013) that uses active and collaborative learning methods and tools to contribute to the student's practical learning and skills development. Its staff consists of instructors composed of programmers and designers, available at the studio daily. For the two-year extension course, 50 graduate students or students to graduate within six months were selected. There are designers, developers, and devigners who are students able to work with both skills as designers and developers (Dors et al. 2020). The Academy uses the Challenge Based Learning (CBL) method to support the mobile application development process, which is based on challenges proposed to students. This is a definition established by Apple as a contractual item. One of the practices associated with CBL is the reflective practice. Reflective practice is a feature of the software studio supported by formal and informal feedback from teachers to students, which can be mentoring or critiquing to improve the outcome (Bull et al. 2013). In this learning environment, students are exposed to social interactions, group work, oral presentations, and discussions of their work with peers (Kuhn et al. 2002). The course instructors use an approach that follows the guidelines established by the partnership and those related to the studio concepts. Thus, students, through workshops, receive theoretical content. Throughout the development of the challenges, instructors use coaching and mentoring reflectively with the students, according to the software studio concepts. The instructors encourage the students to reflect and find solutions independently, i.e., the instructors do not give the answers but the tools and conditions for the students to develop them. The presence and importance of reflective practice are recognized in the software engineering educational literature, according to Dors et al. (2020) and Bull and https://orcid.org/0009-0000-4239-4464 https://orcid.org/0009-0001-8853-3295 https://orcid.org/0000-0003-1167-7685 https://orcid.org/0000-0003-1564-1743 https://orcid.org/0000-0001-6682-3868 https://orcid.org/0000-0002-9508-0888 https://orcid.org/0000-0002-0929-1874 https://orcid.org/0000-0001-9430-7713 Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Whittle (2014). It is a form of reflective-based learning, as the name implies, that ranges from constant questioning, teamwork, peer review, and collaborative learning to group problem-solving. The concept was initially proposed by Donald Schön when observing architecture studios. He suggested thinking about professional practice, relating theory to practice, and generating terms such as reflection- in-action, reflection-on-action, and conversation with the material (Schön 1983). Reflective practice in architecture has proven effective in assisting soft skills development, improving performance, and helping students acquire an artistic talent essential for professional competencies (Schön 1983 and Hazzan 2002). The contributions of such an approach to computer science education are described by Bull and Whittle (2014) regarding the technical and attitudinal skills of the software engineer. The latter are improved decision-making skills, teamwork, communication, planning, and time management. With so many positive results aimed at teaching mobile application development from the software studio education approach, an interest arose in deepening the understanding of the results provided using the reflective practice, extending the study of Dors et al. (2020) to the analysis of a more extensive set of students and projects. Dors et al. (2020) analyzed data from the Academy's 2017-2018 student class, conducting a face-to-face ethnographic observation study. The present study analyzed data from the 2019-2020 and 2021-2022 classes obtained through semi- structured interviews and constituting a longitudinal study. New findings were identified, complementing Dors et al. (2020), as can be seen later in this article. 2 Background The different methodological approaches used by teachers throughout history have intrinsically aimed at the same scenario: to enable the learner to act autonomously in diverse professional situations. However, traditional methods that focus only on lectures are not enough to develop the competencies required by today's society. Active methodologies, which place the student at the center of the learning process, have been widely used in universities worldwide and, in recent years, also in Brazil. They seem to offer better results for the development of the required competencies. Regardless of the methodology used, Ferraz and Belhot (2010) said that it is not enough to focus on the content to be covered to conclude the teaching-learning process efficiently. It is necessary to plan and structure activities to be developed, resources available, methodologies adopted, and evaluation tools used. 2.1 Collaborative Learning Collaborative learning is one way to overcome the challenges faced by traditional teaching methodologies. According to Barkley et al. (2014), a collaborative approach meets the following criteria: (i) the activity design must be intentional and carefully undertaken by the faculty member and not just limited to assigning some group activity; (ii) all group members must effectively engage in the activity and contribute equally to the outcome; and (iii) meaningful learning, related to the learning objectives of the discipline, must occur. Briefly, collaborative learning is about "two or more students working together and sharing the workload equally as they progress toward the intended learning outcomes" (Barkley et al., (2014)). The process of collaborative learning refers partly to the metacognition process, which means getting the students to reflect on their learning process. Besides choosing the technique, implementing collaborative approaches implies properly defining how to organize the groups, encourage collaboration, and conduct the assessment. 2.2 Hard and Soft Skills It is prevalent for undergraduate courses to focus on developing technical skills so that the future professional can work in his/her field. These are also known as hard skills. However, these are insufficient to build an excellent professional to cover current market demands. Behavioral skills, also called soft skills, are equally or even more relevant in this journey. According to Agante (2015), soft skills are non-technical competencies such as communication, empathy creation, trust with groups, and resilience in a work environment. Competency is "a set of capabilities (knowledge, skills, attitudes, and values) mobilized for a delivery, which adds value to both the individual and the organization" (Fernandes 2013). The survey developed in July 2021 by the American company CareerBuilder with 2,138 managers and human resources professionals pointed out that 77% of interviewees believe that soft skills are essential for the job. Carter, Ferzli, and Wiebe (2007) state that although communication skills are vital for an effective professional, these skills usually fall short of employers' expectations of recent technology graduates. Several universities already recognize the need for computer science students to acquire these skills and incorporate teaching methods that favor their development. Studies show the importance of communication in technology because students learn what it means to think like computer scientists and be professionals in the field (Burge et al. 2012). 2.3 Software Studio One of the approaches to developing these competencies is the software studio, which comes from the historical tradition of the École des Beaux-Arts and the Bauhaus and its atelier model (Dors et al. 2020). According to Tomayco (1991), the software studio emphasizes developing reflective skills and sensibilities and is the reflective practice the essence of the atelier concept. Collaborative learning in the studio helps students to develop their skills through practice. Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Furthermore, the dynamic interconnection of elements in a studio, such as people, software tools, development methodologies, processes, techniques, and products, provides a network in which software development knowledge and skills are created (Prior et al. 2014)). Reflection generally occurs in cycles of experience followed by consistent reflection to learn from that experience, during which the developer can explore comparisons, weigh alternatives and diverse perspectives, and generate inferences, especially in new and/or complex situations (Dybå et al. 2014). According to Shön (1983), reflection-in-action occurs during problem-solving, with doing and thinking as complementary ways. Reflection-on-action is about thinking about a different approach to an already executed process. Finally, conversation with the material refers to a conversation with the product that has been developed. Reflection-in-action is the reflective form of knowing-in- action, reflecting during the problem-solving process. In the reflection-in-action process, doing and thinking are complementary. Knowing-in-action is the knowledge built into and revealed by our performance of everyday action routines (Schön 1983). Sometimes it is labeled as intuition, instinct, or motor skills. In such cases, one continually controls and modifies his/her behavior in response to changing conditions (Schön 1987). In such cases, we continually control and modify our behavior in response to changing conditions. “This capacity to do the right thing ... exhibiting the more that we know in what we do by the way in which we do it, is what we mean by knowing-in-action. And this capacity to respond to surprise through improvisation on the spot is what we mean by reflection-in- action". (Schön 1987). Carbone and Sheard (2002) reported the reactions of first- year students to being exposed to a new learning environment that consisted of differentiated physical space, a new teaching approach, IT facilities, and a new assessment method. This space was a workshop whose approach was established in 2000 in the School of Management and Information Systems, Bachelor of Management, and Information Systems (BIMS), at Monash University (Australia). The studio-based teaching and learning approach adopted was based on the Bauhaus School of Design model. The Bauhaus School introduced a radical change from the traditional art education model, completely reshaping the teaching and learning spaces at that time. The atelier aims to allow students to develop strategies to cooperate and collaborate. The authors concluded that, in general, most first-year students enjoyed learning in the studio environment. An unexpected finding was the evidence of students developing metacognitive skills. Danielewicz-Betz and Tatsuki (2014) analyzed reflective practice concerning the outcomes of a software workshop in undergraduate and graduate software courses. The analysis focused on the interaction between students and clients to determine how and to what degree students transformed through collaborative project-based learning. During the final self-reflection, students reported improving their project management, communication, presentation, writing, business, and software development skills. The reflective practice was analyzed and focused on collaborative learning and students' relationships with clients. Prior et al. (2019) described a study based on open-ended interviews and ethnographic observations in studio sessions to understand how this experience impacted students’ employability. Students observed that the studio experience helped enhance their technical and non-technical employability skills. In addition, from interviews with mentors and academics, the study corroborated the students’ views. They concluded that the relevant skills for employability include collaboration and communication, project management, mutual support to solve technical problems with help from industry mentors and academics, social aspects of the work, reflection skills, and technical skills were found to be essential employability skills. According to Marques et al. (2018), adopting reflective practice (Reflexive Weekly Monitoring - RWM) is a way to improve learning for computer science students. The authors followed nine semesters of a project discipline and concluded that the approach effectively improved student coordination, effectiveness, sense of belonging, and satisfaction. 3 Research Method The research was conducted in a case study format based on data collection through semi-structured interviews (Yin 2017). This method constitutes a research strategy that aims to understand the dynamics in a contemporary context over which the researcher has no control. It is appropriate to answer "how" and "why" questions. The study's main objective was to understand the contributions of reflective practice to technical and non- technical skills development in a software study. We followed the steps defined by Yin (2017): (i) Definitions and planning; (ii) Preparation, data collection, and analysis; (iii) cross-analysis and conclusions. The research planning involved case selection and preparation of the research protocol. The Informed Consent Form (ICF) and the Non-disclosure Agreement (NDA) were prepared and signed by all researchers involved in the project. The project went through the analysis by the Research Ethical Committee, receiving its approval (number 4.209.411) on August 12th, 2020. The underlying question for this study was: How is the reflective practice performed in software development studios? The following complementary questions arise from this general question: How is the reflective practice carried out in software studio environments? How can reflective practice contribute to the learning of computer science students? The present study was characterized by a prospective design in a qualitative research format. The first and second collection data rounds occurred between January and May 2021 and referred to the 2019-2020 class. The third collection cycle occurred from December 2021 until January 2022 and referred to the 2021-2022 class. The sample was determined through convenience time series. Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 According to the inclusion and exclusion criteria, individuals considered eligible to participate in the study answered a semi-structured interview. The unit of analysis of this project is the Apple Developer Academy (called Academy), a software studio constituted in the scope of the partnership between Apple and PUCPR. Candidates go through a selection process that identifies the most appropriate profiles. These students then undergo a two-year training period, exposing them to several challenges. Included in the research were Academy students over 18 who agreed to collaborate with the study. As selection criteria, we adopted that: the students should have participated in active learning using reflective practice and attended the class workshop immediately before. The project was divided into three cycles. In the first cycle, ten students participated in the interview. This stage was considered a pilot project and aimed to understand the benefits of reflective practice from the student's point of view. The first author initially developed the semi-structured script, which the other authors later revised. The script contained ten open questions to understand the Academy student's perspective on reflective practice, ranging from usefulness, learning, and future applications outside the academic environment. At the end of this stage, a preliminary data analysis was performed to adjust for the second collection cycle. For the second cycle, the questions were adjusted, allowing a deeper exploration of items that emerged from the first collection. The second cycle comprised eight interviews that took place remotely and synchronously, each lasting 30 to 60 minutes, as had already happened in the first cycle. No adverse effects were perceived because the interviews took place online. The third and last cycle was conducted with twenty students from the 2021-2022 class. The interviews were undertaken in a synchronous remote way in two phases. The interviews for all cycles were recorded in audio format and later transcribed with the interviewee's permission. The information was mapped and analyzed using the support of Atlas.ti tool. The analysis of the results used the open and axial coding of Grounded Theory (Strauss and Corbin 2007). Open coding is a microanalysis of the transcribed interviews, performed line by line, identifying concepts and memos records (researcher's notes) about the meaning of the codes and categories. On the other hand, axial coding refers to grouping codes of the same properties in the form of networks. The results present the behavioral and technical skills acquired from applying reflective practice. The study participants had no direct benefit from the project. However, the research contributed to the planning and development of future actions aimed at improving the skills of technology students in the researched environment. It can be understood that an indirect benefit for the Academy students was their reflection on reflective practice, performed through the researcher's inquiries. 4 Results This section details the process of analyzing and triangulating the collected data, and it is organized into subsections according to the collection cycles. 4.1 First Cycle Ten students between 18 and 24 years old were interviewed in the first cycle. Of these, five were male, and five were female. The interview follows the script shown in Table 1. Table 1. Script for the semi-structured interview - Cycle 1. Script for the semi-structured interview - Cycle 1 Have you done activity reflections before joining the Apple Developer Academy? How was your first reaction when you discovered that you would need to reflect on your challenges? Why? Considering programming, design, and business, what have you learned technically using reflective practice? How do you think reflective practice contributed to developing your behavioral skills? How do you think reflective practice contributed to developing your behavioral skills? How was the critique carried out? Can you explain to me what it was like to receive and give a review of a challenge? After doing some reflection, did you avoid any mistakes in the development of the challenge, and consequently, did you see new attitudes for the following activities? If yes, explain how. How was your last reflection compared to the first one? In the future, do you intend to continue using reflective practice in new projects? Can you explain how you intend to use it? Finally, could you tell me the most significant benefits of reflective practice? Interviewees answered about the process of reflective practice based on a semi-structured script with more comprehensive questions that would allow for more in- depth results. Most students stated that they had conducted reflections before participating in the Academy. When asked about their reactions to finding out that reflection would be mandatory, most students used positive words such as cool and productive. Only three participants demonstrated negative expressions, such as confusing, complicated, or boring. From the open coding analysis, it was obtained the axial network that reflects soft skills learning as presented in Figure 1. This category comprises the following subcategories: leadership, communication, and teamwork. Teamwork was very evident during this study. Students could develop technical and behavioral skills during team interactions, such as collaboration among colleagues, managing time, and improving communication. Examples of these statements can be seen later in this text. Regarding teamwork, we found evidence of learning about conflict management. This emerged from statements concerning arguments and stress between colleagues in the same team. Through reflection, these students were able to manage these disagreements better. Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Figure 1. Soft Skills (behavioral competencies) - Cycle 1. Figure 2 presents the results of the technical knowledge obtained through reflective practice. The most cited terms were technical learning, writing learning, and project management. It was also possible to analyze that many students could identify the depth of their knowledge about programming after doing the reflections. Figure 2. Hard Skills (Technical Competences) - Cycle 1. The technical competence considered most relevant by the interviewees was project management. Usually, students have no previous practical experience with project management except for graduation work. The studio provides them the practical experience throughout the course. They perceived this theme as planning skills, activities standardization, error analysis, problem identification, and time management. Therefore, the results concerning personal development obtained from the reflective practice were: improvement in the student's technical performance, self-knowledge, and learning evolution in hard and soft skills, which led to a continuous improvement process for the students. Students realized that through reflection, self-knowledge could be developed, which brings increased self-confidence. The evolution of learning, whether in the development of soft or hard skills, is also perceived as a cause of personal growth that makes one learn to deal better with feelings. During the interviews, many students stated that they reflected in search of improvements in their technical and behavioral results. With this, they started to analyze themselves more critically, finding mistakes and successes to deal with the following activities differently and obtain future learning and improved performance. Self-knowledge was highly mentioned among the interviewees. Through the practice of reflection, students could know themselves, understand their preferences, and what made them more confident in performing the Academy activities. Table 2 presents some interview quotations regarding the soft skills found in this first cycle. An “S” followed by a number replaced the student's identity (e.g., S1, S2). Table 2. Soft Skills Quotations – Cycle 1. Soft Skills Quotations Self-knowledge "[...] self-knowledge for sure, more patience to understand my process, understand my time, decrease anxiety and stress […]" (S2) "[...] I believe the greatest benefit that reflective practice has brought to me is the self- knowledge [...]" (S6) Planning "[...] It is assuredly very favorable to prepare yourself for the next challenge better [...]" (S8) Self-confidence "[...] Fell me more confident regards the skills that I possessed." (S1) Time Management "[...] Learn how to manage time [...]" (S8) Communication "[...] And another thing was a personal relationship. I believe you think about what interests you, what you talk about, the way you talk. I learned better this relationship with other people […]" (S8) "[...] learn how to communicate verbally better [...]" (S2) Learning Evolution "[...] I think a very cool thing was seeing the evolution over time [...]" (S1) "[...] not making the same mistake twice [...]" (S2) Personal Development Improvement “[…] creating a concept, doing the more artistic part, doing the development part, doing the presentation later, so you interact with various aspects. And I think through reflective practice. I was able to understand better how my development process was […]” (S2) Technical Development Improvement “[…] think, learn to communicate better verbally, […], not making the same mistake twice, improve what you do, like something that worked […]” (S5) Critical Thinking "[...] to make that change in behavior which is in line with the critical analysis [...]" (S3) The encodings were performed by one of the authors and reviewed by the others. These can be grouped in the category that Donald Schön (1983) defines as a reflection- on-action, the reflection after the action is performed. 4.2 Second Cycle In the second cycle, eight more students were interviewed. The basis for these interviews was the revised script shown in Table 3. Table 3. Semi-structured Interview Script – Cycle 2. Script for the semi-structured interviews - Cycle 2 Have you ever done any reflection before joining the Apple Developer Academy? What was your first reaction when discovering that you had to reflect on your challenges? Why was that? Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 In the day-to-day life of the Academy, did you notice yourself reflecting during the execution of your activities? How was it? What were the moments when you needed to record the reflections? What was the difference between reflecting on the short-term and the long-term challenge? What are the differences between doing reflections with guidelines for their execution and those with a free format? What are the benefits of both types of reflections? What are the differences between learning when working as a team and individually? How were the reflections during the planning of the challenges? What about development? And in the delivery of the final product? There were only sessions with the instructor to reflect on the progress during the development of challenges. How were these moments? What was it like to receive feedback from the instructor? What skills have you developed to perform presentations in public sessions? What kind of learning or skill was developed or required to perform the division of tasks in teams? What kind of challenge reflections, learnings, or experiences was helpful in another challenge? What benefits have you brought to your professional life using reflective practice? In addition to encouraging individual reflection, the Academy has specific moments for students to reflect critically on other students'/teams' work. What was it like to make and receive criticism about the CBL and Design in the review sessions? How have the reviews contributed to your personal development or development of the challenge/final product? Were you able to avoid any mistakes in any challenge after reflecting and consequently seeing new attitudes to the following activities? If so, explain how. How was your last reflection compared to the first one? Do you continue to use reflective practice in your projects? How did that become part of your day-to-day life? Finally, could you tell me the most significant benefits the reflective practice has brought you? With the changes implemented, nine additional results were found to be added to the networks obtained in Cycle 1. The analysis allowed us to divide them into three technical skills, three behavioral skills, and three classified as the personal benefits of reflective practice. The ability to speak English was the only property of the technical skills that differed from Cycle 1. The other changes were related to project management. The following were cited as relevant points: clarity in the execution of project activities and division of tasks, as shown in Figure 3. As can be seen, the network was refined due to the best understanding of the scenario. Figure 3. Hard Skills (Technical Competences) - Cycle 2. Figure 4 presents the soft or behavioral skills mapped in Cycle 2. It is possible to observe an outstanding factor that refers to knowing how to listen to colleagues once listening to different opinions is fundamental for good communication. All interviewees mentioned this skill. In addition, this ability is in great demand in the professional market and is essential in personal development through feedback. The students identified that improving the organization of presentations results in behavioral skills development, especially those that facilitate communication. This was a skill developed because they had to make several presentations of their projects throughout the course. Figure 4. Soft Skills (Behavioral Competencies) - Cycle 2. The reflective practice significantly impacted the students' personal evolution and behavior improvement. The active methodology also helped the students to evolve their learning through reading their classmates' reflections and personal development. As can be seen, it was possible to evolve the networks to contemplate the findings made through the analysis of the Cycle 2 interviews to find the hard and soft skills developed from reflective practice. Table 4 presents the quotations extracted from the interviewees' speeches and represents the most frequently mentioned skills in the interviews in Cycle 2. Knowing how to listen to one's colleagues was highly cited among the interviewees in this second cycle. Through reflection, students could better understand their colleagues and the importance of listening to them. Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Table 4. Hard Skills Quotations - Cycle 2. Hard Skills Quotations Active Listening “[...] it is a lot about listening to other people you know, and understanding their thinking in a good way..." (S13) “[...] you learn to listen to what people want..." (S15) “[...] what I learned most from it was to listen to others before doing" (S17) Presentation Organization "I learned to organize for the presentations right, and go-getting the hang of getting better [...]" (S17) Behavior Improvement “I learned personally because we are dealing with people. So, I learned how to express myself or the form of my posture […] with people [...]" (S11) Technical Performance Improvement “I was able to improve my results, so, so for me, it's one of the main things, to improve results effectively, quickly […]" (S14) Learning by reading colleagues' reflections “[...] for me, learning from the mistake of others is also valid. I read the reflections of people who seemed interested in learning from them, so sometimes someone wrote and seemed dissatisfied, or someone who seemed very satisfied. I liked to read these reflections [...]" (S12) Ideas Sharing “[...] I knew then to present my thoughts [...]" (S12) “[...] I was able to expose better what I was thinking." (S15) “[...] I think the skill that I learned the most for the presentation, like this, was losing the shame, you know. Lose the fear of presenting or speaking my ideas.” (S13) English Speaking “[...] I developed a little bit in how to present in English, keep learning, and lose the fear right." (S18) Another interesting issue that emerged when analyzing the interviews was the observation that it is also relevant to learn through the mistakes and successes of colleagues, which was possible by reading the reflections made by the colleagues. The ability to organize also appears in the reflections about improving oral presentations. The students learned self-organization to show their work to colleagues and realized this throughout the reflection process. It is interesting to note that in the students' perception, by reflecting and analyzing the situations in which they were involved with their colleagues, they report having learned to deal with people and adopt a more appropriate posture towards the team. 4.3 Third Cycle The third cycle refers to 10 students participating in the 2021- 2022 Academy class. This cycle aimed to identify the contributions from each reflective practice concept, such as reflection-in-action, reflection-on-action, conversation with the material, and knowing-in-action. An expansion was made to a more significant number of quotations and subsequent refinement of the second cycle networks to meet the interrelationship of codes with the third cycle, so new codes emerged from the interpretation. In this collection cycle, ten students answered questions from a semi-structured script about their studio Showcase Experience from the perspective of reflective practice contributions (see Table 5). The Showcase is a studio session where students present their projects to other studio students. The best project receives an award. They were also interviewed in this cycle to investigate the conversation with the material. Table 5. Semi-structured Interview Script - Cycle 3. Script for the semi-structured interviews - Cycle 3 Please comment on your experience with reflections on challenges. Have you ever been introduced to reflection as a teaching methodology throughout your education? What aspects would your formal education have been different if you had used reflective practice? How do you believe that reflections with colleagues help create more interesting projects? Has this ever happened to you? What techniques would you use in case of imminent team conflict? How did the reflections affect the relationship between the team members? What technical skills did you develop? What interpersonal skills do you think you developed throughout studio activities? How will the materials produced, such as projects, codes, slides, presentations, and so on, influence the development of future materials? What is your process of revisiting the materials produced at the Academy like? Have you changed the materials produced due to reflections between Challenges? How have the team reflections helped to develop your creativity? What are the main lessons or learnings from the studio? Table 6 shows some of the quotations extracted from the interviews in Cycle 3. Table 6. Quotations - Cycle 3. Code Quotation Decision Making "[…] I believe would have... take more assertive decisions […] I had wasted less time because I would have had to stop to really focus on my ability to think about what's going on [...]" (S19) "[…] I think it brought me these insights into what I should do from now on, […] (S20) Conflict Management "[…] but basically, it was through conversations that we solved these problems [...]the third person who was the one who brought the conflict, admitted that they could have brought the situation in another way" (S24) “Then through the reflections, I realized that in most cases, this was not a good alternative, and I started opting for these conversations 100% of the times [...]" (S20) Reflection on mistakes and successes "[…] I think they kind of force you to look at everything you've done, look at all you did, and analyze what you did right or wrong. So, analyzing these practices, you can think on what keep doing, or what behaviors should I stop doing […]" (S22) Active Listening "[…] learning to give and receive feedback, to stop and hear feedback, was something I already was working on before, but the Academy gave me interesting ways to develop this [...]" (S19) Collaboration "[...] I think collaboration is a major one, work as a Team, [...]" (S21) Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Communication "[...] I learned a lot about how to communicate myself, [...]" (S22) Self-confidence "[...] also works for me to have confidence and believe in my own potential, [...]" (S21) Motivation "[…] It influenced me a lot to keep myself motivated […]" (S19) Upcoming Artifacts Production "[...] I think it influences a lot; I think that everything is a reference [...]" (S25) Professional Impacts “This helps a lot in the development of the personal portfolio.” (S22) “[...] when you need to reference these projects in some professional opportunity.” (S26) Academic Impacts “[...] like a portfolio, this is where you expose your projects, but more than that, you delve into a retrospective of how the project was carried out (why and how was the project developing). And this helps you not only document all the process[...]". (S29) “[...] something for the future, something I would apply to a future project or team.” (S24) “The biggest influence is on future projects, where I can use what was written and learned during the projects I participated in.” (S20) Self-knowledge “[...] to see better this way, how we are doing in a specific subject, to dedicate ourselves more or better understand what we like, and what we do not want to work on.” (S22) Synthesis Skill "[...] the skills I developed in the construction of this activity is self-knowledge and synthesis. [...]" (S29) Priority Management “I started to apply this model of reflection […] not only in my work but also in my study things. Then I realized that I had a much clearer vision of what I had to do or the alternatives I would prioritize in the steps forward.” (S20) From the analysis of the interviews, it was possible to observe that reflection-in-action promotes the behavioral improvement and the student’s continuous development. Throughout the challenge development, students had to manage situations of divergence of ideas, conflict of leadership, and the teams’ expectations, improving their conflict management skills. Students said this usually occurs at the beginning of the challenge while doing the project design. Conflict resolution comes through conversations and sometimes using the voting strategy. The students stated that, based on reflection-in-action, they realized they could make more assertive decisions regarding the project and their behaviors. It also develops leadership, which contributes to engagement and teamwork. Students pointed out reflection-on-action as a powerful tool to better understand their motivations, interests, and capabilities, contributing to their self-knowledge development. Students wrote a self-reflection at the end of each challenge to stimulate their reflection, promoting reflection- on-action. Reflection-on-action promotes self-knowledge, as well as self-reflection and creativity. These self- reflections promote work process comprehension, professional development monitoring (which stands for the ability to keep track of what is learning), reflections on mistakes and successes (which helped the students not to repeat the same mistakes and emulate behaviors that had positive results in the past) and, creativity. The reflective practice supported by studio sessions incentives having multiple views on a given theme, which promotes creativity. Interaction with creative colleagues contributes to developing creativity. Sharing ideas throughout the studio sessions stimulates teams to be open to different ideas. In addition, creativity is responsible for the creation of innovative projects. Figure 5 shows these reflection-on-action findings. Figure 5. Reflection-on-action - Cycle 3. The third collection cycle’s purpose was to explore the conversations with the material and knowing-in-action, other reflective practice concepts not covered in Cycle 2. From knowing-in-action, we could realize hard and soft skills development, as shown in Figure 6 and Figure 7. Concerning hard skills, as shown in Figure 6, knowing- in-action promotes the development of project management, project development, and presentation organization skills. Regarding the development of the project was noticed the design skills development. Some students had never made a mobile app design before the Academy course. In addition, they started to learn about code reuse-oriented development, software architecture, database development, App prototyping for different platforms such as MIRO, FIGMA, or Adobe Package, algorithmic logic and programming, object-oriented programming, and programming in Swift/ Swift UI. Figure 6. Knowing-in-action – Hard Skills - Cycle 3. Figure 7 shows the knowing-in-action findings regarding soft skills. They actively listen to colleagues’ ideas, communicate, collaborate, and self-criticism. Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Figure 7. Knowing-in-action – Soft Skills - Cycle 3. In this cycle, students reported soft skills that had not appeared in the previous cycle: responsibility and autonomy, storytelling, and communicating feedback. There was an evolution in their commitment to the project regarding deadlines and the execution of assigned tasks, improving responsibility and autonomy skills. Another soft skill developed by the students was storytelling. Students developed the art of telling stories while preparing app design and project presentations. They had to create an appealing backstory for the app and even engage colleagues in their presentations. The Academy’s activities are collaborative by nature, and the other new skills the students present relate to this specific characteristic. Through the development of collaborative tasks, students showed considerable improvement in communicating feedback to their colleagues, solving issues, and maintaining a respectful work environment. As shown in Figure 8, the Showcase stimulates the conversation exercise with the materials to studio students. As a result, the showcase experience positively influenced Upcoming Artifacts Production and promoted priority management, synthesis skill, motivation, self-confidence, learning by shared experience, and self-knowledge, which positively influenced professional and academic contexts. Figure 8. Conversation with the material - Cycle 3. Since the students were supposed to create brief presentations about complex projects, they had to develop priority management and understanding of the presentations' purpose and structure. Not only prioritizing the most relevant items but using synthesis skills to tell a story as effectively as possible. Enrolling in such activities helped students to boost their motivation and self-confidence. Students sharing experiences in the Showcase help them to learn from colleagues’ experiences, which positively influences professional and academic contexts. Written self-reflection supports reflective practice in the studio and can be carried out in two ways: free format or with guiding questions. The student perceives these skills development throughout the exercise of his metacognition. Metacognition is ”being aware of and able to monitor the development of one’s own learning and the application of that learning to their practice.” (Parson; Stephenson, 2005). The difficulties in applying the Reflective Practice found in the analysis are related to the student's lack of experience and physical fatigue. The latter is because students tend to work hard to complete the project on time. Consequently, they get tired after the delivery, and writing the reflections becomes difficult. 5 Discussion This study aimed to identify the contributions of reflective practice in a software studio, analyzing the benefits of mobile applications development and the acquisition of professional skills demanded by the market. The CBL active methodology is reflection-based learning that uses the students' relationship with their experiences. It was identified that students reflect to find new results and knowledge, so the practice aims to improve the student's abilities for the following activities. The results showed that reflective practice positively affects software development and the acquisition of professional skills. The study highlighted that collaborative learning helps students develop their own skills through practice and that groups interested in other teams' work acquire new knowledge and skills. In addition, among the main contributions of reflective practice are skills development, like teamwork, collaboration, communication, time management, planning, problem identification, decision-making, and self- knowledge. Therefore, these contributions are crucial for a computer science practitioner to succeed. 5.1 Related Work Compared to other literature studies, it was possible to notice that conflict management and time management are essential skills in executing a project. These points were also cited in the work of Dors et al. (2020) and were confirmed in the present study results. The findings of this research confirm and extend the results obtained by Dors et al. (2020). The authors' main results were that reflective practice promotes the emergence of new ideas and contributes to the practice and development of skills, such as collaboration, oral or written communication, commitment, interpersonal relationships, adaptability, flexibility, and teamwork. It also develops problem-solving, decision-making, planning, project management, time management, scope management, outsourcing development management, and new technical skills. In addition, the reflective practice emphasizes hands- Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 on learning, supports the development of technical skills, and appears to be an authentic environment of the relationship between academic disciplines and real-world experiences, where students can practice and learn by doing, preparing students for the real world. The interviewees in this study highlighted the importance of behavioral competencies for their development. In this study, students emphasized the relevance of self- knowledge, knowing how to listen, awakening behavioral improvement, confidence, and communication as benefits of reflective practice. The second cycle highlighted communication as a fundamental part of personal development and teamwork (Carter Ferzli and Wiebe (2007)). On the other hand, in the present study regarding the question, it was observed that students recognize that public speaking and listening skills are equally relevant to personal and professional life. The analysis of the results of this study showed that students perceived that decision-making is improved with each new reflection made since pondering one's virtues and weaknesses are critical to improving the timing and quality of personal deliberations. This is consistent with what was obtained by Dors et al. (2020). This research made it possible to observe the students' acceptance and motivation to use reflective practice. These findings differ from the study by Prior et al. (2016), who presented the results of action research conducted at the Sydney University of Technology with three software development studios. The main challenges in terms of motivation that the authors encountered were i) time pressures that made it difficult to record the journals; ii) difficulties in making the journal entries. Some students only did the reflections when reminded, and others refused. They also identified the following patterns of students: i) Refusers (do not write reflections), ii) Recounters (difficulties in doing reflection), and iii) Instinctive Reflectors (able to reflect naturally). One intervention that proved to be effective was the 10-minute reflection sessions, which consisted of having students express how their written reflections were, any problems, and a stimulus question, which was answered during the session. In the case of the study reported in this article, this difficulty was not observed because the students felt motivated by the reflective practice. In this research, it was possible to observe improvement in technical performance. That is reflective practice positively impacted student performance. This finding is consistent with the research conducted by Nylén et al. (2017), in which the authors studied how students approach the reflective practice task. Two categories were identified in students' recording of critical incidents: Progress and Expansion. Progress refers to progress reporting, divided into "What I am doing," status reports, and daily type categories. These subcategories grow in sophistication, respectively. Expansion indicates students' reports on learning items and reflections on those items. This category is divided into keywords (how-to, knowledge about generic, personal, and theoretical language). The authors concluded that journal recording induces reflection on learning and positively affects students' awareness of their professional knowledge. On the other hand, students found it challenging to identify learning. On this last topic, this did not occur with the present study, as students could clearly identify their growth through the application of reflective practice. 5.2 Threats to Validity According to Yin (2017), research developed using the case study method can be evaluated under four criteria: (i) construct validity; (ii) internal validity; (iii) external validity; and (iv) reliability. A threat to the construct validity of this study was the use of narratives to identify the acquisition of professional competencies through reflective practice. It was recognized that the narrative approach is compatible with the need to assess the complexity of organizations. However, it was necessary to rely on the interviewees' memories to understand this practice's benefits in both professional competencies and application and software development in general. Another limitation of the narrative approach is that people often rationalize the facts while telling the story. The construction of meaning from the facts causes individuals to interpret past events, try to find explanations for what happened, and perhaps confuse what occurred. To lessen this threat, students were asked to recount specific moments to confirm their interpretation of the facts concerning their reflection during the interviews. Similarly, the validation performed with Academy students also helped validate the perception of their interpretations of the sequence of occurrences, understanding the benefits and uses of reflective practice. Regarding external validity, since this study is about a specific environment in specific circumstances, the ability to generalize is limited. It is possible that similar results could be obtained when studying teaching and innovation environments that operate under close conduction, for example, using the software studio concept and reflective practice. 5.3 Implications to software engineering education, industry, and academia From the point of view of software engineering education, we were able to observe that the CBL method, especially reflexive practice, is an effective means of developing technical and non-technical skills. Providing students with challenges that provoke them to go further, search for answers, create, and reflect can lead to valuable knowledge. This can inspire educators worldwide to rethink their educational practices in the classroom. For industry, our study reveals that CBL and reflective practice lead to the development of highly demanded soft skills such as communication, conflict resolution, autonomy, and responsibility, among others. These methods can be used to develop such skills in the academic environment or professional education. Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 From the academic perspective, we could present a model that describes our findings expressed in the form of a relationships network shown in the previous sections. 6 Conclusion Technology changes continuously, so computer professionals must increasingly deal with new methods, tools, platforms, user expectations, and software markets. Thus, more advanced education is needed to prepare these professionals for the coming decades and new demands. In this respect, reflective practice has proven to be an effective method to help develop hard and soft skills, improving performance and assisting students to acquire talents essential for professional competencies. The analyses performed in this study refer to the students who participated in the software studio from 2019-2020 and 2021-2022. Part of the activities of these students occurred in a face-to-face manner, and part happened remotely. This may have caused some effects unknown to the researchers. A further study with the cohort 2021-2022 is already being initiated. It may reveal whether, by entering in a fully remote manner, the results obtained by the students will be different from those obtained in the present study. Acknowledgments We thank the Apple Developer Academy for allowing us to conduct the study, the students who participated in the interviews for making their time available, and CNPQ for providing the grant for this study. References Agante, L. (2015). A importância das soft skills na vida profissional. Dinheiro Vivo. Disponível em: https://www.dinheirovivo.pt/gestao-rh/a-importancia-das- soft-skills- na-vida-profissional-12665712.html. Access in 04th July 2021. Barkley, E. F.; Major, A. H.; Cross, K. P. (2014). Collaborative Learning Techniques – A Handbook for College Faculty. 2nd ed. San Francisco: Jossey-Bass – A Willey Brand, page 417. Burge, J. E.; Gannod, G. C.; Anderson, P. V.; Rosine, K.; Vouk, M. A.; Carter, M. (2012). Characterizing communication instruction in computer science and engineering programs: Methods and applications. In: Frontiers in Education Conference Proceedings, pp. 1-6, DOI: 10.1109/FIE.2012.6462496. Bull, C.; Whittle, J. (2014). Supporting reflective practice in software engineering education through a studio-based approach. IEEE Software, v.31, n.4, pages 44-50. Bull, C. N.; Whittle, J.; Cruickshank, L. (2013). Studios in Software Engineering Education: Towards an Evaluable Model, in International Conference on Software Engineering (ICSE 13), pages. 1063–1072. Carbone, A.; Sheard, J. (2002). A studio-based teaching and learning model in IT. In: Proceedings of the 7th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE'02), v. 34, n. 4, pp. 213-217. Carter, M., Ferzli, M., and Wiebe, E. N. (2007). Writing to learn by learning to write in the disciplines. Journal of Business and Technical Communication, v.21, n3, pages 278-302. Danielewicz-Betz and Tatsuki, (2014). Danielewicz-betz, Anna; Kawaguchi, Tatsuki. Gaining hands-on experience via collaborative learning: Interactive Computer Science Courses. In: 2014 International Conference on Interactive Collaborative Learning (ICL), IEEE, December, pp. 403- 409. Dors, T.M.; Van Amstel, FMC; Binder, F.; Reinehr, S.; Malucelli, A. Reflective Practice in Software Development Studios: Findings from an Ethnographic Study. In 2020 IEEE 32nd Conference on Software Engineering Education and Training (CSEE&T). Dybå, T.; Maiden, N.; Glass, R. (2014). The Reflective Software Engineer: Reflective Practice, IEEE Software, v.31, n4, pages 32-36. Fernandes, B. H. R. (2013). Gestão estratégica de pessoas com foco em competência. Rio de Janeiro: Elsevier. Ferraz, A. P.; Belhot, R. V. (2010). Taxonomia de Bloom: revisão teórica e apresentação das adequações do instrumento para definição de objetivos instrucionais. Gestão da Produção, v.17, n.2, pages 421-431. Hazzan, O. (2002). The reflective practitioner perspective in software engineering education. Journal of Systems and Software, v. 63, n. 3, pages 161-171. Lima, T.; Porto, J. B. (2019). Análise de soft skills na visão de profissionais da engenharia de software. In: Workshop sobre Aspectos Sociais, Humanos e Econômicos de Software (WASHES), 4., (2019). Belém. anais [...]. Porto Alegre: Sociedade Brasileira de Computação, pages 31-40. DOI: https://doi.org/10.5753/washes.2019.6407. Marques, M.; Ochoa, S. F.; Bastarrica, M. C.; Gutierrez, F. (2018). Enhancing the Student Learning Experience in Software Engineering Project Courses. IEEE Transactions on Education, v. 61, n. 1, pages 63-73. Nylén, A.; Isomöttönen, V. Exploring the Critical Incident Technique to Encourage Reflection during Project-Based Learning. In Proceedings of Koli Calling 2017, Koli, Finland, November 16–19, 2017, 10 pages. Parsons, M.; Stephenson, M. (2005) Developing reflective practice in student teachers: collaboration and critical partnerships, Teachers and Teaching: theory and practice, 11:1, 95-116. Prior, J.; Connor, A.; Leaney, J. (2014). Things coming together: learning experiences in a software studio. In: Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education, pp. 129-134. Prior, J.; Suman, L.; Leaney, J. (2019). What is the Effect of a Software Studio Experience on a Student´s Employability? In: Proceedings of the 21st Australasian Computing Education Conference (ACE'19), ACM. Sydney, NSW, Australia, pp. 28-36. Prior, J.; Ferguson, S.; Leaney, J. Reflection is Hard: Teaching And Learning Reflective Practice in a Software https://doi.org/10.5753/washes.2019.6407 Education, Innovation and Software Production: the contributions of the Reflective Practice Andrade et al. 2023 Studio. ACSW '16: Proceedings of the Australasian Computer Science Week Multiconference February 2016. doi- org.ez433.periodicos.capes.gov.br/10.1145/2843043.2843 346. Schön, D. A. (1983). The reflective practitioner: how professionals think in action. New York: Basic Books Inc., 374 p. Schön, D. A. (1987). Teaching artistry through reflection-in- action, In Educating the Reflective Practitioner: Toward a new design for teaching and learning in the professions.1st Ed., San Francisco, CA, US: Jossey-bass. Strauss, A.; Corbin, J. (2007). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 3rd ed. London: Sage Publications. Tomayco, J.E. (1991). Teaching software development in a studio environment. In: Proceedings of the twenty-second SIGCSE technical symposium on Computer science education (SIGCSE '91), v. 23, n. 1, pp. 300-302. Kuhn, S.; Hazzan, O.; Tomayko, J. E.; Corson, B. (2002). The software studio in software engineering education. In: 15th Conference on Software Engineering Education and Training (CSEE&T 2002), Proceedings, Kentucky, USA, pages 236-238. Yin, R. (2017). Case Study Research: Design and Methods (Applied Social Research Methods), 6th ed. Los Angeles: Sage Publications. Education, Innovation and Software Production: the contributions of the Reflective Practice in a Software Studio 1 Introduction 2 Background 2.1 Collaborative Learning 2.2 Hard and Soft Skills 2.3 Software Studio 3 Research Method 4 Results 4.1 First Cycle 4.2 Second Cycle 4.3 Third Cycle 5 Discussion 5.1 Related Work 5.2 Threats to Validity 5.3 Implications to software engineering education, industry, and academia 6 Conclusion Acknowledgments References