Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 1 Co-organising hackathons with students: opportunities and best practices Nuno N. Correia University of Greenwich, UK and ITI/LARSyS, Portugal Abstract We identify problems regarding teaching in higher education, particularly in computing subjects: short attention spans; lack of focus on cutting-edge topics; lack of attention to relevant issues for students; and disconnect from research. We propose that co-organising hackathons with students can assist in addressing these issues. We present a literature review, our experience in organising three hackathons and the resulting feedback gathered from participants. Based on that experience, and in articulation with related literature, we suggest best practices for organising these events. We also discuss applicability to other fields beyond computing. Keywords: hackathons, technology, computing, student engagement, working with students Introduction Lecturers are often faced with students who disengage from modules, with possible serious consequences for their degree performance. This is particularly serious in certain topics within the field of computing that are crucial to building technical skills. In contemporary society, attention spans are getting shorter as smart phones, social media and other technologies compete for our attention (Lorenz-Spreen et al., 2019). Sustaining student engagement in longer modules is therefore problematic: often, modules do not tackle issues that students consider relevant to them; modules are sometimes not agile enough to address emergent technologies and knowledge; students are often disconnected from our research activities in the field of computing. These issues present an opportunity for organising more condensed events and exploring new methods and techniques in computing and related subjects, with both staff and students involved. Such events could have positive outcomes for both research and teaching. Literature review Hackathons are intensive design-and-development events where participants gather to “conceptualise, prototype, and make (mostly digital) products and services” (Lodato and DiSalvo, 2016). They are an exchange between organisers and participants: the former offer logistics (space, electricity, wireless internet and sometimes food), while the latter bring computer devices, their skills and their “undivided attention” (Irani, 2015). Hackathons can differ greatly in their themes – from a corporate setting (Nolte, Chounta and Herbsleb, 2020) to digital arts (Correia and Tanaka, 2015), user interface design (Correia and Tanaka, 2017), and humanitarian aims (Linnell at al., 2014) – but they follow a similar structure: hackathons last for a day or two; challenges are presented to the participants, who form teams around those; the teams engage in a “fervour of activity” to develop solutions; and, at the end of the Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 2 event, the teams show their work, judges judge and awards are given (Lodato and DiSalvo, 2016). In recent years, hackathons have been successful in attracting a wider audience beyond computing and technology communities (Taylor and Clarke, 2018). Six factors for the success of hackathons have been identified (Mohajer Soltani et al., 2014): (i) how well the problem area is defined; (ii) what sort of return is offered to the winning solution(s); (iii) how diversified each team member’s competences and skills are; (iv) whether mentors are in place and how well they communicate their expertise; (v) whether jury members understand well enough the area(s) in focus; (vi) the entry requirements of the competition. Hackathons organised and participant feedback We believe that co-organising hackathons with students can complement existing modules and address the identified problems: short attention spans; lack of focus on cutting-edge topics; insufficient attention to issues that are relevant to students; and disconnect from research. During the last two academic years, we have collaborated with two student societies at the University of Greenwich – ‘GreDevJams’ and ‘CompSoc’ – in the organisation of three half-day hackathons: • Tech4Good (with GreDevJams, on diversity and inclusion in smart cities; 8 November 2018; fifty-four students) • Crack the Code (with CompSoc, on cybersecurity; 28 February 2019; twenty-two students) • Hack the Planet! (with CompSoc and on cybersecurity; 28 February 2020; twenty- three students). Students were grouped into teams of three or four participants. More information on the hackathons organised can be found in our blog, Hack Central1. Feedback on Tech4Good This was the first hackathon we co-organised with students and feedback was not collected in a systematic way. However, we received a good amount of informal positive feedback. We report two spontaneous feedback statements from students: “It was great! I enjoyed taking part and realised that I can actually produce something presentable in a short time.” “I found it challenging, but everyone was helpful and made me feel I can do it.” Standard questionnaire for feedback Following our first hackathon, we designed a questionnaire, intending it to be implemented as a standard procedure in subsequent hackathons. It contained the following questions: 1 https://blogs.gre.ac.uk/hackcentral/ https://blogs.gre.ac.uk/hackcentral/ Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 3 1. How enjoyable was the event? 2. Was it a useful learning experience? 3. What aspects did you like the most? 4. What could be improved? 5. Any other comments? Questions 1 and 2 are Likert-scale questions, with a scale of 1 (not at all enjoyable / useful) to 5 (very enjoyable / useful). The other questions allow for open-ended answers. Feedback on Crack the Code Regarding feedback for the Crack the Code hackathon, we obtained seven anonymous responses to our online feedback form out of twenty-two (approximately one third of the participants). Two of the participants considered it very enjoyable, four found it enjoyable and one gave a neutral answer. Four participants considered it very useful, two found it useful and one considered it not useful at all. Figure 1 summarises these results. Participants liked the flexibility offered by the hackathon set-up (specifically, that any programming language could be used to solve problems). Two participants enjoyed the level of the challenge: it was considered difficult and intriguing, hard enough but not impossible, and with stimulating content. Meeting other students and “the freebies” were also mentioned as positive factors, as was the friendliness of the organisers. Regarding negative aspects, two participants complained about the difficulty level and another two stated that the level of programming experience required for the event had not been adequately described in the advertising for it. Figure 1: Charts summarising feedback from the Crack the Code hackathon Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 4 Feedback on Hack the Planet For the Hack the Planet hackathon, twelve participants out of twenty-three (approximately half) filled in the feedback questionnaire. By comparison with the previous hackathon, the results improved (see figure 2). Eight participants evaluated it as very enjoyable, three as enjoyable and one as not enjoyable. Seven found it very useful, four useful and one not useful at all. In terms of positive aspects, collected from the open-ended answers, the hackathon challenge was mentioned by four respondents, while another three mentioned other aspects, such as the teamwork, the friendly atmosphere and the helpful staff. Regarding suggestions for improvement, two participants suggested having free food and drink as in previous events; two mentioned that information on the tools to be used could be improved; and one student mentioned the highly demanding Linux skills required. Figure 2: Charts summarising feedback from the Hack the Planet hackathon Discussion We identify best practices that could be useful for lecturers planning to organise hackathons with students; they align with the above factors for success presented by Mohajer Soltani et al. (2014): i) Allow the students to pick the problem area, but assist in refining it, to achieve an adequate scope for the timeframe of the hackathon. ii) Offer certificates of participation for all students attending and prizes for the top teams. For example, we have offered gift vouchers, bought in advance, to the winning and runner-up teams. We recommend that these are handed out at the end of the event, following a swift deliberation process by the jury – mirroring established practices for these events. iii) If possible, allow for students from different backgrounds to participate, integrating multi-disciplinary teams (for example, Computer Science and Digital Media students). Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 5 iv) Request that organising students engage with a member of staff knowledgeable in the field of the hackathon, who would act as a mentor and who should be part of the jury. v) Following the previous point (iv), the jury should include a member of staff specialising in the topic of the hackathon. vi) When recruiting participants, the level of difficulty, the prior knowledge required and the target audience should all be made clear (for example: students from all levels or final-year students only). We would add a seventh recommendation, which does not align with the six factors of success presented above. It may seem trivial compared to the other factors, but it helped us to improve upon previous events: vii) Collect feedback from participants in a systematic way, with a consistent questionnaire, allowing comparison of results from a sequence of events. Hackathons are “brief bursts of collocated activity” (Trainer et al., 2016), where participants usually show their “undivided attention” (Irani, 2015). As such, they address the identified problem of short attention spans. From our observation, students concentrate better in these events because of a tight deadline for finishing the task. The rewards gained are also motivational: a sense of self-achievement, camaraderie generated by intense group work, recognition from peers and lecturers and the actual prize. We believe that the intensity of the event and participant awareness that it is short and significant encourage students to focus well and to postpone possible distractions to afterwards. In our collaboration with student societies, the students themselves pick the challenge for the hackathon – there is therefore a bottom-up approach in terms of the theme of the event, ensuring its relevance to the students. Additionally, we request the involvement of a member of staff who is an expert in that topic, which fulfils a double objective: on the one hand, the quality assurance of the hackathon task and outcomes; on the other, a learning experience for the organisers by collaborating with an expert in the field. In turn, this specialist member of staff becomes more aware of issues that students are interested in and so constructive adaptations to curricula may possibly take place as a result. The feedback we have collected from participants is in tune with some of the best practices above. Participants enjoyed the prizes and rewards – recommended in best practice ii) above. In the second hackathon, we did not follow best practice vi) in the above list – the difficulty level was not properly advertised and so led to negative feedback. In the third hackathon, we took care to make the difficulty level clear in the promotional materials, resulting in more positive feedback from students about the clarity of description of the challenge level. (However, information issues were not entirely solved, according to feedback.) More generic recommendations that emerge from the positive feedback – that organisers of such events should take into account – are to facilitate socialisation and to create a friendly and supportive atmosphere. However, we did not include this in our proposed best practices, as it is a broad, commonsense observation. Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 6 Conclusions and limitations With this paper, we have aimed to raise awareness about the potential of co-organising hackathons with students, particularly in computing-related subjects. Using experience gained from running three hackathons, we have also extended existing literature on the topic by identifying best practices. The feedback we collected from students supports several of our findings. The best practices identified through hackathons can be applicable not only to computing students, but also to other disciplines applying technology for a specific purpose, such as social issues and business (Trainer et al., 2016). Regarding the limitations (identified from student feedback) of the hackathons conducted, we highlight the lack of adequate prior information about the difficulty level and the tools involved. Despite our efforts to make those elements clearer in promotional material, participants still considered the information insufficient. It remains a challenge for our future hackathon information provision. Reference list Correia, N.N. and Tanaka, A. (2015) ‘Prototyping Audiovisual Performance Tools: A Hackathon Approach.’ In: Proceedings, NIME 2015, Baton Rouge, Louisiana, 1-3. [online] Available at: http://www.nime.org/proceedings/2015/nime2015_230.pdf (Accessed: 13 November 2015). Correia, N. N. and Tanaka, A. (2017) ‘AVUI: Designing a Toolkit for Audiovisual Interfaces.’ In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, Denver, CO, USA, ACM. [online] Available at: https://dl.acm.org/doi/abs/10.1145/3025453.3026042 (Accessed 14 August 2020). Irani, L. (2015) ‘Hackathons and the Making of Entrepreneurial Citizenship.’, Science, Technology & Human Values, 40(5), 1-26. [online] Available at: http://sth.sagepub.com/content/early/2015/04/07/0162243915578486 (Accessed 11 March 2016). Linnell, N., Figueira, S., Chintala, N., Falzarano, L. and Ciancio, V. (2014) ‘Hack for the homeless: A humanitarian technology hackathon.’ In: Proceedings of the 2014 IEEE Global Humanitarian Technology Conference (GHTC), 577–584. [online] Available at: https://ieeexplore.ieee.org/document/6970341 (Accessed: 11 June 2021). Lodato, T.J. and DiSalvo, C. (2016) ‘Issue-oriented Hackathons as Material Participation.’ New Media & Society, 18(4), 539-557. [online] Available at: http://nms.sagepub.com/content/early/2016/02/10/1461444816629467 (Accessed: 22 July 2016). Lorenz-Spreen, P., Mønsted, B.M., Hövel, P. and Lehmann, S. (2019) ‘Accelerating Dynamics of Collective Attention.’ Nature Communications, Nature Publishing Group, 10(1), 1-9. [online] Available at: https://www.nature.com/articles/s41467-019-09311-w (Accessed: 9 October 2020). http://www.nime.org/proceedings/2015/nime2015_230.pdf https://dl.acm.org/doi/abs/10.1145/3025453.3026042 http://sth.sagepub.com/content/early/2015/04/07/0162243915578486 https://ieeexplore.ieee.org/document/6970341 http://nms.sagepub.com/content/early/2016/02/10/1461444816629467 https://www.nature.com/articles/s41467-019-09311-w Case Studies Compass: Journal of Learning and Teaching, Vol 14, No 2, 2021 7 Mohajer Soltani, P., Pessi, K., Ahlin, K. and Wernered, I. (2014) ‘Hackathon – A Method for Digital Innovative Success: A Comparative Descriptive Study.’ In: Proceedings of the 8th European Conference on IS Management and Evaluation – ECIME 2014, Ghent, Academic Conferences and Publishing International Limited, 367-373. [online] Available at: http://urn.kb.se/resolve?urn=urn:nbn:se:miun:diva-22258 (Accessed: 22 November 2018). Nolte, A., Chounta, I.-A. and Herbsleb, J.D. (2020) ‘What Happens to All These Hackathon Projects? Identifying Factors to Promote Hackathon Project Continuation.’ Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2), 145:1-145:26. [online] Available at: https://doi.org/10.1145/3415216 (Accessed: 16 November 2020). Taylor, N. and Clarke, L. (2018) ‘Everybody’s Hacking: Participation and the Mainstreaming of Hackathons.’ In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, New York, NY, USA, Association for Computing Machinery, 1- 12. [online] Available at: https://doi.org/10.1145/3173574.3173746 (Accessed: 22 February 2021). Trainer, E.H., Kalyanasundaram, A., Chaihirunkarn, C. and Herbsleb, J.D. (2016) ‘How to Hackathon: Socio-technical Tradeoffs in Brief, Intensive Collocation.’ In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, CSCW ’16, San Francisco, CA, USA, Association for Computing Machinery, 1118–1130. [online] Available at: https://doi.org/10.1145/2818048.2819946 (Accessed: 16 November 2020). http://urn.kb.se/resolve?urn=urn:nbn:se:miun:diva-22258 https://doi.org/10.1145/3415216 https://doi.org/10.1145/3173574.3173746 https://doi.org/10.1145/2818048.2819946