Formative assessment in apprenticeship Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Using mobile technology to facilitate formative assessment and support learning in apprenticeship programmes Richie Ryan, Gina Noonan, Eddie McElheron Institute of Technology Carlow, Ireland Abstract Regular formative assessments can encourage students to spread learning effort throughout entire programmes and afford more feedback opportunities to tutors (Gibbs and Simpson, 2004; Nicol and Macfarlane‐Dick, 2006), while the proliferation of smart mobile phones has contributed to an increasing trend for ‘bring-your-own-device’ (BYOD) activities in higher education (Johnson et al., 2015; Merga, 2016; Sundgren, 2017). As students take more responsibility for their own learning, technology has a greater role to play in facilitating and supporting solutions that can provide more flexible learning opportunities. Quizlet - one such technology, is a web-enabled mobile learning application that repurposes study-sets for use in any of its eight different quiz modes. For a relatively modest time-investment by the tutor, students are provided with a re-usable, interactive, mobile learning resource that can be personalised to suit specific learning needs. This technology review focuses on Quizlet and our experience of using it to increase levels of student engagement and improve learning outcomes in craft apprenticeship programmes. Keywords: Formative assessment; Technology enhanced learning; Quizlet. Introduction Formative assessment is an important part of learning and can provide essential feedback to both students and teachers. The use of technology to enhance learning and facilitate formative assessment activities, both inside and outside the classroom, is becoming increasingly popular and the range of mobile applications to support such activities continues to grow. Quizlet is a mobile learning application that provides a variety of quiz modes based on the creation of a single deck of flashcards. It automatically repurposes the content for use in any of five ‘study’ modes and three ‘play’ modes (Figure 1). ‘Study’ modes present the information in ways that offer choice in the level of difficulty while gradually shifting the focus from learning to more formal assessment, whereas ‘Play’ modes introduce fun, engaging, competitive and collaborative elements to learning. Figure 1: Quizlet - eight modes of learning from the web-based platform Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 The web-based version of Quizlet provides access to all quiz modes, whilst Android and iOS applications are adapted to facilitate mobile and offline access to five of the eight modes. Although the basic version of Quizlet is free, upgrading to Quizlet Teacher* provides additional functionality, including the facility to analyse and track learner progress. Apprenticeship programmes afford limited independent study time for students to reflect on learning outside class contact hours and Quizlet was used to support and encourage more convenient and flexible approaches to self-directed study. Quizlet ‘study-sets’† were introduced on a weekly basis during classroom activities and students used the application to engage with learning exercises outside the classroom in preparation for summative assessments. Although there are similarities with other mobile applications such as Kahoot!, Socrative and Quizizz, Quizlet requires very little effort to set up and generates a resource for the student that encourages independent learning facilitated by a flexible approach to study. The platform employs cognitive strategies associated with spaced learning and retrieval practice, which have been proven to strengthen memory and reduce the likelihood of subsequent forgetting (Roediger and Karpicke, 2006). Quizlet allows users to create, share and personalise content while providing a search function to over 200 million user-created study- sets. Each quiz mode is designed to facilitate different approaches to learning which can benefit and challenge students in a variety of ways. The cards below provide an overview of a selection of quiz modes and describe how these were used by students‡ and embedded into classroom learning activities by tutors. Learn This mode initially presents terms in an easy-answer format, but gradually adapts the level of difficulty in accordance with student progression through each round of learning. In our case, this mode was used predominantly by students for self-directed learning outside class time. * Quizlet Teacher requires an annual subscription of €34.99. † ‘Study-set’: The term used to describe a deck of flashcards in Quizlet. ‡ Quizlet analytics and learner feedback surveys provided data in relation to usage. Learner application Learn mode identifies strength/weakness, while customised study-sets help to focus learning. Responses to questions are self-assessed at this stage, meaning that the exact spelling and terminology does not pose a barrier to progression. Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Flashcards Flashcards present a term (question) on one side of the card and the definition (answer) on the other. The activity can begin with the term or with the definition, if a ‘Jeopardy!’§-style activity is desired. Write Write mode is a self-assessed activity in which students' responses are automatically graded for answers identical to that in the deck. § Jeopardy! is a US television game show in which contestants are presented with an answer and must phrase their responses in the form of question. Learner application An effective way to check levels of understanding in a self-assessed examination-style activity. Progress meters indicate real-time success rates and incorrect responses are automatically retained for subsequent quiz rounds. Tutor application I (Beginning of class) Older quiz-sets provide opportunities to review previously-covered material. New quiz sets focus learners’ attention on upcoming material and outline the goals for the session. Tutor application II (End of class) • Quick-fire question-and-answer format provides a summary of the lesson. • Learner responses are used as an exit pass. Learner application In our case, learners tended to use this mode in preparation for other learning modes and as a quick-fire content reminder in advance of formal summative assessments. Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Spell Spell requires students to type what they hear in response to the question posed. Although rarely used with apprenticeship students, it could have benefits for improving spelling and language development in other fields of education. Test ‘Test’ presents the study-set using a range of different question styles and automatically grades learners based on the accuracy of their responses. Feedback, including a score and correct/incorrect responses, is provided at the end of the test. Learner application Students with dyslexia have indicated a learning benefit using this mode and – more specifically – with the optional audio narration associated with most quiz modes. Tutor application Quizlet Teacher allows the tutor to record her/his own voice, adding more authenticity to this activity. Tutor application Results from test mode are recorded, providing rudimentary analytics on student performance and presenting an opportunity to identify learning shortfalls and at-risk students. Customised tests can be easily printed for manual completion/correction and provide excellent opportunities for peer-assessment/review activities. Learner application Learners can customise the test and the level of difficulty by choosing which questions to include and deciding on which question-types to incorporate. Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Match This timed game mode requires learners to match a selection of terms to their corresponding definitions. Notifications are automatically issued to students when their score has been overtaken and performance is displayed on a class leader-board. Live Live is a collaborative classroom-based game mode which can be accessed on any web- enabled device using a unique game code. Team groups are automatically created and must work together to complete a communal multiple-choice quiz. Penalties are incurred for incorrect responses to improve teamwork and discourage guesswork. Learner application We found that students enjoyed the competitive nature of this quick-thinking game mode and many learners engaged with other study modes to improve their ‘match’ mode scores. Tutor application Used in our case as a fun weekend activity to engage learners outside the classroom; the automated email notifications ensured continued interactions right up to the following classroom session when the leader board was displayed. Tutor Application The most engaging classroom-based game mode on the platform. It relies on collaboration and communication to promote competitive classroom learning scenarios. Scenario I: Mid-lesson At this stage, previous learning is reinforced, and a roadmap provided for where the lesson is going - prompting intense team discussions, particularly when teams are competing closer to the finish. C la s s ro o m : Q u iz le t L iv e . M a rc h 2 0 1 7 Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Advantages Quizlet offers a large degree of learner autonomy outside the classroom, encouraging students to take more responsibility for their learning and facilitating different rates of learner progression. Although Quizlet was primarily designed as an assessment tool for self-directed learning, the competitive and collaborative nature of play modes offers many opportunities to embed learning into assessment activities which extend beyond the confines of the classroom. Creating, customising and sharing study-sets is quick and easy and the application provides immediate feedback to learners. The facility to track student progress can also provide important analytical data to tutors. Disadvantages The structure of the learning platform – along with the limitations placed on the question format – is such that it may encourage a rote-style approach to learning that values memorisation over understanding. Learner application Students reported that this mode represents the most fun learning activity within Quizlet. (This became apparent to us from the very animated reactions and celebrations we witnessed during live classroom quiz sessions.) C la s s ro o m : Q u iz le t L iv e . M a rc h 2 0 1 7 Scenario III: Silent mode Although live mode generally requires learners to sit together, silent live mode requires that team members remain seated and unable to communicate. This introduces a whole new dynamic to the activity and limits the impact of dominant team members while also helping in situations where the classroom layout makes it difficult to move about. Scenario II: End-lesson An enjoyable way to end classroom sessions with lively group discussion in a fun and competitive environment. Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 We found that negative learner feedback tended to focus on limiting the mobile application to five of the eight modes of study and students expressed frustration at the requirement, in some quiz modes, for learner input to match exactly that of the assessor. With Quizlet and BYOD, the main logistical disadvantage associated with the web-based content is a dependence on network coverage and the need for mobile device power sources. Conclusions / further plans Quizlet engages students through a medium that appeals to their mobile nature and utilises proven educational methodologies to engage learners in a variety of competitive, collaborative and self-directed learning activities. It puts students in control of their learning while offering interactive opportunities for tutors to embed enjoyable learning activities into a formative assessment strategy. Although apprenticeship students found many aspects of Quizlet to be very enjoyable, competitive and initially quite novel, we would caution against overuse and recommend it as part of a broader assessment strategy to engage learners. Further studies relating to Quizlet are planned, to investigate the relationship between student engagement and the learning gains associated with different quiz modes, while the potential learning involved with student-created quiz content will also be explored.** Reference list Gibbs, G. and Simpson, C. (2004) ‘Conditions Under Which Assessment Supports Students’ Learning.’ Learning and Teaching in Higher Education, 1(1), 3-31. Available at: https://goo.gl/wnsznF (Accessed: 04 March 2017). Johnson, L., Adams Becker, S., Estrada, V. and Freeman, A. (2015) NMC Horizon Report: 2015 Higher Education Edition. Austin, Texas: The New Media Consortium. Available at: http://cdn.nmc.org/media/2015-nmc-horizon-report-HE-EN.pdf (Accessed: 15 April 2017). Merga, M.K. (2016) ‘Bring Your Own Device: Considering potential risks to student health.’ Health Education Journal, 75(4), 464-473. Available at: https://doi.org/10.1177/0017896915599563 (Accessed: 28 March 2018). Nicol, D.J. and Macfarlane‐Dick, D. (2006) ‘Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice.’ Studies in Higher Education, 31(2), 199-218. Available at: http://www.psy.gla.ac.uk/~steve/rap/docs/nicol.dmd.pdf (Accessed: 22 November 2017). Roediger, H.L. and Karpicke, J.D. (2006) ‘Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention.’ Psychological Science, 17(3), 249-255. Available at: http://learninglab.psych.purdue.edu/downloads/2006_Roediger_Karpicke_PsychSci.pdf (Accessed: 24 Febuary 2017). ** All Quizlet images and logos have been reproduced with the expressed permission of Quizlet.com https://goo.gl/wnsznF https://doi.org/10.1177%2F0017896915599563 http://www.psy.gla.ac.uk/~steve/rap/docs/nicol.dmd.pdf http://learninglab.psych.purdue.edu/downloads/2006_Roediger_Karpicke_PsychSci.pdf Technology Reviews Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Sundgren, M. (2017) ‘Blurring time and place in higher education with bring your own device applications: a literature review.’ Education and Information Technologies, 22(6), 3081-3119. Available at: https://link.springer.com/article/10.1007/s10639-017-9576-3 (Accessed: 02 Feburary 2017). https://link.springer.com/article/10.1007/s10639-017-9576-3