untitled 1 Case studies Evaluation Of Web-Based Learning Sets For A Postgraduate Community Pharmacist Programme Shivaun Gammie Medway School of Pharmacy, University of Greenwich and University of Kent A Pharmacy White Paper (Department of Health, 2008) identified the need for changes in education and training to ensure that pharmacists have the clinical competencies to deliver more patient-focused services. The Postgraduate Diploma in General Pharmacy Practice (Community Pharmacy) is a new, work-based postgraduate programme utilising self-directed learning defined as, “pharmacy practitioners are personally responsible for achieving the required competencies in the workplace to ensure fitness to practise, underpinned by a supporting infrastructure”, (Jubraj, 2009). The programme builds on an established programme for hospital pharmacists and utilises the general level framework (GLF), a validated competency framework (Competency Development and Evaluation Group, 2007), which has been shown to improve and sustain pharmacist performance in both hospital and community pharmacy (Mills et al., 2005). The programme started in 2009–10 with eight (four male and four female) mature students with the equivalent of an MPharm degree, registered as pharmacists and with experience of working as a community pharmacist. Students were a mixture of newly qualified and experienced pharmacists working in a range of community pharmacy settings. A significant challenge during programme development was how to provide an infrastructure to support workplace learning for community pharmacists who tend to be remote, often single-handed practitioners, working extended hours Monday to Sunday. The eight students were geographically spread across England (one Manchester, one Brighton, one Kent and five within M25) working a variety of shift patterns. The Postgraduate Diploma in General Pharmacy Practice (Community Pharmacy) programme included: ● Significant, structured work-place learning supported by a pharmacist, practice tutor leading to the development of a portfolio ● Five face-to-face study days ● Web-based, facilitated, interactive, learning sets held at convenient times for participants (each learning set was held twice; once on a Tuesday evening and repeated the following Sunday morning) ● Written and online resources to support learning which included practitioner completed workbooks prior to each study day and web-based learning set. This case study describes the use of web-based learning sets within this programme. Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 2 Planning the introduction of web-based learning sets On the recommendation of a university technologist, dimdim® (dimdim Inc, 2010) was chosen as the virtual classroom software as it allowed: ● Audio-visuals through webcams and microphone ● Whiteboard for both presenter and participants to write on ● File viewing e.g. Powerpoint® slides, pdf files ● Links to websites for participants to use ● Participant viewing of presenter’s computer screen ● Discussion thread to enable group and one-to-one communication during the session. Please note: dimdim® (dimdim Inc, 2010) is no longer available. However, this case study illustrates the use of virtual learning technology and its findings can be applied when considering the use of web-based learning sets. The technology was piloted within the postgraduate team and the following decisions were made: ● Web-based learning sets would last for a maximum of two hours ● A workbook and lesson plan would support each learning set ● Participant instructions for using the virtual classroom technology were written ● A classroom session and individual student technical checks to support student use of the technology would be used to ensure that the technology worked. Introducing web-based learning sets The virtual classroom was introduced to the students during a face-to-face day held in one computer teaching room. All students were able to access dimdim® and a 30 minute teaching session was undertaken without the sound option (as this would not work in an open-classroom setting). Student feedback (verbal and written evaluation form) was positive. No one felt that dimdim® was complicated to use or distracted them from learning and everyone was looking forward to the next session. Ideas for lesson plans and teaching strategies within a virtual classroom setting were developed. Each practitioner was booked for an individual 30 minute slot to test dimdim® and instructions provided. The technical tests were frustrating and stressful as problem after problem occurred. Practitioners could frequently not hear the tutor or the tutor could not hear them and there were numerous connection losses. Technical support was sought and the following changes made: ● The webcam function was removed as this might slow the system ● The participant instructions were substantially updated ● Practitioners were supplied with information on the internet and computer requirements to use dimdim® as it was suspected that those with poor broadband speeds experienced more problems Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 3 ● dimdim® Pro was used in an attempt to improve the service ● Teleconferencing was arranged as a back-up The problems encountered during the technical checks continued into the learning sets; interactivity was limited (e.g. speaking rights had to be assigned) and the system was unreliable (difficulties logging in and frequent lost connections). A summary of the technical problems experienced is shown in Table 1. As a result of these technical problems, teleconferencing, the planned back-up system, was ultimately used for three of the four learning sets. Table 1 Summary of the technical problems experienced with dimdim® ● Unreliability of the system – despite technical checks only one of four web-based learning sets proceeded as a web-based learning set which resulted in a reliance on teleconferencing which was expensive (a bill for the first teleconference was in excess of £90) ● dimdim® made discussion difficult as echoes and time delays occurred when two people had a microphone at the same time. This necessitated the development of a system to assign microphones ● Security concerns – unknown participants joined the web-based learning sets ● Record option not used – this was a key requirement for the programme to allow access to participants who were unable to attend at the designated time ● Isolation of the tutor/lack of infrastructure leading to an extremely stressful and frustrating teaching experience ● Lack of access to technical support when problems arose. The programme was subsequently reviewed and alternatives to web-based learning sets using online discussion groups, supported by teleconferencing, were introduced to replace web-based learning sets. Student and tutor feedback Verbal and written feedback was obtained from tutors and students. Students were asked to complete a written evaluation form after the classroom session (six completed forms were received, Table 2) and remote learning sets (three completed forms were received, Table 3). Comments were received verbally during teaching sessions and in free text on the evaluation forms (Table 4). Tutors summarised their observations after the four learning sets that attempted to use the technology. Table 2 Participants attitudes following the classroom web-based learning set Agree or strongly agree Neither agree or disagree Disagree or strongly disagree I like using new technologies 5 (83%) 1 (17%) I am looking forward to participating in web based learning sets 6 (100%) Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 4 Agree or strongly agree Neither agree or disagree Disagree or strongly disagree dimdim® is complicated to use 6 (100%) Using dimdim® distracted me from learning 6 (100%) I would prefer face-to-face learning sets 4 (67%) 2 (33%) Web-based learning sets are useful because I would find it difficult to attend face to face learning sets 2 (33%) 2 (33%) 2 (33%) I am looking forward to the next web-based learning set 6 (100%) Table 3 Participants attitudes following remote web-based learning sets Agree or strongly agree Neither agree or disagree Disagree or strongly disagree I like using new technologies 3 (100%) I enjoyed participating in the web based learning sets 3 (100% dimdim is complicated to use 2 (67%) 1 (33%) Using dimdim distracted me from learning 3 (100%) I would prefer face-to-face learning sets 1 (33%) 1 (33%) 1 (33%) Web-based learning sets are useful because I would find it difficult to attend face to face learning sets 1 (33%) 2 (67%) I am looking forward to the next web-based learning set 3 (100%) I did not like using a combination of dimdim® and telephone conferencing (Sunday’s learning set only)* 1 (33%) 1 (33%) * One form was blank Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 5 Table 4 Example student comments ● “Quite easy to use considering I have very limited IT skills (after classroom session).” ● “I did not enjoy the full benefits of dimdim because we did not use it a lot in the learning set because of the problems some people encountered and we had to resort to telephone conferencing. I am looking forward to using it fully in the next learning set. (after remote learning set).” ● “It can be very frustrating when finding it difficult to use the microphone or hear one talking. (after remote learning set).” ● “When it works it is the best technology. Anybody will enjoy using it. (after remote learning set).” Table 5 Tutors’ observations following the learning sets Tutor 1 (responsible for face-to-face session, technical checks and present at all learning sets) ● Chaotic and stressful ● Disappointing that dimdim® couldn’t be used ● Sound checks took a lot of time (probably because of problems) but had to make a quick decision to go to teleconferencing ● Felt extremely vulnerable when my internet connection failed – would have been more difficult if had not had second tutor to inform participants to go to teleconferencing ● Have to be careful that failure in technology does not impact on the reputation of this course ● Do we pursue the use of web-based technology? ● Teleconferencing is turning out to be very expensive (approx £100 per session) – add to this using two tutors per session it is becoming a very expensive option do we need to rethink? Tutor 2 (not present at face-to-face days and first web-based learning set) ● Disappointing that dimdim® didn’t work for either session ● The learning sets are designed to be interactive but dimdim® appears to be a more didactic medium ● The teleconferencing appeared more interactive but expensive ● Difficult to identify who was talking from voices ● Worked for four but would it work for eight (or ideally more) participants? Despite the negative feedback, tutors identified the following benefits of web-based learning sets, if the technical problems could be resolved: ● Widening access to postgraduate education to community pharmacists supporting them to provide increased clinical services ● Providing flexible postgraduate education to fit community pharmacists’ working lives ● High student acceptability. Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 6 Evidence of student transformation and enhanced teaching As web-based learning sets had to be abandoned in this programme evidence of transformation resulting from the technology is not available. However, students believed that they had benefitted from the programme as a whole as evidenced by the following quotes: “It is programme every community pharmacist could pursue to improve the practice of pharmacy in the Community. My money is well spent.” “This course has improved my knowledge and confidence so that I enjoy practising as community pharmacist now. I now see myself as a professional and now, speaking to and answering queries from GPs, is no longer a problem but a part of my work. It has helped me to do my enhanced and advanced services better. I will therefore recommend any pharmacist who wants to develop as a pharmacy practitioner to undertake this course.” Academic and practice tutors have observed the students’ professional and clinical development throughout the programme. Teaching has been enhanced by the development of teaching resources and skills to support remote community pharmacy students, for example student workbooks, structured lesson plans, communication, strategies to involve all students in an unseen environment. Reflection This project has indicated that web-based learning sets were well received by pharmacists and remote learning appeared acceptable and fitted into the busy schedules of community pharmacists. Anderson (2008) indicated that synchronous virtual classrooms are familiar to students and provide increased access. Two of the eight practitioners enrolled on this programme lived more than 50 miles from the campus. Davis et al., (2008) highlighted the importance of understanding the needs of potential students with regard to prior learning and experiences with technology. This project did this through discussions with practitioners, baseline questionnaires, introduction to the technology in a classroom setting and technical tests. The feedback obtained suggests that if the technology could work it would be highly acceptable to the practitioners. However, the need to resort to teleconferencing caused problems. Salmon (2000) produced a five step model for computer mediated communication: 1) Access and motivation 2) Online socialisation 3) Information exchange 4) Knowledge construction 5) Development. Unfortunately, the present study failed at the first step as technical issues prevented practitioners consistently accessing dimdim. However, when the technology did work there was evidence of practitioner motivation, online socialisation, information exchange and knowledge construction. Therefore, it is suggested that if the technical issues could be resolved then web-based learning sets could provide a valuable learning experience. Davis et al., (2008) suggest that, “the ideal online learning and teaching system is developed from scratch, with no restrictions on costs and staffing, and uninhibited by resistance to change from previous practices”, whilst recognising that the implementation of online learning involves trial and error. This project used a commercially available system that failed to meet the needs of the participants; namely technical reliability and promotion of interactive discourse. The trial of dimdim® has been valuable in identifying the requirements Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 7 of a system to support interactive web-based learning but suggests that alternatives to dimdim® should be investigated. Lalos et al., (2009) undertook an evaluation of dimdim® and an alternative synchronous tool with 12 attendees and four presenters and concluded that dimdim® was simple to use. They did not report the level of technical problems experienced in this study but reported that audio performance was less than the comparator. Schullo (2007) reported an approach to selecting a virtual classroom system which could be used to evaluate available systems. dimdim® was chosen in this project on the recommendation of a learning technologist as it was outside the expertise of the author to undertake technical reviews of systems. However, the experiences from this project could support a technical review of systems. This was a new form of teaching for the tutors involved. Throughout the project the technical issues were considerable but educational challenges were identified and addressed through the development of more structured lesson plans and promoting interactivity in the absence of non-verbal communication. Anderson (2008) sates that, “one does not have to be a technical expert to be an effective online teacher”, which is reassuring. He further identifies that during the development of online programmes the tutor needs to demonstrate, “resilience, innovativeness and perseverance”, which is consistent with the experiences in this project. A particular finding is the intrusion of web-based teaching into the tutor’s home life, e.g. the use of personal computers and telephones as well as the need to run teaching sessions during weekends and evenings. This should be addressed in any expansion of web-based learning sets. Davis et al., (2008) suggests that a relationship between academic and administrative computer teams, which promotes a joint vision for the technological architecture to provide flexibility and sustainability, is required. The findings from this project indicate that there is a risk of tutors feeling isolated and vulnerable when delivering web-based learning. If web-based learning is to expand it is believed essential that a supporting infrastructure is developed. As a result of this case study recommendations were made for the future use of web-based learning sets (Table 6). Table 6 Recommendations for the future use of web-based learning sets ● A review of available virtual classrooms is required to determine whether there is a cost-effective, reliable system for participants with standard, domestic broadband access which allows: - Two or more participants to speak at the same time - Documents, slides and websites to be shared - Discussion threads - Recordable version for those practitioners unable to join the event live ● An infrastructure should be established to support the use of web-based learning sets for post-graduate education recognising that learning events are likely to occur outside of normal working hours. This should include: - Reliable virtual classroom systems (as above) - Computers set up for web-based learning in areas of the campus used outside of normal working hours e.g. library with access to back-up systems e.g. telephone, teleconferencing - Technical expertise and support - Recognition of revised working patterns in staff contracts. Compass: The Journal of Learning and Teaching at the University of Greenwich, Issue 5, 2012 8 References Anderson, T. (2008) Teaching in an Online Learning Context. In: Anderson, T. (ed.) The Theory and Practice of Online Learning. 2nd ed. pp. 343–365. Athabasca University: AU Press. Competency Development and Evaluation Group (CoDEG). 2007. General Level Framework a Framework for Pharmacist Development in General Pharmacy Practice. [Online]. Available at: www.codeg.org/ fileadmin/codeg/pdf/glf/GLF_October_2007_Edition.pdf [Accessed 28 March 2011]. Davies, et al. 2008. Developing an Infrastructure for Online Learning. In: Anderson, T. (ed.) The Theory and Practice of Online Learning. 2nd ed. Athabasca University: AU Press, pp. 121–142. Department of Health. 2008. Pharmacy in England: Building of Strengths – Delivering the Future. [Online]. Available at: www.official-documents.gov.uk/document/cm73/7341/7341.pdf [Accessed 28 March 2011] dimdim Inc. 2010. dimdim©. [Online]. Available at: www.dimdim.com [Accessed 28 March 2011]. Jubraj, B. (2009) Developing a Culture of Self-Directed Workplace Learning in Pharmacy. Pharmaceutical Journal, 283, pp. 47–48. Lalos, P. et al. 2009 Discovering DIMDIM: A Heuristic Evaluation of MOODLE’s synchtonous Open Source Perspective. [Online]. Available at: http://journal.webscience.org/115/1/websci09_submission_13.pdf [Accessed 13 January 2012]. Mills, E. et al. (2005) Development of An Evidence-Led Competency Framework for Primary Care and Community Pharmacists. Pharmaceutical Journal, 275, pp. 48–52. Salmon, G. (2000) A Model for CMC in Education and Training. In: E-Moderating: the Key to Teaching and Learning Online. 1st ed. pp. 22–37. London Sterling (USA): Kogan Page. Schullo, S. et al. 2007 Selecting a virtual classroom system: Elluminate Live vs. Macromedia Breeze (Adobe Acrobat Connect Professional). Merlot Journal of Online Learning and Teaching, 3 (4). [Online]. Available at: http://jolt.merlot.org/vol3no4/hilbelink.htm (Accessed 13 January 2012). http://www.codeg.org/ http://www.official-documents.gov.uk/document/cm73/7341/7341.pdf http://www.dimdim.com/ http://journal.webscience.org/115/1/websci09_submission_13.pdf http://jolt.merlot.org/vol3no4/hilbelink.htm