583 “Changing the Way We Talk”: Developing Librarians’ Competence in Emerging Technologies through a Structured Program Mark Pegrum and Ralph Kiel Mark Pegrum is Associate Professor in the Graduate School of Education at The University of Western Australia; e-mail: mark.pegrum@uwa.edu.au. Ralph Kiel is University Librarian at Victoria University; e-mail: Ralph.Kiel@vu.edu.au. We are grateful to three anonymous College and Research Libraries reviewers for their insightful comments on an earlier version of this manuscript. © Mark Pegrum and Ralph Kiel This article reports on the implementation of a professional develop- ment program in emerging technologies for librarians at the University of Western Australia. Set up in 2008 to address a lack of pedagogically grounded training in this area, it has now graduated three cohorts. A quali- tative analysis of participants’ online contributions and course projects, complemented by quantitative survey data, reveals that most librarians acquired new understandings of both pedagogy and technology; many were able to apply newly gained skills in the workplace; and some went on to create pedagogically grounded, technologically enabled resources of ongoing value to the library. A more intangible change in the discourse around new technologies was also observed. Limitations and challenges are identified and discussed. n a 2006 review of knowledge and skills required for the fu- ture, the University of West- ern Australia (UWA) Library identified e-learning as a key area. On- line courses previously developed by the library had been designed for passive use and were not grounded in contemporary pedagogy. Mr. Ralph Kiel, the Associate Librarian responsible for Information Systems, invited Associate Professor Mark Pegrum, who lectures in e-learning in the Graduate School of Education (GSE) at UWA, to work with library management to create and deliver a pro- fessional development course tailored to staff needs. Initially entitled simply E-learning but later renamed Emergent Technologies in Education, this course had a dual focus on the new generation of Web 2.0 technologies and on appropriate pedagogical frameworks for their use. The aim was not to produce e-learning experts or educational designers but to give librarians from all parts of the library an understanding of pedagogical approaches relevant to e-learning; the skills to develop some e-learning resources using Web 2.0 software; and the ability to act as informed participants and content advisers in larger e-learning programs. crl-190 584 College & Research Libraries November 2011 The course graduated its first cohort in 2008; its second in 2009; and its third in 2010. Drawing primarily on qualitative data in the form of participants’ online contributions and course projects, but complemented by quantitative survey data, this article uses Kirkpatrick and Kirkpatrick’s (2006) four-level Evaluating Training Programs model as a starting point for assessing the success of the course. It explains how challenges have been addressed over time and outlines limitations that remain. Background E-learning has become a prominent focus for many educational institutions due to the proliferation of user-friendly Web 2.0 tools like blogs, wikis, social networking sites, and RSS; the compatibility of Web 2.0 with social constructivism and associ- ated active, collaborative pedagogical ap- proaches;1 increasing student familiarity with and expectations of technology use in education;2 increasingly experienced staff who appreciate the ways in which technol- ogy can enhance education; and the begin- nings of a consensus on best practices.3 Given the central role of information and communication in libraries, as well as libraries’ long-standing progressive ap- proach to the adoption of relevant new technologies, e-learning has also emerged as a key focus for librarians.4 There is a growing body of literature linking librar- ies and Web 2.0,5 including work focusing in detail on the use of specific technologies like blogs,6 microblogs,7 social network- ing,8 RSS,9 and virtual worlds.10 The Emergent Technologies course was designed in line with the principles of social constructivism,11 arguably the domi- nant progressive pedagogical approach of our time, which promotes the social con- struction of understanding in collabora- tive, learner-centred contexts. Participants, who started out with widely varying levels of technological and pedagogical knowl- edge, skills and experience, were able to teach and learn from each other, collab- oratively building understanding under the guidance of the lecturer. The face-to- face component of the course consisted of thematic sessions to build theoretical understanding of key technological trends, pedagogical approaches and social issues, and tools sessions to build knowledge of Web 2.0 and Web 2.0–related tools ranging from blogs, wikis, and social networking sites to folksonomies, RSS, podcasting, and virtual worlds (for a full overview, see the course homepage at http://e-language. wikispaces.com/emergent-technologies). These were followed by computer lab sessions, where participants undertook guided explorations of the tools. The course was underpinned by the public E-language wiki (http://e-language.wikispaces.com/), which participants used as a base for their explorations of Web 2.0, and the class wiki, a password-protected, private space where participants recorded reflections and inter- acted with peers in the intervals between face-to-face sessions. Wikis are often viewed as the ultimate collaborative Web 2.0 tool,12 and it was on the class wiki that the social constructivist approach came into its own as participants contributed to the collective intelligence13 of their learning community, or “community of practice.”14 In the process, they experienced firsthand the theoretical approaches to pedagogy they were learning about in the course. With the help of ongoing feedback from peers and the lecturer on the wiki, each participant developed an e-learning re- source relevant to his or her professional context and submitted it, along with a statement of rationale, for assessment after the course. The first course ran over five weeks in early 2008. Later versions were extended to 10 weeks to provide more time for reflection on course materials and inter- action on the class wiki: the second ran in late 2008, concluding in 2009; and the third ran in late 2009, concluding in 2010. In this article, participants will be referred to as the 2008, 2009, and 2010 cohorts based on their finishing dates. The first cohort consisted of 14 UWA librarians along with one representative from stu- “Changing the Way We Talk” 585 dent services; the second, of 13 librarians, two student services staff, and one staff development officer, all from UWA; the third, of nine UWA librarians, two stu- dent services staff, and one policy officer, all from UWA, as well as two Murdoch University librarians and two private attendees. The majority group, UWA librarians, included several managers of larger subject libraries as well as supervi- sors of smaller, single-discipline libraries or key library areas, particularly in the 2008 cohort, many of whose members had more than 20 years of library experience. The average amount of library experience and the average age fell in subsequent cohorts, while the amount of prior online experience increased. Staff were initially recommended by line managers, based on their current or anticipated involvement in library e-learning projects or training. Those who agreed to participate were expected to attend all sessions except in the case of illness or prearranged com- mitments. Methodology Kirkpatrick and Kirkpatrick’s Evaluating Training Programs model,15 widely used for appraising training courses,16 includ- ing in library contexts,17 was selected as a starting point for our evaluation, partly for the comprehensive overview it builds, but also because its four levels “serve more than the obvious purpose of evaluating training after the fact; they are a great model for developing programs.”18 This helped us make ongoing improve- ments over the three years. The four levels of the model are as follows: • Reactions: participants’ satisfaction, that is to say, what they thought and felt about the course; • Learning: the increase in partici- pants’ knowledge and skills, as well as changes in attitudes; • Behavior: changes in participants’ on-the-job behavior; • Results: organizational results achieved due to changes in partici- pants’ behavior. In part because many course par- ticipants held management positions and were well qualified to assess their own progress, and in part because of the dif- ficulty of obtaining external assessments, our data are drawn primarily from partici- pants’ self-assessments, complemented by more objective evidence of learning and organizational changes as seen in course assignments. Qualitative data were col- lected systematically during each iteration of the course in the form of participants’ wiki postings, supplemented by other electronic communications, and provided some evidence of reactions and learning, as well as initial evidence of changes in behavior. The statements of rationale ac- companying projects submitted within two to four months of course completion provided evidence of learning and changes in behavior as well as, through the projects themselves (a few of which had already been integrated into library practices), organizational results. Two anonymous surveys administered around six months after the completion of each course (con- taining Likert-scale items as well as open questions) provided a longer-term view of changes in behavior and results. While nonlibrarians’ feedback is integrated with that of librarians in much of our data, the final survey in each course was restricted to UWA librarians. Comments by nonli- brarians in the current article are followed by an asterisk, while only library projects have been selected for discussion. Findings Reactions Overall assessments of the course were very positive, as reflected in participants’ readiness to recommend it to others (see figure 1). A typical comment by an experi- enced librarian in the first cohort described it as “the best course I have completed at Tertiary level—by a long shot” (e-mail, 2008), while, at the other end of the scale of experience, a young librarian wrote: I have become more interested in new technologies and their uses, 586 College & Research Libraries November 2011 0 20 40 60 80 100 120 Strongly Disagree Disagree Neutral Agree Strongly Agree 2008 2009 2010 % R es p o n se both professionally and personally, as a result of this course. It was a pleasure to attend this course—I enjoyed every session and it’s been one [of] the highlights of my first year at UWA. (final survey, 2008) Many participants stressed their— sometimes unanticipated!—enjoyment of the material covered, as in this e-mail received from a participant who had to miss a class due to a previously booked overseas holiday: “[T]he emergent tech- nologies course is the only work-related activity that I have thought about while on holiday and that’s because it’s more like fun than work ;-)” (e-mail, 2010). There was appreciation of the wide range of reference materials provided, with individuals differing as to which resources they found most useful—or most challenging. Multimedia resources were especially popular, being described as a “good start to the non-linear learning process” (wiki, 2008) and even as “ad- dictive” (wiki, 2009). There were varying assessments of Web 2.0 tools, reflecting differences in individual preferences, learning styles, and work contexts. Par- ticipants commented on the benefits of being required to complete a course assignment and valued the detailed feed- back provided: “The most comprehensive and useful feedback on a project that I can ever remember receiving,” said one (e-mail, 2008). In the first cohort, there was almost universal agreement on the need to space out the classes more, meeting fortnightly rather than weekly, and to run the course at a less busy time of year, with most sug- gesting the end rather than the start of the academic year. These changes were made in time for the second iteration of the course, with a resulting improvement in the evaluation of these aspects, though a number of participants in 2009 and 2010 continued to note that time was an issue. Aside from the change in scheduling and the regular updating of the course to include new content and materials as they became available, other aspects such as the structure, delivery, and assessment remained similar over the three years. Consequently, the slightly less positive evaluation of the course by the 2009 cohort—also reflected in this cohort’s assessments of learning and behavioral Figure 1 responses to survey item: ‘i would recommend this class to other students’ [all participants] “Changing the Way We Talk” 587 change, as seen below—seems to have been due to individual personalities and the constraints of particular work roles. Learning According to Kirkpatrick and Kirkpat- rick, learning can take the form of im- proved knowledge, increased skill, and/or changed attitudes.19 Evidence of improved knowledge—and even skill—can be seen in the final survey results in figure 2. Par- ticipants were very much aware of their own increasing knowledge levels; as one put it humorously: “I finally know what all this gen-Y speak is about” (wiki, 2009). While older participants in the first cohort commented more often on learning about the technology, many in the second and third cohorts found there was still much to discover about tools they had not yet experienced or concepts they had not quite grasped. As one librarian put it: Week 1 gave me the explanations I needed of concepts and terms that I was constantly coming across but never felt I had the time to grab hold of and explore by myself. (wiki, 2010) Improved knowledge was closely tied to increased skill developed through prac- tical exposure. Another participant wrote: I just signed up for Twitter, which is something I thought I’d never do! […] I feel that this course is already opening me up to new possibilities and it’s quite exciting!* (wiki, 2010) As can be seen in this example, changes in knowledge and skill were linked to changes in attitude; a different participant wrote of the same service: “I was dismis- sive of Twitter at the time, for one, but I hadn’t appreciated its use beyond the ‘what are you doing now?’ aspect” (wiki, 2010). That social or entertainment tools can be used for pedagogical purposes was an important realization for many. It was clear across all the cohorts that, whatever their level of knowledge about technology, most participants were more interested in the relevant pedagogy. A typical comment read: The most useful knowledge I gained from the course was a basic under- standing of pedagogical theory. Figure 2 Responses to final survey items: ‘I gained useful insight into E-learning theory/practice’ [uWA librarians only] 0 20 40 60 80 100 120 Disagree Strongly Disagree Neutral Agree Strongly Agree 2008 (theory) 2008 (prac�ce) 2009 (theory) 2009 (prac�ce) 2010 (theory) 2010 (prac�ce) % R es p o n se 588 College & Research Libraries November 2011 […] The most important skill I de- veloped (or just began to develop) was to assess material and decide on appropriate educational strategies using a range of different technolo- gies. (final survey, 2009) Participants quickly took on board one of the core messages of the course, namely that “the pedagogy must always come before the technology”* (rationale, 2010). Of course, the two are not neces- sarily separate, as one librarian noted in respect of animated Voki avatars: Certainly it could be used in an edu- cational context […] The animation is just a really cool way to package/ brand your message. It’s like using a big juicy prawn instead of crusty whitebait when fishing—lure stu- dents with something entertaining and visually appealing and then WHAMMO(!) before they know it they’ve received some educational piece of information. (wiki, 2008) Significantly, when asked in the final class to select one new term that had made an impression on them and that they themselves would now use, the majority of participants across all three cohorts chose a term relating to pedagogy (figure 3A). This was most commonly “social constructivism” or “constructiv- ism” (figure 3B). Participants were certainly not just absorbing theory but were also reflecting on how it related to their own personal learning—and ultimately, teaching—ap- proaches: Week 2 started off with Jonas- sen’s definitions of Objectivism, Constructivism and constructivist criteria. Right from the beginning I began to relate the definitions to how they personally affected me in regards to how I learn. (wiki, 2008) I have started observing myself as an individual actively constructing knowledge. (wiki, 2010) Practical exposure to Web 2.0 was a key course component. At the start of every course, but particularly the first, a number of individuals felt uncomfortable about participating on the class wiki. As one wrote: “It was a new experience for me to have somebody read my still half- baked thoughts before I thought they were ready for ‘public’ viewing” (wiki, 2008). However, it did not take long for participants to see the value of being forced to use unfamiliar technologies Figure 3A And 3B 3A: Categories of new terms learned on the course, 2008–2010 [all participants] 3B: New pedagogical terms learned on the course, 2008–2010 [all participants] Pedagogy 60%Tools 17% Other 23% Social Construc�vism 52% Other 24% Construc�vism 24% “Changing the Way We Talk” 589 and interact—with the technologies and each other—in unfamiliar ways. Another librarian commented: Although it is still early days, I am most surprised that at the end of Week One (a mere 6 days into this course) I feel more confident using this form of communication. I can already see how beneficial it is to learn and experience how students themselves are communicating with each other using these tools. (wiki, 2008) Over time, each cohort built a socially and intellectually supportive online com- munity. Participants frequently expressed their appreciation of peers’ contributions: What is really useful is looking at others’ discussion of their read- ings—I have found [name] and [name] have both made some in- sightful summaries of their readings. Thanks for that guys! (wiki, 2008) You’ve put a lot of thought into this, [name], you’re well ahead of me but you are helping me clear my think- ing! (wiki, 2010) In all three cohorts, participants posted links to informative articles on the class wiki or even inserted relevant RSS feeds into their wiki pages, while a 2009 partici- pant set up a dedicated wiki page entitled “Toolbox” as a place to share useful links. This was a good way of communicating new information of direct professional relevance to complement some of the more general e-learning input received in the face-to-face classes. Beyond this, participants showed great willingness to work with the social constructivist principles underpinning the course and to actively build on each other’s contributions. When, for example, one 2008 participant posted a message ex- plaining what he had learned about social networking by observing his son’s group of friends, a second participant’s reply made a link with Vygotsky’s work on the Zone of Proximal Development, and a third par- ticipant’s reply made a link with Putnam’s work on bonding and bridging capital. Or, when a 2010 participant created an avatar and posted it on his wiki page along with a link to the service he’d used to create it, two other participants commented that they wanted to create their own avatars but hadn’t been able to follow the instruc- tions provided by the service. He replied with his own instructions and, a day later, a second participant posted her avatar, explaining how she’d followed a slightly different pathway to create it. By the time the class met again face-to-face, four mem- bers had avatars—and they were able to reflect on this wiki-based conversation as an illustration of a social constructivist ap- proach in action. In fact, it did not take long for participants in any of the cohorts to perceive the parallels between the theories they were learning about and the learning community they were building: a place where group members could collabora- tively improve knowledge, increase skill, and change attitudes. As one person put it: A good example of [social construc- tivism] is our class wiki, it is a web 2.0 technology which allows participants to both experientially learn on an in- dividual level through [their] use and manipulation of the wiki technology, and also share their new experiences/ learning through reflection and dis- cussion. The wiki itself then came to represent the collective intelligence of the group. (rationale, 2008) Behavior Behavioral changes were reflected in the final post-course survey where—despite a significant number of exceptions—the majority of UWA librarians indicated that they had been able to implement their new learning in the workplace (see figure 4). As might be expected, many librarians focused on pedagogy: 590 College & Research Libraries November 2011 [The course] made me realise that I had been trying to teach without the necessary pedagogical knowledge to underpin my teaching. I have used the knowledge I gained about the pedagogy to re-evaluate my teach- ing, both face-to-face and online […] (final survey, 2008) I now look for opportunities to make any teaching activity more interactive and less teacher centred. (survey, 2009) A greater familiarity with key Web 2.0 tools was also valued, especially by mem- bers of the 2008 cohort. A typical participant described an “increased confidence in rela- tion to web 2.0 technologies” and a “willing- ness to take on any projects involving them” (final survey, 2008), while another wrote: The course equipped me with skills that allowed me to contribute to a Faculty wiki which was terrific! In many cases I felt that I was more knowledgeable than the academic staff I was working with. (final survey, 2008) Members of all cohorts indicated that, even more than class wiki participation, it was being “forced” (wiki, 2008) to complete an assessed course project— entailing sustained engagement with a particular tool or set of tools—that led to the greatest changes in behavior: Although the assessment was chal- lenging and time-consuming, I think it is the only way to really learn the skills & understand the theories—by actually producing something. (final survey, 2008) Everyone is time-poor [and] people can be quite resistant to adopt new technologies and practices even when they suspect there is value. It’s been good, therefore, to have had the incentive of this course to reflect on what might be of use and then have to implement the practical example as well as reflect on the utility, the challenges and value of the application chosen. The combination of theory, practice and reflection creates a powerful learn- ing experience.* (rationale, 2009) Figure 4 Responses to final survey item: ‘I have implemented in the workplace some of the things I learnt on this course’ [UWA librarians only] 0 10 20 30 40 50 60 Strongly Disagree Disagree Neutral Agree Strongly Agree 2008 2009 2010 % R es p o n se “Changing the Way We Talk” 591 In all three cohorts, the limitations on long-term behavioral changes reported by librarians fell into three main categories. First, time and workload were seen as obstacles for many, including those who re- ported in the final survey that they had not (yet) been able to implement changes in the workplace. Lack of time also contributed to the limited use and eventual abandonment of the Facebook group, set up as an ongoing forum at participants’ request at the end of the first course, and replaced in 2009 with a Twitter feed that made participation less demanding (but also less trackable). Librar- ians’ comments often reflected a general sense of overload that went beyond day- to-day work demands: I feel that I am always teetering on the brink of the time rift. There is a real feeling of anxiety about not be- ing able to cope with the incredible amount of interesting information available. The question that con- stantly plagues me is “how much am I missing out on?” (wiki, 2009) Remaining up to date and familiaris- ing myself with new technologies is an ongoing task. I often find myself overwhelmed trying to keep up with professional development “tweets,” “blogs,” wikis etc. (rationale, 2010) Second, librarians felt their behavior was constrained by certain aspects of their professional roles. While most indicated they were keen on interactive, student- centred approaches, especially after exploring contemporary pedagogy on the course, they had limited contact time with students—a situation, they said, that pushed them toward transmission modes: “Social constructivism” and “Com- munities of Practice” are ideals to strive for but […] remain difficult to achieve in the Library context where we see students once only and are teaching a limited range of skills. (wiki, 2008) [O]ne of the challenges for libraries is how to build these communities [of practice] when you don’t have regular contact with particular groups of students. (wiki, 2010) Echoing the first comment above, it was noted by some that librarians’ role is primarily to teach skills rather than academic subjects and that, although the scope for “sharing and building up collective knowledge” (wiki, 2008) does exist, it is often restricted. Significantly, several saw their roles as limited by aca- demics who gave them little educational autonomy, mandating specific outcomes in face-to-face teaching and “a linear, sequenced approach” (wiki, 2008) in on- line courses. Nevertheless, others found themselves in more flexible contexts where they were in fact able to make their teaching and/or resources more student- centred and interactive (see examples of projects described below). Third, institutional constraints were highlighted by four librarians on the 2008 course, who expressed frustration at what they saw as a lack of cooperation by IT staff. This was attributed to security concerns or simply a lack of precedents. The feeling of frustration was even more widely echoed in 2009 (as also reflected in figure 4 above) by librarians keen to experiment with Web 2.0, with typical final survey comments including: […] it doesn’t seem that we are able to set-up such tools in the Library without IT resources. We seem to be moving forward at full pace with the training and education, but lagging behind in the practical implementa- tion of what we have learnt. (final survey, 2009) The current IT environment is far too restrictive. […] The course is excellent but the learning shouldn’t stop there. (final survey, 2009) Significantly, in both 2009 and 2010, 592 College & Research Libraries November 2011 librarians were more likely to reflect on larger institutional issues going beyond IT per se. Web 2.0, much like the pedagogical approaches it complements, may not sit comfortably with traditional organiza- tional structures: [There is] the hierarchy of control that is so entrenched in large insti- tutions. Everyone wants to jump on the Web 2.0 bandwagon because they want to reach people—but I think there has to be a real com- mitment to sharing control—of community ownership, otherwise I can’t see that it will amount to much. (wiki, 2009) Interestingly, many of these tech- nologies involve not just a shift in technology, but a shift in organisa- tional attitudes. Are organisations like Libraries ready to put up what they may consider flakey software to allow students to say what ever they want. Lots of trust issues here and how much the Library is pre- pared to let go. (wiki, 2010) As implicitly recognized in the above comment, there may be tension between the need for institutions to approach change conservatively and the radically new possibilities offered by emerging technologies. Some participants came to see the need to deal with this tension as part of their learning process: By undertaking a project which had the endorsement of the Library and which will form a significant part of its information literacy and training effort, we also learned about the practicalities of working in a large corporate environment. In this environment factors such as sustainability and cost become significant. Balancing these against educational “ideals” required com- promise and adjustment of our aim. Learning how to deal with the tension between ideals and realities was a valuable experience. (ratio- nale, 2009) Results Despite the contextual limitations dis- cussed above, librarians’ comments on changes in their own behavior suggest that institutional results are beginning to follow from the course. The most tan- gible results to date take the form of the resources built by participants for their course projects, some of which have al- ready been used in the library or related contexts, while others are ready to be used, either in their current form or as models, if and when the opportunity arises. Resources designed for staff typically provided spaces for collaborative learn- ing or knowledge building. The 23 Things blog (2009), for example, was set up as a group learning platform for participants on the UWA Library’s own version of this popular course. The Science IRIS Staff Wiki (2008) was set up with the dual aim of cap- turing tacit organizational knowledge and providing a “valuable ‘on the run’ training tool” to which all librarians involved with this project could contribute (note that all quotes in this section are drawn from the relevant project statements of rationale). The Wiki for New Librarians at UWA (2008) had a similar dual focus on introducing new staff to core competencies for librar- ians and “facilitating knowledge sharing between new and existing librarians.” The Library Lingo Wiki (2009) offered new librarians the chance to tap into a vocabulary database built collaboratively by all librarians, and the ALIA WA Library Technicians’ Group Wiki (2010) provided an interactive group space for its members. The best-used of these resources have been those, like the Science IRIS Staff Wiki, whose creators had considerable autonomy to choose their own internal group communication tools, or which, like the 23 Things blog or ALIA Wiki, were clearly integrated with institutional or organizational initiatives. Resources that lacked clear institutional support were “Changing the Way We Talk” 593 software purchased by both the UWA and Murdoch Libraries in September 2008 and used as a base platform by a number of course participants to create resources on subjects as diverse as Finding Legislation and Secondary Sources in 2009 or Case Law and EndNote in 2010. The best-used of these resources have been those that, like Vet Search, were tightly integrated with academic courses and promoted by aca- demic staff; that, like the many LibGuides, were built in a library-supported format and integrated into library information provision strategies; or that functioned as stand-alone learning objects that could be used in varying contexts and even embed- ded in different platforms. The tangible results outlined above were complemented by other, less tan- gible results. Asked whether the course had facilitated productive interactions with colleagues in the application of e- learning, the majority of UWA librarians agreed (see figure 5). Further elucidation of this point was provided by comments that make it clear that the course led to a change in librarians’ discourse around new technologies. As a 2009 participant expressed it in his final presentation, the Emergent Technologies course gave librarians the vocabulary they previously lacked to discuss pedagogical issues in relation to new technologies. Or, as a 2008 participant, herself a subject library manager, put it when asked by senior management about the ultimate effect of the course: “It’s changed the whole way we talk about technology.” Discussion Reactions. While positive reactions don’t guarantee learning, according to Kirkpatrick and Kirkpatrick, negative reactions make it less likely.20 In this case, participants’ overwhelmingly positive reactions—resulting, it seems, both from the intrinsic appeal of the subject matter (once initial fears had been overcome) and its presentation in a “fun” (in other words, exploratory and experimental) manner—provided fertile ground for less likely to be used and more likely to be rendered obsolete by management decisions on organization-wide training approaches or software purchases. Resources designed for students typically sought to enhance static, print- based, transmission materials with regular updates (for instance, through RSS feeds from blogs or media sites), multimodal content (such as podcasts, vodcasts, or animated avatars), interactive spaces for consulting with librarians (like chat windows or discussion boards), or interactive spaces for building knowledge with other students (using wiki pages or collaboratively edited tag clouds, to name two). For example, the Research Success! blog (2008), with integrated YouTube videos and a tag cloud, offered post- graduates in Social Work an additional communication channel with the library, one predicated on a social constructivist approach encouraging them to “pool knowledge and compare notes with oth- ers.” Similarly, the Murdoch-based Vet Search wiki (2010) offered “collaborative help for resource discovery” to which “both librarian[s] and students could contribute.” With the rapid spread of user-friendly video editing software, it became more common in each successive cohort for participants to create their own videos, animations, and screencasts tai- lored to particular informational contexts. Some, like the Legal Research Video (2008), Podcasting at UWA (2009), or Why Use Academic Articles? (2009), were designed as stand-alone learning objects that could be used in a variety of contexts, and some were additionally hosted on services like YouTube. By 2010, it had become normal practice to embed videos in other plat- forms, such as the Capzles digital story Endnote XI (2010) or the Evidence-Based Dentistry Wiki (2010). This was part of a broader move over three years toward layering Web 2.0 tools over one another, which typically involved embedding videos, avatars, tag clouds, and RSS feeds in, for example, a Web site, blog, wiki, or LibGuide. The last of these is proprietary 594 College & Research Libraries November 2011 learning. In successive iterations of the course, the range of resources, especially in multimedia formats, was gradually ex- panded, as was the range of tools covered, which helped cater to a wider range of participant interests and learning styles. Timetabling changes, such as altering the time of year and spacing out lessons, partly alleviated the concerns over time pressure expressed by 2008 course partici- pants, while a 2009–2010 request to line managers to grant librarians some addi- tional time to devote to course readings and activities during regular working hours helped in some cases. Nonethe- less, time pressure is likely to remain an issue with any intensive course integrated into participants’ regular schedules, with those able and willing to devote some personal time to it being likely to gain the greatest benefit. Learning. It is clear that changes took place in all three of Kirkpatrick and Kirkpatrick’s categories of learning, namely knowledge, skill, and attitudes. As expected, there was much for partici- pants to learn, theoretically and practi- cally, about new technologies, including for younger and more digitally aware cohorts. An unexpectedly strong trend observed across all cohorts was a greater interest in pedagogy than technology per se, suggesting that librarians might well appreciate more general educational input as a foundation not only for their use of technology but for their various instructional activities. For many par- ticipants, “forced” interaction with and via Web 2.0 tools, while challenging, was one of their most important learning experiences, justifying the strong practi- cal focus of the course. Moreover, using a class wiki as the basis for the online learning component gave participants firsthand experience not only of the tech- nology itself, but of a social constructivist educational approach facilitated by the technology. All three cohorts came to function well as learning communities, or communities of practice, with members actively and collaboratively building new perspectives as they scaffolded each other ’s understandings and developed the group’s collective intelligence. This tight integration of theory and practice was noted and valued by participants and appears to have been one of the strongest features of the course. Figure 5 Responses to final survey statement: ‘The course has facilitated productive interactions with colleagues in the application of E-Learning’ [UWA librarians only] 0 10 20 30 40 50 60 70 80 Strongly Disagree Disagree Neutral Agree Strongly Agree 2008 2009 2010 % R es p o n se “Changing the Way We Talk” 595 Behavior. According to Kirkpatrick and Kirkpatrick, four conditions are nec- essary for behavioral change: a person must want to change, must know how to do so, must work in the right climate, and must be rewarded for changing.21 While the first two conditions seem to have been met for many participants during the Emergent Technologies course, the third and fourth conditions typically fall outside the parameters of any given course. As seen above, contextual constraints did in fact place some limits on the extent of behav- ioral change resulting from the course. While changes certainly occurred, notably in librarians’ pedagogical approaches but also in their readiness to engage with Web 2.0 projects—as borne out, indeed, by the Web 2.0 projects in which some were and are engaged—three main kinds of limita- tions were identified by participants. The first limitation, time and workload, may be linked to the looming societal is- sue of information or attention overload, particularly for knowledge workers, which is the subject of a rapidly growing body of literature.22 While global issues are unlikely to be solved locally, pos- sible steps at an institutional level might include carving out some time for past cohorts to participate in ongoing post- course forums (building on participants’ wishes to maintain momentum through the ultimately failed Facebook forum, and perhaps supplementing the current transmissive Twitter feed with a more collaborative space) or organizing short follow-up courses. The second limitation, the restricted scope of librarians’ teach- ing roles, is, again, difficult to address locally, but it was notable that in some cases participants were able to integrate constructivist approaches and/or Web 2.0 tools into their teaching and resources, often by taking a lateral approach to the possibilities. Perhaps there is a role for both course and institutional leadership in encouraging more “thinking outside the box.” The third limitation, a lack of IT and/or broader institutional support, may reflect a particular moment in time when the UWA Library, like other similar organizations, has had the foresight to train staff in emerging technologies, but an environment conducive to Web 2.0 experimentation has yet to fully evolve. Recent reports suggest there is in fact an international, sector-wide problem with organizational IT restrictions cramping educational innovation.23 On the other hand, institutions do have a responsibil- ity to consider security and related issues before unleashing large-scale change. Certainly, there would seem to be a need for dialogue between librarians, IT staff, and library management about how to balance security against flexibility. Such dialogue can be facilitated by strategies such as asking the library for suggested topics for course projects, as was done in 2009, and inviting managers as well as past participants to attend the final pre- sentations session in each course to view and discuss the projects undertaken and the resources developed. Results. Each course produced tan- gible results in the form of educational resources, often begun as part of course projects. Somewhat unusually, the proj- ects meant that institutional results began to occur within the parameters of the course itself, reinforcing the value of set- ting participants such hands-on tasks. The real test, however, was the ongoing use— or the creation of modified versions—of resources after the end of each course. This occurred in a number of cases. The relative success of these resources often depended on their tight integration with either academic programs or library initiatives, suggesting the need for more collaborative dialogue around the use of new technologies and confirming the value of involving library management in suggesting course projects and viewing course results. Other successful resources took the form of artifacts that could be reused in a variety of ways. Despite the limitations of generic learning objects,24 this suggests that useful artifacts can be both tailored to their contexts (that is, in terms of subject areas, student levels, or 596 College & Research Libraries November 2011 task types) and flexible enough to be em- bedded in different platforms or to serve a range of needs within a given course or discipline. The course also produced intangible results, notably a change in discourse around new technologies. It is interesting to speculate on whether such a change would have occurred if the course had only run once or whether it was neces- sary to build up a certain critical mass of past course participants who shared a new language before an effect on broader institutional discourse could emerge. The latter interpretation is supported by the following comment from a librarian on the 2010 course: I feel that the Library is starting to embrace new technologies a lot more now and is encouraging its staff to do the same, I believe that has a lot to do with the Emergent Technologies course and the people who have attended the course. (final survey, 2010) Given that organizational results may take a long time to manifest,25 intangible changes could well be more important in the long run than short-term, tangible changes—and might indeed lead to fur- ther tangible changes in the future. In the meantime, placing more emphasis on ongoing collaborative forums or follow-up meetings might help to sustain the emerging language, better embed changed or changing perspectives, help new collaborative projects to emerge, foster dialogue between librarians, IT staff, and management, and even provide a springboard from which to launch new professional development initiatives. Conclusion Feedback from the three cohorts indicates that, despite certain limitations, the Emer- gent Technologies course produced desir- able improvements in knowledge, skill, and attitudes; changes in behavior; and some initial institutional results, including a change in discourse. This gives the li- brary a base on which to continue to build staff capacity in the pedagogically effec- tive integration of digital technologies. Thanks to an Australian Learning & Teaching Council grant, the Emergent Technologies course is being reconfigured in 2010–2011 as a broader course for pro- fessional staff, coupled with the develop- ment of an online network for past attend- ees and others who are interested in the use of digital technologies in educational institutions. Through the course and the network, it is intended to promote both cascade and viral models of knowledge and skills dissemination.26 Librarians from a number of higher education in- stitutions will be among the participants on the new courses. It is hoped that any who may initially be hesitant about digital technologies will come, in time, to share the sentiments of a senior librarian from the very first cohort: My lack of knowledge, fear of, and overall inexperience with, Web 2.0 technologies created a real barrier to entry. I was very hesitant and spent a lot of time reading as much as I could before I started. Now I’m addicted! (wiki, 2008) Notes 1. Mark Pegrum, “Communicative Networking and Linguistic Mashups on Web 2.0,” in Handbook of Research on Web 2.0 and Second Language Learning, ed. Michael Thomas (Hershey, Pa.: Information Science Reference, 2009), 20–41; Mark Pegrum, From Blogs to Bombs: The Future of Digital Technologies in Education (Crawley, W.A..: UWAP, 2009). 2. Gregor E. Kennedy, Terry S. Judd, Anna Churchward, Kathleen Gray, and Kerri-Lee Krause, “First Year Students’ Experiences with Technology: Are They Really Digital Natives?” Australasian Journal of Educational Technology 24, no. 1 (2008): 108–22, available online at http://www.ascilite. org.au/ajet/ajet24/kennedy.pdf [accessed 4 September 2010]; Shannon D. Smith, Gail Salaway, and “Changing the Way We Talk” 597 Judith Borreson Caruso, The ECAR Study of Undergraduate Students and Information Technology, 2009 (Boulder, Co: EDUCAUSE Center for Applied Research, 2009), available online at http:// net.educause.edu/ir/library/pdf/ers0906/rs/ERS0906w.pdf [accessed 4 September 2010]. 3. The emerging consensus is reflected in the publication of frameworks and handbooks for e-learning: for example, The Sage Handbook of E-learning Research, eds. Richard Andrews and Caroline Haythornthwaite (London: Sage, 2007); D.R. Garrison and Terry Anderson, E-learning in the 21st Century: A Framework for Research and Practice (London: Routledge Falmer, 2003); Robin Mason and Frank Rennie, E-learning and Social Networking Handbook: Resources for Higher Education (New York: Routledge, 2008). 4. Supporting E-learning: A Guide for Library and Information Managers, ed. Maxine Melling (London: Facet, 2005). 5. Phil Bradley, How to Use Web 2.0 in Your Library (London: Facet, 2007); Information Literacy Meets Library 2.0, eds. Peter Godwin and Jo Parker (London: Facet, 2008); Ellyssa Kroski, Web 2.0 for Librarians and Information Professionals (New York: Neal-Schuman, 2008); Web 2.0 and Librar- ies: Impacts, Technologies and Trends, eds. Dave Parkes and Geoff Walton (Oxford, U.K.: Chandos, 2010); Medical Librarian 2.0: Use of Web 2.0 Technologies in Reference Services, ed. M. Sandra Wood (Binghamton, N.Y.: Haworth Information Press, 2007). 6. Karen A. Coombs and Jason Griffey, Library Blogging (Columbus, Ohio: Linworth, 2008); Walt Crawford, Academic Library Blogs: 231 Examples (Mountain View, Calif.: Cites & Insights, 2008); Michael P. Sauers, Blogging and RSS: A Librarian’s Guide (Medford, N.J.: Information Today, 2006). 7. Ellyssa Kroski, “All a Twitter: Want to Try Microblogging?” School Library Journal, July 1, 2008, available online at http://www.schoollibraryjournal.com/article/CA6573999.html [accessed 4 September 2010]. 8. Linda Berube, Do You Web 2.0? Social Networking and Library Services (Oxford, U.K.: Chan- dos, 2010); Laurie Charnigo and Paula Barnett-Ellis, “Checking Out Facebook.com: The Impact of a Digital Trend on Academic Libraries,” Information Technology and Libraries 26, no. 1 (2007): 23–34, available online at http://www.ala.org/ala/mgrps/divs/lita/ital/262007/2601mar/charnigo. pdf [accessed 4 September 2010]; Lee Rainie, Friending Libraries: Why Libraries Can Become Nodes in People’s Social Networks (Washington, D.C.: Pew Internet & American Life Project, 2009), available online at http://www.pewinternet.org/Presentations/2009/8-Friending-libraries.aspx [accessed 4 September 2010]. 9. Sauers, Blogging and RSS. 10. Ilene Frank, “Librarians in Virtual Worlds: Why Get a Second Life?” First Monday 13, no. 8 (2008), available online at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/ view/2222/2010 [accessed 4 September 2010]. 11. Constructivism and the Technology of Instruction: A Conversation, eds. Thomas M. Duffy and David H. Jonassen (Hillsdale, N.J.: Lawrence Erlbaum, 1992); Glenn Finger, Glenn Russell, Romina Jamieson-Proctor, and Neil Russell, Transforming Learning with ICT: Making IT Happen (Frenchs Forest, NSW: Pearson, 2007); L.S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes (Cambridge, Mass.: Harvard University Press, 1978). 12. Jane E. Klobas, Wikis: Tools for Information, Work and Collaboration (Oxford, U.K.: Chandos, 2006); Stewart Mader, Wikipatterns: A Practical Guide to Improving Productivity and Collaboration in Your Organization (Indianapolis, Ind.: Wiley, 2008); James A. West and Margaret L. West, Using Wikis for Online Collaboration: The Power of the Read-Write Web (San Francisco, Calif.: Jossey-Bass, 2009). 13. Henry Jenkins, Convergence Culture: Where Old and New Media Collide, new ed. (New York: New York University Press, 2008). 14. Etienne Wenger, Communities of Practice: Learning, Meaning, and Identity (Cambridge, U.K.: Cambridge University Press, 1998). 15. Donald L. Kirkpatrick and James D. Kirkpatrick, Evaluating Training Programs: The Four Levels, 3rd ed. (San Francisco, Calif.: Berrett-Koehler, 2006); Donald L. Kirkpatrick and James D. Kirkpatrick, Implementing the Four Levels: A Practical Guide for Effective Evaluation of Training Programs (San Francisco, Calif.: Berrett-Koehler, 2007). 16. Jody L. Fitzpatrick, James R. Sanders, and Blaine R. Worthen, Program Evaluation: Alternative Approaches and Practical Guidelines, 3rd ed. (Boston, Mass.: Pearson, 2004); Questionmark, Assess- ments and the Kirkpatrick Model: Using Assessments to Evaluate Training (2004), available online at http://tryout.questionmark.com/ learningcafe/1919374234/Assessments_and_Kirkpatrick_Model. ppt [accessed 4 September 2010]; Elaine C. Winfrey, “Kirkpatrick’s Four Levels of Evaluation,” in Encyclopedia of Educational Technology, ed. B. Hoffman, available online at http://edweb.sdsu. edu/eet/articles/k4levels/start.htm [accessed 4 September 2010]. 17. For example, John H. Heinrichs and Bin Li, “Integrating Electronic Notebooks in Daily Work at Wayne State University,” in An Introduction to Staff Development in Academic Libraries, 598 College & Research Libraries November 2011 ed. Elizabeth Connor (New York: Routledge, 2009), 134–53; Maggie Z. Saponaro, M. Sue Baugh- man, and Jennifer Kinniff, “You Came for the Snacks, But What Have You Learned? Evaluation of a Staff Learning Program at the University of Maryland Libraries,” in An Introduction to Staff Development, ed. Connor, 201–19. 18. Donald L. Kirkpatrick and James D. Kirkpatrick, Transferring Learning to Behavior: Using the Four Levels to Improve Performance (San Francisco, Calif.: Berrett-Koehler, 2005), 8. 19. Kirkpatrick and Kirkpatrick, Evaluating Training Programs, 22. 20. Ibid., 22. 21. Ibid., 23. 22. Maggie Jackson, Distracted: The Erosion of Attention and the Coming Dark Age (Amherst, N.Y.: Prometheus, 2009); John Naish, Enough: Breaking Free from the World of More (London, U.K.: Hodder & Stoughton, 2008); David Shenk, Data Smog: Surviving the Information Glut (San Francisco, Calif.: Harper Edge, 1997); Andrew Whitworth, Information Obesity (Oxford, U.K.: Chandos, 2009). 23. L. Johnson, A. Levine, and R. Smith, The Horizon Report: 2008 Australia–New Zealand Edi- tion (Austin, Tex.: The New Media Consortium, 2008), 3, available online at http://www.nmc.org/ pdf/2008-Horizon-Report-ANZ.pdf [accessed 4 September, 2010]; Strategic ICT Advisory Service, Department of Education, Employment and Workplace Relations [Australia], Collaboration in Teaching and Learning (Dulwich, S.A.: Education.au, 2009), 29–30, available online at http://www. educationau.edu.au/sites/default/files/2009_SICTAS_CTL_1.pdf [accessed 4 September 2010]. 24. Sami Nurmi and Tomi Jaakkola, “Promises and Pitfalls of Learning Objects,” Learning, Media and Technology 31, no. 3 (2006): 269–85; Patrick E. Parrish, “The Trouble with Learning Objects,” Educational Technology, Research and Development 52, no. 1 (2004): 49–67; Michael Shaw, “(Contextual and Mutated) Learning Objects in the Context of Design, Learning and (Re)Use,” Teaching & Learning with Technology 8 (2003), available online at http://www.shawmultimedia.com/ edtech_oct_03.html [accessed 4 September 2010]; David Wiley et al., “Overcoming the Limitations of Learning Objects,” Journal of Educational Multimedia and Hypermedia 13, no. 4 (2004): 507–21. 25. Kirkpatrick and Kirkpatrick, Implementing the Four Levels, 111. 26. For further information, see the ALTC-Viral Leadership Project Web site, available online at http://www.altc-viral.groupsite.com [accessed 4 September 2010].