Microsoft Word - EducationLIbrariesSpring2008A.doc Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 51 Developing a website directory for young people: A case study using graduate students in library and information science By Enid Irwin and Ken Haycock Abstract KidsClick! is a web-based instructional resource designed for K-12. In 2006 it was transferred to the San Jose State School of Library and Information Science, and was subsequently refocused for grades 4-9, ages 10-14, post-reading, and pre-adult reading level phases. This article describes three parts of the redesign project - content, interface, and back end – explains methods and procedures, and analyzes results. Finally, we list lessons learned and recommendations for using graduate students on long term projects “across time and space.” KidsClick! (www.kidsclick.org) is a directory of websites organized by subject and offers easy access to a controlled set of 5,400+ resources selected by librarians to meet the needs of young users. It was started by a group of librarians in The Ramapo Catskill Library System (RCLS) in Middletown, New York, with a $12,000 grant from the Library Services and Technology Act in 1997. Development and maintenance then transferred to the Colorado State Library and Department of Education in June, 2001 (Crotty, 2007). In 2003 KidsClick! earned a Top Twenty rating for its variety of search methods, relatively clear navigation, age appropriate material, and lack of advertising (Haycock, Dober, & Edwards, 2003). In addition Danny Sullivan's Search Engine Watch in 2005 included KidsClick.org in a list of "major children’s directories that filter websites to include those best suited to the younger audience" (Crotty, 2007, p. 1). During the spring of 2006, San Jose School of Library and Information Science (SLIS) purchased KidsClick.org. Although the website averaged 59,000 hits and 3,000 unique visitors a day, their vision was to modernize the search tool and change its primary audience. The initial application for a California State University (CSU) research grant, subsequently funded, stated: “The purpose of this application is to apply research-based principles to the evaluation, selection and organization of web resources, specifically for young people ages 10-14, and to design an age- appropriate portal, subject directory and search engine.” The KidsClick! redesign project started in February, 2007. The objectives of the project to refocus the target audience fell into four groups as follows: Under the Hood • Update the technology. • Add web 2.0 interactivity and features. Design • Create a modern interface that appeals to ages 10-14, grades 4-9. Content • Limit the scope of the site to ages 10-14, grades 4-9. Future Services • Add a spell checker. • Publish an automated weekly newsletter. • Investigate use of Spanish language sites. • Include relevance and rankings of search results. • Include user recommendations. • Establish evaluation criteria for websites. Redesign objectives focused on youth with post-ability to read, pre-adult reading, and search engine ability, in part because this group was under-represented by search engines and directories targeting youth (Haycock, 2006). An assumption was that teachers, librarians, and parents for these students will also use the website. In addition to refocusing the target audience, a second major category of goals centered on using graduate students to: • Evaluate and categorize the websites in the collection. • Assign readability levels to each selected website. • Design an age-appropriate and appealing search engine portal. • Market the website to professionals as well as parents and students. • Develop assessment tools to assess the effectiveness of the redesign. http://www.kidsclick.org Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 52 Skills relating to these tasks are taught at the School (http://slisweb.sjsu.edu/classes/coursedesc.htm) so projects could be covered within class assignments or students who completed specific courses recruited to work on KidsClick! Using seed money from the CSU mini-grant, the San Jose School of Library and Information Science (SLIS) coordinated the development and ongoing maintenance of the site and also looked for external funding. The School believed the project had potential for graduate students to train teachers, students and others in effective searching and information literacy. A School goal was for the website to bring favorable publicity to the University and the School among elementary and middle school students as well as teachers and administrators, so over the longer term, the School planned KidsClick! as a showcase for the work of professional librarians in the electronic age. Redesign Procedures Organization In order to start the redesign and oversee development, the school hired a project manager. The salary was originally funded by the school and subsequently through a grant from the Juniper Networks Foundation (https://www.juniper.net/). The redesign was divided into three separate major tasks, including content, interface, and back end. Content involved reviewing existing websites in KidsClick.org and setting up a collection development policy. Interface centered on developing a modern look and feel to the website. Back end included establishing maintenance procedures, reviewing search engines, and investigating web 2.0 applications such as blogs, wikis, and surveys. Two student assistants with technology experience prior to enrollment at the school were hired to document back end maintenance procedures and catch up on tasks including a backlog of 400 emails accumulated during the transfer to SLIS. They also hired two additional student assistants to assist with additional tasks, including program logic mapping, web page coding, and survey processing. Since the school offers a wide range of classes teaching skills needed to redesign the website, they decided to involve graduate students in the redesign, and consequently recruited eight graduate students who had excelled in an information retrieval class taught by the project manager to form a KidsClick! Design Group (KCDG) to tackle tasks related to the content and interface design. Also recruited was a graphic arts student and faculty adviser from California State University, Monterey Bay (CSUMB) to use KidsClick! as a yearlong Capstone graduation project. As the project progressed, the school sought and accepted other partnerships outside of the school benefiting the redesign as they presented themselves. One such collaboration was creating comic book style information literacy lessons using Bill Zimmerman's Comic Generator (http://www.makebeliefscomix.com/). Communication As several tasks began simultaneously and SLIS students had more than one responsibility, communication proved vital. Also, since students were scattered throughout the country, meeting face to face was rarely possible, so many discussions were handled online using an online classroom called Blackboard (http://www.blackboard.com/us/index.Bb) for the Design Group and student assistants. There they posted documentation, procedures, research articles, and files, as well as used discussion forums for brainstorming, reviewing work, and presenting research results. Blackboard emails allowed for quicker communication when needed for reminders and task updates or changes. The Design Group was responsible for finding solutions for content and interface needs while the student assistants were tasked with implementing any technology the solutions required. Blackboard and the application Elluminate facilitated working together and giving everyone an awareness of the project's scope. Elluminate (http://www.elluminate.com/) is a real- time virtual classroom environment designed for distance education and collaboration. It facilitated the group in face-to-face brainstorming, decision making, planning, instruction, and document sharing. Specific Project Procedures Content. The Design Group decided to develop a survey to use for evaluating approximately 5400 websites in the database. After researching evaluation techniques, they chose selection criteria (http://www.kidsclick.org/selection.html) for KidsClick.org and the Librarians’ Index to the Internet (http://lii.org/pub/htdocs/selectioncriteria.htm). Criteria from KidsClick.org were used to disqualify websites, and criteria from the Librarians' Index (LII) were used for overall evaluation. The assumption was made that initial reviews of all websites were by librarians using standard yet undocumented collection development criteria; therefore, the primary purpose of the review was to determine if the website had changed. http://slisweb.sjsu.edu/classes/coursedesc.htm https://www.juniper.net/ http://www.makebeliefscomix.com/ http://www.blackboard.com/us/index.Bb http://www.elluminate.com/ http://www.kidsclick.org/selection.html http://lii.org/pub/htdocs/selectioncriteria.htm Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 53 A major factor in the survey design was to make the survey simple to understand and easy to complete. This allowed inexperienced SLIS students to do the surveys as well as increase objectivity in the evaluations. The second design goal was to establish a survey that could be eventually expanded for augmenting the KidsClick! Collection. The final objective was to set up a review process for the surveys to ensure consistency and quality in the evaluations. After beta testing the survey within the Design Group and correcting problems, the group recruited volunteers and special study students for the Summer and Fall 2007 semesters. In addition they set up a Blackboard site for the evaluators to learn about website assessment from research articles, ask questions, and suggest new websites. A survey tutorial was recorded in Elluminate for on demand viewing. One student assistant was tasked with being the Survey Coordinator to keep track of progress, assist evaluators, and oversee the survey process. (Please see Figure 1 at end of Article) The evaluation surveys were fully assigned midway through the Spring 2007 semester and the student assistants’ focus switched to reviewing completed surveys. In addition, members of the Design Team started a pilot study for collection development, and applied for and were awarded a second California State University research grant for student assistant salaries. The San Jose School of Library and Information Science students now reviewed new websites suggested by librarians and teachers for addition to the collection. Magnolia (http://ma.gnolia.com/), a social tagging program, was used to tag both website suggestions and reviewed websites, which prevented duplication of effort. This particular application allowed first-level category tagging for the database record in the case that the website was added to the collection. Finally, a student assistant who was assigned the role of Collection Development Coordinator created training materials, and will set up quality control and review processes before new websites are added to KidsClick.org after the Spring 2008 semester. Interface Initially, members of the Design Group researched how youth in the target group learn new ideas and search the Internet. This research included usability and navigation problems, as well as how these fourth and ninth graders used alternative learning methods such as comics and games. These findings were shared with the graphic artist who integrated them into the prototype for the new web interface. Other input for designing the new interface prototype came from informal focus groups of classes in the target audience. The students looked at the current KidsClick.org in March, 2007. Design Group members working in schools as teachers or librarians developed questions and ran the discussions. They interviewed twenty fourth and sixth graders were at one school and thirty ninth and tenth graders at a second school. Besides the new interface prototype, the CSUMB graphic artist created a new logo with input from the Design Group. Other collaborative tasks relating to the new interface design included marketing materials: an advertisement for conference proceedings, flash movies introducing the new logo and redesign project, t-shirts, and bookmarks. Design Group members wrote information literacy comic book lessons with a comic generator and used flash to develop an animated Dewey D character as an instructional tool. A website was set up to share the materials and get feedback (http://www.kidsclick.org:8080/conferences/resources.html). When the prototype was finished in December, 2007, two Design Group members conducted additional focus groups in eight classes, four classes of ninth graders taking French (94 students), two classes of sixth graders (26 students), and two classes of seventh/eight graders (70 students) using a survey to collect data. The ninth graders completed individual surveys although they chatted amongst themselves while the lower grades worked together on the computers as well as submitted individual surveys. Back End Besides checking broken links, replying to emails, and updating database records, student assistants mapped the logic of the search engine calls and link paths in the current website. They then used the logic maps to code http://ma.gnolia.com/ http://www.kidsclick.org:8080/conferences/resources.html Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 54 the prototype as well as help the Design Group research search engine costs and features. Because the same group of people initially worked on LII.org and KidsClick.org, Librarians' Index personnel graciously supplied search engine advice and expertise about their recent redesign. The SLIS systems administrator set up a development area on the school server for web development and after the prototype design was completed, two student assistants started coding the new interface, Work will continue during the Spring, 2008 semester and usability testing is planned in June, 2008. Whether or not a new search engine is added depends on funding and the success of grant applications. First Year Progress Content By the end of the Spring 2007 semester, 5,476 websites were either assigned to evaluators or deleted from the database because of permanently broken links. In addition to the Design Team and volunteers, students in Information Retrieval (LIBR 202), Advanced Information Resources, and Services (LIBR 228) evaluated 5,058 websites. Some websites were assigned twice to use as a quality control check and the results are yet to be determined. Only 244 assigned websites were returned unfinished for various reasons and they will be completed by Information Retrieval students during the Spring 2008 semester (See Table 1). One serious technological issue showed up when the surveys were first rolled out after beta testing. The problem was that when the browser back button or histories were clicked, bookmarks or favorites used, or previous surveys edited and resubmitted, the survey was not stored as a new record. Instead it wrote over the previous survey submitted (L. Crotty & S. Laufer personal communication, March 2007). The problem was discovered when a number of students submitted their completed list of links, and only one survey was found although the evaluators had completed many more. After researching the problem with the SLIS systems administrator and trouble shooters from the survey software company (http://www.chumpsoft.com/ products/phpq/), the problem was discovered. Since surveys are generally submitted once by an individual rather than multiple times, KidsClick! was trying to use the surveys in a manner that they were not designed to be used. The problem was quickly solved through a group e- mail to all involved, announcements on discussion boards, and updating instructional materials. This approach eliminated the problem except for a few random occurrences. The pilot study for collection development using surveys and Magnolia did not have any problems with 250 websites surveyed and tagged. Interface Informal focus group discussions in March, 2007 confirmed that youth in the target audience thought that the current KidsClick.org website (http://www.kidsclick.org/), as well as the mascot logo was childish and boring. They remarked on the lack of color, images, and small point size of the text, and wanted features that included their opinions and website rankings. Although they preferred a Google style interface, they were more concerned about how the website looked rather than with quality or appropriateness of search results. Entering words into the search box was the overwhelming choice over browsing A-Z links and advanced search features (K. Gialdini & E. Scherer, personal communication, March 2007). These focus group results were integrated into the prototype, and yet when surveys from the focus groups on the prototype (http://www.kidsclick.org:8080/ conferences/newKC.pdf) in December, 2007 were reviewed, the initial reaction of new focus groups was surprise. Now younger students (sixth to eight grade) complained about too much color, the lack of a mascot, and too many menu choices. A number of factors may have contributed to the contradictory results. Although the questions were similar, current KidsClick.org focus groups utilized an open discussion style, and the focus groups on the new interface prototype used anonymous surveys. How the questions were asked may also have elicited different responses than the same questions in a written format. Students’ identity in the group discussions were known while survey answers were not linked to specific students. Finally, younger students may not have as clear-cut opinions as older students, which might have resulted in their Youth in the target audience thought that the current KidsClick.org website, as well as the mascot logo was childish and boring. They remarked on the lack of color, images, and small point size of the text, and wanted features that included their opinions and website rankings. Although they preferred a Google style interface, they were more concerned about how the website looked rather than with quality or appropriateness of search results. http://www.chumpsoft.com/products/phpq/ http://www.chumpsoft.com/products/phpq/ http://www.kidsclick.org/ http://www.kidsclick.org:8080/conferences/newKC.pdf http://www.kidsclick.org:8080/conferences/newKC.pdf Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 55 changing their minds more often. Reasons for the differences between the two sets of results on color, mascots, and menu choices will require further investigation. If the variation was caused by the wide range of maturity and learning skills within the target group, redesigning the interface will be a greater challenge. But if differences were caused by focus group technique, procedures would need to be improved. Although many remarks in the December focus groups contradicted statements in the earlier focus groups, older students in the ninth grade French classes consistently remarked favorably on organization of the items on the website, as well as grouping of functions by color. One other significant finding was that approximately half of the students said they would use the School Projects search feature that is planned for the redesign but is not available in the current KidsClick.org. The next popular search choice in the prototype was the Extreme Search, which in the earlier groups and under the former name of Advanced Search had few takers (Irwin, 2007). Back end. Student assistants successfully worked out the maintenance procedures on the database and wrote up documentation. They continued e-mail correspondence to KidsClick! and drafted forms to be personalized for common requests. They also discovered one technological issue in a search script after the redesign project started, which had not been fixed and prototype coding was slower than expected. Plans for a new search engine need funding, and so the current strategy is to attach the prototype design to the existing back end. If a new search engine is added later, the coding for the new search engine will be swapped for existing code. Web 2.0 features have not been added to the current version in order to cut down on duplicate efforts, but book review blogs, surveys, and a newsletter are planned for the prototype. Summary of Results The redesign project accomplished a number of goals during the last year: • Established a standardized method for evaluating websites. • Received 95% of the website evaluation surveys assigned to reviewers. • Started a collection development project to expand offerings. • Updated approximately 45% of the database records. • Obtained a prototype and marketing materials from the partnership with California State University, Monterey Bay. • Developed alternative instructional materials. • Started a documentation website for KidsClick.org procedures. Tasks that need more effort are: • Improve progress in coding the prototype interface for testing in June 2008. • Examine the results of the informal focus groups and survey in greater depth. • Improve focus group techniques in order to yield more consistent data. • Work with students to have their reports published. • Acquire grant funding in order to implement a search engine that allows web 2.0 features such as folksonomies, user rankings, and improved retrieval. In 2006, when the school assumed responsibility for KidsClick.org, the website averaged 59,000 hits by 3,000 unique visitors a day. By the end of 2007, use had increased to 166,700 hits a day by 4,200 unique visitors. Almost 75% of users reside in the United States, with an additional 13% in Canada, Australia, New Zealand and the United Kingdom. Remaining users span the globe. The increase in usage was attributed to marketing efforts that started during the Fall 2007 semester. Lessons Learned Using graduate students. Students involved with the surveys and enrolled in an Information Organizations and Management class did a case study of the survey process. They reported that the survey helped staff successfully complete a key task in the re-design project. Moreover the survey team model provided opportunity for enrichment of the SLIS graduate student experience. Students performed activities on an actual project with real-time specifications such as survey development, decision-making associated with protocol and best practices, and website evaluations (Crotty et al., 2007, p. 15). By involving graduate students in the redesign, SLIS achieved its goal of expanding the learning experience of its students. Searching habits of youth Research by the Design Group turned up several important concepts relating to youth learning and searching behaviors. Many of the concepts identified were incorporated into either the prototype design or Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 56 plans for the website. Because children are more successful in categorizing topics for concrete concepts than for abstract ones (Bilal. et al., 2005), KidsClick! plans to add tags to allow searching by school projects. In addition, providing them with the capability to apply their own tags to websites will also improve retrieval. Although children want to use keyword searching, they also like graphical interfaces and multimedia (Broch, 1999). Graphical aids are planned in the redesign for users to maximize their search strategies, sound files will be used as search aids to describe what is included in various categories, and more websites using multimedia will be added to the collection. Youth in the target group like to browse and a design that displays the hierarchy of related information aids their browsing (Bilal et al., 2005). Students are often frustrated by their search results, so if librarians and teachers work with them to plan their queries at the beginning of assignments, searchers will be less pressured and more engaged in learning process (Kuhlthau, 1999). Furthermore children lack a developed recall memory which makes it difficult to modify a search (Broch, 1999). To assist youth in altering search strategies, KidsClick! plans to what are called “breadcrumbs” to track searches. Focus group comments showed children are less concerned with the quality of search results and more attracted by interactivity on the website. One suggestion was to increase effort in not only teaching information literacy but also to make searching a more engaging and attractive activity. Improving engagement calls for designs that motivate and involve youth in the search process, which requires more than entertainment. Specifically, curiosity and challenge are components of motivation recommended for optimal engagement (Chiasson & Gutwin, 2001). When websites have visual and auditory content that youth find interactive and motivation, they become immersed in the learning activity (Somekh, 2003). The purpose for adding engaging activities to KidsClick.org is to actively involve youth in the search process which results in an improvement of information literacy, and consequently learning. Focus group strategies Using children in focus groups can be a challenge because they require rigorous screening and a skilled facilitator who can establish rapport with them. In addition youth in the target audience often exhibit age- related behavior such as silliness and “group think” (Krueger & Casey, 2000). Consequently the redesign project used SLIS students who also worked as teachers or librarians and already knew the participants. The expectation was that they would ensure rapport and improve results. The first focus groups on the current KidsClick.org were informal class discussions, and reports by SLIS facilitators indicated that “group think” and “follow the leader” behavior entered into the comments (K. Gialdini & E. Scherer, personal communication, March 2007). Since these first focus groups were used to confirm published reviews of KidsClick.org, student behavior was not a considered a drawback because their comments did reinforce previous conclusions. To minimize these behaviors for the more significant focus groups on the new interface prototype, a survey was used for the December focus groups. SLIS facilitators also discovered giving middle school students an anonymous survey appeared to bring out the silliness factor in many sixth through eighth graders. On the other hand, students in a ninth grade French class submitted more thoughtful and useful comments (K. Gialdini & S. Oremland, personal communication, December 2007). Despite these problems, researchers obtained useful information from the focus groups and was utilized in the prototype design. Nevertheless more research is needed on techniques for running focus groups with youth. If funding allows, a professional focus group facilitator will be engaged for beta testing. Another alternative is to give SLIS students a short yet intensive course on running small focus groups and usability studies with children. Technology requirements In many projects, working with technology takes the most time and has the greatest cost. Plans need to be flexible in order for a project to achieve its major goals if optimum funding does not materialize. Time lost on Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 57 unexpected events can be minimized by design simplification for the technical components and by adjustments to other non-technical yet related projects; for example, other tasks can be started sooner. When technology projects involve geographically scattered graduate students, increased time is needed for communication. In addition, people often work at different times, and so consequently, problems are often more difficult to spot and solve. Two factors are optimal for successful interactions with graduate students on large or complex technology projects. First, faculty must be hands-on, and involved with the students in a collaborative partnership of informal, as needed instruction and team member mentoring. Second, a low student-to-faculty ratio is strongly recommended. An informal academic atmosphere generates greater productivity than more traditional instruction methods (CSUMB faculty, personal communication, November 2007). Recommendations for using students for long-term projects: • Design a process for recruiting students that can be repeated easily in subsequent semesters • Make sure new recruits are engaged with and committed to the project goals and mission. They should understand that decisions that they make in this program can not always be changed later. • Establish consistent and orderly turnover from one semester to the next. • Allow for instruction time during planning tasks because students may not have all the skills needed, especially with technology- oriented activities. • Give students opportunities to be creative and contribute their own ideas to add value and enrich the project. • Remind students that excellence and quality are not optional because the project is a real world situation and not a class assignment. • Reward students for their work and commitment. • Increase retention and improve productivity by creating an organization chart so students can advance and learn new skills. Working with students on long-term or large projects requires a greater commitment than regular classroom or online instruction. The best results are obtained through a collaborative partnership that requires a dedicated investment of time and administrative support. Using graduate students on long-term projects is an area of library education that calls for greater investigation, but the potential rewards for both students and schools or libraries are great. The most significant insight uncovered during the KidsClick.org redesign is the expectations of youth when using search engines. Their interest appears to be with website interactivity rather than the quality of search results. Because of this, librarians and teachers will want to take a greater role in developing query and evaluation skills. As new methods for teaching information literacy develop, they will also need to more actively engage youth in the search process. Rather than merely displaying information, search portals such as KidsClick! must encourage users to engage with the information via interactive and collaborative experiences. The primary question is: will encouraging youth to engage in the search process through web-based activities improve learning? Evaluation Group Number of evaluators Surveys completed KCDG 11 715 LIBR 298 25 1820 LIBR 228 2 250 LIBR 202 29 815 Volunteers 35 1458 TOTALS 120 5058 References Barry-Rodriguez, H. (1999). Kid’s catalogue and kid’s catalogue web; A review of Carl’s OPACs for children. Retrieved February 8, 2007, from the University of North Carolina, Chapel Hill, School of Information & Library Science: http://www.ils.unc.edu/~barrh/kidscat.html Bilal, D., & Wang, P. (2005). Children's conceptual structures of science categories and the design of web directories. Journal of the American Society for Information Science and Technology, 56(12), 1303- 1313. Retrieved February 18, 2007, from Library Literature & Information Science database. 1303-1313. Retrieved February 18, 2007, from Library Literature & Information Science database. Broch, E. (1999). Children’s search engines from an information search process perspective: Retrieved February 5, 2007, from http://www.ala.org/ala/aasl/aaslpubsandjournals/ slmrb/slmrcontents/volume32000/childrens.htm#Top Chaisson, S., & Gutwin, C. (2005). Testing the media equation with children. Proceedings of the ACM http://www.ils.unc.edu/~barrh/kidscat.html http://www.ala.org/ala/aasl/aaslpubsandjournals/ Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 58 Conference on Human Factors in Computing Systems, 829-838. Crotty, L. (2007). From East to West – the KidsClick! story. Unpublished manuscript, San Jose School of Library and Information Science. Crotty, L., Eseltine, J., Halarnkar, S., & Norton, K. (2007). A survey process for website analysis: Implications for the KidsClick! search engine re-design project. Unpublished manuscript, San Jose School of Library and Information Science. Haycock, K. (2006). California State University grant proposal. Unpublished manuscript, San Jose School of Library and Information Science. Haycock, K., & Dober, M., & Edwards, B. (2003). Authoritative guide to kid's search engines, subject directories, and portals. New York: Neil-Schuman. Irwin, E. (2007). [Focus group survey comments on KidsClick prototype design]. Unpublished raw data. Krueger, R., & Casey, M. (2000). Focus groups: A practical guide for applied research. (3rd ed.). London: Sage. Kuhlthau, C.C. (1999). Accommodating the user's information search process: Challenges for information retrieval system designers. Bulletin of the American Society for Information Science, 25(3), 12-16. Somekh, B. (2003). Children exploring a 'fun' web-site: Sites of learning and roles of being. Proceedings of IFIP Workshop Group 3.5 Conference on Young People and Learning Technologies, University of Western Sydney, Paramatta, July, 2003. Sullivan, D. (2005, April). Kids’ Search Engines. Search Engine News. Retrieved July 7, 2007, from http://searchenginewatch.com/showPage.html?page=2156191 Enid Irwin is a lecturer on Information Retrieval and is the Project Manager of the KidsClick.org redesign project for San Jose State School of Library and Information Science in California. She is also an assistant reference librarian at California State University, Monterey Bay. She has worked with electronic databases and web interface design for ten years in both commercial and academic libraries. SLA enhances her professional life because membership helps keep her current with her interests in information literacy and learning games. In her free time, Enid volunteers with the City of Monterey in various projects in the marina such as invasive seaweed research and sea lion tracking. enid@slis.sjsu.edu Books in Brief - By Jacqueline Snider Book Review & Liaison Editor Performance-Based Learning: Aligning Experiential Tasks and Assessment to Increase Learning Berman, S. (2008). Performance-Based Learning: Aligning Experiential Tasks and Assessment to Increase Learning. 2nd edition. Thousand Oaks: CA: Corwin Press. 168 p. ISBN: 978-1-4129-53092 (cloth) $66.95; 978-1-4129-53108 (pbk.) $30.95. The second edition of Performance Based Learning for the Multiple Intelligences Classroom combines theory and practical applications in one text. The first chapter covers the groundwork behind performance based learning, and describes the importance of authentic and cooperative learning by tying practical experiences, practice, and performance to knowledge acquisition. Berman provides concrete examples, lessons, tasks, timelines, rubrics, and most importantly, reflections. Each lesson includes the grade level, and targets novice, advanced beginner, competent user, proficient performer, and expert levels. Topics range from constitution day, science projects, telling stories, and music. This book is recommended for all academic institutions with education programs, and for staff development collections. http://searchenginewatch.com/showPage.html?page=2156191 mailto:enid@slis.sjsu.edu Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 59 Table 1 Website Evaluation Surveys Completed by Session and Evaluation Group Spring 2007 (Apr-May) Summer 2007 (June-Aug) Fall 2007 (Sept-Dec) Evaluation Group Surveys completed Number of evaluators Surveys completed Number of evaluators Surveys completed Number of evaluators KCDG 366 8 299 2 50 1 LIBR 298 1145 9 675 11 LIBR 228 250 25 LIBR 202 590 20 225 9 Volunteers 100 2 744 17 614 16 TOTALS 1056 30 2188 28 1814 62 Total websites evaluated 5058 Number of "broken" websites deleted without evaluation 174 Number of websites remaining 244 5476 Duplicate sites <54> Number of websites from March 2007 5424 Education Libraries: Childrens Resources, Volume 31, No. 3, Spring 2008 60 Figure 1 Website Evaluation Survey Process Flow Chart