Evidence Based Library and Information Practice Evidence Based Library and Information Practice 2010, 5.3 20 Evidence Based Library and Information Practice Article Learning in Simulations: Examining the Effectiveness of Information Literacy Instruction Using Middle School Students’ Portfolio Products Terrance S. Newell Assistant Professor University of Wisconsin-Milwaukee School of Information Studies Milwaukee, Wisconsin, United States of America Email: newellt@uwm.edu Received: 30 Mar. 2010 Accepted: 13 Aug. 2010 2010 Newell. This is an Open Access article distributed under the terms of the Creative Commons-Attribution- Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one. Abstract Objective –This study compared the effectiveness of simulation-based and didactic instructional approaches in improving students’ understanding of information literacy (IL) concepts and practices. Methods – The instructional approaches were implemented with two groups of middle school students (i.e., seventh and eighth grades) over a 4-week period. During the implementation period, all students were required to maintain a portfolio of their work. The portfolios were designed to capture students’ actions as they engaged in a common set of information-based problems. The contents of the portfolios were analyzed to examine the research questions that guided the study. Contingency tables demonstrated observed patterns of difference from week 1 to week 4. Chi-square analysis helped to determine whether a significant relationship existed between instructional approach and shifts in IL proficiency levels at the .05 level. Results – There was a significant relationship between the simulation-based approach and increases in students’ ability to 1) recognize the need for information, 2) formulate specific questions that would help in finding needed information, 3) identify a range of information sources for meeting needs, 4) explain successful strategies for accessing needed information, 5) judge the accuracy, relevance and completeness of sources and 6) analyze information from a variety of sources to determine its applicability to a specific problem. Four major distinctions are believed to have caused the students mailto:newellt@uwm.edu Evidence Based Library and Information Practice 2010, 5.3 21 within the simulated instructional environment to experience more proficiency level shifts: situated practice, authenticity, community of practice and an expanded landscape of resources. Conclusion – The results of this study suggest that simulation-based instructional approaches have the potential to augment IL learning. The technology-based approaches may provide powerful learning environments (virtual worlds) that allow students to engage in the activities and practice of information specialists, instead of simply learning the facts associated with the discipline. Introduction Educational scholars are only beginning to understand the benefits and pitfalls of learning via simulated learning encounters. Scholars are currently conducting research, building prototypes, and developing theoretical frameworks to assist educators in understanding how students learn within these technology-based environments and how effective the environments are within particular disciplinary contexts (e.g., history). The term digital simulation refers to a computer-based, problem solving situation with multiple interacting variables and a landscape of information resources (Gredler, 2004). There are various types of simulations (e.g., social process, system, diagnostic, data management and laboratory). However, all simulations have four basic characteristics: 1) learners are placed within an environment that mirrors a complex real-world situation; 2) roles are assigned to learners; 3) an expanded information landscape is represented; and 4) feedback mechanisms are employed. The research literature indicates a potential in the role that simulations could play in mediating learning, and as a vital part of the K-12 instructional mission, U.S. school libraries are beginning to explore the potential role of digital simulations. It is imperative that school librarians gain evidence based insights into the effectiveness of simulation-based approaches in teaching information-oriented concepts and practices. Literature Review Over the last three decades, the use of digital simulations has increased within the general educational landscape. Gredler (2004) illuminated five dominant types of simulations tied to educational environments: social process, diagnostic, data management, system, and laboratory simulations. A social process simulation is a complex, discipline- specific situation that provides learners with a finite field of contingencies for the occurrences inscribed within the simulation. A diagnostic simulation presents learners with realistic problem scenarios and requires them to make optimal, information-based decisions at sequential points. Data-management simulations are collaborative exercises that require learners to manage financial variables under varying conditions. Laboratory simulations place learners within discipline- specific, virtual worlds, and learners then fill the roles of researchers within the simulations. System simulations also place learners within discipline-specific, virtual worlds. In system simulations learners play the role of problem- solvers. Theoretical Impetus for the use of Digital Simulations The use of simulations within the educational landscape is directly tied to ecological (e.g., Gibson, 1986; Shaw & Bransford, 1977), constructivist (e.g., Dewey, 1933; Inhelder & Piaget, 1958; Kelly, 1963; Bruner, 1977) and situative perspectives of learning (e.g., Brown et al., 1989; Kirshner & Whitson, 1998). “Ecological perspectives” of learning illuminate the role that contextually sensitive activity (e.g., interacting with visual, auditory, and tactile data in the environment) plays during the learning process (e.g., Gibson, 1986). “Constructivist perspectives” of Evidence Based Library and Information Practice 2010, 5.3 22 learning advance the notion that learning is an active process of personal meaning-making, a concept grounded in experience (Dewey, 1916). “Situative perspectives view learning as a process of 1) active meaning-making within a student’s mind and 2) enculturation into the disciplinary practices of literate people within a particular domain of study (Cobb, 1996). These three perspectives situate learning (or cognitive functioning) within environmental and experiential frameworks, which create a theoretical grounding for the use of digital simulations. Virtual Reality as a Dominant Technology for Simulation Design Simulations can be brought to life using a variety of technologies. Schools serving students from kindergarten through twelfth grade are primarily exploring the use of desktop virtual reality (VR). Desktop VR is a relatively inexpensive computer-based technology that allows learners to examine three-dimensional, highly interactive, multi- sensory, simulated environments using a computer screen and a keyboard (McLellan, 2004; Focier, 1999). Educational settings are currently using VR in various ways: 1) feedback on student performance, 2) training systems, 3) experiential learning spaces, 4) virtual field trips and 5) rehabilitation spaces (McLellan, 2004). A number of VR environments have been employed within educational settings such as the Virtual European Schools project (Bouras, Fotakis, Kapoulas, Koubek, Mayher & Rehatscheck, 1999), Intelligent Distributed Virtual Training Environment (Bouras, Triantafillou & Tsiatsos, 2001), Educational Virtual Environments Project (Bouras, Giannaka & Tsiatsos, 2003), Quest Atlantis (Barab, Thomas, Dodge, Carteaux & Tuzun, 2005), Active Worlds Educational Universe (Corbit, 2002), Second Life (Andreas, Tsiatsos, Terzidou & Pomportsis, 2010), and C-VISIONS (San Chee & Meng Hooi, 2002). Effectiveness of Simulations in Educational Environments Empirical research studies show that simulation approaches are just as effective as traditional teaching approaches in teaching basic math and reading comprehension skills (Rosas et al., 2003; Laffey et al., 2003), mathematical problem solving skills (Van Eck & Dempsey, 2002), basic logic (Costabile et al., 2003), geographical content knowledge (Wiebe & Martin, 1994; Virvou, Katsionis & Manos, 2005) and vocabulary skills (Malouf, 1988). The simulation and learning literature has also illuminated particular advantages associated with these learning environments, such as the development of critical thinking skills (Rieber, 1996), problem solving abilities (Rieber, 1996; Gorriz & Mediana, 2000; Prensky, 2001), visual/spatial skills (Greenfield et al., 1994), cognitive strategies (Gredler, 1996), discovery learning (Prensky, 2001), interactivity (Rosas et al., 2003; Price, 1990) and motor skill growth. The research literature indicates the potential role that simulations may play in mediating learning. This article examines the persuasive, academic position that constructs digital simulations as dynamic educational spaces that could facilitate learning (Shaffer, Squire, Halverson & Gee, 2005). It also contributes to the current empirical record on learning via simulations within both the general educational landscape and the school library context by providing evidence into the effectiveness of these approaches in improving students’ understanding of IL concepts and practices. Aims This study compared the effectiveness of simulation-based and didactic instructional approaches in improving students’ IL proficiency levels by examining the following questions: 1. After a 4 week course, does a significant relationship exist between instructional approach (simulation and didactic) and the number of students who were able to achieve a higher proficiency level in: Evidence Based Library and Information Practice 2010, 5.3 23 a. recognizing the need for information b. formulating questions c. identifying a variety of potential information sources d. developing and using location strategies e. determining the accuracy, relevance, and comprehensiveness of information f. selecting appropriate information 2. What are the patterns of difference between the instructional approaches and number of shifts across six information-oriented variables (i.e., needs recognition, question formulation, identifying information sources, information location, information evaluation, and resource selection)? Methods Research Site This study was conducted in a middle school (i.e., a school serving grades 7 and 8) in a U.S. Midwestern city of approximately 200,000 residents. The school’s educational philosophy was based on two guiding principles: curricular integration and the use of technology to support learning. The school had three computer labs, a library computer center, and fully networked classroom computers. The school’s student-to-computer ratio was 4 to 1, with a total enrollment of 240 students. Sample A total of 54 students recruited from the school’s four computer literacy classes participated in this study. The didactic group was composed of 27 students (12 seventh grade and 15 eighth grade students) and 27 students (11 seventh grade and 16 eighth grade students) in the simulation group. Controlling for Confounds There are, potentially, many confounding variables (e.g., teacher qualities and educational backgrounds) in a study attempting to compare instructional approaches. Two primary strategies were used in an attempt to control for confounding variables and to minimize their role in observed effects: a) using random assignments, and b) keeping educators and instructional tasks constant for both instructional approaches. Students within the research site were unsystematically assigned— except for grade level—to the classes participating in this study. There were various learning styles within each class -- computer skill levels, genders, races, and intelligences. The four computer literacy classes were given unique numbers and a random numbers table was used to assign classes to instructional approaches. Random assignment to learning models increased the probability that differences were due to the intended variables. In an attempt to keep some factors the same for every group, only one teacher and one school librarian were used for the study. Therefore, any observed differences could not be attributed to differences in educator experience, personality, or educational background. Furthermore, although the learning approaches differed for the two groups, each used the same information- oriented learning problems, tasks, and problem solving strategies. Students engaged a common set of problems and thinking strategies for information-based problem solving. Development of Learning Tasks Both instructional approaches used the same information-oriented learning problems/tasks. The initial development of all information- oriented, learning tasks was performed by the researcher (Figure 1). A variety of information literacy and information problem solving textbooks were analyzed and used in the construction of tasks. However, the school librarian and the technology teacher at the Evidence Based Library and Information Practice 2010, 5.3 24 research site reviewed and added tasks prior to the beginning of the study. Approaches to Instruction Simulation-Based Approach A three-dimensional (3D) simulation was used as one of the instructional approaches (Figure 2). The 3D interface of the simulation was developed using authorware similar to the design tools utilized in Second Life (Secondlife.com), the online virtual world. The interface was highly interactive and allowed students to navigate the 3D environment from every perspective using the computer screen, mouse, and keyboard. The simulated environment consisted of a middle school library (Figure 3), high school library (Figure 4), informal environments (see Figure 5) and electronic environments (Figure 6). The various parts of the simulated enwere designed to represent a small town. Students could virtually walk from the middle school library to the high school library within the 3D simulation. The 3D simulation technology also enabled the construction of virtual information objects, artifacts, and resources (e.g., books, computers, televisions and people). Within the simulated environment, students could improve freely, communicate using chat, and use a variety of information resources. Within the simulation, students learned information-oriented practices through simulated experiences, interactions, and communities of practices. Moreover, students learned by actively participating as information literacy (IL) apprentices within a computer-generated community of information professionals. As IL apprentices, students assisted the simulated information professionals in meeting the needs of other computer-generated characters within the simulation (i.e., people within the small simulated town) using novice information- oriented practices (Wenger, 1998). Through participation students gradually developed information-seeking skills. Moreover, student learning was guided using three techniques: scaffolds, communities of practice, and cognitive process frameworks. Instructional scaffolding aided students as they learned through active participation. , These learning supports (scaffolds) existed in the form of tutorials (Gee, 2007), information on-demand (Gee, 2007), just-in-time pop-ups (Gee, 2007), modeling (Collins et al., 1989), exploration (Collins et al., 1989), and questioning (Gallimore & Tharp, 1990). A community of practice (CoP) also aided the development of practices. Information professionals—a group that is bounded by best practice approaches to information problem solving—constituted the community of practice. Within the 3D simulation, students could work with members of this CoP (e.g., computer-generated librarians, real librarians, and other student apprentices) to solve problems. Third, as students engaged tasks, they used a cognitive process framework to cognitively structure the stages of information problem solving and the thinking strategies related to the different stages. Fig. 1. Sample portfolio task. Thanks for your help! I am Jaime. I am thinking about buying a vehicle like this one, but I don’t know how to ask the right questions or how to find the answers to those questions. I think that I need information related to the price, warranty and safety. I may also need information related to consumer and expert reports. Could you help me answer these questions and find comprehensive visual information on the vehicle? Could you also develop and answer four additional questions for me? Evidence Based Library and Information Practice 2010, 5.3 25 Fig. 2. 3D simulation. Fig. 6. Simulated accessing an electronic resource. Didactic Approach The second instructional approach was primarily a didactic format of teaching incorporating problem-based activities for students to solve. The didactic teaching context consisted of a real-world computer lab, middle school library, and the information objects within them (e.g., library catalogs, books, and computers). The middle school library and computer lab were connected, and the computer lab housed enough computers for each student to have a personal computer during the learning periods. During instruction, the technology teacher and the school librarian supported student learning using lectures and problem-based activities. Students engaged in the problem-based activities in pairs, and used a generic, process- oriented framework to guide them through the stages of information problem solving and thinking strategies related to various stages. Fig. 3. Simulated middle school library. Fig. 4. Simulated high school library. Fig. 5. Simulated informal information environment. Evidence Based Library and Information Practice 2010, 5.3 26 The framework was the same as the one used in the simulation approach. Assessment of Student Portfolios Although the learning approaches differed for the two groups, both learning environments used the same information-oriented learning tasks. Therefore, portfolios could be used to capture students’ activities as they engaged this common set of problems. The portfolios were not simply folders filled with work samples after the 4-week implementation period. Instead, the portfolios facilitated the collection of students’ information problem solving activities across seven areas (i.e., task identification, search strategy initiation, information access, information evaluation, information use, information communication, and problem solving evaluation). The portfolio document was printed for students before the instructional periods began. The documents had 10 sections, representing the 10 required tasks for the 4-week period. As students addressed each of the tasks, they were required to record their information problem solving activities in designated parts of the portfolio. With the exceptions of the first, second, and last tasks, the tasks could be completed in any order. The second and last tasks were predetermined for content analysis purposes. Data Analysis A content analysis performed on week 1 and week 4 portfolio products examined the seven research questions guiding the study. Content analysis is an empirical procedure for quantifying a field of representation, and the procedures of content analysis provided the researcher with a framework for classifying and quantifying the student portfolios as content. The empirical procedures also allowed the researcher to make statements about aspects of representation (e.g., shifts in information evaluation abilities) that are based on observable evidence. Variables and Values A content analysis classifies selected texts along specified dimensions known as variables and values. Variables refer to dimensions of representation that are of the same type. Values are mutually exclusive and there are exhaustive classification options for variables. For example, a school library (variable) could be classified as: elementary school (value 1), middle school (value 2), high school (value 3), alternative school (value 4), charter school (value 5), or private school (value 6). For the purposes of this study, the identification of variables and values was based on the Standards for Information Literacy Learning (AASL & AECT, 1998). Standards for Information Literacy Learning advances three IL standards (i.e., access, evaluation, and use standards), and thirteen demonstration indicators of information literacy across the standards (AASL & AECT, 1998). The K-12 standards also provide three levels of proficiency (i.e., basic, proficient, and exemplary) for each demonstration indicator. Using the Standards for Information Literacy Learning (AASL & AECT, 1998), the variables were constructed using the content area indicators, and values were defined using the levels of proficiency tied to each indicator. Variables and values were placed in a coding protocol to help guide the coders in applying consistent criteria as they assessed the portfolios. The variables and values are listed in Appendix A. Comparison Procedures and Reliability The researcher and another university level professor served as coders for this content analysis following a typical content analysis procedure (Leedy & Ormrod, 2001). First, the coders practiced using the coding protocol on five student work samples that were similar to—but not part of—the official field of bound-texts to refine clues, signifiers and definitions of variables/values. Second, inter- rater agreement of the coded work samples was assessed. Inter-rater agreement was initially 92 percent; however, a discussion of disagreements and further training resulted in Evidence Based Library and Information Practice 2010, 5.3 27 a final inter-rater agreement of 100 percent. Third, the coders used the coding protocol to consistently evaluate student portfolios along the identified dimensions (variable and values). Each coder evaluated half of the didactic portfolios and half of the simulation portfolios. Each portfolio was coded separately, and coders concealed student names during coding to minimize bias. Fourth, after coding all portfolios, the coders produced a numerical set of results. Contingency tables presented observed patterns of difference from week 1 to week 4. Chi-square analysis determined whether a significant relationship existed between instructional approach and shifts in IL proficiency levels at the .05 level. Results The results of the content analysis follow. First, are the results from week 1. Second are the results from the comparison of week 1 portfolios to week 4 portfolios according to the six variables that guided this study. The 27 didactic student portfolios from week 1 were compared to the same 27 portfolios from week 4 (the end of the course) to identify whether proficiency had changed across the six variables that guided this study. The 27 simulation portfolios from week 1 were also reexamined at the end of the course to evaluate changes in proficiency across the six variables. Week 1 The content analysis clearly showed six major findings during the first week of implementation that demonstrated effectiveness (Table 1). First, 33% of the didactic students and 37% of the simulation students did not consider whether additional information—beyond their own knowledge— was needed during the problem solving process. Second, 93% of the didactic students and 100% of the simulation students could form only one broad question (or none at all) to guide them through the problem solving process. Third, 74% of the didactic students and 93% of the simulation students could generate only two sources of information that could potentially be used to answer their question(s). Fourth, 93% of the didactic students and 100% of the simulation students could either 1) not list any ideas for identifying and finding needed information, or 2) list only vague ideas about their potential strategies for locating information. Fifth, 96% of the didactic students and 100% of the simulation students sparingly considered the concepts of accuracy, relevance, and comprehensiveness—if at all. Sixth, 86% of the didactic students and 96% of the simulation students either did not identify information to address the problem or used only one or two information sources to address the problem. Evidence Based Library and Information Practice 2010, 5.3 28 Table 1 Week 1 Didactic (n=27) and Simulation (n=27) Groups Didactic Group: Number of Students Across Proficiency Levels Not represented Basic Proficient Exemplary Variables Variable 1. Recognizing need for information 9 (33%) 16 (59%) 2 (7%) Variable 2. Formulating questions based on information needs 4 (15%) 21 (78%) 2 (7%) Variable 3. Identifying a variety of potential sources of information 5 (18%) 15 (56%) 7 (26%) Variable 4. Developing and using successful strategies for locating information 10 (37%) 15 (56%) 2 (7%) Variable 5. Determining the accuracy, relevance, and comprehensiveness of information 21 (78%) 5 (18%) 1 (4%) Variable 6. Selecting information appropriate to the problem or the question at hand 13 (49%) 10 (37%) 4 (15%) Simulation Group: Number of Students Across Proficiency Levels Not represented Basic Proficient Exemplary Variables Variable 1. Recognizing need for information 10 (37%) 17 (63%) Variable 2. Formulating questions based on information needs 4 (15%) 23 (85%) Variable 3. Identifying a variety of potential sources of information 8 (30%) 17 (63%) 2 (7%) Variable 4. Developing and using successful strategies for locating information 15 (56%) 12 (45%) Variable 5. Determining the accuracy, relevance, and comprehensiveness of information 25 (93%) 2 (7%) Variable 6. Selecting information appropriate to the problem or the question at hand 17 (63%) 9 (33%) 1 (4%) Recognizing the Need for Information In the didactic group, the proficiency levels of 6 students experienced a positive change, whereas, the proficiency levels of 19 students experienced a positive change in the simulation group (Table 2). A chi-square test was performed to determine if a significant relationship existed between instructional approach and the number of students who were able to recognize the need for information at a higher proficiency level. This indicated a significant relationship for the simulation approach in improving proficiency levels at recognizing a need for information, [X2 (2, N=54) = 12.58, p < .05], but not the didactic approach (Table 2). Evidence Based Library and Information Practice 2010, 5.3 29 Formulating Questions Based on Information Needs In the didactic group, the proficiency levels of 2 students experienced a positive change and the proficiency levels of 26 students experienced a positive change in the simulation group, (Table 3). A chi-square test indicated that there was a significant relationship between the simulation approach and the ability to improve students’ proficiency levels in formulating questions [X2 (2, N=54) = 42.725, p < .05] but not the didactic approach (Table 3). Identifying a Variety of Potential Sources of Information Proficiency levels of 6 students experienced a positive change in the didactic group and 26 students experienced a positive change in the simulation group (Table 4). A chi-square test indicated this was a significant relationship for the simulation approach [X2 (2, N=54) = 30.682, p < .05] but not the didactic approach and the ability to improve proficiency levels in identifying sources of information (Table 4). Developing and Using Successful Strategies for Locating Information In the didactic group, the proficiency levels of 4 students experienced a positive change whilst the proficiency levels of 22 students experienced a positive change in the simulation group (Table 5). A chi-square test indicated that this was a significant change in the simulation group [X2 (2, N=54) = 24.033, p < .05] (Table 5). Table 2 Recognizing the Need for Information: Proficiency Level Improvements Between Weeks 1 and 4 Variable 1: Recognizing the need for information Didactic Group 6 didactic students improved from a basic to a proficient level 21 didactic students retained their week 1 proficiency levels Simulation Group 8 simulation students improved from a basic to a proficient level 11 simulation students improved from proficient to exemplary 8 simulation students retained their week 1 proficiency levels Table 3 Formulating: Proficiency Level Improvements Between Weeks 1 and 4 Variable 2: Formulating questions based on information needs Didactic Group 2 didactic students improved from a “not represented” proficiency to a basic level 1 didactic student regressed from a proficient level to a basic level 24 didactic students retained their week 1 proficiency levels Simulation Group 3 simulation students improved from “not represented” to a basic level 8 simulation students improved from a basic to a proficient level 15 simulation students improved from a basic to an exemplary level 1 student retained her week 1 proficiency level Evidence Based Library and Information Practice 2010, 5.3 30 Table 5 Locating Information: Proficiency Level Improvements Between Weeks 1 and 4 Variable 4: Developing and using successful strategies for locating information Didactic Group 4 didactic students improved from a “not represented” proficiency to a basic level 1 didactic student regressed from a proficient to a basic level 22 didactic students retained their week 1 proficiency levels Simulation Group 1 simulation student improved from a “not represented” proficiency to a basic level 11 simulation students improved from “not represented” to a proficient level 1 simulation student improved from a basic to a proficient level 9 simulation students improved from a basic to an exemplary level 5 simulation students retained their week 1 proficiency levels Table 6 Accuracy, Relevance and Comprehensiveness: Proficiency Level Improvements Between Weeks 1 and 4 Variable 5: Determining the accuracy, relevance and comprehensiveness of information Didactic Group 6 didactic students improved from a “not represented” proficiency to a basic level 1 didactic student regressed from a proficient to a basic level 20 didactic students retained their week 1 proficiency levels Simulation Group 12 simulation students improved from a “not represented” proficiency to a basic level 5 simulation students improved from “not represented” to a proficient level 10 simulation students retained their week 1 proficiency levels Determining the Accuracy, Relevance and Comprehensiveness of Information The proficiency levels of 6 students in the didactic group and 17 students in the simulation group experienced a positive change (Table 6). A chi-square test indicated that this was a significant improvement in the simulation group [X2 (2, N=54) = 9.164, p < .05] for determining the accuracy, relevance and comprehensiveness of information (Table 6). Selecting Information Appropriate for the Problem The proficiency levels of 8 students in the didactic group and 20 students in the simulation group experienced a positive change (Table 7). A chi-square test determined a significant relationship for the simulation approach and the number of students who were able to select appropriate information at a higher proficiency level [X2 (2, N=54) = 10.681, p < .05]. There was no significant relationship between the didactic approach and the ability to improve proficiency levels (Table 7). Evidence Based Library and Information Practice 2010, 5.3 31 Table 7 Selecting Information: Proficiency Level Improvements Between Weeks 1 and 4 Variable 6: Selecting information appropriate to the problem or question at hand Didactic Group 2 didactic students improved from a “not represented” proficiency to a basic level 6 didactic students improved from a basic to a proficient level 19 didactic students retained their week 1 proficiency levels Simulation Group 4 simulation students improved from “not represented to a proficient level 13 simulation students improved from “not represented to an exemplary level 5 simulation students improved from a basic to an exemplary level 7 simulation students retained their week 1 proficiency levels Discussion A comparison of students’ proficiency between weeks 1 and 4 indicates that the simulation was more likely to improve students’ ability levels across all 6 variables. This means that there was a significant relationship between the simulation approach and increases in students’ abilities to:  recognize the need for information,  formulate specific questions to help find needed information,  identify a range of information sources,  explain successful strategies for accessing needed information,  judge the accuracy, relevance, and completeness of sources, and  analyze information from a variety of sources to determine its applicability to the specific problem. These findings suggests that while there are surface level theoretical homologies between different types of information literacy and instructional environments that seem to assign similar roles to information educators, teachers, and students during the course of learning, there are profound distinctions. Moreover, both instructional approaches featured the same content, information-based problems, and educators. However, the ways in which the content was experienced by the students suggested significant distinctions between the approaches. Those distinctions are believed to have caused the students in the simulated instructional environment to experience more proficiency level shifts. The distinctions can be organized around 4 dimensions:  situated practice  authenticity  community of practice  an expanded landscape of resources. Situated Practice Both the didactic and simulation environments shared the same information- oriented content and practices; however, the delivery of content within the didactic environment was more abstract and decontextualized. Moreover, units of information content were primarily transmitted from educators to students via lectures. Although the educators incorporated learning activities that required students to engage in complex problems, the problems did not become the context for teaching and learning. In other words, teaching and learning did not occur primarily within the problem contexts. Rather, students went through the traditional process of classroom learning and then demonstrated abilities during an activity session. Within the simulation environment, students learned practices through the process of solving complex problems and participating in a virtual community of practice (Shaffer, Squire, Halverson & Gee, 2005). In other words, Evidence Based Library and Information Practice 2010, 5.3 32 students actively participated as members of a disciplinary community (information specialists) while addressing community- specific tasks/problems. As students worked toward the completion of tasks and problems, they were building an epistemic frame that was specific to the community. In addition, students worked toward the completion of tasks with distinct ways of acting, interacting, valuing, feeling, and knowing that constituted an epistemic frame. The simulation approach situated knowledge development within legitimate participation and practice. Authenticity Learning environments could be considered authentic to educators or students (Brown et al., 1989). Both the didactic and simulation environments were considered authentic by the educators because the content, practices, problems, and resources were reflective of real world, disciplinary contexts and activities. However, for students, authenticity refers to the nexus between content or practices and the degree to which they value those practices during activities that require their use (Barab & Duffy, 2000). The student portfolios suggest that the simulation learners perceived a stronger relationship between information -oriented practices and their value during use whereas portfolios from the didactic learners suggested that they did not value the cognitive process framework (a set of information problem solving practices) that they were taught. Instead, the didactic portfolios show that students eliminated many practices (e.g., information evaluation, selection, and organization) from the information problem solving equation. They began to focus more upon quick solutions generated outside of the problem solving framework and its practices, reflected in portfolio products that focused less on the process of information problem solving and more on the creation of products. In other words, they became more product- oriented and less process-oriented. Portfolios from the simulation learners suggested that they valued the cognitive process framework (a set of information problem solving practices) that had been taught. Instead of cutting practices, simulation students began to augment the initial 7-stage cognitive framework with an additional chain of practices: 1) defining the problem through a multimodal and keyword generating process; 2) questioning the current information environment for needed information; 3) searching for different versions of information skills; and 4) thinking about resources that were not in their current environment. Within the immersive environment, the improvement from conscious information practices to automatic (almost unconscious) operations was disrupted because of perceived limitations in the information problem solving process and its chain of practices. Simulation students not only valued the content during activities that required their use, but they also attempted to improve the practices. Expanded Landscape of Resources The didactic portfolios did not reflect a strong acknowledgment or understanding of different types of information and multiple information environments. Instead, they were Internet focused, and this was possibly the result of the learning context. Moreover, the classroom and its computers served as the primary context for information problem solving activities within the didactic environment. As didactic students encountered problem based activities that required them to deconstruct both the problem and its surrounding context and to solve the problem using information tools, they also encountered a simplification of real- world information complexities. No matter what scenario, the classroom and computer represented the primary informational landscape for problem solving. Simulation students, on the other hand, contextualized the use of processes within a VR space that recreated the information complexities of the real world. The simulation reinforced the idea that school libraries are preparing students to navigate, evaluate, and use diverse print and electronic environments, some directly located Evidence Based Library and Information Practice 2010, 5.3 33 within or linked to institutions, and others independent from time and place constraints. Therefore, simulation students were afforded the opportunity to employ processes, practices, and knowledge across information environments and artifacts. The diversity within the simulation manifested itself in student products that acknowledged different types of formats and information environments. Community of Practice The didactic approach created a teacher- centered or lecture-based environment. The simulation environment, on the other hand, created a simulated community of practice (CoP). A CoP is a disciplinary community of expert practitioners that are bound by: a common task; a shared set of standards; and a common understanding of cognitive and social practices. A CoP shares knowledge and practices with students during an instructional process that requires inexperienced members of a discipline (e.g., students) to learn under the guidance of specialists within the discipline (e.g., experienced information specialists) during their legitimate participation in authentic, disciplinary activity. The same content and processes were delivered by both the CoP and the lecture- based environment. However, with a few exceptions, those processes were represented only within the simulation portfolios. Although this was a simulated CoP that could not replicate all real life nuances, the participatory storyline that inscribed authentic, purposeful interactions between students and experts across multiple tasks, artifacts, and environments made the simulated CoP more authentic and may have increased the representation of content and processes in the portfolios of the simulation group. Limitations There are benefits and pitfalls for all types of assessment, including portfolio assessment. Portfolio assessment allows for the purposeful collection of students’ work, and it illuminates their demonstrated efforts and abilities. This form of assessment aligns well with the major strength of learning via simulation —that is, the development of skills through practice and engagement. However it may not have illuminated the major strengths of didactic instruction, the recall of disciplinary content. In other words, didactic students could have developed an exemplary level of understanding of content across the six information literacy domains (i.e., needs identification, question formulation, source identification, information seeking, information evaluation, and source selection) and yet failed in the actual application of the content. On the other hand, the simulation group could have developed an exemplary level of application and failed to answer complex questions about content (e.g., “What is the difference between external, internal, and negotiated needs?”). Fixed assessment designs such as multiple choice and short answer questions are more appropriate for the assessment of content knowledge. Conclusion The results of this study suggest that simulation approaches have the potential to augment information literacy teaching and learning practices. These technology-based approaches may provide powerful learning environments (virtual worlds) that allow students to engage in the activities and practices of information specialists, instead of simply learning the facts associated with the discipline. Digital simulations also allow for the creation of simulated environments that allow students to engage discipline-specific, literacy activities requiring distinctive ways of using language, objects, processes, information, and knowledge. Within such simulated learning environments students can inhabit roles such as engineers or information specialists, that may be inaccessible within the traditional teaching and learning landscape. These simulated learning environments can also use their distinct features (e.g., situated practice, authenticity, community of practice, and expanded resource landscapes) to aid in creating a nexus between the understanding of Evidence Based Library and Information Practice 2010, 5.3 34 abstract ideas within roles and the use of concrete practices to produce meaning within roles (Shaffer et. al., 2005). References American Association of School Librarians & Association for Educational Communications and technology. (1998). Information power: Building partnerships for learning. Chicago:, IL American Library Association. Andreas, K., Tsiatsost, T., Terzidou, T., & Pomportsis, A. (Sept. 2010). Fostering collaborative learning in Second Life: Metaphors and affordances. Computers & Education, 55(2), 603-615. Barab, S., & Duffy, T. (2000). From practice fields to communities of practice. In D. Jonassen & S. Land (Eds.) Theoretical foundations of learning environments (pp. 25-56). Mahwah, NJ: Erlbaum. Barab, S., Thomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53(1), 86-107. Bouras, C., Fotakis, D., Kapoulas, V., Koubek, A., Mayher, H., & Rehatscheck, H. (1999). Virtual European School-VES. In Proceedings of the IEEE multimedia systems ’99: special session on European projects, 1055–1057. Bouras, C., Triantafillou, V., & Tsiatsos, T., (2001). Aspects of collaborative learning environment using distributed virtual environments. In Proceedings of the world conference on educational multimedia, hyper-media & telecommunications, 173–178. Bouras, C., Giannaka, E., & Tsiatsos, T., (2003). Virtual collaboration spaces: the EVE community. In Proceedings of the 2003 symposium on applications and the Internet, 48–55. Brown, J. S., Collins, A., & Duguid, P. (Jan. Feb. 1989). Situated cognition and the culture of learning. Educational Research, 18(1), 32-42. Bruner, J. (1977). The process of education. Cambridge, MA: Harvard University Press. Chen, S. 1993. A study of high school students’ online catalog searching behavior. School Library Media Quarterly 22: 33- 40. Cobb, P. (1996). Where is the mind? A coordination of sociocultural and cognitive constructivist perspectives. In C. T. Fosnot (Ed.), Constructivism: Theory, perspectives, and practice (pp. 34-54). New York, NY: Teachers College Press. Collins, A., Brown, J., & Newman, S. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing and mathematics. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor of Robert Glaser (pp. 223–53). Hillsdale, N.J.: Erlbaum. Corbit, M. (2002). Building virtual worlds for informal science learning in the Active Worlds Educational Universe (AWEDU). Presence 11, 55–67. Costabile, M., De Angeli, A. Roselli, T., Lanzilotti, R. & Plantamura, P. (2003). Evaluating the educational impact of a tutoring hypermedia for children. Information Technology in Childhood Education Annual, 289-308. Dewey, J. (1933). How we think. Lexington, MA: Heath. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. New York, NY: Macmillan. Evidence Based Library and Information Practice 2010, 5.3 35 Dunn, K. (2002). Assessing information literacy skills in the California State University: A progress report. Journal of Academic Librarianship, 30, 26-35. Focier, R. C. (1999). The computer as an educational tool: Productivity and problem solving. Columbus, OH: Merrill. Gallimore, R., & Tharp, R. (1990). Teaching mind in society: teaching, schooling and literate discourse. In L. C. Moll (Ed.), Vygotsky and education: Instructional implications and applications of sociohistorical psychology (pp. 175–205). Cambridge, U.K.: Cambridge. Gee, J. (2007). What video games have to teach us about learning and literacy. New York, NY: Palgrave Macmillan. Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Erlbaum. Goniz M, & Medina C.. (2000). Engaging girls with computers through software games. Communications of the ACM, 43(1), 42-49. Gredler, M. E. (2004). Games and simulations and their relationships to learning. In D H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 571-581). Mahwah, NJ: Erlbaum. Hirsh, S. (1998). Relevance determinations in children’s use of electronic resources: A case study. Proceedings of the 61st ASIS Annual Meeting, 35, 63-72. Inhelder, B., & Piaget, J. (1958). The growth of logical thinking: From childhood to Adolescence: An essay on the construction of formal operational structures. New York, NY: Basic Books. Kelly, G. A. (1963). A theory of personality: The psychology of personal constructs. New York, NY: Norton. Kirshner, D., & Whitson, J. A. (1997). Situated cognition: Social, semiotic, and psychological perspectives. Mahwah, NJ: Erlbaum. Laffey, J., Espinosa, L., Moore, J., & Lodree, A. (2003). Supporting learning and behavior of at-risk young children: Computers in urban education. Journal of Research Technology in Education, 35, 423-440. Loerke, K. (1994). Teaching the library research process in junior high. School Libraries in Canada, 14, 23-26. Malouf, D. (Win.) 1987-1988). The effect of instructional computer games on continuing student motivation.” The Journal of Special Education, 21(4), 27- 38. Mark, B., & Jacobson, T. (1995). Teaching anxious students skills for the electronic library. College Teaching, 43, 28-31. McLellan, H. (2004). Virtual realities. In D H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 461-497). Mahwah, NJ: Erlbaum. Pitts, J. (1995). Mental models of information: The 1993-1994 AASL/Highsmith research award study. School Library Media Quarterly 23, 177-184. Price, R. 1990. Computer-aided instruction: A guide for authors. Pacific Grove, CA: Brooks/Cole. Prensky, M. (2001). Digital game-based learning. New York, NY: McGraw-Hill. Rieber, L. (1996). Seriously considering play: Designing interactive learning Evidence Based Library and Information Practice 2010, 5.3 36 environments based on the blending of microworlds, simulations, and games. Educational Technology Research and Development (44), 43-58. Rosas, R., Nussbaum, M., Cumsille P., Marianov,V., Correa, M., Flores P., Grau V., Lagos F., López, X., López V., Rodriguez P., & Salinas, M. (Jan. 2003). Beyond Nintendo: Design and assessment of educational video games for first- and second-grade students. Computers & Education, 40(1), 71-94. Shaffer, D., Squire, K., Halverson, G., & Gee, J. (2005). Video games and the future of learning. Phi Delta Kappan, 87, 104–11. San Chee, Y., & Meng Hooi, C. (2002). C- VISions: socialized learning through collaborative, virtual, interactive simulations. In Proceedings of conference on computer support for collaborative learning, 687–696. Shaw, R. E., & Bransford, J. (1977). Introduction: Psychological approaches to the problem of knowledge. In R. E. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing (pp. 1-39). Hillsdale, NJ: Erlbaum. Van Eck, R., & Dempsey J. (2002). The effect of competition and contextualized advisement on the transfer of mathematics skills in a computer- based instructional simulation game. Educational Technology Research and Development, 50(3), 23-41. Virvou, M., Katsionis, G., & Manos, K.. (2005). Combining software games with education: Evaluation of its educational effectiveness. Educational Technology & Society, 8(2), 54-65. Wiebe, J., & Martin, N. J. (1994). The impact of a computer-based adventure game on achievement and attitudes in geography. Journal of Computing in Childhood Education, 5(1), 61-71. Evidence Based Library and Information Practice 2010, 5.3 37 Appendix A Variables and Values Variable 1. Recognizing the need for information Value 1.0 Not represented. Value 1.1 Basic—when faced with the information problem, the student did not consider whether additional information (beyond one’s own knowledge) was needed to resolve it. Value 1.2 Proficient—when faced with the information problem, the student recognized the need for additional information as a central feature in the problem solving process. Value 1.3 Exemplary—the student identified the primary problem and a range of connected problems. S/he also assessed whether the problems could be resolved based on one’s own knowledge or whether additional information was required. Variable 2. Formulating questions based on information needs Value 2.0 Not represented Value 2.1 Basic—the student stated one or two broad questions that would help in finding needed information. Value 2.2 Proficient—the student stated both broad questions and sub-questions that would help in finding needed information. Value 2.3 Exemplary—the student used a flexible questioning style that allowed him/her to revise, add, and delete questions within a given problem situation. Variable 3. Identifying a variety of potential sources of information Value 3.0 Not represented. Value 3.1 Basic—the student listed one or two sources of information and generally explained the kind of information found in each. Value 3.2 Proficient—the student brainstormed a range of sources of information that would meet an information need. Value 3.3 Exemplary—the student used a full range of information sources to meet differing information needs. Variable 4. Developing and using successful strategies for locating information Value 4.0 Not represented. Value 4.1 Basic—the student listed basic ideas for how to identify and find needed information. Value 4.2 Proficient—the student articulated and applied a plan to access needed information. Value 4.3 Exemplary—the student formulated and revised plans for accessing information for a range of needs and situations. Variable 5. Determining the accuracy, relevance, and comprehensiveness of information Value 5.0 Not represented. Value 5.1 Basic—the student generally understood the concept of accuracy, relevance, and comprehensiveness, but considered them sparingly. Value 5.2 Proficient—the student compared and contrasted sources related to the primary problem to determine which were more accurate, relevant, and comprehensive. Value 5.3 Exemplary—the student judged the accuracy, relevance, and completeness of information in relation to a range of problems and sub-problems. Evidence Based Library and Information Practice 2010, 5.3 38 Variable 6. Selecting information appropriate to the problem or question at hand Value 6.0 Not represented Value 6.1 Basic—the student identified information (one or two types) applicable to a specific information problem or question Value 6.2 Proficient—the student analysed information from a variety of sources to determine its applicability to the specific information problem. Value 6.3 Exemplary—the student integrated accurate, relevant, and comprehensive information to resolve the information problem.