Australasian Journal of Educational Technology, 2018, 34(6). 17 Reflective renovation: Insights from a collaborative and active learning space project evaluation Bing Mei, Lawrence May University of Auckland With growing knowledge about the effectiveness of collaborative or team-based learning in developing important skill sets among students, the development of collaborative and active learning space (CALS) classrooms has gained increasing momentum in recent years. However, there currently exists a paucity of research evaluating the impact of these CALS projects, especially from the perspective of the staff within the institutions that implement them. In view of this gap, this study, using secondary data, reports an institutional evaluation of a CALS project from the teachers’ perspectives. Both quantitative and qualitative results suggest that overall, the CALS project was positively viewed by the teachers. Nonetheless, challenges were also revealed such as classroom settings, digital infrastructure, and technical support. The findings suggest that instead of viewing digital technology as a panacea, the implementation of a CALS project should be regarded as a systematic project, which involves stakeholders across an educational institution, including administrative staff, teaching staff, support staff, and students. Introduction With the rapid proliferation of information and communication technology (ICT), the landscape of traditional classrooms has changed remarkably (Fisher, 2016). In place of teacher-centred approaches, student-centred learning activities (e.g., collaborative and group projects) are, instead, increasingly the primary pedagogical approach taken in physical classrooms (Staley & Freeman, 2017). Amid this change, there emerges a trend for collaborative and active learning space classrooms (CALS) that are technology- enhanced, flexible, and adaptive to a wide range of teaching and learning activities and cater to the students’ need to collaborate in groups (Benade, 2017). Some exemplary CALS examples include the Student- Centred Activities for Large Enrollment Undergraduate Programs (SCALE-UP) project at North Carolina State University (Beichner, Saul, Allain, Deardorff, & Abbott, 2000), Active Learning Classrooms (ALC) at the University of Minnesota, (The ALC Pilot Evaluation Team, 2007), the Technology Enabled Active Learning (TEAL) project at The Massachusetts Institute of Technology (Dori et al., 2003), and the Hasso Plattner Institute of Design at Stanford University (commonly known as the d. school) (Doorley & Witthoft, 2012). Prior research (e.g., Brooks, 2011; Kolb & Kolb, 2005; Tanner, 2000) has revealed a wide range of potential benefits afforded by CALS developments, which include improving students’ engagement level, promoting collaboration among students, facilitating communication between students and teachers, and enhancing learning outcomes. However, despite such reported benefits, and while educational institutions are increasingly pursuing the development of such spaces, it is obvious that there is a need for improved measurement of their efficacy, particularly from the perspective of staff (Fisher, 2016). Given the nature of CALS developments, which are costly, experimental, and subject to fast-changing technological evolution, effective post-occupancy evaluation is a prerequisite to ensure the sustainable success of these projects. To this end, drawing on secondary data, this article reports findings from an evaluation of a pilot CALS project at a university in New Zealand. We hope that this report can generate more discussion on best practices in CALS projects and thus lead to more effective implementation of these projects in the near future. Literature review Given that CALS projects typically involve significant levels of financial commitment, educational institutions have long emphasised the importance of evaluation to achieve intended outcomes. To date, a wide range of measures have been proposed worldwide (ranging from international organisations to local governmental departments and individual institutions) to provide evidence on which to make strategic decisions about CALS projects. The programme on educational building from the Organisation for Economic Co-operation and Development (OECD), (Organisation for Economic Co-operation and Development, 2008), for example, presents a platform for the demonstration of exemplary learning spaces from its member countries against five criteria – (1) flexibility, (2) community needs, (3) sustainability, (4) safety and security, and (5) alternative financing – providing a wealth of concrete examples to support Australasian Journal of Educational Technology, 2018, 34(6). 18 the design, use, and management of learning spaces. In a similar vein, the United Kingdom-based higher education and digital technology expert body Jisc (2016) hosts an online repertoire of case studies and photos showcasing instances of popular designs. It also contains a set of criteria that should be considered when assessing a CALS project. Similarly, the non-profit organisation Educause (2017), in the United States, offers a learning space rating system to provide measurable guidance on the planning and design of new learning spaces and the renovation of existing spaces. The scoring system consists of two parts, with the first section focusing on operational contexts and the second centring around functional aspects of the classroom. Alongside these international efforts, evaluation of CALS projects has started to gain momentum among local governmental departments and individual institutions in recent years. For instance, in 2013, Education Queensland proposed an evaluation scheme, which covers four aspects: (1) environmental, (2) functional, (3) emotional, and (4) pedagogical (Germany, 2014). Similarly, the Ministry of Education (2017) in New Zealand has also released the Innovative Learning Environment Assessment Tool to guide schools’ evaluation of CALS projects. Furthermore, at university level in the Australasian context, the University of Melbourne has set up the Learning Environments Applied Research Network (LEaRN, 2018), aiming to use research evidence to inform the ongoing development of burgeoning CALS projects. In short, our review suggests that educational institutions are becoming increasingly aware of the urgency of evaluating CALS projects to ensure their evolution and that the research on CALS evaluation has started gaining momentum in recent years (Imms, Cleveland, & Fisher, 2016). Nevertheless, due to a wide range of contextual complexities and obstacles, to date no consensus has been reached on an effective evaluation scheme that can cater to different educational institutions and their varied contexts. Moreover, the review also reveals that the current evaluation schemes primarily focus on the technical standards of the equipment deployed and integrated into spaces, or the relationship between investment and learning outcome, which arise mainly from the perspectives of students or designers of these classrooms. Given that the implementation of CALS projects usually involves multiple stakeholders, especially teachers, there is a need to know how they, as the main users of these classrooms, view these projects. About the CALS project In view of the widespread adoption of CALS design in the educational sector, and particularly in New Zealand’s compulsory schooling sector (comprised of primary, intermediate, and secondary schools, teaching students from 5 to 19 years of age), a pilot CALS project was initiated in 2012 at the Faculty of Education (today the Faculty of Education and Social Work) at the University of Auckland in New Zealand. The project aimed to develop new pedagogies, promote collaborative, active project-based learning, and foster ICT literacy in both students and staff at the institution. In order to develop an effective implementation strategy that catered to a wider range of practical needs, four existing traditional classrooms were chosen and converted into CALS classrooms. During the process, relevant literature was drawn on for reporting of prior successful experiences. On the basis of layout of the extant classrooms, efforts were mainly focused on the following six aspects of the spaces: 1. deployment and arrangement of flexible and moveable furniture so that the classroom could change easily from lecturing mode to a more collaborative mode, 2. mounting of multiple screens around the classrooms so that all participants could easily view the presentation materials without turning to a specific direction, 3. provision of in-classroom iPads so that students could use them for online collaborative activities, 4. design of a dual display system so as to ensure classroom screens are capable of providing both unanimous and alternate views, and broadcasting students’ screens, 5. enhancements to the existing wireless network so that students would feel welcome to bring their own devices, and 6. installation of appropriate amplification systems so that clear sound could be evenly distributed. Figure 1 provides an overview of a typical CALS classroom in this project. Australasian Journal of Educational Technology, 2018, 34(6). 19 Figure 1. The layout of a CALS classroom To support teachers’ effective use of the CALS classrooms, measures were also taken to streamline the use of available functions of the spaces. For example, drawing upon prior successful experience, technical support was provided, including one-on-one training, instructional signage, and staffing resource provided in the form of a small team of learning technology assistants who would check the CALS classrooms regularly and respond to in-class technical support needs. As part of the CALS project, teachers were assigned mobile computing devices (laptop computers and Apple iPad tablets) and encouraged to experiment with digital pedagogies. These devices in particular were selected for provision on the basis of both institutional procurement policies as well as one technological component of the CALS spaces’ fit- out (Apple TV devices that allow live wireless screen-mirroring, but only from devices using Apple’s iOS operating system). Altogether, 189 devices were allocated during 2013–2017. With regard to the actual use of CALS classrooms, classroom booking records shows that these four CALS classrooms were consistently booked and used over this 5-year period. Prior to the construction of the CALS classrooms, a number of courses had been identified by the faculty for the consolidation of their multiple, small (approximately 20 student) tutorial classes into single, larger (approximately 60 student) workshop classes. These courses were proactively timetabled into the CALS classrooms for the three semesters of 2013, and teaching staff not associated with these particular courses were invited to make use of these spaces in the remaining periods of availability. This pattern continued in 2014, and in following years the balance of bookings shifted primarily to teaching staff electing to use the spaces rather than staff observing an administrative mandate placed on their course. The faculty uses an institution-wide online platform for booking teaching spaces, which teaching staff typically access themselves for one-off or ad hoc space use, but defer to departmental administrators for regularly timetabled teaching. The learning design team held, until the beginning of 2017, direct responsibility for approving CALS classroom bookings and as a matter of policy required staff new to the spaces to undergo an induction workshop. Beyond such orientation, no requirement was made of teachers to undertake teaching in a particular collaborative or active mode in order to secure their use of a CALS classroom. As of May 2017, a total of 122 lecturers had taught in one of the four available CALS classrooms. Over the past 5 years, communication with teachers about students’ classroom-related feedback in the end- of-semester course evaluation suggest that the CALS project was popular among students. This reporting from teaching staff (the institution’s learning design team do not have direct access to evaluation data) suggests that students enjoy the collaborative atmosphere afforded by the CALS classrooms. However, no formal evaluation had been conducted to examine the teachers’ experience and perception of the CALS classrooms. In view of this gap, an evaluation of the CALS project was conducted into the multifaceted nature of CALS projects in 2017. Australasian Journal of Educational Technology, 2018, 34(6). 20 The evaluation aimed to answer the following two questions so as to inform the future development of the project: 1. How do teachers perceive the CALS project? 2. How effective are the measures taken to encourage teachers’ exploration of, and experimentation with, teaching and learning possibilities within the CALS classrooms? The measures considered as part of the second research question encompassed communications to staff, professional learning and development events, digital resources and print collateral, the consultative services provided by technical and pedagogical support staff, the design of the spaces themselves, the furniture within the spaces, and technologies installed and implemented as part of the fit-outs. Methods The study used secondary data analysis to evaluate the pilot CALS project at the Faculty of Education and Social Work, University of Auckland in New Zealand. The evaluation was conducted in mid-2017, aiming to solicit the underlying views among teachers who have been involved in teaching using the institution’s CALS classrooms to date, so as to inform future developments of the project. Prior to the study, permission to use the data for research purposes was obtained from the Associate Dean for Teaching and Learning of the faculty, where the data was held. Participants and procedure All teachers who had ever booked CALS classrooms for teaching purposes were invited to participate in a voluntary online survey hosted on the Qualtrics platform. To maximise the response rate, the survey was set to be open for 3 weeks. During the 3-week period, three invitation emails were sent to the target staff as reminders. In these emails, potential respondents were briefed about the purpose and significance of the evaluation. Finally, 52 teachers responded to the survey, representing an overall response rate of 43%, which met the standard for similar organisational survey research recommended by Baruch and Holtom (2008). Instrument On the basis of prior studies (e.g., Wilson & Randall, 2012; Dane, 2016), the questionnaire was formulated around the following four major themes: (1) pedagogy, (2) alignment with the institution’s strategic teaching and learning vision, (3) technical support, and (4) digital infrastructure and furniture. There were two parts to the questionnaire. Part 1 was demographic in nature, collecting information related to device ownership and teachers’ agreement level with the classroom settings. Part 2 aimed to ascertain teachers’ level of agreement with the pre-determined four themes. To address the potential problem of respondents having a positive inclination towards the novelty associated with the CALS project, a 6-point positively- packed self-reported rating scale (Brown, 2004) was used for items in Part 2 (1 = strongly disagree, 2 = mostly disagree, 3 = slightly agree, 4 = moderately agree, 5 = mostly agree, and 6 = strongly agree). Furthermore, four open-ended questions related to the four themes were also designed to solicit qualitative feedback from the teachers. In order to ensure its content validity, two experts with psychometric research backgrounds were consulted to judge the relevance and overall quality of those items. In addition, before the questionnaire was administered, a pilot test was conducted among a group of learning designers (n = 5), and adjustments including the layout and the wording of some items were made accordingly. Data analysis All quantitative data were imported into SPSS for statistical analysis. Descriptive statistics were first summarised and checked for validity and reliability. Following this step, an independent t-test was performed to check whether there existed mean differences in their perceptions of the CALS project between those issued a mobile computing device and those not issued one. Meanwhile, all qualitative data were imported into NVivo 10 and a directed content analysis approach was employed to analyse the data (Hsieh & Shannon, 2005). During the process, an initial code template around the pre-determined four themes was used to code the data. After coding all the transcripts, each theme was re-examined to check the common patterns. Australasian Journal of Educational Technology, 2018, 34(6). 21 Results Quantitative results First, examination of teachers’ voluntary responses to the questions contained within Part 1 demonstrated that 33 participants (64%) agreed with the use of moveable, flexible furniture, suggesting that they generally agreed with the furniture arrangement. Moreover, among 42 teachers who responded to the question regarding device ownership, 32 (76%) had been recipients of a mobile computing device issued by the faculty. However, with regard to teachers’ responses to the slider question of how often they used them for teaching and learning in the CALS classrooms, the self-reported data showed that the average usage percentage for CALS-related purposes was only 31%, indicating that these devices are not used as expected as part of everyday teaching within the redeveloped classrooms. Next, descriptive statistics for the questionnaire items measuring teachers’ agreement level of CALS- related themes were checked. As shown in Table 1, all means were above 3.00, ranging from 3.00 to 5.10. The standard deviations ranged from 1.01 to 1.64, indicating a fair spread around the mean. The inspection of the univariate normality of these items showed that univariate skew and kurtosis statistics were from - 1.57 to 0.66 and -0.98 to 2.02 respectively, suggesting they were normally distributed. All alpha values were above the threshold value of 0.70 recommended by Nunnally (1978), suggesting the presence of internal consistency. Table 1 Descriptive statistics of part 2 questionnaire items (N = 52) Code Questionnaire Items M SD Skewness Kurtosis Pedagogy (α = .88) Q1_1 The physical space in the CALS classrooms has prompted me to think about the different pedagogies that are available to me. 4.61 1.15 -0.97 1.09 Q1_2 I’ve experimented with different pedagogies as a result of having access to the CALS classrooms. 4.45 1.29 -0.98 0.69 Q1_3 I’ve intentionally redesigned my course, or aspects of it, as a result of having access to the CALS classrooms. 3.90 1.38 -0.44 -0.87 Q1_4 My teaching includes collaborative and active elements as a result of having access to the CALS classrooms. 4.10 1.39 -0.65 -0.40 Q1_5 The CALS classrooms allow me to model the kind of pedagogies and skills that pre-service professionals will be expected to exhibit in schools/practice settings. 4.16 1.35 -0.66 -0.27 Q1_6 Learning in the CALS classrooms prepares students well for work in schools/practice settings. 3.90 1.25 -0.46 -0.40 Q1_7 I'm more confident teaching with digital technology as a result of having access to the CALS classrooms. 4.08 1.37 -0.80 0.20 Faculty vision of teaching and learning (α = .95) Q2_1 The CALS classrooms are clearly aligned with the faculty vision. 4.30 1.30 -0.73 -0.06 Q2_2 The CALS classrooms are clearly aligned with current strategic plan of the faculty. 4.45 1.15 -0.88 1.10 Q2_3 The CALS classrooms are clearly aligned with the current teaching and learning plan of the faculty. 4.38 1.23 -0.95 0.61 Q2_4 The CALS classrooms are clearly aligned with the practice in schools or professional settings. 4.02 1.29 -0.32 -0.62 Support (α = .90) Australasian Journal of Educational Technology, 2018, 34(6). 22 Code Questionnaire Items M SD Skewness Kurtosis Q3_1 I find one-on-one sessions with Learning Designers useful for teaching in classrooms. 4.77 1.07 -0.70 -0.18 Q3_2 I find the IT support for the CALS classrooms useful. 4.95 1.29 -1.57 2.02 Q3_3 I find the pyramids on the desks useful for supporting students with getting online and mirroring. 3.64 1.36 -0.10 -0.48 Q3_4 I find the posters in the CALS classrooms useful to help understand the technology. 3.40 1.11 0.37 0.61 Q3_5 I find the workshops and events related to the CALS classrooms useful. 4.13 1.14 0.07 -0.91 Q3_6 I find the learning technology assistants a useful source of support when teaching in a CALS classroom. 5.00 1.27 -1.51 1.93 Q3_7 I find the support mobile app and YouTube videos useful for understanding the CALS classrooms. 3.44 1.30 0.09 -0.02 Technology and furniture (α = .76) Q4_1 I frequently use the iPads in the CALS classrooms. 3.29 1.40 0.13 -0.57 Q4_2 Students are able to display their work on the screens for me has proven to be a vital feature of the CALS classrooms. 3.60 1.56 0.20 -0.98 Q4_3 The technology in the CALS classrooms is robust and reliable. 4.24 1.27 -1.16 1.27 Q4_4 I am comfortable operating the control panel to control the displays. 3.95 1.54 -0.46 -0.74 Q4_5 The document camera is an important part of my teaching. 3.00 1.64 0.66 -0.68 Q4_6 The TV screens are a useful feature. 5.10 1.01 -0.80 -0.50 The self-reported responses from teachers to the pedagogy section show that teachers generally agreed that the CALS project had a positive influence on their pedagogy, particularly in stimulating their new pedagogical ideas (M = 4.61) and affording more opportunities to experiment (M = 4.45). However, teachers did tend to agree less with its influence in promoting course redesign (M = 3.90) and preparing students for better practices (M = 3.90). Furthermore, data suggest that teachers agreed that the CALS project aligned with the institution’s strategic plans and the teaching and learning vision articulated within them (Ms > 4.00), indicating that it is generally acknowledged that the project is an integral part to achieving the faculty’s strategic goals. As for the forms of support, the most useful support identified by the respondents included learning technology assistants (M = 5.00), support from the institution’s traditional information technology support teams (M = 4.95), learning designers (M = 4.77), and workshops and events (M = 4.13). Some of the forms of support that teachers considered less useful included classroom posters, desktop pyramids (small cardboard pyramids placed on each classroom table with common technical procedures pertinent in the CALS explained for staff and students), mobile apps, and YouTube videos produced by the spaces’ various support teams (Ms < 4.00). When asked to evaluate technology in the CALS, respondents indicated that the technology in the CALS was robust and reliable (M = 4.24) and that they valued the visibility and accessibility for students afforded by multiple television screens (M = 5.10). However, some equipment – notably the document camera, classroom iPad sets, and Extron screen consoles (control pads for the audio-visual matrix switcher underlying the ability for teachers to customise displays and inputs for the rooms’ multiple screens) – were deemed neither useful nor user-friendly (Ms < 4.00). Further, to see the possible effect of the ownership of faculty-issued digital devices, the 42 teachers who indicated their faculty issued device ownership were split into two groups: those with devices issued by the Australasian Journal of Educational Technology, 2018, 34(6). 23 faculty (n = 32) and those without one (n = 10). Independent samples t-tests were conducted to compare the means of the four themes between the two groups. As is shown in Table 2, results suggest that one statistically significant difference was detected for the support theme, which indicated those who own faculty-issued devices tended to be more aware of the support for the CALS project. Table 2 Independent t-test results (n = 42) Device ownership Yes (n = 32) No (n = 10) Theme M SD M SD t p Pedagogy 4.39 0.87 3.77 1.18 1.80 > .05 Faculty vision 4.48 1.11 3.88 1.21 1.46 > .05 Support form 4.37 0.83 3.46 1.16 2.75 < .05* Technology 3.99 0.83 3.43 1.21 1.66 > .05 Qualitative results In order to gain an in-depth understanding of the pilot CALS project, teachers’ suggestions for future improvement were also analysed on the basis of the four pre-determined themes. Overall, qualitative analysis results also suggest that teachers generally held positive attitudes towards the CALS project as it affords teachers more opportunities to experiment with new pedagogies, and the spaces and their equipment are generally easy to engage with and teach within. They also agreed that the project was in line with the institution’s strategic vision and that it also contributes to the improvement of the faculty’s image and public profile. Nevertheless, their feedback highlighted some aspects that deserve attention in relation to future developments. Firstly, instead of demanding more advanced technology, teachers asked for reliable technology and timely technical support. Most stated that they would feel assured if they knew that timely technical support was available or if someone they were familiar with could attend their teaching sessions. Teachers’ suggestions also focused on the physical aspects of a CALS classroom. Some teachers indicated that in designing a CALS classroom, the physical shape of the classroom should be considered. They expressed their dissatisfaction with the irregular shape of one particular CALS classroom (which bears a longer and narrower rectangular footprint than its counterparts), which hindered their ability to make eye contact with all students in the space. Some further suggested adjusting the furniture arrangement to allow for easier collaboration amongst three to four students (rather than only the six-student collaborations privileged by the particular furniture deployed in these CALS). Lastly, some participants also emphasised the importance of considering students’ practical needs, which include the provision of both air conditioning and greater numbers of easily accessible power outlets in the CALS classrooms. Discussion and implication The aim of the study was to gain insights from the first-stage implementation process of a CALS project so as to inform its future improvement. These insights are valuable to a wide audience of educators involved in the design, development and ongoing support of collaborative, active learning spaces and environments. In line with prior reporting of students’ favourable attitudes towards CALS classrooms (e.g., Rands & Gansemer-Topf, 2017), both quantitative and qualitative results showed that although they were reluctant to fully embrace the digital pedagogy embedded within the CALS project, teachers generally held positive perceptions towards the project and the retrofitted teaching spaces. Further, resonating with previous studies on the relationships between physical learning spaces and teachers’ teaching practice (Ochola & Achrazoglou, 2015), the findings suggest that the CALS project provides teachers with more pedagogical options (in terms of not only collaborative and active teaching approaches, but also technology-enhanced activity), and helps to foster a shared vision regarding institutional uptake of educational technology. Institutional cultural change is a slow process, but in this project the CALS classrooms proved to play an important and expedient role in translating practical experiences of teaching to a sense of understanding of the ambitions for change articulated by the institution at a strategic level. New spaces (in this case retrofitted) and spatial possibilities make strategic objectives and teaching and learning concepts concrete for their users and can play a part in catalysing wider change. The clear challenges for a tertiary institution in Australasia in capitalising upon this potential are twofold: the resource (financial, spatial, and political) Australasian Journal of Educational Technology, 2018, 34(6). 24 required to construct or retrofit further spaces, and also embracing best practice in collaborative, active classroom design as a fluid and developing model. Considering the rapid development of modern learning environments at the primary and secondary school levels, there is a need to further promote the CALS projects at the university level, so that current students can adjust to the change and educational institutions can be well prepared for the future. Cohorts of students enrolling at Australasian universities today are increasingly unlikely to have undergone schooling in classroom spaces that look, feel, or function in the same ways as traditional university lecture halls and tutorial rooms. It is equally important to note, however, that not all of the current and future intake of tertiary students will have experienced modern learning environments, nor spaces attuned to collaborative and active pedagogies. The CALS project discussed in this paper responded to two key contextual factors: the institution’s mandate to consolidate smaller classes together, and the growth of modern learning environments in New Zealand’s compulsory schooling sector. Future CALS projects must ensure contextual relevance, rather than simply to dogmatically promote a particular teaching and learning model. For institutions such as this faculty, which has a core focus on the undergraduate and graduate preparation of teacher education graduates, the further development and evaluation of CALS projects reactive to local contexts can aid in aligning student experience with the evolving practical realities of the school sector they will enter upon qualification and employment. Echoing prior research findings on collaborative and active learning (e.g., Laurillard, 2001; Mumtaz, 2000), the results further suggest there exists a gap between teachers’ needs and the CALS-related support. Teachers value the support they receive from learning designers, information technology support staff, and learning technology assistants, which facilitates their exploration of the potentials of these classrooms. However, they also indicated that they felt they were not adequately informed of the capacity for various modes of teaching and activity afforded by the digital technology and infrastructure of the CALS classrooms. At the time the CALS classrooms launched, a number of the faculty’s courses had already been identified to use the spaces on the basis of their student numbers, and there was limited lead time provided to prepare these staff for their CALS-based delivery. Initial communication and sharing of pedagogical and technical possibilities (in the form of email communication, digital resources, and hands-on workshops conducted within the CALS) in these classrooms was consequently focused on the staff involved in the delivery of those particular classes, rather than the wider teaching community. As this initial cohort of teaching staff settled into and became familiar with the CALS classrooms, communication was widened to the broader faculty, but teachers’ responses highlight clearly that there is a need to rethink the technical support mode and engage with the varied and evolving teaching and learning possibilities contained in these spaces more proactively. While learning designers were able to ensure all teaching staff using CALS had been inducted into the required technical knowledge of the classrooms (through their gatekeeping of the spaces within the institution’s room booking platform), and had been given insight into some of the pedagogical possibilities, this did not necessarily allow an ongoing and evolving sense of the modes of teaching and learning encouraged by CALS to be shared across the faculty teaching community. More communication channels should be opened so as to facilitate the dissemination of knowledge among a broader spread of teachers, including those who are teaching in traditional spaces. Deliberate communication efforts and network- building (facilitated, for example, by learning designers) could also aid in the development of communities of practice to critically reflect and develop emerging pedagogical opportunities and challenges in the CALS. Authentic communities of practice also cultivate mutual trust between teachers, learning designers and other support staff and ensure all stakeholders in the CALS are able to remain current with research- informed best practices in ways that static, one-way technical and pedagogical support resources cannot. Critically, these processes and venues for sharing and developing teacher practice need to be embedded early on in the lifespan of CALS classrooms and made available to as wide a range of teaching stakeholders within the institution as possible. As for technology, results suggest that the technology that teachers use in these spaces should be easy to use, reliable, and pedagogically relevant. This is in line with McNeely’s (2005) suggestion, and implies that CALS design should be guided by the growing body of research in this area and that technology should not be viewed as a panacea. Before introducing technology into a CALS classroom, its potential to afford innovation, and to enable pedagogies that are collaborative or active is only one consideration. Also significant is the the reliability of these technologies, as once in operation in a CALS classroom they may Australasian Journal of Educational Technology, 2018, 34(6). 25 be accessed by numerous users throughout a teaching day, are likely to run continuously, and must integrate with other elements of institutional infrastructure. Moreover, voices from different sections of the faculty such as information technology support, library teams, and managers should be consulted in order to ensure the alignment of technology and pedagogy. Lastly, only slight differences in attitude and teaching practice were found between those staff issued a mobile computing device by the institution and those not. Furthermore, the self-reported usage data suggest that these devices are not used to facilitate CALS-based teaching as frequently or as meaningfully as expected. More sustainable approaches could be considered in the future, therefore, instead of allocating mobile computing devices to teachers. This might include offering funds for purchasing various technologies desired by teaching staff rather than direct provision of predetermined devices, or requiring that participants propose and articulate possible teaching and learning practices to pursue with such tools before making provision of them. Such decisions by staff and information technology procurement stakeholders could again be informed by the understandings of CALS teaching and learning approaches, and the practical requirements of these, developed within communities of practice associated with the classroom spaces. Limitations and recommendations for future research Although empirical support is present, the study does have limitations. Due to contextual constraints, the sample size of the evaluation is relatively small, which may reduce the generalisability of the results to other educational institutions. Thus, future studies should be carried out to further ascertain teachers’ perceptions of CALS projects. Furthermore, classroom observations could also be carried out to avoid the gap between self-reports and actual practices of teachers in the CALS classrooms. References Baruch, Y., & Holtom, B. C. (2008). Survey response rate levels and trends in organizational research. Human Relations, 61(8), 1139–1160. https://doi.org/10.1177/0018726708094863 Beichner, R. J., Saul, J. M., Allain, R. J., Deardorff, D. L., & Abbott, D. S. (2000). Introduction to SCALE-UP: Student-centered activities for large enrollment university physics. Retrieved from https://files.eric.ed.gov/fulltext/ED459062.pdf Benade, L. (2017). Is the classroom obsolete in the twenty-first century? Educational Philosophy and Theory, 49(8), 796–807. https://doi.org/10.1080/00131857.2016.1269631 Brooks, D. C. (2011). Space matters: The impact of formal learning environments on student learning. British Journal of Educational Technology, 42(5), 719–726. https://doi.org/10.1111/j.1467- 8535.2010.01098.x Brown, G. T. L. (2004). Measuring attitude with positively packed self-report ratings: Comparison of agreement and frequency scales. Psychological Reports, 94(3), 1015–1024. https://doi.org/10.2466/pr0.94.3.1015-1024 Dane, J. (2016). The effective teaching and learning spatial framework. In W. Imms, B. Cleveland, & K. Fisher (Eds.), Evaluating learning environments (pp. 211–228). Rotterdam: Sense. https://doi.org/10.1007/978-94-6300-537-1_15 Doorley, S., & Witthoft, S. (2012). Make space: How to set the stage for creative collaboration. Hoboken, NJ: Wiley. Dori, Y. J., Belcher, J., Bessette, M., Danziger, M., McKinney, A., & Hult, E. (2003). Technology for active learning. Materials Today, 6(12), 44–49. https://doi.org/10.1016/S1369-7021(03)01225-2 Educause (2017). Learning space rating system. Retrieved from https://www.educause.edu/eli/initiatives/learning-space-rating-system Fisher, K. (Ed.) (2016). The translational design of schools: An evidence-based approach to aligning pedagogy and learning environments. Rotterdam: Sense. https://doi.org/10.1007/978-94-6300-364-3 Germany, L. M. (2014). Learning space evaluations: Timing, team, techniques. In K. Fraser (Ed.), The future of learning and teaching in next generation learning spaces (pp. 267–268). Bingley: Emerald. https://doi.org/10.1108/S1479-362820140000012018 Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687 https://doi.org/10.1177/0018726708094863 https://files.eric.ed.gov/fulltext/ED459062.pdf https://doi.org/10.1080/00131857.2016.1269631 https://doi.org/10.1111/j.1467-8535.2010.01098.x https://doi.org/10.1111/j.1467-8535.2010.01098.x https://doi.org/10.2466/pr0.94.3.1015-1024 https://doi.org/10.1007/978-94-6300-537-1_15 https://doi.org/10.1016/S1369-7021(03)01225-2 https://www.educause.edu/eli/initiatives/learning-space-rating-system https://doi.org/10.1007/978-94-6300-364-3 https://doi.org/10.1108/S1479-362820140000012018 https://doi.org/10.1177/1049732305276687 Australasian Journal of Educational Technology, 2018, 34(6). 26 Imms, W., Cleveland, B., & Fisher, K. (Eds.). (2016). Evaluating learning environments: Snapshots of emerging issues, methods and knowledge. Rotterdam: Sense. https://doi.org/10.1007/978-94-6300- 537-1 Jisc (2016). Evaluating and designing learning spaces. Retrieved from https://www.jisc.ac.uk/guides/evaluating-and-designing-learning-spaces Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning & Education, 4(2), 193–212. https://doi.org/10.5465/amle.2005.17268566 Laurillard, D. (2001). Rethinking university teaching: A conversational framework for the effective use of learning technologies. London: Routledge. LEaRN (2018). LEaRN. Retrieved from https://research.unimelb.edu.au/learnetwork/home McNeely, B. (2005). Using technology as a learning tool, not just the cool new thing. In D. Oblinger, & J. L. Oblinger (Eds.). Educating the net generation (pp. 41–51). Boulder, CO: Educasue. Retrieved from https://www.educause.edu/research-and-publications/books/educating-net-generation/using- technology-learning-tool-not-just-cool-new-thing Ministry of Education (2017). Innovative learning environment assessment tool. Retrieved from https://www.education.govt.nz/school/property/state-schools/design-standards/flexible-learning- spaces/assessment-tool/ Mumtaz, S. (2000). Factors affecting teachers’ use of information and communications technology: A review of the literature. Journal of Information Technology for Teacher Education, 9(3), 319–342. https://doi.org/10.1080/14759390000200096 Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill. Ochola, J. E., & Achrazoglou, G. J. (2015). Maximizing opportunities: Smart learning spaces, smarter interactions and collaboration. Journal of Education and Human Development, 4(1), 121–132. https://doi.org/10.15640/jehd.v4n1a12 Organisation for Economic Co-operation and Development (2008). PEB exchange, programme on educational building. Retrieved from https://www.oecd-ilibrary.org/education/peb-exchange- programme-on-educational-building_16097548/datedesc#collectionsort Rands, M., & Gansemer-Topf, A. (2017). The room itself is active: How classroom design impacts student engagement. Journal of Learning Spaces, 6(1), 26–33. Retrieved from http://libjournal.uncg.edu/jls/article/view/1286 Staley, B., & Freeman, L. A. (2017). Digital storytelling as student-centred pedagogy: Empowering high school students to frame their futures. Research and Practice in Technology Enhanced Learning, 12(21). https://doi.org/10.1186/s41039-017-0061-9 Tanner, C. K. (2000). The influence of school architecture on academic achievement. Journal of Educational Administration, 38(4), 309–330. https://doi.org/10.1108/09578230010373598 The ALC Pilot Evaluation Team (2007). Active learning classrooms pilot evaluation: Fall 2007 findings and recommendations. Retrieved from https://classroom.umn.edu/sites/classroom.umn.edu/files/2017- 09/alc_report_final.pdf Wilson, G., & Randall, M. (2012). The implementation and evaluation of a new learning space: A pilot study. Research in Learning Technology, 20(2), 1–17. https://doi.org/10.3402/rlt.v20i0.14431 Corresponding author: Lawrence May, l.may@auckland.ac.nz. Australasian Journal of Educational Technology © 2018. Please cite as: Mei, B., & May, L. (2018). Reflective renovation: Insights from a collaborative and active learning space project evaluation. Australasian Journal of Educational Technology, 34(6), 17-26. https://doi.org/10.14742/ajet.4476 https://doi.org/10.1007/978-94-6300-537-1 https://doi.org/10.1007/978-94-6300-537-1 https://www.jisc.ac.uk/guides/evaluating-and-designing-learning-spaces https://doi.org/10.5465/amle.2005.17268566 https://www.educause.edu/research-and-publications/books/educating-net-generation/using-technology-learning-tool-not-just-cool-new-thing https://www.educause.edu/research-and-publications/books/educating-net-generation/using-technology-learning-tool-not-just-cool-new-thing https://www.education.govt.nz/school/property/state-schools/design-standards/flexible-learning-spaces/assessment-tool/ https://www.education.govt.nz/school/property/state-schools/design-standards/flexible-learning-spaces/assessment-tool/ https://doi.org/10.1080/14759390000200096 https://doi.org/10.15640/jehd.v4n1a12 https://www.oecd-ilibrary.org/education/peb-exchange-programme-on-educational-building_16097548/datedesc#collectionsort https://www.oecd-ilibrary.org/education/peb-exchange-programme-on-educational-building_16097548/datedesc#collectionsort http://libjournal.uncg.edu/jls/article/view/1286 https://doi.org/10.1186/s41039-017-0061-9 https://doi.org/10.1108/09578230010373598 https://classroom.umn.edu/sites/classroom.umn.edu/files/2017-09/alc_report_final.pdf https://classroom.umn.edu/sites/classroom.umn.edu/files/2017-09/alc_report_final.pdf https://doi.org/10.3402/rlt.v20i0.14431 mailto:l.may@auckland.ac.nz https://doi.org/10.14742/ajet.4476 Literature review About the CALS project Methods Participants and procedure Instrument Data analysis Results Quantitative results Qualitative results Discussion and implication Limitations and recommendations for future research