Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 7/23/2020 Facebook: /EPAAA Revisions received: 12/22/2020 Twitter: @epaa_aape Accepted: 12/24/2020 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 29 Number 45 April 5, 2021 ISSN 1068-2341 Faculty Attitudes toward Technology-Driven Instruction in Developmental Mathematics Jenna W. Kramer RAND Corporation Stephany Cuevas Chapman University & Angela Boatman Boston College United States Citation: Kramer, J. W., Cuevas, S., & Boatman, A. (2021). Faculty attitudes toward technology- driven instruction in developmental mathematics. Education Policy Analysis Archives, 29(45). https://doi.org/10.14507/epaa.29.5843 Abstract: Innovation in instructional technology has contributed to the rapid implementation of technology-driven instructional platforms, particularly in developmental math coursework (Bickerstaff et al., 2016). In this phenomenological study, we investigate how faculty perceive and respond to a mandated, technology-driven instructional model for developmental math coursework at public colleges in Tennessee. Through interviews with faculty members across four colleges, we find that many faculty agreed that technology helped them to better track student performance, provide more targeted assistance, and communicate directly with studen ts. Faculty also expressed concerns that technology provides the opportunity or temptation to game the system, interfering with true learning, and that students with the greatest needs may not be well served by the http://epaa.asu.edu/ojs/ https://doi.org/10.14507/epaa.29.5843 Education Policy Analysis Archives, Vol. 29 No. 45 2 instructional model. We draw policy implications related to the role of educators in the development and implementation of curricular policy, provision and requirements for ongoing professional development, and postsecondary learning accountability. Keywords: educational policy; remedial education; remedial mathematics; college faculty; computer assisted instruction; qualitative research Actitudes de los profesores hacia la instrucción tecnológica en matemáticas del desarrollo Resumen: La innovación en la tecnología de instrucción ha contribuido a la rápida implementación de plataformas de instrucción impulsadas por la tecnología, particularmente en los cursos de matemáticas de desarrollo (Bickerstaff et al., 2016). En este estudio fenomenológico, investigamos cómo los profesores perciben y respo nden a un modelo de instrucción obligatorio e impulsado por la tecnología para los cursos de matemáticas del desarrollo en las universidades públicas de Tennessee. A través de entrevistas con miembros de la facultad en cuatro universidades, encontramos que muchos profesores estuvieron de acuerdo en que la tecnología los ayudó a rastrear mejor el desempeño de los estudiantes, brindar asistencia más específica y comunicarse directamente con los estudiantes. Los profesores también expresaron su preocupación de que la tecnología brinde la oportunidad o la tentación de jugar con el sistema, interfiriendo con el verdadero aprendizaje, y que los estudiantes con las mayores necesidades pueden no estar bien atendidos por el modelo de instrucción. Extraemos las implic aciones de las políticas relacionadas con el papel de los educadores en el desarrollo e implementación de la política curricular, la provisión y los requisitos para el desarrollo profesional continuo y la responsabilidad del aprendizaje postsecundario. Palabras-clave: política educativa; educación correctiva; matemáticas correctivas; profesores universitarios; instrucción asistida por computadora; investigación cualitativa Atitudes dos professors em relação ao ensino baseado na tecnologia em matemática de desenvolvimento Resumo: A inovação em tecnologia instrucional tem contribuído para a rápida implementação de plataformas instrucionais voltadas para a tecnologia, particularmente em cursos de matemática de desenvolvimento (Bickerstaff et al., 2016). Neste estudo fenomenológico, investigamos como os professores percebem e respondem a um modelo instrucional orientado para a tecnologia obrigatório para cursos de matemática de desenvolvimento em universidades públicas no Tennessee. Por meio de entrevistas com membros do corpo docente em quatro faculdades, descobrimos que muitos professores concordaram que a tecnologia os ajudou a acompanhar melhor o desempenho dos alunos, fornecer assistência mais direcionada e se comunicar diretamente com os alunos. Os professores também expressaram preocupação com o fato de que a tecnologia oferece a oportunidade ou a tentação de manipular o sistema, interferindo no verdadeiro aprendizado, e que os alunos com as maiores necessidades podem não ser bem atendidos pelo modelo instrucional. Traçamos implicações políticas relacionadas ao papel dos educadores no desenvolvimento e implementação da política curricular, provisão e requisitos para o desenvolvimento profissional contínuo e responsabilidade da aprendizagem pós-secundária. Palavras-chave: política educacional; educação corretiva; matemática corretiva; professores universitários; instrução assistida por computador; pesquisa qualitativa Faculty attitudes toward technology-driven instruction 3 Faculty Attitudes toward Technology-Driven Instruction in Developmental Mathematics In response to historically low levels of student success in developmental math courses, many colleges and college systems have begun to change the way faculty approach developmental education (Jaggars & Bickerstaff, 2018). Recently, states and institutions have harnessed technology in their reimagined vision for how they teach developmental courses: instead of relying solely on lecture-based instruction, increasing numbers of colleges have adopted various forms of technology to facilitate instruction and learning both in the classroom and outside of it. Advances in educational technology have contributed to the rapid implementation of technology-driven instructional platforms, particularly in developmental math coursework (Bickerstaff et al., 2016). These instructional models, which range from in-person instruction aided by technology to entirely online, asynchronous courses, fundamentally change the pedagogy and patterns of interaction in developmental math classrooms (Boatman & Kramer, 2019). While college faculty determine their own instructional style in the classroom, decisions about a college or college system’s overall instructional model may be made at the state or institutional level. When institutions make the transition to technology-driven instruction in developmental courses without input from faculty, faculty may be concerned about an inevitable change in the nature and quality of the academic experience. Primary and secondary education research has documented the disconnect between policies demanding academic change and how teachers envision best meeting the needs of students (Greene et al., 2008; Stevenson, 2008). For example, when teachers are required to focus primarily on raising student test scores, most report feeling less student-centered in their teaching and unsure of how to provide their students with a rich educational experience (Greene et al., 2008). While we have evidence of these perceptions in the K- 12 literature, our understanding of how college instructors experience mandated curriculum change is limited. Insights into faculty’s views toward a new instructional policy could illuminate areas of promise or concern when it comes to student learning and the quality of teaching. Thus, it becomes important to consider the experiences of instructors responsible for implementing changes to their teaching, particularly given the important role faculty play in shaping students’ campus experiences (Astin, 1977; Pascarella & Terenzini, 1976). Our study illustrates how higher education administrators and faculty perceive a mandated, technology-driven instructional change in developmental math courses. Drawing from a larger mixed-method, multi-site study on the effectiveness of the Emporium Model, a technology-based curriculum for remedial mathematics in the state of Tennessee, this phenomenological study is one of the first to describe how instructors experience policies that mandate higher education curriculum changes. We answer the following question, leveraging the experiences and perspectives of 15 administrators and faculty in four community colleges and public universities across Tennessee: How do faculty respond to mandated, technology-driven instructional change in developmental math? Our findings illuminate how a curriculum redesign policy changes both the traditional instructional model and the traditional power that faculty hold in the domain of curricular design. Specifically, respondents described their concerns about how a technology-based curriculum influenced both their pedagogy and their students’ learning. We find that when instructors are required to make curriculum changes without considering their best pedagogical practices, there is a mismatch between their expertise and their content delivery. Specifically, faculty were concerned that the technology platform did not encourage or support true learning and allowed students to game the system and/or cheat. While the faculty in this study described positive aspects to the change to technology based-curriculum, including faster grading and facilitated communication, Education Policy Analysis Archives, Vol. 29 No. 45 4 overall, concerns about meeting all student needs prevailed. The majority of participants believed the new model did not best support students’ need for differential learning. These findings have important implications for higher education policy and practice: to best meet student needs, widespread instructional changes require drawing upon the expertise of faculty. Policy Context In Tennessee, the adoption of a technology-driven instructional model replaced traditional developmental math courses at all public colleges and universities in 2013. The decision to reform remedial education began in the early 2000s when state data revealed large numbers of students enrolling in developmental courses combined with low numbers of students passing the courses. In 2000, more than 70% of the students enrolled at one of the state’s 13 community colleges were placed into developmental math courses, along with more than half of students at one of the state’s six four-year public universities (Gray-Barnett, 2001). The failure/withdrawal rate in developmental math courses averaged 45%, compared to 26% in college-level math courses. In 2005, the Tennessee Board of Regents provided small pilot grants to four institutions who agreed to adopt technology- centered learning as part of the state’s Developmental Studies Redesign Initiative. Descriptive summaries from these four pilot colleges suggested that the new model was helping students to complete developmental and college-level math (Twigg, 2011). In 2012, the Board of Regents voted to scale up the adoption of technology-driven instruction into developmental math, reading, and writing courses at all public colleges in the state by the fall of 2013. This is the policy change we examine in our study. In 2013, the public colleges in Tennessee adopted a technology-centered instructional model known as the Emporium Model. The Emporium Model is a computer-based developmental math curriculum that allows students to work at their own pace in a computer lab, as opposed to a traditional lecture-style course. Students meet for class, typically three times per week, and work through a set of math modules, completing an exam at the end of each module. The role of the faculty member is substantially different in an emporium-style course versus a traditional classroom. Under this new method of instruction, faculty act more as tutors, answering student questions individually as opposed to teaching through large-group instruction. They respond to student questions both in the lab and out of class, largely through email and messaging offered in the online platform. Prior research has shown that students experience faculty to be more accessible and approachable under the Emporium Model (Boatman & Kramer, 2019), but we are aware of no studies examining the reactions of faculty to this new method of teaching. Literature Review Faculty Voices in Existing Literature Existing higher education research on college faculty predominantly focuses on job satisfaction, diversity and inclusion, and the factors likely to influence the adoption of new teaching practices. Yet, this literature falls short in exploring how faculty respond, experience, or make sense of mandated changes to their teaching practices. Several studies have examined the characteristics of faculty that predict their willingness to adopt certain teaching practices, albeit not necessarily practices they are required to adopt. This is particularly salient in research related to instructional technology adoption. As faculty begin to utilize technology in their instructional practice, the factors that influence its adoption become important to understand. Factors found to predict technology usage among community college faculty include prior teaching experience in online courses (Paver et al., Faculty attitudes toward technology-driven instruction 5 2014) and the offer of extrinsic rewards (Akroyd et al., 2013). Studies have also found that faculty are more likely to favorably adopt a new instructional model when they perceive it as better than the traditional model, when they perceive consistency with their past experiences and the needs of students, when it is not overly complex, and when they have the opportunity to experiment with it on a limited basis (Tabata & Johnsrud, 2008). An increasing reliance on technology also raises concerns about faculty autonomy. Autonomy is an important characteristic of the academic profession, and heavy reliance on technology as an instructional tool has the potential to restrict college faculty’s autonomy (Kim et al., 2008; Levin et al., 2006). For colleges to be successful in expanding technology-based teaching, they must promote technology use across all faculty, not just fulltime faculty or those teaching in online courses (Jackowski & Akroyd, 2010). Higher education faculty voices and perspectives have also been captured in studies of teaching quality and quality audits; although the findings are mixed (see Acevedo, 2019). While these studies are predominantly based on traditional, face-to-face courses, some find that quality assurance mechanisms, such as evaluative rubrics, are associated with techniques of power manifested through coercive accountability and control (Shore & Wright, 2000; Worthington & Hodgson, 2005). Other scholars have documented how faculty subjected to quality assurance practices resisted these practices through game-playing, passive compliance, and, in some cases, outright refusal (Anderson, 2006; Newton, 2000, 2002; Worthington & Hodgson, 2005). The experiences of faculty members under a new instructional model are rarely universal, and can be, at times, varying and even conflicting (Acevedo, 2019). Faculty Response to Mandated Instructional Change Research on faculty attitudes about changes in instruction typically examines reactions to potential changes in classroom teaching; however, the body of literature related to curricular reform in higher education is quite small relative to K-12 education. The limited existing higher education literature affirms the role of instructional quality in the success of mandated instructional reform (Tabata & Johnsrud, 2008) and primarily examines the adoption of technology-driven instruction in college classrooms. While faculty may be asked their opinion when it comes to the adoption of a new technological system or teaching tool, the final decision to adopt such technologies typically lies with the administration. In such cases, faculty, who may be accustomed to making their own instructional decisions, are asked to adapt their teaching to accommodate the new models. In one case of a mandated switch to e-learning in the classroom, faculty attitudes were found to shift from resistance to curiosity to acceptance, with questions about teaching load, support, and recognition, and concerns about technical support and professional development (LeBaron & McFadden, 2008). K-12 education has been the site of many more mandated policy changes than higher education, allowing for further examination into the attitudes of teachers in response to such changes. In research on the federal No Child Left Behind Act of 2002, teachers reported frustration with the prescriptive nature of the policy, thereby limiting their autonomy as professionals and shifting decision making away from teachers to administrators (Roellke & Rice, 2008). When teachers did not adopt technology in their classrooms as mandated by their districts, it often resulted in inadequate access to equipment, inability to troubleshoot minor technology problems, and the absence of training in learning activities (Davidson et al., 2014). Few studies have examined faculty attitudes toward student learning and concerns about learning in response to mandated policy changes. In a 2005 survey of community college psychology faculty, the majority agreed that computer-assisted instruction enhanced teaching, was an effective tool, and improved education (Glasgow & Keim, 2005). But these attitudes may differ when faculty, rather than considering a hypothetical scenario, are in actuality faced with the adoption of such Education Policy Analysis Archives, Vol. 29 No. 45 6 methods. Additionally, these perceptions may change over time as technology advances to become more adaptive to student learning. More recently, research has examined student-focused concerns about learning. In interviews with and observations of 56 science and math faculty, researchers found that the two most common beliefs about student learning from faculty are that students learn best through repeated practice, and that students have different learning styles (Hora, 2014). Generally, there is a lack of empirical work on the nature of higher education faculty beliefs as they relate to student learning (Hora, 2014). Research also notes that teachers are concerned about pedagogy and student learning as they implement mandated instructional changes. For instance, in the case of mandated changes in response to accountability-focused instructional priorities, middle school teachers reported feeling less satisfied and less student-centered in their practice (Greene et al., 2008). Further, when implementing mandated instructional changes, teachers express concerns that student comprehension and learning may be adversely affected. These concerns may be particularly salient for instructors of students from historically underserved and marginalized groups (Jackson et al., 2017). The implementation of instructional reforms depends on teacher perceptions of students’ difficulties in mathematics (Jackson et al., 2017). Conceptual Framework To center the experience and perspectives of the individuals putting into practice the instructional reforms, this study focuses on faculty and administrative voices. Taking into consideration the lessons learned from K-12 research on teachers’ responses to mandated policy changes, this study’s conceptual framework centers on the premise that faculty perspectives in the implementation of educational reforms and policies are essential. Thus, building from the literature reviewed above, our study is grounded in three notions: (1) faculty’s perceptions of technology- driven education depend on previous experience with and comfort using technology (Akroyd et al., 2013; Paver et al., 2014); (2) mandated curricular changes pose a challenge to faculty autonomy for those accustomed to making their own instructional decisions (Kim et al., 2008; Levin et al., 2006), and (3) student learning remains at the center of faculty concerns (Jackson et al., 2017). Together, these notions help conceptualize a deeper understanding of higher education faculty experiences with mandated technology-driven curriculum changes: they explore faculty perceptions of technology-based education, the process by which this change occurred (i.e., mandated policy), and how they perceive the changes impacting their students. Research Design and Methodology This paper is part of a larger study that focused on analyzing the effectiveness of a mandated technology-based curriculum for remedial mathematics, the Emporium Model, by the Tennessee Board of Regents (TBR). This study’s methodology uses methods adopted from phenomenology. Phenomenology describes the meaning several individuals give to a particular lived experience, or a phenomenon, and identifies the commonalities of this experience (Patton, 2002). Since we were interested in exploring how faculty experienced a mandated change to their curriculum and pedagogy, which in this case is the phenomenon in question, this approach is well suited to our analysis; it aligns with our conceptual framework (Creswell, 2014; Patton, 2002; Ravitch & Riggan, 2012). Thus, for this analysis, we specifically examine the experiences of faculty at four institutions with the aim of achieving maximum variation along two dimensions: implementation level and institutional type (Patton, 2002). We concentrate our analysis on the faculty perspectives at two community colleges and two baccalaureate institutions that have implemented technology-driven Faculty attitudes toward technology-driven instruction 7 instruction to different degrees. We will refer to these institutions by the following pseudonyms: Foothills State Community College, Valley State Community College, Urban State University, and Lakes State University. Sites The institutions are located in localities of varying sizes, ranging from a remote town (Lakes State) to a large city (Urban State). One of the 2-year institutions (Foothills State) and one of the 4- year institutions (Urban State) had 5,000-9,999 students (Carnegie classification sizes of large 2-year and medium 4-year, respectively) and the other institutions had 10,000-19,999 students (large). These four institutions collectively enroll a large number of students of color and low- income students, similar to Jackson et al. (2017). In Fall 2019, the student body across all four institutions was 31% non-white, 39% Pell eligible, and 22% adult students (Tennessee Higher Education Commission, 2020). At the four-year institutions, 42% of students were non-white, 40% were Pell eligible, and 13% were adult students (Tennessee Higher Education Commission, 2020). At the two-year institutions, 22% of undergraduate students were non-white, 39% were Pell eligible, and 29% were adult students (Tennessee Higher Education Commission, 2020). Foothills State Community College Foothills State Community College implemented the Emporium Model in Fall 2009. Institutional adoption of technology-driven instruction was comprehensive. Invested department leaders and administrators, along with adequate resources (i.e., space, financial) facilitated the transition. Prior to the adoption of the Emporium Model, faculty used computer-based instruction as a supplemental tool rather than the primary tool for content delivery. During data collection, there was heavy computer usage across the math curriculum, which was supported through a large, well-staffed math lab. All developmental and corequisite college-level coursework was technology- facilitated, with instruction primarily on the computer. Homework and testing took place exclusively on computers. Students had time to work on assignments in the computer lab both during the scheduled class time and outside of it. Instructors provided timely and individualized instruction. Exams were taken at-will on the computer during a testing period and were proctored in a controlled testing area of the math lab. At Foothills State, 51-60% of entering students took developmental math coursework, and class sizes ranged from 10-25 students per class (mean 20). In the expansion of technology-based instruction, faculty utilized online resources and created videos for students. Adjunct faculty taught approximately 15% of Emporium classes. Valley State Community College Valley State Community College began to gradually pilot the Emporium Model early in Fall 2007, only fully adopting the model seven years later (Fall 2014). Leadership turnover at the department level accelerated the transition. Technology was not used in coursework before the adoption of the Emporium model. After 2014, Valley State transitioned from a full Emporium model to a hybrid model after students reported frustration with a fully technology-driven instructional environment. At Valley State, roughly half of entering students took developmental math, and class sizes ranged from 10-25 students per class (mean 15). During data collection, developmental sections were largely self-paced and technology-facilitated with some group instruction, while college-level instruction was primarily in a guided lecture format. Approximately 25% of Emporium instructors were adjunct faculty. Education Policy Analysis Archives, Vol. 29 No. 45 8 Urban State University At Urban State University, over-enrollment in the Fall of 2014 prompted the adoption of technology in the classroom. While some faculty had begun to use an e-book in Spring 2014, there was significant skepticism in the math department about the role of technology in math learning. However, when 300 students required math remediation in Fall 2014, Urban State began piloting e- books, web-based homework, and online exam-taking in order to ease the burden of grading. Faculty at Urban State had the autonomy to choose how they instructed their courses, and some web-based resources were available. Classes were largely lecture-based across all sections. Most instructors used an online platform for posting notes or slides, and many used WebAssign for homework. Course exams were taken online during class time, but the final exam was paper-based and taken during a set exam period. Adjunct faculty taught approximately 15% of Emporium classes. Lakes State University Lakes State University adopted technology-driven instruction later than most public institutions in Tennessee, but quickly became highly-invested in the Emporium model. Institutional executive turnover and an educator-administrator partnership contributed to adoption in the Fall of 2012. The merging of the Learning Support and Math Departments facilitated implementation at Lakes State, with full implementation reported by the Fall of 2013. Prior to adoption, Lakes State used technology widely for homework and testing. After the adoption of the Emporium model, "heavy" computer usage in the institution's new math lab was facilitated by instructors through short guided practice with the bulk of the class period reserved for computer-facilitated instruction with on-demand, one-on-one help. Nearly a quarter of incoming students enrolled in developmental math. Lakes State faculty conducted their technology-driven developmental math courses in the institution's math computer lab. The space could serve up to 90 students per class, with an average class size of around 50 students. Instruction was primarily technology-driven but included a classroom discussion that covered study skills and time management, along with an "overview" of the work to be completed during the upcoming week. Full-time faculty taught all math courses, with no adjunct faculty assigned to developmental courses. Study Sample For this analysis, our sample consisted of 15 faculty and administrator participants at four institutions: two 2-year institutions (Foothills State Community College and Valley State Community College) and two 4-year institutions (Urban State University and Lakes State University). Table 1 presents a description of the interview sample. Interview participants were overwhelmingly white (n=12), and most were female (n=9). They held various roles at their institutions, including instructor, department chair, developmental math coordination, and math lab coordinator, and many had more than one position. While we did not ask the interview participants their ages, we estimate that they ranged from 30-60, with a left-skewed distribution. Data Collection Mixed-methods in design, we collected data for the larger project in two phases: survey data collection and campus site visits (Creswell, 2014). In the first phase, we administered an online survey via Qualtrics during the summer of 2016 to math department administrators of each of the 19 public community and state colleges in Tennessee. We received a response from 18 of the 19 colleges via the survey platform; an administrator from the final institution contacted the research team to answer the questions over the phone. Faculty attitudes toward technology-driven instruction 9 Table 1 Description of Site Visit Sample Faculty Respondent Characteristics Pseudonym Sector Urbanicity Institutional Size Total Race/Ethnicity Gender Roles Valley State Community College Public, 2-year Suburb: Large 10,000 - 19,999 2 White (2) Female (2) Faculty, Department Chair, Developmental Math Coordinator Foothills State Community College Public, 2-year City: Midsize 5,000 - 9,999 2 White (2) Female (2) Faculty, Math Lab Coordinator Lakes State University Public, 4-year or above Town: Remote 10,000 - 19,999 5 White (5) Female (2); Male (3) Faculty, Department Chair, Math Lab Coordinator Urban State University Public, 4-year or above City: Large 5,000 - 9,999 6 Black/African- American (3); White (3) Female (3); Male (3) Faculty, Department Chair, Math Lab Coordinator The purpose of the survey was to collect background information on the institutions, the initial implementation of the state - mandated instructional change, and ongoing use of technology-driven instruction to inform our future data collection efforts. We asked participants for rote information (instructional platform used; average developmental mathematics class size; semester in which implementation began; degree of computer usage in the classroom; frequency of ongoing professional development; etc.) as well as their perspective on the implementation, professional development and challenges and benefits for students, faculty, and administrators related to the redesign. Education Policy Analysis Archives, Vol. 29 No. 45 10 We reviewed the survey responses for trends and outlying cases of implementation. We then contacted a purposive subgroup of institutions to inquire about site visits, focus groups with students and faculty, and one-on-one interviews with administrators and faculty (Creswell, 2014; Patton 2002). We selected sites to achieve maximum variation in timing and degree of implementation of the technology-driven model to explore the potential heterogeneity of student experiences in the instructional model under different conditions (Creswell, 2014; Patton 2002). From January to March of 2017, we conducted hour-long interviews with an administrator and faculty members, observed a developmental math class, and facilitated a 60-minute focus group with 2-8 students. In total, we collected thirty hours of qualitative data, averaging five hours per site. The interviews with administrators and faculty at Foothills State Community College, Valley State Community College, Urban State University, and Lakes State University are the focus of this paper and analysis. During interviews with administrators and faculty, we raised questions centered on the implementation of the Emporium Model, including planning and implementation of the redesign, the evolution of instruction, organizational structure, and shared governance. Interviews were conducted on-campus, in offices and classroom spaces. Appendix A presents the protocol questions for our semi-structured faculty interviews. Data Analysis We transcribed all interviews with administrators and faculty and used qualitative software, Dedoose, for analysis. Based on our conceptual framework and the theoretical underpinnings of phenomenology, we first identified every statement in which participants discussed their experiences and perspectives on the implementation of the new instructional policy (Creswell, 2014; Patton, 2002; Ravitch & Riggan, 2012). We began with data horizontalization to understand how faculty experienced the mandated change in curriculum (Maxwell, 2013; Patton, 2002; Saldaña, 2015). Through this open coding, we realized that faculty conversations could be grouped into three broader categories about their experiences: (1) the assumptions faculty made about how their pedagogy and students’ learning could be impacted by the curriculum change; (2) their attitudes and beliefs about how their pedagogy and students’ learning was impacted by the curriculum change, which were based on their lived experiences; and (3) the behaviors and practices they engaged in as a result of the curriculum change. Within each of these categories, we developed codes to explore different aspects that were impacted by the change in instruction. Thus, our closed coding focused on the assumptions and attitudes faculty had towards their institutions, their instruction, and their studentse also developed codes that described their classroom practices, or their behaviors. In order to describe how faculty respond to mandated, technology-driven instructional change in developmental math, we focused our analysis on the codes that describe instructors’ concerns about pedagogy and concerns about student learning. Throughout data analysis, our research team wrote analytic memos detailing the coding process and data interpretation (Maxwell, 2013; Patton, 2002; Saldaña, 2015). We also established inter-coder reliability by coding multiple transcripts together and discussing discrepancies in developing the codebook. Memo-writing and establishing inter-coder reliability are essential qualitative data analysis practices (Maxwell, 2013; Patton, 2002; Saldaña, 2015). Table 2 provides examples of data coding and emergent themes. The themes presented below and their corresponding interview excerpts are representative of this focused coding. Faculty attitudes toward technology-driven instruction 11 Table 2 Examples of Data Coding and Theme Formation Codes Explanation of Codes Theme Content delivery Faculty express concerns that computer-, rather than instructor- delivered, content changes pedagogy and presents challenges for students. Concerns about pedagogy Uniformity between sections Faculty articulate that requirements for uniformity between sections lead them to implement changes to instruction and assessment that are at odds with their pedagogical approach. Platform capabilities Faculty identify limitations to the functionality of the instructional platform that limit their ability to teach students in a way that is consistent with their instructional philosophy. Learning Faculty express that students do not achieve deep learning and skills mastery when primary instruction is computer-based. Concerns about students Gaming and cheating Faculty articulate that the nature of assessment under a hybrid Emporium instructional model presents opportunities to, intentionally or unintentionally, misrepresent mastery, Education orientation Faculty interpret student actions and course performance as indicative of their orientation toward learning and education. Findings Concerns about Pedagogy Our analysis of the data highlighted instructors’ concerns about changes to pedagogy after the mandated switch to computer-based instruction. Interview respondents expressed attitudes and described behaviors that were rooted in their concerns that the shift in content delivery, the degree of autonomy to shape the curriculum, and the capabilities of the online instructional platforms in use at their institutions changed the pedagogy of foundational math coursework at their institutions. Content Delivery In our discussions of the mandated instructional policy change, faculty described their experiences with content delivery prior to and after the Emporium Model implementation. They expressed concerns that computer-delivered, rather than instructor-delivered, content changes pedagogy and presents challenges for students. Their experiences with the instructional platform informed their attitudes about the platform’s efficacy in supporting students in the development of Education Policy Analysis Archives, Vol. 29 No. 45 12 math skills mastery. Faculty reported mixed experiences with the hybrid teacher-platform content delivery of the Emporium Model: while some teachers expressed serious reservations about the technological delivery of content, many were enthusiastic about the hybrid model of delivery. Regardless of their attitude toward content delivery, faculty expressed that delivery of content by the instructional platform, rather than by teachers, changed the nature of instruction and questioning. One instructor at Foothills State Community College described her traditional teaching prior to the policy change as “very Socratic… trying to get them [students] to think about it. Rather than giving them their information, I’m pulling answers out of them.” With the switch to online delivery, this instructor felt that she no longer had much meaningful contact with her students. She articulated that, moving forward, it is important to maintain some teacher instruction as a part of the hybrid curricular model rather than shifting toward purely computer-driven instruction. She articulated, “we still need those three contact hours…when we’ve got them as a group that you could sit down and work with them…where we can actually give them more of that one-on-one time.” This faculty member’s trepidation regarding the diminished role of in-person instructors in content delivery is representative of many of the participants. Across institutions, participants maintained that whereas their in-person instruction was generally adaptive and critically engaged, the computer-driven platform was repetitive and artificial. For example, a number of instructors conveyed that the questions in the instructional modules were not sufficiently realistic, applied, or age appropriate. One instructor at Valley State Community College articulated that, “I thought some of [the activities] were kind of mamby pamby that, you know, we do need to go out in the real world and find some situations and stuff like that more than pulling up elementary school stuff for our college students.” Over half of the interviewees expressed concerns about adequately serving the particular population of developmental mathematics students. This concern was particularly expressed by women faculty. Faculty also expressed that the structure of computer-driven instructional classes makes it hard to incorporate sufficient and relevant teacher-driven instructional time and assessment. Instructors described their efforts to make alterations to incorporate instruction and assessment that are not a part of the model to make the course better align with their pedagogy. For instance, some instructors added paper-based practice assignments and quizzes to their courses. However, faculty also found themselves limited by the tension between time and course requirements. Students must complete the sequence computer-based modules (and corresponding practice problems) to complete the course. Consequently, instructors have limited flexibility for the time they can reclaim for direct instruction, even if they identify that their students would benefit from additional faculty-led instruction. One instructor at Foothills State Community College articulated this tension, saying, “I mean, the ideal situation would be that we’d lecture for part of the class and then students could work for part of the class hopefully on what we just lectured on. But that’s (laughs) rarely the case.” Degree of Autonomy to Shape the Curriculum Faculty also expressed that requirements for uniformity between different course sections led them to implement changes to instruction and assessment that are at odds with their pedagogical approach. In spite of the aforementioned opportunities for additions, there are limits to time and customizability given the uniformity required by institutions. One Foothills State Community College instructor reported, “So I don’t have the latitude to go into my own course, to match my own philosophy on that, you know…So like I said, there’s pros and cons to it. We’re cut down on our academic freedom but then it’s also pretty convenient that it’s set up for you, too.” Faculty at Lakes State University expressed a similar sentiment with regard to assessment. When the interviewer inquired as to the degree of flexibility in exam administration during a focus group, the Faculty attitudes toward technology-driven instruction 13 instructors clarified that they do not have any with uniformity requirements put in place under the model, and their nonverbal communication, including facial expressions, posture, and gestures, made clear their distaste for this loss of autonomy. As an example, we noted the frustration of participants related to the limits on their autonomy: P1: Each instructor that’s teaching the class will be able to give their own exam in the classroom… I: So individual teachers can choose how they would like to administer [exams]? (P1 & P2 exchange exasperated looks) P2: Well, no. Not for College Algebra. All the teachers [have to do the same thing]. In spite of the limitations with regard to mode of delivery and assessment, a positive aspect for some faculty was that they had the opportunity to fill gaps in the published curriculum through the development of content, practice problems, and videos. One Valley State Community College instructor shared that “Certain labs, they had to be written because of what we wanted them to know and [platform publisher] just didn’t have it.” Overall, there was an undertone of dissatisfaction with the more limited degree of autonomy that faculty have in the computer-driven instructional model. Platform Capabilities We also found evidence of instructor perceptions that technology platform capabilities shape pedagogy and student learning experience. Faculty expressed that the functionality of the instructional platform is limited in its ability to teach students in a way consistent with faculty’s instructional philosophy. For example, different platform publishers (with whom institutions contract to deliver the course content) have adopted various structures for modules, practice, and assessment. Something as simple as the number of practice problems that populate after a student mistake can shape the student learning experience. Indeed, we heard from instructors that the practice and assessment loops of certain programs promote better training than other programs. One Urban State University instructor’s comment captures a common complaint about the nature of practice and the lack of variety in problem types and structure: I’m not a huge fan of [our current platform] because I don’t think it does the explaining [as well as another platform] we used before. [The prior platform] wasn’t perfect either…but they learned more. This one, they’ll give you a problem. If you miss it, you can still do a similar problem. It’s the exact same wording. It’s just going to change one number in it. So if they just figure out the process, then they move on. In this quote, the faculty member expresses dissatisfaction with the two platforms the institution has used since the instructional model changed: the prior platform offers inadequate practice and assessment, while the present platform insufficiently explains concepts. While this Urban State faculty member’s experience with each platform was unique, the excerpt captures the popular sentiment that computer-based instructional platform capabilities shape teaching pedagogy, and, consequently, student learning. The issue of technology malfunction emerged as a common experience that informed teacher attitudes toward computer-driven instruction. When content is supposed to be delivered on the computer, it can derail the semester if a provider is having trouble with the platform. During the semester of data collection, we heard from multiple colleges that their software provider had been experiencing crashing issues. Instructors reported crashing issues that affected both instruction and assessment. In one instance, the platform was down for two days. Even when students had access to their modules, some instructors had issues with accessing their homepage, where they track student Education Policy Analysis Archives, Vol. 29 No. 45 14 progress and access messages from students. The instructor platform came back online at different times for different instructors, in one instance locking instructors out of their portal for two weeks. In a traditional lecture classroom, instruction is unlikely to be substantially affected for multiple class periods by technological malfunction and faculty access to electronic communication may be less central to course functioning. On the other hand, faculty articulated appreciation for dimensions they could control. For instance, faculty gave examples of their use of the platform that reflected their gratitude for greater control over test administration, in terms of both version control and the testing environment. In some cases, faculty spoke of assignments or even modules they added to the standard contracted course. At one institution, the instructors agreed to incorporate an applied capstone assignment with discrete project elements for students to complete after each module. Some departments created in- house instructional videos and interactive graphs to supplement those provided by the publisher. Faculty acknowledged that the instructional platform might be capable of accommodating other changes that would at least marginally improve overall pedagogy. The suggestion to turn off informational prompts during practice problems surfaced in multiple interviews. While faculty had not yet figured out how to make this change, they described the benefits to pedagogy and learning that could result from this change. In spite of abundant faculty assertions related to concerns about shifts in pedagogy, comments also captured instructors’ beliefs that changes to pedagogy as a function of mandated instructional reform were not entirely negative. The instructor portal facilitates tracking of student performance and earlier intervention. As a result, faculty can be more responsive to student performance. One Foothills State Community College instructor captured this functionality by sharing, “I think though what the technology allows us to do is track a little better to see…who’s falling behind, who’s not understanding, who’s doing the minimum amount of homework. In other words…we can actually get with them individually…whereas we didn’t have that access before.” Relatedly, the instructor portal may increase the ease with which faculty and students can communicate and, in so doing, facilitate faculty responsiveness and support. Similarly, one Valley State Community College faculty member described the benefits of being able to check performance and contact students about concerns: You can actually e-mail a bunch of students based on their performance…we can give them a gentle reminder that says, ‘Hey, I notice you haven't been to class. What are your plans for the course?’ or ‘I noticed you haven't been doing the assignments. Could you please come see me?’ You know, there's better ways now that we can communicate with our students, because it's all in one place. Faculty across institutions expressed concerns that their method of instruction shifted in the wake of mandated instructional change in developmental mathematics. Teaching practice in the hybrid Emporium Model shaped the reflections that instructors shared about the delivery of content, uniformity between course sections, and the role that the capabilities of the instructional platform played in instruction and faculty-student engagement. Concerns about Students Our analysis revealed that faculty’s concerns about pedagogy were rooted in their apprehension about the implications of this change for students, particularly students’ learning and future academic progress, as well as their overall orientation toward education. Faculty attitudes toward technology-driven instruction 15 Learning Faculty across colleges conveyed the concern that students do not achieve deep learning and skills mastery when primary instruction is computer-based. This concern was rooted in both apprehensions about the platform as well as about the suitability for the platform for the students served. Platform-specific concerns centered on the notion that computer-based instruction and practice does not encourage true learning. For instance, one Urban State University faculty member articulated that because of the repetitive nature of questioning, “[Students] just figure out the process, then they move on” rather than developing true understanding of the concept and process. Another dimension of faculty concerns regarding learning centered on perceived misalignment with the needs of the student population. One Foothills State Community College instructor put it so: “I could not and still cannot wrap my brain around that, how those students are struggling, can get by with no instruction. Are they not the ones that need it the most?” Instructors across settings expressed a need for teacher-delivered instruction presented greater opportunity for students to raise questions and for teachers to identify areas of need with regard to both study skills and substantive skills mastery. However, a number of participants expressed doubts that the model could adequately serve the population, even with additional, strategic supports. Another Foothills State Community College instructor added: “I just, I think they need more time. I think they need more attention. I think they need more education and study skills. None of which we can give them really under the current model.” According to the faculty participants, the computer-driven instructional model moves away from personalization and limits instructor ownership of and time spent teaching students the particular skills they need. Faculty commented that a computer-based instructional model may poorly serve students who intend to continue their studies along a math-intensive pathway. While the instructional model may work for a blitz approach to developing baseline skills for those who will not continue using math, it is not a good approach if students expect to continue taking math coursework. In particular, the faculty believed the corequisite remediation model in combination with computer-driven instruction may be particularly ill-suited to preparing students for future math study. In addition to being misaligned with the needs of the student population, faculty expressed that the assessments built into the computer-based instructional platform may not truly identify when students do not understand the material. Further, whereas in an instructor-led model with paper-based assessments students may be able to self-identify as struggling and seek help, the feedback of misaligned assessments may mask for students how little they understand and how reliant they are on the “crutches” built into the instructional model. While the bulk of participants expressed concerns about diminished student learning under the computer-based instructional model, women participants in particular also expressed concerns about diminished ability to observe student learning. Some faculty expressed that, with greater control over the speed and intensity of their learning, students may not make choices that are in the best interest of development understanding. One Urban State University faculty member gave the example that, when students do not understand a particular concept, “All they have to do is keep hitting ‘Similar Problem’ and they can have like 50 of the same problem if they want it…But they choose not to because they just want to get through it.” Many faculty participants articulated the assumption that under the computer-based instructional model, students are inclined to get through as quickly as possible, sacrificing understanding. On the other hand, faculty also expressed a number of ways in which the instructional model may be a boon for student learning. First, they generally expressed the belief that the model afforded greater access to practice and help. One Valley State Community College faculty member listed the Education Policy Analysis Archives, Vol. 29 No. 45 16 benefits: “Students can go to the homework. Students can contact their teacher in time. We have several labs that students can go and get help with. So, I think all of that has helped our success rate.” Another positive dimension of the model mentioned by faculty is the way it feeds students’ curiosity and ambition. Faculty expressed that under technology-driven instruction, students have better access to instruction and feedback on their performance. Additionally, faculty commented that the platform can better accommodate individual students’ needs for flexibility, for instance if their work schedule changes unexpectedly or if they have greater outside of school demands in other classes during particular weeks. Gaming and Cheating As a corollary of concerns about true learning after the mandated instructional change, faculty widely expressed concerns that the nature of assessment under a hybrid emporium instructional model presents opportunities to, intentionally or unintentionally, misrepresent mastery. Some faculty members expressed that the nature of the assessment sequences may lead to the gaming of the system, whereby students can learn to answer a type of question rather than developing a deep understanding of the concept and how to apply it. For some faculty, they saw this as unconscious, while others thought it was a function of the rush to get through modules. Faculty members contrasted the computer-based assessment with paper-based work. A Foothills State Community College instructor noted, “If you just had paper and pencil and had to look at the back of the textbook; you’ve got to go get help to figure it out…I think [computer-based instruction] has been a detriment in some cases because the resources are not used appropriately.” By contrast, some faculty found that the computer-based platform provides the opportunity, and perhaps additional temptation, to cheat. Faculty described that the ability to have open resources while completing homework, or to have another person log into your profile to take your quiz, led to cheating among course enrollees. One Valley State Community College faculty member described that, “They do everything online, even their benchmark quizzes…Everything online, non- proctored. And so while some students were honest about it and did it on their own, many of them were not.” This assumption about or knowledge of student deception often informed faculty assumptions about the investment of students in their learning. Education Orientation Faculty interpreted student actions and course performance as indicative of their orientation toward learning and education. Faculty grouped students into two groups: those invested in their education and thus, those who “care”, and those who do not care. Their comments conveyed that they based the formation of these opinions about students based on the degree to which students contact them through the platform and seek their help in the computer lab or during office hours. One Urban State University faculty member articulated that the students who use the in-person resources “care more. They care about their grades. They don’t care if they’re just getting by.” In many cases, faculty made comments about students that reflected assumptions about the permanence of these educational attitudes. One Foothills State Community College instructor shared, “The students who are going to do it, they’re going to do it. The students that aren’t, aren’t. And it doesn’t matter, you know, whether I keep track of things or not.” These comments fail to acknowledge the different demands that students have on their time, which may limit their ability to avail themselves of supports beyond the instructional platform. Overall, teaching with the platform does not solve challenges in the classroom; instructors using the platform face similar challenges as those who do not. For example, some students appear more invested than others and take better advantage of resources, while those who likely need the most support are less likely to reach out or leverage available resources. Faculty attitudes toward technology-driven instruction 17 Discussion Institutions of higher education are constantly seeking effective ways to improve how they offer developmental education. The transformation of in-person courses to remote or hybrid courses has emerged as a popular innovation in recent years, to say nothing of the necessity of such moves during a pandemic or times of great social unrest. A popular option is the adoption of technology and innovative pedagogical tools for instructors. Yet, as this study has illustrated, the context in which these changes occur is important to their success—how curricular changes are developed, introduced, implemented in colleges and universities, and who has a say in the redesign is essential to consider. The faculty in this study experienced a curriculum redesign policy that they had little or no say in. This challenged both the traditional instructional model and the traditional power that faculty hold in the domain of curricular design. As we have illustrated above, faculty expressed concerns about their pedagogical practices and how students were interacting with the technology-based curriculum. For example, consistent with prior literature (O’Meara et al., 2019; Webber & Rogers, 2018), in our interviews, women generally voiced greater dissatisfaction with the new instructional model and cited concerns about diminished ability to observe student learning, in particular. Consequently, these types of concerns informed their assumptions and beliefs about the curriculum they were mandated to implement, which in turn, influenced their behaviors in their classrooms. In other words, a cyclical process informed how faculty interfaced with students and shaped students’ academic experiences. As colleges and universities contemplate shifting to technology-based curriculum and pedagogy for developmental education, particularly in the wake of the COVID-19 pandemic, it is important to consider how this process is developed and implemented. Similar to their primary and secondary school teachers (Davidson et al., 2014; Roellke & Rice, 2008), our study demonstrates that sudden and mandated changes may not align with best pedagogical practices for instructors in higher education. Put differently, when requiring instructors to make instructional changes without considering their pedagogical best practices or expert input, there is a mismatch between their expertise and their content delivery. Instructors in our study noted that the Emporium Model asked them to change the way they taught—they had to adjust, and sometimes limit or expand, their pedagogy to meet the needs of the technology platform. The Emporium Model restrained instructor’s time and customizability in their teaching, changed the nature of their communication with students, and dramatically changed content delivery. These pedagogical changes led faculty to consider the implications for their students, especially in regard to their learning and education orientation. Specifically, faculty were concerned that the technology platform did not encourage or support true learning and allowed students to game the system and/or cheat. While the faculty in this study described positive aspects to the change to technology based- curriculum, including faster grading and facilitated communication, overall, concerns about meeting all student needs prevailed. The majority of participants believed the model did not support students’ need for differential learning. It is interesting to compare these findings to recent research on the learning outcomes of students in technology-driven developmental math courses. When examining the pass rates of students in these courses, research has demonstrated negative effects of technology-driven instruction on students’ subsequent academic performance (Boatman, 2019; Kozakowski, 2019). The perceptions of faculty members regarding student learning are particularly relevant here. Without faculty input in the development and design of curriculum changes in higher education, meeting students’ needs remains a challenge. Additionally, by considering the experiences of the people who carry out mandated changes, this study also identifies the pedagogical limitations Education Policy Analysis Archives, Vol. 29 No. 45 18 to technology-based education. These findings, then, have important implications for higher education practice in developmental education and technology-driven curriculum changes. This is specifically important to consider with regard to educational access and equity for all students. There are a number of related research questions that would lend policy- and practice- oriented contributions in this area but that were outside of the scope of this research. Mixed methods research could contribute to greater understanding of the implementation of top-down instructional reform and implications for student outcomes. For example, future research could estimate the correlation between the attitudes and experiences of faculty members and student success in the wake of policy-driven instructional change. Additionally, institutions and faculty may dedicate differential resources, time, and effort to learning and leveraging novel instructional tools; institutional resources dedicated to implementation, as well as faculty use of online tools and temporal investment may be related to their faculty attitudes and experiences, students’ grades, and perceptions of Emporium coursework. Policy Implications Include Educators in Decisions about Curricular Policy and the Implementation of Redesigns Faculty expertise should be central to the discussion of institution- and system-wide curricular decisions, as well as to the implementation of any instructional redesign. Top-down policy mandates fail to leverage instructor expertise regarding pedagogy and how best to serve the students at their particular institutions. Community college faculty report that extrinsic factors, such as mandates perceived to originate outside the institution, influence both their job dissatisfaction and their ability to form social connections with students and fellow faculty (Rodriguez & Rima, 2020). Our findings suggest that institutions and systems considering instructional changes give experienced instructors in that subject area key roles in the discussion and formulation of any policies. When it comes to implementation, our findings suggest that redesign efforts should include committee participation, as was evident in faculty discussions of implementation in our interview data. Consequently, any institution- or system-level guidance for classroom-level practice should build in collaboration among instructors charged with implementing the mandated change. Collaborative efforts among experienced instructors (those slated to teach the coursework in question) may contribute to instructional and course policy design that is better aligned with student needs, particularly if the committee represents a broad coalition that brings pedagogical expertise, representation of student background and experiences, and knowledge of the circumstances and needs of the student body. We recommend that as faculty implement ongoing changes to improve developmental instruction that they engage with counselors, advisors, and institutional research staff to ensure that their attitudes are informed by a representative picture of the student experience. In addition to including educators in the adoption and implementation stages of the policy process, we also encourage coordination of these efforts through the National Organization for Student Success (NOSS), formerly the National Association for Developmental Education (NADE). In addition to the national organization, NOSS operates statewide chapters in over half of all U.S. states. These organizations have the expertise and content knowledge to assist in implementation of new reforms efforts, either through providing professional development to faculty or in generating buy-in from other state-level organizations and administrators. Collaboration with state and/or national organizations focused on developmental education may aid in the implementation process. Faculty attitudes toward technology-driven instruction 19 Provide Faculty with Ongoing Professional Development Our findings suggest that institutional structure and governance should allow faculty and departments the flexibility to implement redesigns in distinct ways based on their pedagogy and the needs of students. Faculty differ widely in terms of their instructional expertise, from the wealth of knowledge and experience of long-time instructors to the novice instructional training of newly minted PhDs. Therefore, professional development plays a critical role for faculty at all levels. Our participants reported that ongoing professional development for developmental math instructors at their institutions is limited to that provided (free of charge as part of marketing and retention) by the instructional software company. Even so, instructors felt limited in their knowledge of the possibilities of the platforms. As institutions and systems consider policies mandating instructional change and the use of particular models, they should include provisions for funding and structures to support continuing professional development to ensure that pedagogy and implementation do not suffer with the change in authority around instructional delivery. Our interview data also suggested that faculty find that students are limited in their technological skills in ways that are consequential to their learning under a technology-driven instructional model. Decision makers considering instructional design mandates should include provisions for the development and provision of professional development that focuses on facilitating the instruction of baseline technology skills and the development of technology-focused pedagogy and differentiation. As instructors become confident in their application of technology- focused pedagogy, they will also be better equipped to support their struggling students. Consider Innovative Systems of Learning Accountability Our findings suggest that, due to the nature of their design, technology-based teaching platforms may avail themselves to student gaming and/or cheating. These shortcomings can limit or hinder student learning. As educational technology companies develop ways to improve their platforms, colleges and universities using these programs can implement procedures to hold both students and instructors accountable for student learning. We encourage educators using technology-based curricula to consider different ways of testing student knowledge and holding them accountable to true learning. For example, while faculty may depend on computer programs for content delivery and homework completion, they could administer quizzes and exams using scantron technology. We also urge departmental and institutional leaders to allow instructors to apply different techniques to their pedagogy, including testing their students’ learning. Additionally, we call to technology developers to redesign the systems of accountability set within their technology platforms—developers need to consider the implications of students being able to cheat or game the platform and should work with educators to figure out ways to prevent or at least minimize this. Study Limitations As with most studies, there are limitations to the generalizability of our findings. First, the context in which the mandated policy change happens is specific to the state of Tennessee. As a whole, higher education institutions in the state were already thinking about possible changes to their curriculum. This context may have shaped how participants responded in the interviews. Additionally, we were only able to capture the faculty voices of those who volunteered and agreed to take place in the study. We encourage future research to inquire about the experiences with mandated higher education reforms with more diverse sample—this should include faculty across different disciplines and different titles (e.g., adjunct lectures, non-tenure track faculty, tenured faculty). Education Policy Analysis Archives, Vol. 29 No. 45 20 Further, faculty experiences are inevitably influenced by the implementation of changes in response to policy. For instance, in light of changing state requirements, the support institutions provide faculty to navigate policy changes and novel instructional requirements will necessarily shape their attitudes and experiences. Support to navigate these challenges varies by institution. Thus, future research should also consider facets of implementation, including institutional culture and faculty-leadership dynamics. This research did not leverage comparison between in-person and technology-facilitated instruction. The ever-changing nature and fast-paced adoption of online education, particularly in times of crisis, makes it essential to understand potential differences in effectiveness between instructional modalities and how it technology-facilitated instruction compares to traditional in- person courses. Exploring this is beyond the scope of this paper and we recommend that future research also consider how the efficacy of online education implementation is related to the gains and losses experienced by students. While it is important to account for the study’s limitations, we illustrate the importance of considering faculty voices in policy changes. As our findings show, higher education educators face similar challenges their K-12 counterparts experience when forced changes occur: frustration, confinement, and, above all, concerns about student success. Future research and policy development, especially, must include this expertise. Conclusion The experiences and perceptions of the teachers in our study illuminate the implications of state-mandated developmental math instructional reform for college instructors. Traditionally, faculty hold the ultimate authority regarding curricular design and implementation. By taking the form of instruction out of the hands of instructors, the state-mandated instructional redesign contributed to the redistribution of power and limited the role of expert instructor perspectives on shaping the classroom environment and pedagogy. Our study illustrates the importance of considering faculty perceptions and experiences in the development of higher education policies. As institutions of higher education consider how to improve upon existing curricula with technology-based techniques, it is especially important for leaders and policymakers to consider faculty expertise. This is especially true when it comes to the design of remedial courses: instructors of these courses may not wield the same power and influence as instructors of upper-level coursework, but their skillsets are incredibly specialized, and their success undergirds the success of open-access colleges. Even with a collaborative model for institution-level design and implementation, the instructors in our study were concerned about the implications of the instructional model and were dissatisfied with their level of involvement in the redesign. Trends in the field suggest that computer-driven instruction in developmental courses may continue to have increasing prevalence. The ability of instructors to identify and address limitations of the technology will be essential to building and sustaining student success under this model. Our findings suggest that institutions and college systems that consider mandating changes to curricular delivery may find greater buy-in and success in implementation if they dedicate funding or build structures for the continued professional development of instructors. Acknowledgments We thank Treva Berryman at the Tennessee Board of Regents for sharing her knowledge of the curricular redesign process in Tennessee, the learning support coordinators at Tennessee public Faculty attitudes toward technology-driven instruction 21 colleges for their assistance in arranging our visits and interviews, and the interviewed faculty members for sharing their expertise and experiences. We are also grateful to our discussant, Karen Paulson, our fellow panelists, and session participants at ASHE 2019 for their feedback on an early version of this manuscript. The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305C140007 to Teachers College, Columbia University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. References Acevedo, M. (2019). Auditing quality: A critical exploration of faculty members’ experiences with quality matters. (Doctoral dissertation). Florida International University: FIU Digital Commons. Akroyd, D., Patton, B., & Bracken, S. (2013). Factors that predict full-time community college faculty engagement in online instruction. Community College Journal of Research and Practice, 37(3), 185–195. https://doi.org/10.1080/10668926.2013.739512 Anderson, G. (2006). Assuring quality/resisting quality assurance: Academics’ responses to ‘quality’ in some Australian universities. Quality in Higher Education, 12(2), 161-173. https://doi.org/10.1080/13538320600916767 Astin, A. W. (1977). Four critical years. Effects of college on beliefs, attitudes, and knowledge. Jossey-Bass. Bickerstaff, S., Fay, M. P., & Trimble, M. J. (2016). Modularization in developmental mathematics in two states: Implementation and early outcomes. Columbia University Academic Commons. Boatman, A. (2019). Computer-based math remediation: Evidence from technology-centered instruction in two-year and four-year colleges. [Working Paper]. Center for the Analysis of Postsecondary Readiness. https://eric.ed.gov/?id=ED602852 Boatman, A., & Kramer, J.W. (2019). Content and connections: Students’ responses to a hybrid emporium instructional model in developmental mathematics. [Working Paper]. Center for the Analysis of Postsecondary Readiness. https://eric.ed.gov/?id=ED602862] Creswell, J.W. (2014). Research design: Qualitative, quantitative and mixed methods approaches (4th ed.). Sage. Davidson, L. Y. J., Richardson, M., & Jones, D. (2014). Teachers' perspective on using technology as an instructional tool. Research in Higher Education Journal, 24, 1-25. Glasgow, Y., & Keim, M. C. (2005). Community college psychology faculty uses and attitudes toward computer-assisted instruction. Community College Journal of Research and Practice, 29(7), 547–555. https://doi.org/10.1080/10668920590954006 Gray-Barnett, N. K. (2001). An analysis of the academic success achieved by five freshman cohorts through a community college developmental education program (Doctoral dissertation). ProQuest Dissertations and Theses database. (UMI No. ATT 3034561). Greene, W. L., Caskey, M. M., Musser, P. M., Samek, L. L., Casbon, J., & Olson, M. (2008). Caught in the middle again: Accountability & the changing practice of middle school teachers. Middle Grades Research Journal, 3(4), 41-72. Hora, M.T. (2014). Exploring faculty beliefs about student learning and their role in instructional decision-making. The Review of Higher Education, 38(1), 37-70. https://doi.org/10.1353/rhe.2014.0047 Jackowski, M. B., & Akroyd, D. (2010). Technology usage among community college faculty. Community College Journal of Research and Practice, 34(8), 624–644. https://doi.org/10.1080/10668920701831530 Jackson, K., Gibbons, L., & Sharpe, C. J. (2017). Teachers’ views of students’ mathematical https://doi.org/10.1080/10668926.2013.739512 https://doi.org/10.1080/13538320600916767 https://eric.ed.gov/?id=ED602852 https://eric.ed.gov/?id=ED602862 https://doi.org/10.1080/10668920590954006 https://doi.org/10.1353/rhe.2014.0047 https://doi.org/10.1080/10668920701831530 Education Policy Analysis Archives, Vol. 29 No. 45 22 capabilities: Challenges and possibilities for ambitious reform. Teachers College Record, 119, 1– 43. Jaggars, S. S., & Bickerstaff, S. (2018) Developmental education: The evolution of research and reform. In M. Paulsen (Ed.), Higher education: Handbook of theory and research Vol. 33 (pp. 469– 503). Springer. Kim, D., Twombly, S., & Wolf-Wendel, L. (2008). Factors predicting community college faculty satisfaction with instructional autonomy. Community College Review, 35(3), 159–180. https://doi.org/10.1177/0091552107310111 Kozakowski, W. (2019). Moving the classroom to the computer lab: Can online learning with in- person support improve outcomes in community colleges? Economics of Education Review, 70, 159-172. https://doi.org/10.1016/j.econedurev.2019.03.004 LeBaron, J., & McFadden, A. (2008). The brave new world of e-learning: A department’s response to mandated change. Interactive Learning Environments, 16(2), 143–156. https://doi.org/10.1080/10494820701331079 Levin, J. S., Kater, S., & Wagoner, R. L. (2006). Community college faculty at work in the new economy. Palgrave Macmillan. Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Sage Publications. Newton, J. (2000). Feeding the beast or improving quality?: Academics' perceptions of quality assurance and quality monitoring. Quality in Higher Education, 6(2), 153-163. https://doi.org/10.1080/713692740 Newton, J. (2002). Views from below: Academics coping with quality. Quality in Higher Education, 8(1), 39-61. https://doi.org/10.1080/13538320220127434 O’Meara, K., Lennartz, C. J., Kuvaeva, A., Jaeger, A., & Misra, J. (2019). Department conditions and practices associated with faculty workload satisfaction and perceptions of equity. The Journal of Higher Education, 90(5), 744–772. https://doi.org/10.1080/00221546.2019.1584025 Pascarella, E. T., & Terenzini, P. T. (1976). Informal interaction with faculty and freshman ratings of academic and non-academic experiences of college. Journal of Educational Research, 70, 35-41. https://doi.org/10.1080/00220671.1976.10884944 Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Sage. Paver, J., Walker, D. A., & Hung, W.-C. (2014). Factors that predict the integration of technology for instruction by community college adjunct faculty. Community College Journal of Research and Practice, 38(1), 68–85. https://doi.org/10.1080/10668926.2013.799449 Ravitch, S. M., & Riggan, M. (2012). Reason & rigor: How conceptual frameworks guide research. Sage. Rodriguez, C. C., & Rima, B. (2020). The perceptions of community college faculty teaching in learning communities. Community College Journal of Research and Practice, 44(7), 469-481. https://doi.org/10.1080/10668926.2019.1615014 Roellke, C., & Rice, J. K. (2008). Responding to teacher quality and accountability mandates: The perspective of school administrators and classroom teachers. Leadership and Policy in Schools, 7(3), 264-295. https://doi.org/10.1080/15700760701822124 Saldaña, J. (2015). The coding manual for qualitative researchers. Sage. Shore, C., & Wright, S. (2000). Coercive accountability: The rise of audit culture in higher education. In M. Strathern (Ed.), Audit cultures: Studies in accountability, ethics and the academy (pp. 57-89). Routledge. Tabata, L., & Johnsrud, L. (2008). The impact of faculty attitudes toward technology, distance education, and innovation. Research in Higher Education, 49, 625–646. https://doi.org/10.1007/s11162-008-9094-7 https://doi.org/10.1177/0091552107310111 https://doi.org/10.1016/j.econedurev.2019.03.004 https://doi.org/10.1080/10494820701331079 https://doi.org/10.1080/713692740 https://doi.org/10.1080/13538320220127434 https://doi.org/10.1080/00221546.2019.1584025 https://doi.org/10.1080/00220671.1976.10884944 https://doi.org/10.1080/10668926.2013.799449 https://doi.org/10.1080/10668926.2019.1615014 https://doi.org/10.1080/15700760701822124 https://doi.org/10.1007/s11162-008-9094-7 Faculty attitudes toward technology-driven instruction 23 Tennessee Higher Education Commission. (2020). 2020 Tennessee Higher Education Factbook, Table 2.7. https://www.tn.gov/content/dam/tn/thec/bureau/research/other- research/factbook/FB%202020_Full%20Draft_Suppressed_Final.pdf Twigg, C. A. (2011). The math emporium: A silver bullet for higher education. Change: The Magazine of Higher Learning, 43(3), 25–34. https://doi.org/10.1080/00091383.2011.569241 Webber, K.L. & Rogers, S. M. (2018). Gender differences in faculty member job satisfaction: Equity forestalled? Research in Higher Education, 59(8), 1105-1132. https://doi.org/10.1007/s11162- 018-9494-2 Worthington, F., & Hodgson, J. (2005). Academic labor and the politics of quality in higher education: A critical evaluation of the conditions of possibility of resistance. Critical Quarterly, 47(1-2), 96-110. https://doi.org/10.1111/j.0011-1562.2005.00617.x https://www.tn.gov/content/dam/tn/thec/bureau/research/other-research/factbook/FB%202020_Full%20Draft_Suppressed_Final.pdf https://www.tn.gov/content/dam/tn/thec/bureau/research/other-research/factbook/FB%202020_Full%20Draft_Suppressed_Final.pdf https://doi.org/10.1080/00091383.2011.569241 https://doi.org/10.1007/s11162-018-9494-2 https://doi.org/10.1007/s11162-018-9494-2 https://doi.org/10.1111/j.0011-1562.2005.00617.x Education Policy Analysis Archives, Vol. 29 No. 45 24 Appendix A Semi-structured Protocol for Faculty Interviews 1. Please describe the version of the Emporium Model that you use on your campus a. Probe about number and classification of instructors, classroom setting, required hours, assignments, number of students per section, etc. 2. How do you use technology in class? Does this differ from the way in which technology was used in the past? If so, how so? 3. Has instruction changed with the adoption of the Emporium Model? 4. What were the most difficult parts of the redesign process for you? For students? 5. How does your institution determine whether class/lab hours should be required? How many hours should be required? 6. How does your institution/do you encourage students to go to class and/or lab? 7. Should all students be required to spend the same amount of time in the lab? 8. Have you seen evidence that students develop and apply essential knowledge and skills in challenging and meaningful ways? 9. Have you seen evidence that of redundancy or unnecessary overlap within the curriculum school wide? 10. Have you seen evidence that this model supports students of various skill levels and fosters student-driven development? 11. Have you seen evidence that lessons encourage students to develop and apply problem solving abilities? 12. What essential skills and proficiencies in mathematics are being applied and/or developed through the student work? 13. Are skill levels and individual learning styles being incorporated into the lessons? 14. In your experience, does the Emporium Model change the nature or frequency of interaction between students and instructors? 15. Is there a particular learner who most benefits from the Emporium Model? Who is most negatively impacted? 16. How does a redesigned course compare to the time and effort put forth to teach a traditional three-credit course? 17. What sort of training did you receive when making the transition to the Emporium Model? Was it effective? What additional training would be beneficial? How often should training take place? 18. How do we ensure ongoing consistency among instructors? Faculty attitudes toward technology-driven instruction 25 About the Authors Jenna W. Kramer RAND Corporation jkramer@rand.org ORCID: https://orcid.org/0000-0001-7145-9204 Jenna W. Kramer is an associate policy researcher at RAND. Her research examines the implementation and evaluation of interventions and policies targeted for college access and success. Stephany Cuevas Chapman University stcuevas@chapman.edu ORCID: https://orcid.org/0000-0001-6865-9600 Stephany Cuevas an Assistant Professor of Education at Chapman University. Her research focuses on family engagement in students’ higher education access and success. Angela Boatman Boston College boatmana@bc.edu ORCID: https://orcid.org/0000-0003-3890-8164 Angela Boatman is an Associate Professor of Higher Education at Boston College. Her research focuses on the evaluation of college access and completion policies, particularly in the areas of remediation, financial aid, and community college student success. education policy analysis archives Volume 29 Number 45 April 5, 2021 ISSN 1068-2341 Readers are free to copy, display, distribute, and adapt this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, the changes are identified, and the same license applies to the derivative work. More details of this Creative Commons license are available at https://creativecommons.org/licenses/by-sa/4.0/. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank, SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. https://creativecommons.org/licenses/by-sa/4.0/ http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Education Policy Analysis Archives, Vol. 29 No. 45 26 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: Melanie Bertrand, David Carlson, Lauren McArthur Harris, Danah Henriksen, Eugene Judson, Mirka Koro-Ljungberg, Daniel Liou, Scott Marley, Keon McGuire, Molly Ott, Iveta Silova (Arizona State University) Madelaine Adelman Arizona State University Amy Garrett Dikkers University of North Carolina, Wilmington Gloria M. Rodriguez University of California, Davis Cristina Alfaro San Diego State University Gene V Glass Arizona State University R. Anthony Rolle University of Houston Gary Anderson New York University Ronald Glass University of California, Santa Cruz A. G. Rud Washington State University Michael W. Apple University of Wisconsin, Madison Jacob P. K. Gross University of Louisville Patricia Sánchez University of University of Texas, San Antonio Jeff Bale University of Toronto, Canada Eric M. Haas WestEd Janelle Scott University of California, Berkeley Aaron Benavot SUNY Albany Julian Vasquez Heilig California State University, Sacramento Jack Schneider University of Massachusetts Lowell David C. Berliner Arizona State University Henry Braun Boston College Kimberly Kappler Hewitt University of North Carolina Greensboro Noah Sobe Loyola University Casey Cobb University of Connecticut Aimee Howley Ohio University Nelly P. Stromquist University of Maryland Arnold Danzig San Jose State University Steve Klees University of Maryland Jaekyung Lee SUNY Buffalo Benjamin Superfine University of Illinois, Chicago Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Adai Tefera Virginia Commonwealth University Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago A. Chris Torres Michigan State University David E. DeMatthews University of Texas at Austin Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley Chad d'Entremont Rennie Center for Education Research & Policy Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago John Diamond University of Wisconsin, Madison Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Matthew Di Carlo Albert Shanker Institute William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Sherman Dorn Arizona State University Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Michael J. Dumas University of California, Berkeley Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Kathy Escamilla University of Colorado, Boulder Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Yariv Feniger Ben-Gurion University of the Negev Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Melissa Lynn Freeman Adams State College Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Rachael Gabriel University of Connecticut Susan L. Robertson Bristol University Miri Yemini Tel Aviv University, Israel Faculty attitudes toward technology-driven instruction 27 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Coordinador (Español/Latinoamérica): Ignacio Barrenechea (Universidad de San Andrés), Ezequiel Gomez Caride (Universidad de San Andres/ Pontificia Universidad Católica Argentina) Editor Coordinador (Español/Norteamérica): Armando Alcántara Santuario (Universidad Nacional Autónoma de México) Editor Coordinador (Español/España): Antonio Luzon (Universidad de Granada) Editores Asociados: Jason Beech (Monash University), Angelica Buendia, (Metropolitan Autonomous University), Gabriela de la Cruz Flores (Universidad Nacional Autónoma de Mëxico), Alejandra Falabella (Universidad Alberto Hurtado, Chile), Carmuca Gómez-Bueno (Universidad de Granada), Carolina Guzmán-Valenzuela (Universidade de Chile), Cesar Lorenzo Rodriguez Uribe (Universidad Marista de Guadalajara), Antonia Lozano-Díaz (University of Almería), Sergio Gerardo Málaga Villegas (Instituto de Investigación y Desarrollo Educativo, Universidad Autónoma de Baja California (IIDE-UABC)), María Teresa Martín Palomo (University of Almería), María Fernández Mellizo- Soto (Universidad Complutense de Madrid), Tiburcio Moreno (Autonomous Metropolitan University-Cuajimalpa Unit), José Luis Ramírez, (Universidad de Sonora), Axel Rivas (Universidad de San Andrés), Maria Veronica Santelices (Pontificia Universidad Católica de Chile) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') Education Policy Analysis Archives, Vol. 29 No. 45 28 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Coordenadores: Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Editores Associadas: Andréa Barbosa Gouveia (Universidade Federal do Paraná), Kaizo Iwakami Beltrao, (EBAPE/FGVl), Sheizi Calheira de Freitas (Federal University of Bahia), Maria Margarida Machado (Federal University of Goiás / Universidade Federal de Goiás), Gilberto José Miranda (Universidade Federal de Uberlândia) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Geovana Mendonça Lunardi Mendes Universidade do Estado de Santa Catarina Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil